Database query method adaptive to AI computing framework and computer equipment

Abstract

The embodiment of the invention discloses a database query method adaptive to an AI computing framework and computer equipment. The method comprises the following steps: converting a to-be-responded SQL query statement into an abstract syntax tree, replacing the abstract syntax tree by utilizing a bidirectional mapping relation data structure of a database, generating an operation sequence by utilizing data operation corresponding to each node in the replaced target abstract syntax tree, and then generating an execution directed acyclic graph according to the operation sequence. And combining Python codes of the tensor processing function for executing data operation corresponding to each node in the directed acyclic graph to obtain an execution code, and finally, according to a bidirectional mapping relation data structure, replacing an output two-dimensional tensor obtained after the execution code runs to obtain an answer set. Wherein the Python code of the tensor processing function is constructed according to an AI computing framework specified by a user and a heterogeneous accelerator applicable to the AI computing framework, so that the Python code is adapted to the AI computing framework, and the database query efficiency is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of database technology, and in particular relates to a database query method and computer equipment adapted to an AI computing framework. Background Art

[0002] In today's digital age, data has become an important basis for corporate decision-making. Relational database management system is the main tool for data storage and management. It uses structured query language (SQL) to express user access requests to data for query services.

[0003] At present, the mainstream database query method is mainly executed in the CPU (Central Processing Unit), which fails to fully utilize the heterogeneous accelerators in the computer, such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), etc. These heterogeneous accelerators have powerful parallel processing capabilities and can significantly improve the data processing speed. At the same time, AI (Artificial Intelligence) computing frameworks such as PyTorch, TensorFlow, PaddlePaddle and MindSpore are booming, which can fully utilize heterogeneous accelerators to improve the efficiency of deep learning model training and reasoning applications. Since database queries rely on relational algebra operations and need to process complex relational operations such as joins, merges and aggregations, which are significantly different from the tensor operations implemented in the AI ​​computing framework, it is extremely challenging to implement database queries in the AI ​​computing framework. How to adapt the AI ​​computing framework so that database queries can fully utilize heterogeneous accelerators to improve execution efficiency is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide a database query method and computer device adapted to the AI ​​computing framework, which can solve the problem that the database query method in the prior art cannot utilize heterogeneous accelerators to improve execution efficiency.

[0005] To achieve the above object, the present invention provides a database query method adapted to an AI computing framework, the method comprising: Before answering any SQL query statement in a given database, construct a bidirectional mapping relationship data structure between all values ​​of all string type fields contained in the database and serial numbers; Obtaining a SQL query statement to be answered, and converting the SQL query statement into an abstract syntax tree; Using the bidirectional mapping relationship data structure, the field value of the string value node in the abstract syntax tree is replaced with the field value sequence number to obtain a replaced target abstract syntax tree; Traversing all nodes in the target abstract syntax tree, generating an operation sequence using data operations corresponding to all nodes in the target abstract syntax tree, where the data operations correspond to tensor processing functions defined using Python code; Generate an execution directed acyclic graph according to the operation sequence, determine the input two-dimensional tensor parameters of the passive nodes in the execution directed acyclic graph, and obtain the Python code of the tensor processing function corresponding to the data operation of each node in the execution directed acyclic graph according to the user-specified AI computing framework and the user-specified heterogeneous accelerator applicable to the AI ​​computing framework, wherein the passive node refers to a node without any directed edge pointing to it; According to the hierarchical structure of the directed acyclic graph, the Python codes of the tensor processing functions corresponding to the data operations of all the nodes in the directed acyclic graph are merged to obtain the execution code; Calling the Python interpreter to run the execution code to obtain an output two-dimensional tensor; According to the bidirectional mapping relationship data structure, the field value sequence number in the output two-dimensional tensor is replaced with the field value to obtain the answer set of the SQL query statement.

[0006] In the method of the present invention, generating and executing a directed acyclic graph according to the operation sequence includes: According to the principle that each data operation corresponds to one node, and one node A has a directed edge pointing to another node B if and only if the output tensor of the tensor processing function corresponding to the node A is an input parameter of the tensor processing function corresponding to the node B, the operation sequence is converted into a directed acyclic graph; The nodes of the directed acyclic graph are optimized according to preset optimization rules for maintaining query semantics to obtain an execution directed acyclic graph.

[0007] In the method of the present invention, the optimization rules include: If a first selection operation in the directed acyclic graph has a directed edge pointing to a second selection operation and no other directed edges point to other nodes, the first selection operation and the second selection operation are merged into a new selection operation.

[0008] If a first projection operation in the directed acyclic graph has a directed edge pointing to a second projection operation and no other directed edges point to other nodes, the first projection operation and the second projection operation are merged into a new projection operation.

[0009] If a third selection operation in the directed acyclic graph has a directed edge pointing to the third projection operation and no other directed edges pointing to other nodes, and the third projection operation has a directed edge pointing to the fourth selection operation and no other directed edges pointing to other nodes, then the third selection operation and the fourth selection operation are merged into a new selection operation, the directed edges issued by the new selection operation are adjusted to point to the third projection operation, and the directed edges issued by the third projection operation are adjusted to point to all nodes originally pointed to by the fourth selection operation.

[0010] If a fourth projection operation in the directed acyclic graph has a directed edge pointing to the next operation and no other directed edges pointing to other nodes, the next operation has a directed edge pointing to the fifth projection operation and no other directed edges pointing to other nodes, and the next operation does not involve fields in the input parameters of the fourth projection operation and the fifth projection operation, then the fourth projection operation and the fifth projection operation are merged into a new projection operation, the directed edges issued by the new projection operation are adjusted to point to the next operation, and the directed edges issued by the next operation are adjusted to point to all nodes originally pointed to by the fifth projection operation.

[0011] If a certain merge operation in the directed acyclic graph has a directed edge pointing to the fifth selection operation and no other directed edges pointing to other nodes, and all the field names involved in the fifth selection operation that do not appear in the nested subtable are the field names of the left table of the merge operation, or are the field names of the right table of the merge operation, then the specific directed edge originally pointing to the merge operation is adjusted to point to the fifth selection operation, the directed edges issued by the merge operation are adjusted to point to all nodes originally pointed to by the fifth selection operation, and the directed edges issued by the fifth selection operation are adjusted to point to the merge operation, and the merge operation is an inner join operation , any one of the full outer join operation, the left outer join operation and the right outer join operation, the specific directed edge is one of the two directed edges originally pointing to the merge operation; if all the field names involved in the fifth selection operation that do not appear in the nested subtables are the left table field names of the merge operation, then the specific directed edge is the directed edge originally pointing to the merge operation for transferring the left table data structure; if all the field names involved in the fifth selection operation that do not appear in the nested subtables are the right table field names of the merge operation, then the specific directed edge is the directed edge originally pointing to the merge operation for transferring the right table data structure.

[0012] In the method of the present invention, the building of a bidirectional mapping relationship data structure includes: For all string type fields in the database, a string array and a prefix tree of different field values ​​in all the fields are constructed, the string array is used to map integer serial numbers to strings, and the prefix tree is used to map field values ​​to integer serial numbers in the string array. In the method of the present invention, the corresponding tensor processing function of the data operation needs to call a basic tensor processing function, and the basic tensor processing function is defined using an interface function provided in the AI ​​computing framework specified by the user. If the parameters of the basic tensor processing function include operating device parameters, the operating device parameters are set to the identifier of the heterogeneous accelerator specified by the user.

[0013] To implement the above method, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above database query method adapted to the AI ​​computing framework.

[0014] The embodiments of the present invention have the following beneficial effects: The present invention provides a database query method adapted to an AI computing framework, which can convert SQL query statements into a sequence of relational operations, covering five basic relational algebra operations such as union, difference, Cartesian product, projection and selection that are semantically equivalent to SQL, and is applicable to various types of SQL query statements. Furthermore, the present invention pre-constructs a bidirectional mapping relationship data structure between all values ​​and sequence numbers of all string type fields contained in the database, and uses Python code to define a tensor processing function for data operations to adapt to the AI ​​computing framework specified by the user and the heterogeneous accelerator applicable to the AI ​​computing framework, so that the execution process of the SQL query statement can be adapted to different AI computing frameworks and heterogeneous accelerators, thereby improving the execution efficiency of the query statement. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] in: Figure 1 A flowchart of a database query method adapted to an AI computing framework in an embodiment of the present invention; Figure 2 A schematic diagram of a process of generating and executing a directed acyclic graph according to an operation sequence in an embodiment of the present invention; Figure 34 is a structural block diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0018] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0019] The database query method adapted to the AI ​​computing framework in this application is implemented in two stages: the preparation stage and the execution stage, which are described below: 1. Preparation The preparation phase mainly includes two parts. One part is to build a bidirectional mapping relationship data structure between all values ​​and serial numbers of all string type fields contained in a given database before using the database to answer SQL query statements. The other part is to use Python code to define tensor processing functions for data operations based on the user-specified AI computing framework and the user-specified heterogeneous accelerator applicable to the AI ​​computing framework.

[0020] Among them, building a bidirectional mapping relationship data structure of the database includes: for all string type fields in the database, building a string array and a prefix tree of different field values ​​in all these fields, the string array is used to map integer serial numbers to strings, and the prefix tree is used to map field values ​​to integer serial numbers in the string array.

[0021] Specifically, for all string type fields in the database, a string array consisting of all different values ​​of all string type fields is first constructed, each element in the array stores a different field value, and then a prefix tree of all elements in the string array is constructed. Among them, the prefix tree is also called a dictionary tree, which is a multi-way tree structure. Each node in the prefix tree does not store a complete field value, but a string. Each node in the prefix tree actually corresponds to the prefix of a field value. When it is necessary to obtain the prefix of the field value corresponding to a node, it is necessary to traverse from the root node to the node and connect the strings stored in each node on the traversal path. If the field value prefix corresponding to a node is a complete field value, the node also stores the serial number of the complete field value in the string array. When constructing the prefix tree of all elements in the string array, a traditional prefix tree can be constructed, or an optimized prefix tree can be constructed. In a traditional prefix tree, each node stores only one character, while in an optimized prefix tree such as an Adaptive Radix Tree (ART) and a Height Optimized Trie (HOT), each node stores a string. Therefore, the height of the optimized prefix tree is smaller, and the efficiency of querying its serial number according to a given field value is higher. It should be noted that the specific implementation of the prefix tree is not limited in the embodiments of the present application, as long as its serial number can be queried according to a given field value.

[0022] It should be noted that the bidirectional mapping relationship data structure of the database is constructed mainly because the AI ​​computing framework uses tensor operations and requires the use of tensor processing functions. The input parameters and output results of the tensor processing function are both two-dimensional tensors. In order to adapt to the AI ​​computing framework, it is necessary to convert the value of each string type field in the database into a numerical value in a two-dimensional tensor, that is, the above-mentioned field value sequence number. In addition, the output obtained by using the AI ​​computing framework to complete the SQL statement query is the output of the tensor processing function, that is, a two-dimensional tensor. In order to be able to restore it to the value of the string type field, the above-mentioned prefix tree is required. Therefore, it is necessary to construct a bidirectional mapping relationship data structure between all values ​​and sequence numbers of all string type fields in the database.

[0023] In the embodiment of the present application, Python code is also required to define the tensor processing function of data operations. Among them, data operations include selection operations, projection operations, aggregation operations, merge operations and set operations; further, merge operations include inner union operations, full outer union operations, left outer union operations and right outer union operations, and set operations include union operations, intersection operations and difference operations. The above data operations have covered five basic relational algebra operations such as union, difference, Cartesian product, projection and selection that are equivalent to SQL semantics, and are therefore applicable to various types of SQL query statements.

[0024] It should be noted that the input of the data operation is a form data structure, and the output is also a form data structure. The form data structure consists of two parts: a two-dimensional tensor (tensor) and a list of field names (column_names). The following will introduce in detail each data operation and its corresponding tensor processing function defined in Python code.

[0025] (1) Select an operation The input of the selection operation includes the main table data structure in_table, the nested sub-table list nested_tables in the filter condition, the list of field name pairs used for comparison in the filter condition cmp_column_pairs, and the comparison function list cmp_functions in the filter condition. The output of the selection operation is the new table data structure out_table after the selection and filtering. Among them, in the input of the selection operation, the number of elements in the three lists of the nested sub-table list in the filter condition, the list of field name pairs used for comparison in the filter condition, and the comparison function list in the filter condition are consistent. If an element in the nested sub-table list in the filter condition, that is, the nested sub-table, is not None, then the first field in the corresponding field name pair comes from the main table data structure, and the second field comes from the nested sub-table list. Specifically, the Python code definition of the selection operation is as follows: defselect(in_table, nested_tables, cmp_column_pairs, cmp_functions): mask =[1]*len(in_table.tensor) for cmp_pair, cmp_func, nested_table inzip(cmp_column_pairs, cmp_functions, nested_tables): key1_index = get_index(in_table.column_names,[cmp_pair[0]]) Key2_index = get_index(in_table.column_names,[cmp_pair[1]])if nested_table isNoneelse get_index(nested_table.column_names,[cmp_pair[1]]) tensor1 = column_index_select(in_table.tensor, key1_index) tensor2 = column_index_select(in_table.tensor, key2_index)if nested_table isNoneelse column_index_select(nested_table.tensor, key2_index) mask *= cmp_func(tensor1, tensor2) index =[i for i inrange(len(mask))if mask[i]>0] out_table.tensor = row_index_select(in_table.tensor, index) out_table.column_names = in_table.column_names return out_table In the tensor processing function corresponding to the above selection operation, get_index(full_names, given_names) in the select function returns the serial number of each element in the given_names list in the full_names list in the form of a list; column_index_select(tensor, index) selects the column vectors in the two-dimensional tensor tensor whose serial numbers are in the index list to form a new two-dimensional tensor and returns it; row_index_select(tensor, index) selects the row vectors in the two-dimensional tensor tensor whose row serial numbers are in the index list to form a new two-dimensional tensor and returns it. If a row serial number in the index list is -1, the corresponding row vector is set to a vector composed of all None value components; cmp_func(tensor1, tensor2) uses the specific function name cmp_func to distinguish whether tensor2 comes from the main table where tensor1 is located. If tensor2 comes from the main table where tensor1 is located, cmp_func returns a list of cmp_func comparison results for each row of vectors in tensor1 and each row of vectors in tensor2. Otherwise, cmp_func returns a list of cmp_func comparison results for each row of vectors in tensor1 and the entire two-dimensional tensor of tensor2. In addition, the semantics of the remaining functions or symbols are consistent with those of the same name in Python.

[0026] (2) Projection operation The input of the projection operation includes the main table data structure in_table and the projection field name list names. The output of the projection operation is the new table data structure out_table after projection filtering. Specifically, the Python code definition of the projection operation is as follows: defproject(in_table, names): index = get_index(in_table.column_names, names) out_table.tensor = column_index_select(in_table.tensor, index) out_table.column_names = names return out_table In the tensor processing functions corresponding to the above projection operations, get_index and column_index_select in the project function have the same semantics as the functions of the same name in the select function, which will not be repeated here.

[0027] (3) Aggregation operation The input of the aggregation operation includes the main table data structure in_table, the aggregated field name list agg_columns, the aggregate function list agg_functions, the field name list for grouping group_by_columns, and the identifier device of the running device (i.e., heterogeneous accelerator). The output of the aggregation operation is the new table data structure out_table after aggregation. Among them, the number of elements in the two lists agg_columns and agg_functions is the same, and the corresponding elements in the two lists represent the use of an aggregation function to group and aggregate a single field. When group_by_columns is equal to the empty list [], the whole table is aggregated instead of grouped aggregation. Since the grouping operation in standard SQL must be followed by the aggregation operation, the technical solution does not need to define the grouping operation separately. Specifically, the Python code definition of the aggregation operation is as follows: defaggregate(in_table, agg_columns, agg_functions, group_by_columns,device): iflen(group_by_columns)>0: group_index = get_index(in_table.column_names, group_by_columns) group_by_tensor = column_index_select(in_table.tensor, group_index) group_key, index = unique_with_inverse(group_by_tensor, device) else: group_key, index =[None],[0]*len(in_table.tensor) results =[] for i, key inenumerate(group_key): if key isNone: result, group_tensor =[], in_table.tensor else: result, group_tensor = as_list(key), row_index_select(in_table.tensor,[j for j inrange(len(index))if index(j)= i]) for agg_col, agg_func inzip(agg_columns, agg_functions): if agg_col =='*': result.append(agg_func(group_tensor)) else: agg_index = get_index(in_table.column_names,[agg_col]) agg_tensor = row_index_select(group_tensor, agg_index) result.append(mode(agg_tensor, device)if agg_func isNoneelse agg_func(agg_tensor, device)) results.append(result) out_table.tensor = as_tensor(results) out_table.column_names = group_by_columns +[f'{func}_{col}_'for func,col inzip(agg_functions, agg_columns)] return out_table In the tensor processing functions corresponding to the above aggregation operations, unique_with_inverse(tensor, device) in the aggregate function returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor, as well as a list composed of the row numbers of each row vector in the output two-dimensional vector; as_list(vector) treats the vector vector as a list and returns it; as_tensor(list) treats the two-dimensional list list as a two-dimensional tensor and returns it; agg_func(tensor, device) returns the scalar value obtained by the agg_func aggregation function for each unique component in each row of the two-dimensional tensor tensor, where each row vector of tensor is limited to only one component; mode(tensor, device) is a special form of agg_func(tensor, device), which limits the agg_func aggregation function to the mode function and returns the scalar value that appears most frequently in the unique component of each row in the two-dimensional tensor tensor; get_index, column_index_select, and row_index_select have the same semantics as the functions of the same name in the select function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0028] (4) Inner join operation The inner join operation is a type of merge operation. The input of the inner join operation includes the left table data structure in_table1, the left table associated field name key1, the right table data structure in_table2, the right table associated field name key2 and the identifier device of the running device (i.e., heterogeneous accelerator). The output of the inner join operation is the merged new table data structure out_table, where the field name list in out_table is the concatenation of the left table field name list and the right table field name list, and each row vector in out_table is composed of the left table row and the right table row whose associated field values ​​in the left table are the same as those in the right table. Specifically, the Python code definition of the inner join operation is as follows: definner_join(in_table1, key1, in_table2, key2, device): if key1 isNoneand key2 isNone: range1, range2 =range(len(in_table1.tensor)),range(len(in_table2.tensor)) index_pairs =[(i,j)for i in range1 for j in range2] else: key1_index = get_index(in_table1.column_names,[key1]) Key2_index = get_index(in_table2.column_names,[key2]) key_tensor1 = column_index_select(in_table1.tensor, key1_index) key_tensor2 = column_index_select(in_table2.tensor, key2_index) index_pairs = inner_join_index(key_tensor1, key_tensor2, device) left_tensor = row_index_select(in_table1.tensor,[p[0]for p in index_pairs]) right_tensor = row_index_select(in_table2.tensor,[p[1]for p in index_pairs]) out_table.tensor = column_concat(left_tensor, right_tensor) out_table.column_names = in_table1.column_names for name in in_table2.column_names: while name in out_table.column_names: name = name + '_+' out_table.append(name) return out_table In the inner_join function of the tensor processing function of the inner join operation, when the left table associated field name key1 and the right table associated field name key2 are None, the Cartesian product of the left table and the right table is actually calculated, otherwise the standard inner join result is calculated; column_concat(tensor1, tensor2) returns the concatenation result of the two-dimensional tensor tensor1 and the two-dimensional tensor tensor2 in the column direction, that is, assuming that the dimension of tensor1 is m×n1 and the dimension of tensor2 is m×n2, the dimension of the two-dimensional tensor returned by column_concat(tensor1, tensor2) is m×(n1+n2); the specific definition of the inner_join_index function will be given in the subsequent description and will not be repeated here; get_index, column_index_select and row_index_select have the same semantics as the functions of the same name in the select function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0029] definner_join_index(key_tensor1, key_tensor2, device): sorted_key1, key_index1 = sort(key_tensor1, device) sorted_key2, key_index2 = sort(key_tensor2, device) unique_key1, key_count1 = unique_with_counts(sorted_key1, device) unique_key2, key_count2 = unique_with_counts(sorted_key2, device) cumsum_count1 = cumsum_with_leading_zero(key_count1, device) cumsum_count2 = cumsum_with_leading_zero(key_count2, device) index1, index2 = intersect(unique_key1, unique_key2, device) index_pairs =[] for i inrange(len(index1)): for i1 inrange(cumsum_count1[index1[i]], cumsum_count1[index1[i+1]]): for i2 inrange(cumsum_count2[index2[i]], cumsum_count2[index2[i+1]]): index_pairs.append((key_index1[i1], key_index2[i2])) return index_pairs In the above inner_join_index function, sort(tensor, device) returns the sorting result of each row vector in the two-dimensional tensor tensor from small to large, and a list of the corresponding row numbers of each row vector in tensor in the sorting result; unique_with_counts(tensor, device) returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor, and a list of the number of times each row vector in the output two-dimensional tensor appears in tensor; cumsum_with_leading_zero(counts, device) returns the cumulative result list of the numerical list counts, where 0 is added as the first element of the cumulative result list, that is, the cumulative result list is [0, counts[0], counts[0]+ counts[1], counts[0]+ counts[1] + counts[2], …, counts[0] + … + counts[len(counts)-1]]; intersect(tensor1, tensor2, device) returns the row number list of each row vector in tensor1 and the corresponding row number list in tensor2 in the row intersection of the ordered two-dimensional tensors tensor1 and tensor2, in which each row vector is sorted from small to large; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0030] (5) Full external joint operation The full outer join operation is also a type of merge operation. The input and output of the full outer join operation correspond to the input and output of the inner join operation. Among them, the field name list in the output two-dimensional tensor out_table is the concatenation of the left table field name list and the right table field name list. Each row vector in out_table is concatenated by the left table row and the right table row whose associated field value in the left table is the same as the associated field value in the right table, or is obtained by expanding the right table field by the left table row whose associated field value in the left table is not in the associated field of the right table, or by expanding the left table field by the right table row whose associated field value in the right table is not in the associated field of the left table. Specifically, the Python code definition of the full outer join operation is as follows: deffull_outer_join(in_table1, key1, in_table2, key2, device): key1_index = get_index(in_table1.column_names,[key1]) Key2_index = get_index(in_table2.column_names,[key2]) key_tensor1 = column_index_select(in_table1.tensor, key1_index) key_tensor2 = column_index_select(in_table2.tensor, key2_index) index_pairs = full_outer_join_index(key_tensor1, key_tensor2, device) left_tensor = row_index_select(in_table1.tensor,[p[0]for p in index_pairs]) right_tensor = row_index_select(in_table2.tensor,[p[1]for p in index_pairs]) out_table.tensor = column_concat(left_tensor, right_tensor) out_table.column_names = in_table1.column_names for name in in_table2.column_names: while name in out_table.column_names: name = name + '_+' out_table.append(name) return out_table In the above full_outer_join function, the specific definition of the full_outer_join_index function will be given in the subsequent description and will not be repeated here; get_index, column_index_select, row_index_select and column_concat have the same semantics as the functions of the same name in the inner_join function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0031] deffull_outer_join_index(key_tensor1, key_tensor2, device): sorted_key1, key_index1 = sort(key_tensor1, device) sorted_key2, key_index2 = sort(key_tensor2, device) unique_key1, key_count1 = unique_with_counts(sorted_key1, device) unique_key2, key_count2 = unique_with_counts(sorted_key2, device) cumsum_count1 = cumsum_with_leading_zero(key_count1, device) cumsum_count2 = cumsum_with_leading_zero(key_count2, device) index1, index2 = union(unique_key1, unique_key2, device) index_pairs =[] for i inrange(len(index1)): if index1[i]<0: for i2 inrange(cumsum_count2[index2[i]], cumsum_count2[index2[i+1]]): index_pairs.append((-1, key_index2[i2])) elseif index2[i]<0: for i1 inrange(cumsum_count1[index1[i]], cumsum_count1[index1[i+1]]): index_pairs.append((key_index1[i1],-1)) else: for i1 inrange(cumsum_count1[index1[i]], cumsum_count1[index1[i+1]]): for i2 inrange(cumsum_count2[index2[i]], cumsum_count2[index2[i+1]]): index_pairs.append((key_index1[i1], key_index2[i2])) return index_pairs In the full_outer_join_index function above, union(tensor1, tensor2, device) returns the row number list of each row vector in tensor1 and the corresponding row number list in tensor2 in the row union of the ordered two-dimensional tensors tensor1 and tensor2, in which each row vector is sorted from small to large. If a row vector in the row union is not in tensor1 (or tensor2), the row number of the row vector in tensor1 (or tensor2) is set to -1; sort, unique_with_counts, and cumsum_with_leading_zero have the same semantics as the functions of the same name in the inner_join_index function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0032] (6) Left outer join operation The left outer join operation is also a type of merge operation. The input and output of the left outer join operation correspond to the input and output of the full outer join operation. The field name list in the output two-dimensional tensor out_table is the concatenation of the left table field name list and the right table field name list. Each row vector in out_table is concatenated by the left table row and the right table row whose associated field value in the left table is the same as the associated field value in the right table, or is obtained by expanding the right table field by the left table row whose associated field value in the left table is not in the associated field of the right table. Specifically, the Python code definition of the left outer join operation is as follows: defleft_outer_join(in_table1, key1, in_table2, key2, device): key1_index = get_index(in_table1.column_names,[key1]) Key2_index = get_index(in_table2.column_names,[key2]) key_tensor1 = column_index_select(in_table1.tensor, key1_index) key_tensor2 = column_index_select(in_table2.tensor, key2_index) index_pairs = left_outer_join_index(key_tensor1, key_tensor2, device) left_tensor = row_index_select(in_table1.tensor,[p[0]for p in index_pairs]) right_tensor = row_index_select(in_table2.tensor,[p[1]for p in index_pairs]) out_table.tensor = column_concat(left_tensor, right_tensor) out_table.column_names = in_table1.column_names for name in in_table2.column_names: while name in out_table.column_names: name = name + '_+' out_table.append(name) return out_table In the above left_outer_join function, the specific definition of the left_outer_join_index function will be given in the subsequent description and will not be repeated here; get_index, column_index_select, row_index_select and column_concat have the same semantics as the functions of the same name in the full_outer_join function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0033] defleft_outer_join_index(key_tensor1, key_tensor2, device): sorted_key1, key_index1 = sort(key_tensor1, device) sorted_key2, key_index2 = sort(key_tensor2, device) unique_key1, key_count1 = unique_with_counts(sorted_key1, device) unique_key2, key_count2 = unique_with_counts(sorted_key2, device) cumsum_count1 = cumsum_with_leading_zero(key_count1, device) cumsum_count2 = cumsum_with_leading_zero(key_count2, device) rindex =match(unique_key1, unique_key2, device) index_pairs =[] for i inrange(len(unique_key1)): if rindex[i]<0: for i1 inrange(cumsum_count1[i], cumsum_count1[i+1]): index_pairs.append((key_index1[i1],-1)) else: for i1 inrange(cumsum_count1[i], cumsum_count1[i+1]): for i2 inrange(cumsum_count2[rindex[i]], cumsum_count2[rindex[i+1]]): index_pairs.append((key_index1[i1], key_index2[i2])) return index_pairs In the above left_outer_join_index function, match(tensor1, tensor2, device) returns a list of row numbers of each row vector of the ordered two-dimensional tensor tensor1 in the ordered two-dimensional tensor tensor2 whose row vectors are not repeated. If a row in tensor1 does not appear in tensor2, the row number of the row in tensor2 is set to -1; sort, unique_with_counts, and cumsum_with_leading_zero have the same semantics as the functions of the same name in the full_outer_join_index function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0034] (7) Right outer join operation The right outer join operation is also a type of merge operation. The input and output of the right outer join operation correspond to the input and output of the full outer join operation. The field name list in the output two-dimensional tensor out_table is the concatenation of the left table field name list and the right table field name list. Each row vector in out_table is concatenated by the left table row and the right table row whose associated field value in the left table is the same as the associated field value in the right table, or is obtained by expanding the left table field by the right table row whose associated field value in the right table is not in the associated field of the left table. Specifically, the Python code definition of the right outer join operation is as follows: defright_outer_join(in_table1, key1, in_table2, key2, device): key1_index = get_index(in_table1.column_names,[key1]) Key2_index = get_index(in_table2.column_names,[key2]) key_tensor1 = column_index_select(in_table1.tensor, key1_index) key_tensor2 = column_index_select(in_table2.tensor, key2_index) index_pairs = right_outer_join_index(key_tensor1, key_tensor2,device) left_tensor = row_index_select(in_table1.tensor,[p[0]for p in index_pairs]) right_tensor = row_index_select(in_table2.tensor,[p[1]for p in index_pairs]) out_table.tensor = column_concat(left_tensor, right_tensor) out_table.column_names = in_table1.column_names for name in in_table2.column_names: while name in out_table.column_names: name = name + '_+' out_table.append(name) return out_table In the above right_outer_join function, the specific definition of the right_outer_join_index function will be given in the subsequent description and will not be repeated here; get_index, column_index_select, row_index_select and column_concat have the same semantics as the functions of the same name in the full_outer_join function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0035] defright_outer_join_index(key_tensor1, key_tensor2, device): sorted_key1, key_index1 = sort(key_tensor1, device) sorted_key2, key_index2 = sort(key_tensor2, device) unique_key1, key_count1 = unique_with_counts(sorted_key1, device) unique_key2, key_count2 = unique_with_counts(sorted_key2, device) cumsum_count1 = cumsum_with_leading_zero(key_count1, device) cumsum_count2 = cumsum_with_leading_zero(key_count2, device) lindex =match(unique_key2, unique_key1, device) index_pairs = [] for i inrange(len(unique_key2)): if lindex[i]<0: for i2 inrange(cumsum_count2[i], cumsum_count2[i+1]): index_pairs.append((-1, key_index2[i2])) else: for i1 inrange(cumsum_count1[lindex[i]], cumsum_count1[lindex[i+1]]): for i2 inrange(cumsum_count2[i], cumsum_count2[i+1]): index_pairs.append((key_index1[i1], key_index2[i2])) return index_pairs In the above right_outer_join_index function, sort, unique_with_counts, cumsum_with_leading_zero, and match have the same semantics as the functions of the same name in the left_outer_join_index function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0036] (8) Collection and Operation The set union operation is a type of set operation. The input of the set union operation includes the left table data structure in_table1, the right table data structure in_table2, and the identifier of the running device (i.e., heterogeneous accelerator) device. The output includes the data structure out_table, which is the union of the left table row set and the right table row set. The field name list in the left table and the right table must be consistent. The field name list in out_table is the field name list in the left table. The row vectors in out_table are not repeated. These row vectors appear in either the left table or the right table. Specifically, the Python code definition of the set union operation is as follows: defunion(in_table1, in_table2, device): sorted_tensor1, tensor_index1 = sort(in_table1.tensor, device) sorted_tensor2, tensor_index2 = sort(in_table2.tensor, device) unique_tensor1 = unique(sorted_tensor1, device) unique_tensor2 = unique(sorted_tensor2, device) index1, index2 = union(unique_tensor1, unique_tensor2, device) index2 =[index2[i]for i inrange(len(index2))if index1[i]<0and index2[i]>=0] tensor1 = row_index_select(in_table1.tensor, index1) tensor2 = row_index_select(in_table2.tensor, index2) out_table.tensor = row_concat(tensor1, tensor2) out_table.column_names = in_table1.column_names return out_table In the above union function, row_concat(tensor1, tensor2) returns the concatenation result of the two-dimensional tensor tensor1 and the two-dimensional tensor tensor2 in the row direction, that is, assuming that the dimension of tensor1 is m1×n and the dimension of tensor2 is m2×n, the dimension of the two-dimensional tensor returned by row_concat(tensor1, tensor2) is (m1+m2)×n; unique(tensor, device) returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor; sort, union and row_index_select have the same semantics as the functions of the same name in the full_outer_join_index function; the semantics of the remaining functions or symbols are consistent with the functions or symbols of the same name in Python.

[0037] (9) Set intersection operation The set intersection operation is also a type of set operation. The input of the set intersection operation includes the left table data structure in_table1, the right table data structure in_table2, and the identifier of the running device (i.e., heterogeneous accelerator) device. The output includes the data structure out_table of the intersection of the left table row set and the right table row set. The field name list in the left table and the right table must be consistent. The field name list in out_table is the field name list in the left table. The row vectors in out_table are not repeated and appear in both the left table and the right table. Specifically, the Python code definition of the set intersection operation is as follows: defintersect(in_table1, in_table2, device): sorted_tensor1, tensor_index1 = sort(in_table1.tensor, device) sorted_tensor2, tensor_index2 = sort(in_table2.tensor, device) unique_tensor1 = unique(sorted_tensor1, device) unique_tensor2 = unique(sorted_tensor2, device) index1, index2 = intersect(unique_tensor1, unique_tensor2, device) out_table.tensor = row_index_select(in_table1.tensor, index1) out_table.column_names = in_table1.column_names return out_table In the above intersect function, sort, intersect and row_index_select have the same semantics as the functions of the same name in the inner_join_index function; unique has the same semantics as the functions of the same name in the union function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0038] (10) Set difference operation The set difference operation is a type of set operation. The input of the set difference operation includes the left table data structure in_table1, the right table data structure in_table2, and the identifier of the running device (i.e., heterogeneous accelerator device). The output includes the data structure out_table of the difference set of the left table row set and the right table row set. The field name list in the left table and the right table must be consistent. The field name list in out_table is the field name list in the left table. The row vectors in out_table are not repeated. These row vectors appear in the left table but not in the right table. Specifically, the Python code definition of the set difference operation is as follows: defdifference(in_table1, in_table2, device): sorted_tensor1, tensor_index1 = sort(in_table1.tensor, device) sorted_tensor2, tensor_index2 = sort(in_table2.tensor, device) unique_tensor1 = unique(sorted_tensor1, device) unique_tensor2 = unique(sorted_tensor2, device) rindex =match(unique_tensor1, unique_tensor2, device) lindex =[i for i inrange(len(rindex))if rindex[i]<0] out_table.tensor = row_index_select(in_table1.tensor, index1) out_table.column_names = in_table1.column_names return out_table In the above difference function, sort, match, and row_index_select have the same semantics as the functions of the same name in the left_outer_join_index function; unique has the same semantics as the functions of the same name in the union function; the remaining functions or symbols have the same semantics as the functions or symbols of the same name in Python.

[0039] In this application, compared with the five basic relational algebra operations of union, difference, Cartesian product, projection and selection that are equivalent to SQL semantics, the set union operation realizes the union operation, the set difference operation realizes the difference operation, the inner union operation realizes the Cartesian product operation, the projection operation realizes the projection operation, and the selection operation realizes the selection operation. In other words, this application implements the five basic relational algebra operations of union, difference, Cartesian product, projection and selection that are equivalent to SQL semantics by defining data operations using Python code, so the data operations defined above can be applied to various types of SQL queries.

[0040] In addition, for the various data operations described in (1)-(10) above, some functions introduced in the data operations use two-dimensional tensors as input parameters or output results but are not defined using other defined tensor processing functions. Instead, they are directly defined using the interface functions provided in the user-specified AI computing framework. Therefore, they are called basic tensor processing functions. All basic tensor processing functions involved include: 1. column_index_select(tensor, index): selects the column vectors in the index list of the two-dimensional tensor tensor to form a new two-dimensional tensor and returns it.

[0041] 2. row_index_select(tensor, index): selects the row vectors in the two-dimensional tensor tensor whose row numbers are in the index list to form a new two-dimensional tensor and returns it. If a row number in the index list is -1, the corresponding row vector is set to a vector composed of all None value components.

[0042] 3. cmp_func(tensor1, tensor2): cmp_func is replaced with the specific comparison function name. cmp_func is used to distinguish whether tensor2 comes from the main table where tensor1 is located. If tensor2 comes from the main table where tensor1 is located, cmp_func returns a list of cmp_func comparison results for each row vector in tensor1 and each row vector in tensor2. Otherwise, cmp_func returns a list of cmp_func comparison results for each row vector in tensor1 and the entire two-dimensional tensor of tensor2.

[0043] 4. unique_with_inverse(tensor, device):Returns a two-dimensional tensor consisting of different row vectors in the two-dimensional tensor tensor, as well as a list of the row numbers of each row vector in tensor in the output two-dimensional vector.

[0044] 5. as_list(vector): treats the vector vector as a list and returns it.

[0045] 6. as_tensor(list): treats the two-dimensional list list as a two-dimensional tensor and returns it.

[0046] 7. agg_func(tensor, device): agg_func is replaced with the specific aggregation function name, and the scalar value obtained by the agg_func aggregation function for each unique component in each row of the two-dimensional tensor tensor is returned, where each row vector of tensor is limited to only one component.

[0047] 8. sort(tensor, device): Returns the sorted result of each row of vectors in the two-dimensional tensor tensor from small to large, and a list of the corresponding row numbers of each row of vectors in tensor in the sorted result.

[0048] 9. unique_with_counts(tensor, device) : Returns a two-dimensional tensor consisting of different row vectors in the two-dimensional tensor tensor, and outputs a list of the number of times each row vector in the two-dimensional tensor appears in tensor.

[0049] 10. cumsum_with_leading_zero(counts, device): Returns the cumulative result list of the numerical list counts, where the cumulative result list adds 0 as the first element.

[0050] 11. intersect(tensor1, tensor2, device): Returns the row number list of each row vector in tensor1 and the corresponding row number list in tensor2 in the row intersection of the ordered two-dimensional tensors tensor1 and tensor2, in which each row vector is sorted from small to large.

[0051] 12. union(tensor1, tensor2, device): Returns the row number list of each row vector in tensor1 and the corresponding row number list in tensor2 in the row union of the ordered two-dimensional tensors tensor1 and tensor2, in which each row vector is sorted from small to large; if a row vector of the union is not in tensor1, the row number of the row vector in tensor1 is set to -1; if a row vector of the union is not in tensor2, the row number of the row vector in tensor2 is set to -1.

[0052] 13. match(tensor1, tensor2, device): Returns a list of row numbers of each row vector of the ordered two-dimensional tensor tensor1 that appear in the ordered two-dimensional tensor tensor2 without duplicate row vectors; if a row in tensor1 does not appear in tensor2, the row number of the row in tensor2 is set to -1.

[0053] 14. column_concat(tensor1, tensor2): Returns the concatenation of two-dimensional tensor tensor1 and two-dimensional tensor tensor2 in the column direction. That is, assuming that the dimension of tensor1 is m×n1 and the dimension of tensor2 is m×n2, the dimension of the two-dimensional tensor returned by column_concat(tensor1, tensor2) is m×(n1+n2).

[0054] 15. row_concat(tensor1, tensor2): Returns the concatenation of two-dimensional tensor tensor1 and two-dimensional tensor tensor2 in the row direction. That is, assuming that the dimension of tensor1 is m1×n and the dimension of tensor2 is m2×n, the dimension of the two-dimensional tensor returned by row_concat(tensor1, tensor2) is (m1+m2)×n.

[0055] The above-mentioned basic tensor processing functions can all be implemented in the AI ​​computing framework, so the basic tensor processing functions can be translated into tensor operator calls of the AI ​​computing framework. Among them, the operating device parameters in the tensor operator call function of the AI ​​computing framework are filled with the identifier of the heterogeneous accelerator specified by the user during the execution of the code, that is, the corresponding tensor operator can be executed in the specified heterogeneous accelerator. Specifically, the tensor processing function corresponding to the data operation may need to call the basic tensor processing function, or may not need to call the basic tensor processing function. In the case where the tensor processing function corresponding to a certain data operation needs to call the basic tensor processing function, the basic tensor processing function is defined using the interface function provided in the AI ​​computing framework specified by the user. If the basic tensor processing function includes the operating device parameter, the operating device parameter is the identifier of the heterogeneous accelerator specified by the user.

[0056] Furthermore, based on the translation of the above basic tensor processing functions, the corresponding tensor processing functions of each data operation in (1)-(10) are translated into Python codes that the AI ​​computing framework relies on, where one AI computing framework corresponds to a set of Python codes. In this way, heterogeneous accelerators that are adapted to different AI computing frameworks can be used to implement database queries, thereby improving the query execution efficiency.

[0057] 2. Implementation Phase After completing the preparation phase, you can use the bidirectional mapping relationship data structure of the database constructed above and the tensor processing function of data operations defined by Python code to implement data query methods. For details, please refer to Figure 1 This figure is a flow chart of a database query method adapted to an AI computing framework in an embodiment of the present application, including: S101 converts the SQL query statement to be answered into an abstract syntax tree, and replaces the abstract syntax tree using a bidirectional mapping relationship data structure of the database; S102 generating an operation sequence using the data operations corresponding to each node in the replaced target abstract syntax tree; S103 generating an execution directed acyclic graph according to the operation sequence, and merging the Python codes of the tensor processing functions corresponding to the data operations of each node in the execution directed acyclic graph to obtain an execution code; S104 replaces the output two-dimensional tensor obtained after running the execution code to obtain an answer set.

[0058] In the embodiment of the present application, before answering any SQL query statement in a given database, it is necessary to construct a bidirectional mapping relationship data structure between all values ​​of all string type fields contained in the database and the sequence number. The specific content of the bidirectional mapping relationship data structure can be referred to the relevant description in the aforementioned preparation stage, which will not be repeated here.

[0059] In the embodiment of the present application, a SQL parser can be used to convert a SQL query statement into an abstract syntax tree. For example, a PostgreSQL query statement can be converted into an abstract syntax tree using a pglast parser.

[0060] In an embodiment of the present application, when using a bidirectional mapping relationship data structure to replace the field value of a string value node in an abstract syntax tree with a field value serial number, the prefix tree in the bidirectional mapping relationship data structure is specifically used to query the serial number in the string array based on the string value in the string value node, and the queried serial number is used to replace the value in the string field.

[0061] In the embodiment of the present application, after obtaining the target abstract syntax tree of the query SQL, all nodes in the target abstract syntax tree are traversed, and an operation sequence is generated using the data operations corresponding to all nodes in the target abstract syntax tree. The operation sequence refers to a sequence formed by multiple data operations, and each node in the target abstract syntax tree corresponds to a data operation, and the data operation corresponds to a tensor processing function defined using Python code. The types of data operations and their tensor processing functions can refer to (1)-(10) introduced in the aforementioned preparation stage, and will not be repeated here.

[0062] After the operation sequence is generated, a directed acyclic graph (DAG) will be generated based on the operation sequence. Figure 2 This figure is a schematic diagram of a process of generating and executing a directed acyclic graph according to an operation sequence in an embodiment of the present application, including: S201 converting the operation sequence into a directed acyclic graph according to the principle that each data operation corresponds to one node and one node A has a directed edge pointing to another node B if and only if the output tensor of the tensor processing function corresponding to the node A is an input parameter of the tensor processing function corresponding to the node B; S202 Optimize the nodes of the directed acyclic graph according to preset optimization rules for maintaining query semantics to obtain an execution directed acyclic graph.

[0063] In an embodiment of the present application, it is necessary to convert the operation sequence into a directed acyclic graph according to the conversion principle, and the conversion principle is: according to each data operation corresponding to a node, and a node A has a directed edge pointing to another node B if and only if the output tensor of the tensor processing function corresponding to node A is an input parameter of the tensor processing function corresponding to node B. After the conversion, a preliminary directed acyclic graph can be obtained. In order to further improve the query efficiency, the directed acyclic graph can be optimized. Specifically, the nodes in the converted directed acyclic graph can be optimized according to the preset optimization rules for maintaining the query semantics to obtain an execution directed acyclic graph. Among them, the optimization rules include: rules for selecting operation merging, rules for projection operation merging, rules for selecting operation exchange merging, rules for projection operation exchange merging, and rules for changing post-merge filtering to post-filtering merging. They will be introduced separately below.

[0064] The rule for merging selection operations is: if a first selection operation in a directed acyclic graph has a directed edge pointing to a second selection operation and no other directed edges pointing to other nodes, then merge the first selection operation and the second selection operation into a new selection operation. For example, if the selection operation select(in_table_1, nested_tables_1, cmp_column_pairs_1, cmp_functions_1) has a directed edge pointing to the selection operation select(in_table_2, nested_tables_2, cmp_column_pairs_2, cmp_functions_2) but no other directed edges pointing to other nodes, then merge the two selection operations into select(in_table_1, nested_tables_1 + nested_tables_2, cmp_column_pairs_1 + cmp_column_pairs_2, cmp_functions_1 + cmp_functions_2).

[0065] The rule for merging projection operations is: if a first projection operation in a directed acyclic graph has a directed edge pointing to a second projection operation and no other directed edges pointing to other nodes, then the first projection operation and the second projection operation are merged into a new projection operation. For example: if the projection operation project(in_table_1, column_names_1) has a directed edge pointing to the projection operation project(in_table_2, column_names_2) but no other directed edges pointing to other nodes, then the two projection operations are merged into project(in_table_1, column_names_2).

[0066] The rule for exchanging and merging selection operations is as follows: if a third selection operation in a directed acyclic graph has a directed edge pointing to a third projection operation and no other directed edges pointing to other nodes, and the third projection operation has a directed edge pointing to a fourth selection operation and no other directed edges pointing to other nodes, then the third selection operation and the fourth selection operation are merged into a new selection operation, and the directed edges issued by the new selection operation are adjusted to point to the third projection operation, and the directed edges issued by the third projection operation are adjusted to point to all nodes originally pointed to by the fourth selection operation. For example: if the selection operation select(in_table_1, nested_tables_1, cmp_column_pairs_1, cmp_functions_1) has a directed edge pointing to the projection operation A but no other directed edges pointing to other nodes, and the projection operation A has a directed edge pointing to the selection operation select(in_table, nested_tables_2, cmp_column_pairs_2, cmp_functions_2) but no other directed edges pointing to other nodes, then these two selection operations are merged into select(in_table, nested_tables_1 +nested_tables_2, cmp_column_pairs_1 + cmp_column_pairs_2, cmp_functions_1 +cmp_functions_2), the directed edges issued by the projection operation A are adjusted to point to all the nodes originally pointed to by the next selection operation, and the directed edges issued by the merged selection operation are adjusted to point to the projection operation A.

[0067] The rule for exchanging and merging projection operations is: if a fourth projection operation in a directed acyclic graph has a directed edge pointing to the next operation and no other directed edges pointing to other nodes, the next operation has a directed edge pointing to the fifth projection operation and no other directed edges pointing to other nodes, and the next operation does not involve fields in the input parameters of the fourth and fifth projection operations, then the fourth and fifth projection operations are merged into a new projection operation, the directed edges issued by the new projection operation are adjusted to point to the above-mentioned next operation, and the directed edges issued by the next operation are adjusted to point to all nodes originally pointed to by the fifth projection operation. For example: If the projection operation project(in_table, column_names_1) has a directed edge pointing to operation A but no other directed edges pointing to other nodes, operation A does not involve fields in column_names_1-column_names_2, and operation A has a directed edge pointing to the projection operation project(in_table,column_names_2) but no other directed edges pointing to other nodes, then these two projection operations are merged into project(in_table, column_names_2), the directed edges issued by operation A are adjusted to point to all nodes originally pointed to by the next projection operation, and the directed edges issued by the merged projection operation are adjusted to point to operation A.

[0068] The rule for changing post-merge filtering to post-filtering merging is as follows: if a merge operation in a directed acyclic graph has a directed edge pointing to the fifth selection operation and no other directed edges pointing to other nodes, and all field names involved in the fifth selection operation that do not appear in the nested subtable are the left table field names of the merge operation, or are the right table field names of the merge operation, then the specific directed edge originally pointing to the merge operation is adjusted to point to the fifth selection operation, the directed edges issued by the merge operation are adjusted to point to all nodes originally pointed to by the fifth selection operation, and the directed edges issued by the fifth selection operation are adjusted to point to the merge operation, where the merge operation is any one of an inner join operation, a full outer join operation, a left outer join operation, and a right outer join operation.

[0069] The above-mentioned specific directed edge is one of the two directed edges originally pointing to the merge operation. Specifically, if all the field names involved in the fifth selection operation that do not appear in the nested subtable are the field names of the left table of the merge operation, then the specific directed edge is the directed edge originally pointing to the merge operation for transferring the left table data structure; if all the field names involved in the fifth selection operation that do not appear in the nested subtable are the field names of the right table of the merge operation, then the specific directed edge is the directed edge originally pointing to the merge operation for transferring the right table data structure.

[0070] Here is an example of changing the merge-after-filter rule to a filter-after-merge rule: If a join operation join(in_table_1, key_1, in_table_2, key_2, device) has a directed edge pointing to a select operation select(in_table, nested_tables, cmp_column_pairs, cmp_functions) but has no other directed edges pointing to other nodes, and all field names involved in cmp_column_pairs that do not appear in nested_tables are in in_table_1.column_names or in_table_2.column_names, and join is one of inner_join, full_outer_join, left_outer_join, and right_outer_join, then the specific directed edges originally pointing to the join operation are adjusted to point to the select operation, the directed edges emitted by the join operation are adjusted to all nodes originally pointed to by the select operation, and the directed edges emitted by the select operation are adjusted to the join operation. There are originally two directed edges pointing to the merge operation join, which are used to transfer in_table_1 and in_table_2 respectively; if all the field names involved in cmp_column_pairs that do not appear in nested_tables are in in_table_1.column_names, then this specific directed edge is the directed edge that transfers in_table_1, otherwise this specific directed edge is the directed edge that transfers in_table_2.

[0071] In an embodiment of the present application, after the execution directed acyclic graph is obtained by the above-mentioned optimization rules, the input two-dimensional tensor parameters of the passive nodes in the execution directed acyclic graph can be further determined, wherein the passive node refers to a node without any directed edge pointing to it. Specifically, the input string type field of the passive node can be obtained by querying the above-mentioned given database, and then the prefix tree of the database can be queried using the input string type field to obtain the serial numbers corresponding to all values ​​of the input string type field, and the field values ​​in the string type field are replaced with the serial numbers to obtain the above-mentioned input two-dimensional tensor parameters. In addition, the Python code of the tensor processing function corresponding to the data operation of the user-specified AI computing framework, the Python code of the basic tensor processing function, and the identifier of the user-specified heterogeneous accelerator applicable to the AI ​​computing framework will be obtained to obtain the Python code of the tensor processing function corresponding to the type of data operation of each node of the execution directed acyclic graph. After obtaining the Python code for executing the tensor processing function of each node in the directed acyclic graph, further according to the hierarchical structure of each node in the directed acyclic graph, the input two-dimensional tensor parameters and the Python code of the tensor processing function corresponding to the data operation of all nodes in the directed acyclic graph are merged to obtain the execution code, and the Python parser is called to run the execution code to obtain the output two-dimensional tensor, and it is necessary to perform numerical replacement on the output two-dimensional tensor, that is, replace the field value serial number of all string type fields in the output two-dimensional tensor with the original field value. The query of the original field value requires the use of the string array in the bidirectional mapping relationship data structure of the database. After replacing the field value serial number in the output two-dimensional tensor with the field value, the answer set of the SQL query statement can be obtained to complete the query.

[0072] Compared with the prior art, this application has three key innovations. The first is that the database query method provided by the present invention can convert SQL query statements into a sequence of relational operations, effectively covering a variety of basic relational algebra operations such as union, difference, Cartesian product, projection and selection that are equivalent to SQL semantics, and is applicable to various types of SQL query statements. The second is to pre-define the tensor processing function of data operations using Python code to adapt to the user-specified AI computing framework and the heterogeneous accelerator applicable to the AI ​​computing framework, so that the database query method provided by the present invention can execute SQL query statements in the AI ​​computing framework and heterogeneous accelerator, thereby improving the execution efficiency of SQL query statements. The third is that the data query method provided by the present invention also involves converting the operation sequence of relational operations into a directed acyclic graph, and optimizing and adjusting the directed acyclic graph to maintain the SQL query statement based on the optimization rules to obtain an execution directed acyclic graph, so that the SQL query statement can be translated into Python code executed in the specified AI computing framework and heterogeneous accelerator through the finally generated execution directed acyclic graph, so that the existing AI computing framework can be used to efficiently execute SQL query statements.

[0073] Figure 3 The internal structure diagram of a computer device in one embodiment is shown. The computer device can be a terminal or a server. The computer device adopts an AI computing framework, and the AI ​​computing framework has a heterogeneous accelerator (not shown in the figure). Figure 3 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement each step in the above method embodiment. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement each step in the above method embodiment. Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0074] In one embodiment, a computer device is proposed, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps in the above-mentioned database query method adapted to the AI ​​computing framework.

[0075] In one embodiment, a computer-readable storage medium is proposed, storing a computer program. When the computer program is executed by a processor, the processor executes the steps in the above-mentioned database query method adapted to the AI ​​computing framework.

[0076] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0077] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A database query method adapted to an AI computing framework, characterized in that: The method comprises: Before answering any SQL query statement in a given database, construct a bidirectional mapping relationship data structure between all values ​​of all string type fields contained in the database and serial numbers; Obtaining a SQL query statement to be answered, and converting the SQL query statement into an abstract syntax tree; Using the bidirectional mapping relationship data structure, the field value of the string value node in the abstract syntax tree is replaced with the field value sequence number to obtain a replaced target abstract syntax tree; Traversing all nodes in the target abstract syntax tree, generating an operation sequence using data operations corresponding to all nodes in the target abstract syntax tree, where the data operations correspond to tensor processing functions defined using Python code; Generate an execution directed acyclic graph according to the operation sequence, determine the input two-dimensional tensor parameters of the passive nodes in the execution directed acyclic graph, and obtain the Python code of the tensor processing function corresponding to the data operation of each node in the execution directed acyclic graph according to the user-specified AI computing framework and the user-specified heterogeneous accelerator applicable to the AI ​​computing framework, wherein the passive node refers to a node without any directed edge pointing to it; According to the hierarchical structure of the directed acyclic graph, the Python codes of the tensor processing functions corresponding to the data operations of all the nodes in the directed acyclic graph are merged to obtain the execution code; Calling the Python interpreter to run the execution code to obtain an output two-dimensional tensor; According to the bidirectional mapping relationship data structure, the field value sequence number in the output two-dimensional tensor is replaced with the field value to obtain the answer set of the SQL query statement.

2. The method according to claim 1, characterized in that The step of generating and executing a directed acyclic graph according to the operation sequence includes: According to the principle that each data operation corresponds to one node, and one node A has a directed edge pointing to another node B if and only if the output tensor of the tensor processing function corresponding to the node A is an input parameter of the tensor processing function corresponding to the node B, the operation sequence is converted into a directed acyclic graph; The nodes of the directed acyclic graph are optimized according to preset optimization rules for maintaining query semantics to obtain an execution directed acyclic graph.

3. The method according to claim 2, characterized in that The optimization rules include: If a first selection operation in the directed acyclic graph has a directed edge pointing to a second selection operation and no other directed edges point to other nodes, the first selection operation and the second selection operation are merged into a new selection operation.

4. The method according to claim 2, characterized in that: The optimization rules include: If a first projection operation in the directed acyclic graph has a directed edge pointing to a second projection operation and no other directed edges point to other nodes, the first projection operation and the second projection operation are merged into a new projection operation.

5. The method according to claim 2, characterized in that: The optimization rules include: If a third selection operation in the directed acyclic graph has a directed edge pointing to the third projection operation and no other directed edges pointing to other nodes, and the third projection operation has a directed edge pointing to the fourth selection operation and no other directed edges pointing to other nodes, then the third selection operation and the fourth selection operation are merged into a new selection operation, the directed edges issued by the new selection operation are adjusted to point to the third projection operation, and the directed edges issued by the third projection operation are adjusted to point to all nodes originally pointed to by the fourth selection operation.

6. The method according to claim 2, characterized in that The optimization rules include: If a fourth projection operation in the directed acyclic graph has a directed edge pointing to the next operation and no other directed edges pointing to other nodes, the next operation has a directed edge pointing to the fifth projection operation and no other directed edges pointing to other nodes, and the next operation does not involve fields in the input parameters of the fourth projection operation and the fifth projection operation, then the fourth projection operation and the fifth projection operation are merged into a new projection operation, the directed edges issued by the new projection operation are adjusted to point to the next operation, and the directed edges issued by the next operation are adjusted to point to all nodes originally pointed to by the fifth projection operation.

7. The method according to claim 2, characterized in that The optimization rules include: If a certain merge operation in the directed acyclic graph has a directed edge pointing to the fifth selection operation and no other directed edges pointing to other nodes, and all the field names involved in the fifth selection operation that do not appear in the nested subtable are the field names of the left table of the merge operation, or are the field names of the right table of the merge operation, then the specific directed edge originally pointing to the merge operation is adjusted to point to the fifth selection operation, the directed edges issued by the merge operation are adjusted to point to all nodes originally pointed to by the fifth selection operation, and the directed edges issued by the fifth selection operation are adjusted to point to the merge operation, and the merge operation is an inner join operation , any one of the full outer join operation, the left outer join operation and the right outer join operation, the specific directed edge is one of the two directed edges originally pointing to the merge operation; if all the field names involved in the fifth selection operation that do not appear in the nested subtables are the left table field names of the merge operation, then the specific directed edge is the directed edge originally pointing to the merge operation for transferring the left table data structure; if all the field names involved in the fifth selection operation that do not appear in the nested subtables are the right table field names of the merge operation, then the specific directed edge is the directed edge originally pointing to the merge operation for transferring the right table data structure.

8. The method according to claim 1, characterized in that The construction of a bidirectional mapping relationship data structure includes: for all string type fields in the database, constructing a string array and a prefix tree of different field values ​​in all the fields, the string array is used to map integer serial numbers to strings, and the prefix tree is used to map field values ​​to integer serial numbers in the string array.

9. The method according to claim 1, characterized in that: The corresponding tensor processing function of the data operation needs to call a basic tensor processing function, and the basic tensor processing function is defined using an interface function provided in the AI ​​computing framework specified by the user. If the parameters of the basic tensor processing function include operating device parameters, the operating device parameters are set to the identifier of the heterogeneous accelerator specified by the user.

10. A computer device comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 9.

Images