Database Query Method and Computer Device Adapted to AI Computing Framework

Abstract

An embodiment of the present invention discloses a database query method and a computer device adapted to an AI computing framework. The method converts a SQL query statement to be answered into an abstract syntax tree, replaces the abstract syntax tree by using a bidirectional mapping relationship data structure of the database, generates an operation sequence by using data operations corresponding to each node in the replaced target abstract syntax tree, then generates an execution directed acyclic graph according to the operation sequence, merges Python codes of tensor processing functions corresponding to data operations of each node in the execution directed acyclic graph to obtain execution codes, and finally replaces a two-dimensional tensor output after running the execution codes according to the bidirectional mapping relationship data structure to obtain an answer set. Among them, the Python code of the tensor processing function is constructed according to an AI computing framework specified by a user and a heterogeneous accelerator applicable to the AI computing framework, so as to adapt to the AI computing framework and effectively improve the database query efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of database technology, and in particular relates to a database query method and computer equipment adapted to an AI computing framework. Background Art

[0002] In today's digital age, data has become an important basis for corporate decision-making. Relational database management system is the main tool for data storage and management. It uses structured query language (SQL) to express user access requests to data for query services.

[0003] At present, the mainstream database query method is mainly executed in the CPU (Central Processing Unit), which fails to fully utilize the heterogeneous accelerators in the computer, such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), etc. These heterogeneous accelerators have powerful parallel processing capabilities and can significantly improve the data processing speed. At the same time, AI (Artificial Intelligence) computing frameworks such as PyTorch, TensorFlow, PaddlePaddle and MindSpore are booming, which can fully utilize heterogeneous accelerators to improve the efficiency of deep learning model training and reasoning applications. Since database queries rely on relational algebra operations and need to process complex relational operations such as joins, merges and aggregations, which are significantly different from the tensor operations implemented in the AI computing framework, it is extremely challenging to implement database queries in the AI computing framework. How to adapt the AI computing framework so that database queries can fully utilize heterogeneous accelerators to improve execution efficiency is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide a database query method and computer device adapted to the AI computing framework, which can solve the problem that the database query method in the prior art cannot utilize heterogeneous accelerators to improve execution efficiency.

[0005] To achieve the above object, the present invention provides a database query method adapted to an AI computing framework, the method comprising:

[0006] Before answering any SQL query statement in a given database, construct a bidirectional mapping relationship data structure between all values of all string type fields contained in the database and serial numbers;

[0007] Obtain the SQL query statement to be answered, and convert the SQL query statement into an abstract syntax tree;

[0008] Utilize the bidirectional mapping relationship data structure to replace the field value of the string value node in the abstract syntax tree with a field value serial number, and obtain the target abstract syntax tree after replacement;

[0009] Traverse all nodes in the target abstract syntax tree, and generate an operation sequence by using the data operations corresponding to all nodes in the target abstract syntax tree. The data operations correspond to tensor processing functions defined using Python code;

[0010] Generate an execution directed acyclic graph according to the operation sequence, determine the input two-dimensional tensor parameters of the source-free nodes in the execution directed acyclic graph, and obtain the Python code of the tensor processing function corresponding to the data operation of each node in the execution directed acyclic graph according to the user-specified AI computing framework and the user-specified heterogeneous accelerator applicable to the AI computing framework. The source-free node refers to a node without any directed edges pointing to it;

[0011] Merge the Python code of the tensor processing functions corresponding to the data operations of all nodes in the execution directed acyclic graph according to the hierarchical structure of the execution directed acyclic graph to obtain the execution code;

[0012] Call the Python parser to run the execution code to obtain the output two-dimensional tensor;

[0013] According to the bidirectional mapping relationship data structure, replace the field value serial number in the output two-dimensional tensor with a field value to obtain the answer set of the SQL query statement.

[0014] In the method of the present invention, the generating an execution directed acyclic graph according to the operation sequence includes:

[0015] Convert the operation sequence into a directed acyclic graph according to the principle that each data operation corresponds to a node, and there is a directed edge from a node A to another node B if and only if the output tensor of the tensor processing function corresponding to node A is an input parameter of the tensor processing function corresponding to node B;

[0016] Optimize the nodes of the directed acyclic graph according to the preset optimization rules for maintaining query semantics to obtain the execution directed acyclic graph.

[0017] In the method of the present invention, the optimization rules include:

[0018] If there is a directed edge from a certain first selection operation to a second selection operation in the directed acyclic graph and no other directed edges point to other nodes, then merge the first selection operation and the second selection operation into a new selection operation.

[0019] If there is a directed edge from a certain first projection operation to a second projection operation in the directed acyclic graph and no other directed edges point to other nodes, then merge the first projection operation and the second projection operation into a new projection operation.

[0020] If there is a directed edge from a certain third selection operation to a third projection operation in the directed acyclic graph and no other directed edges point to other nodes, and there is a directed edge from the third projection operation to a fourth selection operation and no other directed edges point to other nodes, then merge the third selection operation and the fourth selection operation into a new selection operation, adjust the directed edge emitted by the new selection operation to point to the third projection operation, and adjust the directed edge emitted by the third projection operation to point to all the nodes that the fourth selection operation originally pointed to.

[0021] If there is a directed edge from a certain fourth projection operation to the next operation in the directed acyclic graph and no other directed edges point to other nodes, there is a directed edge from the next operation to a fifth projection operation and no other directed edges point to other nodes, and the next operation does not involve the fields in the input parameters of the fourth projection operation and the fifth projection operation, then merge the fourth projection operation and the fifth projection operation into a new projection operation, adjust the directed edge emitted by the new projection operation to point to the next operation, and adjust the directed edge emitted by the next operation to point to all the nodes that the fifth projection operation originally pointed to.

[0022] If there is a directed edge in a certain merge class operation in the directed acyclic graph that points to the fifth selection operation and there are no other directed edges pointing to other nodes, and all the field names involved in the fifth selection operation that do not appear in the nested sub-table are either the left table field names of the merge class operation or the right table field names of the merge class operation, then the original specific directed edge pointing to the merge class operation is adjusted to point to the fifth selection operation, the directed edges emitted by the merge class operation are adjusted to point to all the nodes that the fifth selection operation originally pointed to, and the directed edges emitted by the fifth selection operation are adjusted to point to the merge class operation. The merge class operation is any one of the inner union merge operation, full outer union merge operation, left outer union merge operation, and right outer union merge operation. The specific directed edge is one of the two directed edges that originally pointed to the merge class operation. If all the field names involved in the fifth selection operation that do not appear in the nested sub-table are the left table field names of the merge class operation, then the specific directed edge is the directed edge that originally pointed to the merge class operation and is used to transfer the left table data structure. If all the field names involved in the fifth selection operation that do not appear in the nested sub-table are the right table field names of the merge class operation, then the specific directed edge is the directed edge that originally pointed to the merge class operation and is used to transfer the right table data structure.

[0023] In the method of the present invention, the construction of the bidirectional mapping relationship data structure includes:

[0024] For all string-type fields in the database, construct a string array and a prefix tree of different field values in all the fields. The string array is used to map integer sequence numbers to strings, and the prefix tree is used to map field values to the integer sequence numbers in the string array.

[0025] In the method of the present invention, the corresponding tensor processing function of the data operation needs to call the basic tensor processing function, and the basic tensor processing function is defined by using the interface functions provided in the AI computing framework specified by the user. If the parameters of the basic tensor processing function include the running device parameter, then the running device parameter is set to the identifier of the heterogeneous accelerator specified by the user.

[0026] To implement the above method, the present invention also provides a computer device, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the above database query method for adapting to the AI computing framework.

[0027] Adopting the embodiments of the present invention has the following beneficial effects:

[0028] The present invention provides a database query method adapted to an AI computing framework, which can convert SQL query statements into a relational operation sequence, covering five basic relational algebra operations including union, difference, Cartesian product, projection, and selection that are semantically equivalent to SQL, and is applicable to various types of SQL query statements. Further, the present invention pre-constructs a bidirectional mapping relationship data structure between all values of all string-type fields included in the database and serial numbers, and defines a tensor processing function for data operations using Python code to adapt to a user-specified AI computing framework and heterogeneous accelerators applicable to the AI computing framework, so that the execution process of SQL query statements can be adapted to different AI computing frameworks and heterogeneous accelerators, thereby improving the execution efficiency of query statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Among them:

[0031] Figure 1 is a schematic flowchart of the database query method adapted to the AI computing framework in the embodiment of the present invention;

[0032] Figure 2 is a schematic flowchart of generating a directed acyclic graph for execution according to the operation sequence in the embodiment of the present invention;

[0033] Figure 3 is a structural block diagram of a computer device in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] In order to make the objectives, technical solutions, and advantages of the present application clearer, the following further details the present application in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0035] It should be noted that if there is no conflict, the various features in the embodiments of the present application can be combined with each other, and all are within the protection scope of the present application. In addition, although functional modules are divided in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the sequence in the flowchart. Furthermore, the terms "first", "second", "third", etc. used in the present application do not limit the data and the execution order, but only distinguish the same items or similar items with basically the same functions and roles.

[0036] The database query method adapted to the AI computing framework in the present application is implemented in two stages: a preparation stage and an execution stage, which will be described separately below:

[0037] I. Preparation stage

[0038] The preparation stage mainly includes two parts. One part is to construct a two-way mapping relationship data structure between all the values and serial numbers of all string-type fields included in a given database before using the database to answer SQL query statements. The other part is to define a tensor processing function for data operations using Python code based on the user-specified AI computing framework and the user-specified heterogeneous accelerator applicable to the AI computing framework.

[0039] Among them, constructing the two-way mapping relationship data structure of the database includes: for all string-type fields in the database, constructing a string array of different field values and a prefix tree in all these fields. The string array is used to map integer serial numbers to strings, and the prefix tree is used to map field values to integer serial numbers in the string array.

[0040] Specifically, for all string-type fields in the database, first construct a string array composed of all different values of all string-type fields. Each element in the array stores a different field value, and then construct a prefix tree for all elements in the string array. Among them, the prefix tree, also known as the trie tree, is a multi-way tree structure. Each node in the prefix tree does not store the complete field value, but stores a string. Each node in the prefix tree actually corresponds to a prefix of a certain field value. When it is necessary to obtain the prefix of the field value corresponding to a node, it is necessary to traverse from the root node to this node and concatenate the strings stored in each node on the traversal path. If the prefix of the field value corresponding to a certain node is the complete field value, then this node also stores the serial number of the complete field value in the string array. When constructing the prefix tree for all elements in the string array, a traditional prefix tree can be constructed, or an optimized prefix tree can be constructed. Each node in the traditional prefix tree only stores one character. In optimized prefix trees such as Adaptive Radix Tree (ART) and Height Optimized Trie (HOT), each node stores a string. Therefore, the height of the optimized prefix tree is smaller, and the efficiency of querying the serial number according to the given field value is higher. It should be noted that the specific implementation of the prefix tree is not limited in the embodiments of the present application, as long as the serial number can be queried according to the given field value.

[0041] It should be noted that the two-way mapping relationship data structure of the database is mainly constructed because the AI computing framework uses tensor operations and needs to use tensor processing functions. The input parameters and output results of the tensor processing functions are both two-dimensional tensors. In order to be able to adapt to the AI computing framework, it is necessary to convert the values of each string-type field in the database into the numerical values in the two-dimensional tensor, that is, the above-mentioned field value serial numbers. In addition, the output obtained by using the AI computing framework to complete the SQL statement query is the output of the tensor processing function, that is, a two-dimensional tensor. In order to be able to restore it to the value of the string-type field, the above-mentioned prefix tree needs to be used. Therefore, it is necessary to construct a two-way mapping relationship data structure between all the values of all string-type fields in the database and the serial numbers.

[0042] In the embodiments of the present application, it is also necessary to use Python code to define the tensor processing function for data operations. Among them, data operations include selection operations, projection operations, aggregation operations, merge operations, and set operations; further, merge operations include inner union merge operations, full outer union merge operations, left outer union merge operations, and right outer union merge operations, and set operations include union operations, intersection operations, and difference operations. The above data operations have covered five basic relational algebra operations equivalent to SQL semantics, namely union, difference, Cartesian product, projection, and selection. Therefore, they are applicable to various types of SQL query statements.

[0043] It should be noted that the input of data operation is in the form data structure, and the output is also in the form data structure. Among them, the form data structure consists of two parts: a two-dimensional tensor (tensor) and a list of field names (column_names). Each data operation and its corresponding tensor processing function defined using Python code will be introduced in detail below.

[0044] (1) Selection operation

[0045] The input of the selection operation includes the main table data structure in_table, the list of nested sub-tables nested_tables in the filtering condition, the list of pairs of field names cmp_column_pairs for comparison in the filtering condition, and the list of comparison functions cmp_functions in the filtering condition. The output of the selection operation is the new table data structure out_table after selection and filtering. Among the inputs of the selection operation, the number of elements in the three lists of the list of nested sub-tables in the filtering condition, the list of pairs of field names for comparison in the filtering condition, and the list of comparison functions in the filtering condition is the same. If an element in the list of nested sub-tables in the filtering condition, that is, the nested sub-table, is not None, then the first field in the corresponding pair of field names comes from the main table data structure, and the second field comes from this list of nested sub-tables. Specifically, the Python code definition of the selection operation is as follows:

[0046] def select(in_table, nested_tables, cmp_column_pairs, cmp_functions):

[0047] mask = [1] * len(in_table.tensor)

[0048] for cmp_pair, cmp_func, nested_table in zip(cmp_column_pairs, cmp_functions, nested_tables):

[0049] key1_index = get_index(in_table.column_names, [cmp_pair[0]])

[0050] Key2_index = get_index(in_table.column_names, [cmp_pair[1]]) if nested_table is None else get_index(nested_table.column_names, [cmp_pair[1]])

[0051] tensor1 = column_index_select(in_table.tensor, key1_index)

[0052] tensor2 = column_index_select(in_table.tensor, key2_index) if nested_table is None else column_index_select(nested_table.tensor, key2_index)

[0053] mask *= cmp_func(tensor1, tensor2)

[0054] index = [i for i in range(len(mask)) if mask[i] > 0]

[0055] out_table.tensor = row_index_select(in_table.tensor, index)

[0056] out_table.column_names = in_table.column_names

[0057] return out_table

[0058] In the tensor processing function corresponding to the above selection operation, in the select function, get_index(full_names, given_names) returns, in the form of a list, the sequence numbers of each element in the given_names list in the full_names list; column_index_select(tensor, index) selects the column vectors with sequence numbers in the index list in the two-dimensional tensor tensor to form a new two-dimensional tensor and returns it; row_index_select(tensor, index) selects the row vectors with row numbers in the index list in the two-dimensional tensor tensor to form a new two-dimensional tensor and returns it. If a certain row number in the index list is -1, the corresponding row vector is set to a vector composed of all None-valued components; cmp_func(tensor1, tensor2) distinguishes whether tensor2 comes from the main table where tensor1 is located through the specific function name cmp_func. If tensor2 comes from the main table where tensor1 is located, cmp_func returns a list of the cmp_func comparison results of each row vector in tensor1 and each row vector in tensor2. Otherwise, cmp_func returns a list of the cmp_func comparison results of each row vector in tensor1 and the entire two-dimensional tensor tensor2. In addition, the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0059] (2)Projection operation

[0060] The input of the projection operation includes the main table data structure in_table and the list of projection field names names. The output of the projection operation is the new table data structure out_table after projection filtering. Specifically, the Python code definition of the projection operation is as follows:

[0061] def project(in_table, names):

[0062] index = get_index(in_table.column_names, names)

[0063] out_table.tensor = column_index_select(in_table.tensor, index)

[0064] out_table.column_names = names

[0065] return out_table

[0066] In the tensor processing function corresponding to the above projection operation, the functions with the same names as those in the select function, such as get_index and column_index_select in the project function, have the same semantics and will not be elaborated here.

[0067] (3)Aggregation operation

[0068] The inputs of the aggregation operation include the main table data structure in_table, the list of aggregation field names agg_columns, the list of aggregation functions agg_functions, the list of field names for grouping group_by_columns, and the identifier of the running device (i.e., the heterogeneous accelerator) device. The output of the aggregation operation is the new table data structure out_table after aggregation. Among them, the number of elements in the two lists agg_columns and agg_functions is the same, and the corresponding elements in the two lists indicate grouping and aggregating a single field using an aggregation function. When group_by_columns is equal to the empty list [], whole-table aggregation rather than grouped aggregation is performed. Since the grouping operation in standard SQL must be immediately followed by the aggregation operation, this technical solution does not require a separate definition of the grouping operation. Specifically, the Python code definition of the aggregation operation is as follows:

[0069] def aggregate(in_table, agg_columns, agg_functions, group_by_columns, device):

[0070] if len(group_by_columns) > 0:

[0071] group_index = get_index(in_table.column_names, group_by_columns)

[0072] group_by_tensor = column_index_select(in_table.tensor, group_index)

[0073] group_key, index = unique_with_inverse(group_by_tensor, device)

[0074] else:

[0075] group_key, index = [None], [0] * len(in_table.tensor)

[0076] results = []

[0077] for i, key in enumerate(group_key):

[0078] if key is None:

[0079] result, group_tensor = [], in_table.tensor

[0080] else:

[0081] result, group_tensor = as_list(key), row_index_select(in_table.tensor, [j for j in range(len(index)) if index(j) == i])

[0082] for agg_col, agg_func in zip(agg_columns, agg_functions):

[0083] if agg_col == '*':

[0084] result.append(agg_func(group_tensor))

[0085] else:

[0086] agg_index = get_index(in_table.column_names, [agg_col])

[0087] agg_tensor = row_index_select(group_tensor, agg_index)

[0088] result.append(mode(agg_tensor, device) if agg_func is None else agg_func(agg_tensor, device))

[0089] results.append(result)

[0090] out_table.tensor = as_tensor(results)

[0091] out_table.column_names = group_by_columns + [f'{func}_{col}_' for func, col in zip(agg_functions, agg_columns)]

[0092] return out_table

[0093] In the tensor processing function corresponding to the above aggregation operation, in the aggregate function, unique_with_inverse(tensor, device) returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor, and a list composed of the row numbers of each row vector in the output two-dimensional vector; as_list(vector) treats the vector vector as a list and returns it; as_tensor(list) treats the two-dimensional list list as a two-dimensional tensor and returns it; agg_func(tensor, device) returns the scalar value obtained by the agg_func aggregation function for each unique component in each row of the two-dimensional tensor tensor, where each row vector of tensor is limited to having only one component; mode(tensor, device) is a special form of agg_func(tensor, device), which limits the agg_func aggregation function to the mode function and returns the scalar value that appears most frequently among the unique components in each row of the two-dimensional tensor tensor; get_index, column_index_select, and row_index_select have the same semantics as the functions with the same names in the select function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0094] (4)Inner union operation

[0095] The inner union operation is a type of merge operation. The inputs of the inner union operation include the left table data structure in_table1, the left table association field name key1, the right table data structure in_table2, the right table association field name key2, and the identifier device of the running device (i.e., the heterogeneous accelerator). The output of the inner union operation is the merged new table data structure out_table, where the list of field names in out_table is the concatenated list of the left table field name list and the right table field name list, and each row vector in out_table is composed of the concatenation of the left table row and the right table row with the same association field values in the left table and the right table. Specifically, the Python code definition of the inner union operation is as follows:

[0096] def inner_join(in_table1, key1, in_table2, key2, device):

[0097] if key1 is None and key2 is None:

[0098] range1, range2 = range(len(in_table1.tensor)), range(len(in_table2.tensor))

[0099] index_pairs = [(i, j) for i in range1 for j in range2]

[0100] else:

[0101] key1_index = get_index(in_table1.column_names, [key1])

[0102] Key2_index = get_index(in_table2.column_names, [key2])

[0103] key_tensor1 = column_index_select(in_table1.tensor, key1_index)

[0104] key_tensor2 = column_index_select(in_table2.tensor, key2_index)

[0105] index_pairs = inner_join_index(key_tensor1, key_tensor2, device)

[0106] left_tensor = row_index_select(in_table1.tensor, [p[0] for p in index_pairs])

[0107] right_tensor = row_index_select(in_table2.tensor, [p[1] for p in index_pairs])

[0108] out_table.tensor = column_concat(left_tensor, right_tensor)

[0109] out_table.column_names = in_table1.column_names

[0110] for name in in_table2.column_names:

[0111] while name in out_table.column_names:

[0112] name = name +'_+'

[0113] out_table.append(name)

[0114] return out_table

[0115] In the inner_join function of the tensor processing function for the inner union operation, when the left table join field name key1 and the right table join field name key2 are None, it actually calculates the Cartesian product of the left table and the right table; otherwise, it calculates the standard inner union result. column_concat(tensor1, tensor2) returns the concatenation result of the two-dimensional tensor tensor1 and the two-dimensional tensor tensor2 in the column direction. That is, assuming the dimension of tensor1 is m×n1 and the dimension of tensor2 is m×n2, then the dimension of the two-dimensional tensor returned by column_concat(tensor1, tensor2) is m×(n1 + n2). The specific definition of the inner_join_index function will be given in the subsequent description and will not be elaborated here. get_index, column_index_select, and row_index_select have the same semantics as the functions with the same names in the select function. The semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0116] def inner_join_index(key_tensor1, key_tensor2, device):

[0117] sorted_key1, key_index1 = sort(key_tensor1, device)

[0118] sorted_key2, key_index2 = sort(key_tensor2, device)

[0119] unique_key1, key_count1 = unique_with_counts(sorted_key1, device)

[0120] unique_key2, key_count2 = unique_with_counts(sorted_key2, device)

[0121] cumsum_count1 = cumsum_with_leading_zero(key_count1, device)

[0122] cumsum_count2 = cumsum_with_leading_zero(key_count2, device)

[0123] index1, index2 = intersect(unique_key1, unique_key2, device)

[0124] index_pairs = []

[0125] for i in range(len(index1)):

[0126] for i1 in range(cumsum_count1[index1[i]], cumsum_count1[index1[i + 1]]):

[0127] for i2 in range(cumsum_count2[index2[i]], cumsum_count2[index2[i + 1]]):

[0128] index_pairs.append((key_index1[i1], key_index2[i2]))

[0129] return index_pairs

[0130] In the above inner_join_index function, sort(tensor, device) returns the sorting result of each row vector in the two-dimensional tensor tensor from smallest to largest, as well as a list composed of the corresponding row numbers of each row vector in tensor in the sorting result; unique_with_counts(tensor, device) returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor, and a list composed of the number of occurrences of each row vector in the output two-dimensional tensor in tensor; cumsum_with_leading_zero(counts, device) returns a list of cumulative results of the numerical list counts, where 0 is added as the first element in the cumulative result list, that is, the cumulative result list is [0, counts[0], counts[0] + counts[1], counts[0] + counts[1] + counts[2], …, counts[0] + … + counts[len(counts)-1]]; intersect(tensor1, tensor2, device) returns a list of row numbers of each row vector in the row intersection of the ordered two-dimensional tensors tensor1 and tensor2 with each row vector sorted from smallest to largest in tensor1 and the corresponding row numbers in tensor2; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same name in Python.

[0131] (5)Full outer join operation

[0132] The full outer join operation also belongs to a type of join operation. The input and output of the full outer join operation correspond to those of the inner join operation respectively. Among them, the list of field names in the output two-dimensional tensor out_table is the concatenated list of the left table field name list and the right table field name list. Each row vector in out_table is formed by concatenating the left table rows and right table rows with the same associated field values in the left table and the right table, or by expanding the right table fields with the left table rows where the associated field values in the left table are not in the right table associated fields, or by expanding the left table fields with the right table rows where the associated field values in the right table are not in the left table associated fields. Specifically, the Python code definition of the full outer join operation is as follows:

[0133] def full_outer_join(in_table1, key1, in_table2, key2, device):

[0134] key1_index = get_index(in_table1.column_names, [key1])

[0135] Key2_index = get_index(in_table2.column_names, [key2])

[0136] key_tensor1 = column_index_select(in_table1.tensor, key1_index)

[0137] key_tensor2 = column_index_select(in_table2.tensor, key2_index)

[0138] index_pairs = full_outer_join_index(key_tensor1, key_tensor2, device)

[0139] left_tensor = row_index_select(in_table1.tensor, [p[0] for p in index_pairs])

[0140] right_tensor = row_index_select(in_table2.tensor, [p[1] for p in index_pairs])

[0141] out_table.tensor = column_concat(left_tensor, right_tensor)

[0142] out_table.column_names = in_table1.column_names

[0143] for name in in_table2.column_names:

[0144] while name in out_table.column_names:

[0145] name = name +'_+'

[0146] out_table.append(name)

[0147] return out_table

[0148] In the above full_outer_join function, the specific definition of the full_outer_join_index function will be given in the subsequent description and will not be elaborated here; the semantics of get_index, column_index_select, row_index_select, and column_concat are the same as those of the functions with the same names in the inner_join function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0149] def full_outer_join_index(key_tensor1, key_tensor2, device):

[0150] sorted_key1, key_index1 = sort(key_tensor1, device)

[0151] sorted_key2, key_index2 = sort(key_tensor2, device)

[0152] unique_key1, key_count1 = unique_with_counts(sorted_key1, device)

[0153] unique_key2, key_count2 = unique_with_counts(sorted_key2, device)

[0154] cumsum_count1 = cumsum_with_leading_zero(key_count1, device)

[0155] cumsum_count2 = cumsum_with_leading_zero(key_count2, device)

[0156] index1, index2 = union(unique_key1, unique_key2, device)

[0157] index_pairs = []

[0158] for i in range(len(index1)):

[0159] if index1[i] < 0:

[0160] for i2 in range(cumsum_count2[index2[i]], cumsum_count2[index2[i+1]]):

[0161] index_pairs.append((-1, key_index2[i2]))

[0162] else if index2[i]<0:

[0163] for i1 in range(cumsum_count1[index1[i]], cumsum_count1[index1[i+1]]):

[0164] index_pairs.append((key_index1[i1], -1))

[0165] else:

[0166] for i1 in range(cumsum_count1[index1[i]], cumsum_count1[index1[i+1]]):

[0167] for i2 in range(cumsum_count2[index2[i]], cumsum_count2[index2[i+1]]):

[0168] index_pairs.append((key_index1[i1], key_index2[i2]))

[0169] return index_pairs

[0170] In the above full_outer_join_index function, union(tensor1, tensor2, device) returns a list of row numbers of each row vector in tensor1 and the corresponding row numbers in tensor2 in the row union of the ordered two-dimensional tensors tensor1 and tensor2 whose row vectors have been sorted from smallest to largest. If a row vector in the row union is not in tensor1 (or tensor2), the row number of that row vector in tensor1 (or tensor2) is set to -1; the semantics of sort, unique_with_counts, and cumsum_with_leading_zero are the same as those of the functions with the same names in the inner_join_index function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0171] (6)Left outer join operation

[0172] The left outer join operation also belongs to a type of merge operation. The inputs and outputs of the left outer join operation correspond to those of the full outer join operation respectively. Among them, the list of field names in the output two-dimensional tensor out_table is the concatenated list of the left table field names and the right table field names. Each row vector in out_table is formed by concatenating the left table row and the right table row with the same associated field values in the left table and the right table, or by expanding the right table fields with the left table rows whose associated field values in the left table are not in the associated fields of the right table. Specifically, the Python code definition of the left outer join operation is as follows:

[0173] def left_outer_join(in_table1, key1, in_table2, key2, device):

[0174] key1_index = get_index(in_table1.column_names, [key1])

[0175] Key2_index = get_index(in_table2.column_names, [key2])

[0176] key_tensor1 = column_index_select(in_table1.tensor, key1_index)

[0177] key_tensor2 = column_index_select(in_table2.tensor, key2_index)

[0178] index_pairs = left_outer_join_index(key_tensor1, key_tensor2, device)

[0179] left_tensor = row_index_select(in_table1.tensor, [p[0] for p in index_pairs])

[0180] right_tensor = row_index_select(in_table2.tensor, [p[1] for p in index_pairs])

[0181] out_table.tensor = column_concat(left_tensor, right_tensor)

[0182] out_table.column_names = in_table1.column_names

[0183] for name in in_table2.column_names:

[0184] while name in out_table.column_names:

[0185] name = name +'_+'

[0186] out_table.append(name)

[0187] return out_table

[0188] In the above left_outer_join function, the specific definition of the left_outer_join_index function will be given in the subsequent description and will not be elaborated here; the semantics of get_index, column_index_select, row_index_select, and column_concat are the same as those of the functions with the same names in the full_outer_join function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0189] def left_outer_join_index(key_tensor1, key_tensor2, device):

[0190] sorted_key1, key_index1 = sort(key_tensor1, device)

[0191] sorted_key2, key_index2 = sort(key_tensor2, device)

[0192] unique_key1, key_count1 = unique_with_counts(sorted_key1, device)

[0193] unique_key2, key_count2 = unique_with_counts(sorted_key2, device)

[0194] cumsum_count1 = cumsum_with_leading_zero(key_count1, device)

[0195] cumsum_count2 = cumsum_with_leading_zero(key_count2, device)

[0196] rindex = match(unique_key1, unique_key2, device)

[0197] index_pairs = []

[0198] for i in range(len(unique_key1)):

[0199] if rindex[i] < 0:

[0200] for i1 in range(cumsum_count1[i], cumsum_count1[i + 1]):

[0201] index_pairs.append((key_index1[i1], -1))

[0202] else:

[0203] for i1 in range(cumsum_count1[i], cumsum_count1[i+1]):

[0204] for i2 in range(cumsum_count2[rindex[i]], cumsum_count2[rindex[i+1]]):

[0205] index_pairs.append((key_index1[i1], key_index2[i2]))

[0206] return index_pairs

[0207] In the above left_outer_join_index function, match(tensor1, tensor2, device) returns a list of row numbers where each row vector of the ordered two-dimensional tensor tensor1 appears in the ordered two-dimensional tensor tensor2 with no duplicate row vectors. If a row in tensor1 does not appear in tensor2, the row number where it appears in tensor2 is set to -1; sort, unique_with_counts, and cumsum_with_leading_zero have the same semantics as the functions with the same names in the full_outer_join_index function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0208] (7)Right outer join operation

[0209] The right outer join operation is also a type of join operation. The inputs and outputs of the right outer join operation correspond to those of the full outer join operation respectively. Among them, the list of field names in the output two-dimensional tensor out_table is the concatenated list of the left table field name list and the right table field name list. Each row vector in out_table is formed by concatenating the left table row and the right table row with the same associated field values in the left table and the right table, or by extending the left table fields with the right table rows whose associated field values are not in the left table associated fields. Specifically, the Python code definition of the right outer join operation is as follows:

[0210] def right_outer_join(in_table1, key1, in_table2, key2, device):

[0211] key1_index = get_index(in_table1.column_names, [key1])

[0212] Key2_index = get_index(in_table2.column_names, [key2])

[0213] key_tensor1 = column_index_select(in_table1.tensor, key1_index)

[0214] key_tensor2 = column_index_select(in_table2.tensor, key2_index)

[0215] index_pairs = right_outer_join_index(key_tensor1, key_tensor2, device)

[0216] left_tensor = row_index_select(in_table1.tensor, [p[0] for p in index_pairs])

[0217] right_tensor = row_index_select(in_table2.tensor, [p[1] for p in index_pairs])

[0218] out_table.tensor = column_concat(left_tensor, right_tensor)

[0219] out_table.column_names = in_table1.column_names

[0220] for name in in_table2.column_names:

[0221] while name in out_table.column_names:

[0222] name = name + '_+'

[0223] out_table.append(name)

[0224] return out_table

[0225] In the above right_outer_join function, the specific definition of the right_outer_join_index function will be given in the subsequent description and will not be elaborated here; the semantics of get_index, column_index_select, row_index_select, and column_concat are the same as those of the functions with the same names in the full_outer_join function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0226] def right_outer_join_index(key_tensor1, key_tensor2, device):

[0227] sorted_key1, key_index1 = sort(key_tensor1, device)

[0228] sorted_key2, key_index2 = sort(key_tensor2, device)

[0229] unique_key1, key_count1 = unique_with_counts(sorted_key1, device)

[0230] unique_key2, key_count2 = unique_with_counts(sorted_key2, device)

[0231] cumsum_count1 = cumsum_with_leading_zero(key_count1, device)

[0232] cumsum_count2 = cumsum_with_leading_zero(key_count2, device)

[0233] lindex = match(unique_key2, unique_key1, device)

[0234] index_pairs = []

[0235] for i in range(len(unique_key2)):

[0236] if lindex[i] < 0:

[0237] for i2 in range(cumsum_count2[i], cumsum_count2[i + 1]):

[0238] index_pairs.append((-1, key_index2[i2]))

[0239] else:

[0240] for i1 in range(cumsum_count1[lindex[i]], cumsum_count1[lindex[i + 1]]):

[0241] for i2 in range(cumsum_count2[i], cumsum_count2[i + 1]):

[0242] index_pairs.append((key_index1[i1], key_index2[i2]))

[0243] return index_pairs

[0244] In the above right_outer_join_index function, the semantics of the functions sort, unique_with_counts, cumsum_with_leading_zero, and match are the same as those of the functions with the same names in the left_outer_join_index function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0245] (8)Set union operation

[0246] The set union operation is a type of set operation. The inputs of the set union operation include the left table data structure in_table1, the right table data structure in_table2, and the identifier device of the running device (i.e., the heterogeneous accelerator). The output includes the data structure out_table of the union of the left table row set and the right table row set, where the list of field names in the left table and the right table must be the same, the list of field names in out_table is the list of field names in the left table, and the row vectors in out_table are non-repeating, and these row vectors either appear in the left table or in the right table. Specifically, the Python code definition of the set union operation is as follows:

[0247] def union(in_table1, in_table2, device):

[0248] sorted_tensor1, tensor_index1 = sort(in_table1.tensor, device)

[0249] sorted_tensor2, tensor_index2 = sort(in_table2.tensor, device)

[0250] unique_tensor1 = unique(sorted_tensor1, device)

[0251] unique_tensor2 = unique(sorted_tensor2, device)

[0252] index1, index2 = union(unique_tensor1, unique_tensor2, device)

[0253] index2 = [index2[i] for i in range(len(index2)) if index1[i] < 0 and index2[i] >= 0]

[0254] tensor1 = row_index_select(in_table1.tensor, index1)

[0255] tensor2 = row_index_select(in_table2.tensor, index2)

[0256] out_table.tensor = row_concat(tensor1, tensor2)

[0257] out_table.column_names = in_table1.column_names

[0258] return out_table

[0259] In the above `union` function, `row_concat(tensor1, tensor2)` returns the concatenation result of two-dimensional tensor `tensor1` and two-dimensional tensor `tensor2` in the row direction. That is, assuming the dimension of `tensor1` is `m1×n` and the dimension of `tensor2` is `m2×n`, the dimension of the two-dimensional tensor returned by `row_concat(tensor1, tensor2)` is `(m1 + m2)×n`; `unique(tensor, device)` returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor `tensor`; `sort`, `union` and `row_index_select` have the same semantics as the functions with the same names in the `full_outer_join_index` function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0260] (9)Set intersection operation

[0261] The set intersection operation is also a type of set operation. The inputs of the set intersection operation include the left table data structure `in_table1`, the right table data structure `in_table2`, and the identifier `device` of the running device (i.e., the heterogeneous accelerator). The output includes the data structure `out_table` of the intersection of the left table row set and the right table row set, where the list of field names in the left table and the right table must be the same. The list of field names in `out_table` is the list of field names in the left table, and the row vectors in `out_table` are not repeated and appear in both the left table and the right table. Specifically, the Python code definition of the set intersection operation is as follows:

[0262] def intersect(in_table1, in_table2, device):

[0263] sorted_tensor1, tensor_index1 = sort(in_table1.tensor, device)

[0264] sorted_tensor2, tensor_index2 = sort(in_table2.tensor, device)

[0265] unique_tensor1 = unique(sorted_tensor1, device)

[0266] unique_tensor2 = unique(sorted_tensor2, device)

[0267] index1, index2 = intersect(unique_tensor1, unique_tensor2, device)

[0268] out_table.tensor = row_index_select(in_table1.tensor, index1)

[0269] out_table.column_names = in_table1.column_names

[0270] return out_table

[0271] In the above intersect function, the semantics of sort, intersect, and row_index_select are the same as those of the functions with the same names in the inner_join_index function; the semantics of unique are the same as those of the function with the same name in the union function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0272] (10) Set difference operation

[0273] The set difference operation is a type of set operation. The inputs of the set difference operation include the data structure of the left table in_table1, the data structure of the right table in_table2, and the identifier device of the running device (i.e., the heterogeneous accelerator). The output includes the data structure out_table of the difference set between the row sets of the left table and the right table. The list of field names in the left table and the right table must be the same. The list of field names in out_table is the list of field names in the left table. The row vectors in out_table are not repeated, and these row vectors appear in the left table but not in the right table. Specifically, the Python code definition of the set difference operation is as follows:

[0274] def difference(in_table1, in_table2, device):

[0275] sorted_tensor1, tensor_index1 = sort(in_table1.tensor, device)

[0276] sorted_tensor2, tensor_index2 = sort(in_table2.tensor, device)

[0277] unique_tensor1 = unique(sorted_tensor1, device)

[0278] unique_tensor2 = unique(sorted_tensor2, device)

[0279] rindex = match(unique_tensor1, unique_tensor2, device)

[0280] lindex = [i for i in range(len(rindex)) if rindex[i] < 0]

[0281] out_table.tensor = row_index_select(in_table1.tensor, index1)

[0282] out_table.column_names = in_table1.column_names

[0283] return out_table

[0284] In the above difference function, the semantics of sort, match, and row_index_select are the same as those of the functions with the same names in the left_outer_join_index function; the semantics of unique are the same as those of the function with the same name in the union function; the semantics of the remaining functions or symbols are the same as those of the functions or symbols with the same names in Python.

[0285] In this application, compared with the five basic relational algebra operations of union, difference, Cartesian product, projection, and selection that are semantically equivalent to SQL, the set union operation implements the union operation, the set difference operation implements the difference operation, the inner union operation implements the Cartesian product operation, the projection operation implements the projection operation, and the selection operation implements the selection operation. That is to say, this application realizes the five basic relational algebra operations of union, difference, Cartesian product, projection, and selection that are semantically equivalent to SQL by using Python code to define data operations. Therefore, the above-defined data operations can be applied to various types of SQL queries.

[0286] In addition, for the various data operations described in (1)-(10) above, for some of the functions introduced in the data operations, some functions use a two-dimensional tensor as an input parameter or output result but are not defined by using other defined tensor processing functions. Instead, they are directly defined using the interface functions provided in the user-specified AI computing framework, so they are called basic tensor processing functions. All the basic tensor processing functions involved include:

[0287] 1. column_index_select(tensor, index): Selects the column vectors with serial numbers in the index list in the tensor two-dimensional tensor to form a new two-dimensional tensor and returns it.

[0288] 2. row_index_select(tensor, index): Selects the row vectors with row serial numbers in the index list in the tensor two-dimensional tensor to form a new two-dimensional tensor and returns it. If a row serial number in the index list is -1, the corresponding row vector is set to a vector composed of all None values.

[0289] 3. cmp_func(tensor1, tensor2): Replace cmp_func with the specific comparison function name. Distinguish whether tensor2 comes from the main table where tensor1 is located through cmp_func. If tensor2 comes from the main table where tensor1 is located, cmp_func returns a list of the cmp_func comparison results of each row vector in tensor1 and each row vector in tensor2. Otherwise, cmp_func returns a list of the cmp_func comparison results of each row vector in tensor1 and the entire tensor2 two-dimensional tensor.

[0290] 4. unique_with_inverse(tensor, device): Returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor, and a list composed of the row serial numbers of each row vector in tensor in the output two-dimensional vector.

[0291] 5. as_list(vector): Treats the vector vector as a list and returns it.

[0292] 6. as_tensor(list): Treats the two-dimensional list list as a two-dimensional tensor and returns it.

[0293] 7. agg_func(tensor, device): Replace agg_func with the specific aggregation function name. It returns the scalar value obtained by applying the agg_func aggregation function to the unique components in each row of the two-dimensional tensor tensor, where each row vector in tensor is limited to having only one component.

[0294] 8. sort(tensor, device): It returns the sorting result of each row vector in the two-dimensional tensor tensor from smallest to largest, and a list composed of the row numbers corresponding to each row vector in tensor in the sorting result.

[0295] 9. unique_with_counts(tensor, device): It returns a two-dimensional tensor composed of different row vectors in the two-dimensional tensor tensor, and a list composed of the number of occurrences of each row vector in the output two-dimensional tensor in tensor.

[0296] 10. cumsum_with_leading_zero(counts, device): It returns a list of cumulative results of the numerical list counts, where 0 is added as the first element in the cumulative result list.

[0297] 11. intersect(tensor1, tensor2, device): It returns a list of row numbers of each row vector in the row intersection of the ordered two-dimensional tensors tensor1 and tensor2 (where each row vector has been sorted from smallest to largest) in tensor1 and the corresponding row numbers in tensor2.

[0298] 12. union(tensor1, tensor2, device): It returns a list of row numbers of each row vector in the row union of the ordered two-dimensional tensors tensor1 and tensor2 (where each row vector has been sorted from smallest to largest) in tensor1 and the corresponding row numbers in tensor2; if a row vector in the union is not in tensor1, the row number of that row vector in tensor1 is set to -1; if a row vector in the union is not in tensor2, the row number of that row vector in tensor2 is set to -1.

[0299] 13. match(tensor1, tensor2, device): It returns a list of row numbers where each row vector in the ordered two-dimensional tensor tensor1 appears in the ordered two-dimensional tensor tensor2 with no repeated row vectors; if a row in tensor1 does not appear in tensor2, the row number of that row in tensor2 is set to -1.

[0300] 14. column_concat(tensor1, tensor2): Returns the concatenation result of two-dimensional tensor tensor1 and two-dimensional tensor tensor2 in the column direction. That is, assuming tensor1 has a dimension of m×n1 and tensor2 has a dimension of m×n2, the two-dimensional tensor returned by column_concat(tensor1, tensor2) has a dimension of m×(n1 + n2).

[0301] 15. row_concat(tensor1, tensor2): Returns the concatenation result of two-dimensional tensor tensor1 and two-dimensional tensor tensor2 in the row direction. That is, assuming tensor1 has a dimension of m1×n and tensor2 has a dimension of m2×n, the two-dimensional tensor returned by row_concat(tensor1, tensor2) has a dimension of (m1 + m2)×n.

[0302] The above basic tensor processing functions can all be implemented in the AI computing framework. Therefore, the basic tensor processing functions can be translated into calls to tensor operators in the AI computing framework. Among them, the running device parameter filled in the tensor operator call function of the AI computing framework is the identifier of the heterogeneous accelerator specified by the user during the execution of the code, that is, the corresponding tensor operator can be executed in the specified heterogeneous accelerator. Specifically, the tensor processing function corresponding to a data operation may or may not need to call the basic tensor processing function. In the case where the tensor processing function corresponding to a data operation needs to call the basic tensor processing function, the basic tensor processing function is defined using the interface function provided in the AI computing framework specified by the user. If the basic tensor processing function includes a running device parameter, then the running device parameter is the identifier of the heterogeneous accelerator specified by the user.

[0303] Furthermore, based on the translation of the above basic tensor processing functions, the corresponding tensor processing functions for each data operation in (1)-(10) above are translated into Python code dependent on the AI computing framework, where one set of Python code corresponds to one AI computing framework. Through the above method, it is possible to adapt the heterogeneous accelerators of different AI computing frameworks to implement database queries, achieving the purpose of improving query execution efficiency.

[0304] II. Execution Phase

[0305] After completing the work in the preparation phase, the two-way mapping relationship data structure of the constructed database and the tensor processing functions of data operations defined using Python code can be used to implement the method of data query. Specifically, please refer to Figure 1This figure is a schematic flowchart of a database query method adapted to an AI computing framework in an embodiment of the present application, including:

[0306] S101 Convert the SQL query statement to be answered into an abstract syntax tree, and replace the abstract syntax tree using the bidirectional mapping relationship data structure of the database;

[0307] S102 Generate an operation sequence using the data operations corresponding to each node in the replaced target abstract syntax tree;

[0308] S103 Generate an execution directed acyclic graph according to the operation sequence, and merge the Python code of the tensor processing function corresponding to the data operation of each node in the execution directed acyclic graph to obtain the execution code;

[0309] S104 Replace the output two-dimensional tensor obtained after running the execution code to obtain the answer set.

[0310] In an embodiment of the present application, before answering any SQL query statement in a given database, it is necessary to construct a bidirectional mapping relationship data structure between all the values of all string-type fields included in the database and the serial numbers. For the specific content related to the bidirectional mapping relationship data structure, reference can be made to the relevant description in the foregoing preparation stage, and details are not described here.

[0311] In an embodiment of the present application, an SQL parser can be used to convert an SQL query statement into an abstract syntax tree. For example, a PostgreSQL query statement can use the pglast parser for abstract syntax tree conversion.

[0312] In an embodiment of the present application, when replacing the field value of the string value node in the abstract syntax tree with the field value serial number using the bidirectional mapping relationship data structure, specifically, the prefix tree in the bidirectional mapping relationship data structure is used to query the serial number of the string value in the string array according to the string value in the string value node, and the obtained serial number is used to replace the value in the string field.

[0313] In an embodiment of the present application, after obtaining the target abstract syntax tree of the query SQL, all nodes in the target abstract syntax tree will be traversed, and an operation sequence will be generated using the data operations corresponding to all nodes in the target abstract syntax tree. Among them, the operation sequence refers to a sequence formed by multiple data operations, and each node in the target abstract syntax tree corresponds to a data operation, and the data operation corresponds to a tensor processing function defined using Python code. The types of data operations and their tensor processing functions can refer to (1)-(10) introduced in the foregoing preparation stage, and details are not described here.

[0314] After generating the operation sequence, a Directed Acyclic Graph (DAG) for execution will be further generated according to the operation sequence. Specifically, please refer to Figure 2 . This figure is a schematic diagram of the process of generating a DAG for execution according to the operation sequence in the embodiments of the present application, including:

[0315] S201 Convert the operation sequence into a directed acyclic graph according to the principle that each data operation corresponds to a node, and there is a directed edge from a node A to another node B if and only if the output tensor of the tensor processing function corresponding to node A is an input parameter of the tensor processing function corresponding to node B;

[0316] S202 Optimize the nodes of the directed acyclic graph according to the preset optimization rules for maintaining query semantics to obtain a DAG for execution.

[0317] In the embodiments of the present application, the operation sequence needs to be converted into a directed acyclic graph according to the conversion principle, which is: each data operation corresponds to a node, and there is a directed edge from a node A to another node B if and only if the output tensor of the tensor processing function corresponding to node A is an input parameter of the tensor processing function corresponding to node B. After conversion, a preliminary directed acyclic graph can be obtained. In order to further improve the query efficiency, the directed acyclic graph can be optimized. Specifically, the nodes in the directed acyclic graph obtained by conversion can be optimized according to the preset optimization rules for maintaining query semantics to obtain a DAG for execution. Among them, the optimization rules include: rules for merging selection operations, rules for merging projection operations, rules for swapping and merging selection operations, rules for swapping and merging projection operations, and rules for changing the merge after filtering to the merge after filtering. They will be introduced separately below.

[0318] The rule for merging selection operations is as follows: If there is a directed edge from a first selection operation to a second selection operation in a directed acyclic graph and no other directed edges point to other nodes, then merge the first selection operation and the second selection operation into a new selection operation. For example, if there is a directed edge from the selection operation select(in_table_1, nested_tables_1, cmp_column_pairs_1, cmp_functions_1) to the selection operation select(in_table_2, nested_tables_2, cmp_column_pairs_2, cmp_functions_2) but no other directed edges point to other nodes, then merge these two selection operations into select(in_table_1, nested_tables_1 + nested_tables_2, cmp_column_pairs_1 + cmp_column_pairs_2, cmp_functions_1 + cmp_functions_2).

[0319] The rule for merging projection operations is as follows: If there is a directed edge from a first projection operation to a second projection operation in a directed acyclic graph and no other directed edges point to other nodes, then merge the first projection operation and the second projection operation into a new projection operation. For example, if there is a directed edge from the projection operation project(in_table_1, column_names_1) to the projection operation project(in_table_2, column_names_2) but no other directed edges point to other nodes, then merge these two projection operations into project(in_table_1, column_names_2).

[0320] The rules for selective operation swapping and merging are as follows: If there is a directed edge from a certain third selective operation in a directed acyclic graph to a third projection operation and no other directed edges to other nodes, and there is a directed edge from this third projection operation to a fourth selective operation and no other directed edges to other nodes, then merge this third selective operation with this fourth selective operation into a new selective operation. Adjust the directed edge emitted by the new selective operation to point to the third projection operation, and adjust the directed edge emitted by this third projection operation to point to all the nodes that the fourth selective operation originally pointed to. For example: If there is a directed edge from the selective operation select(in_table_1, nested_tables_1, cmp_column_pairs_1, cmp_functions_1) to the projection operation A but no other directed edges to other nodes, and there is a directed edge from the projection operation A to the selective operation select(in_table, nested_tables_2, cmp_column_pairs_2, cmp_functions_2) but no other directed edges to other nodes, then merge these two selective operations into select(in_table, nested_tables_1 +nested_tables_2, cmp_column_pairs_1 + cmp_column_pairs_2, cmp_functions_1 +cmp_functions_2), adjust the directed edge emitted by the projection operation A to point to all the nodes that the latter selective operation originally pointed to, and adjust the directed edge emitted by the merged selective operation to point to the projection operation A.

[0321] The rules for swapping and merging projection operations are as follows: If there is a directed edge from a certain fourth projection operation in a directed acyclic graph to the next operation and no other directed edges pointing to other nodes, and there is a directed edge from this next operation to a fifth projection operation and no other directed edges pointing to other nodes, and this next operation does not involve the fields in the input parameters of the fourth and fifth projection operations, then the fourth and fifth projection operations are merged into a new projection operation. The directed edge emitted by the new projection operation is adjusted to point to the above-mentioned next operation, and the directed edge emitted by this next operation is adjusted to point to all the nodes that the fifth projection operation originally pointed to. For example: If the projection operation project(in_table, column_names_1) has a directed edge pointing to operation A but no other directed edges pointing to other nodes, operation A does not involve the fields in column_names_1 - column_names_2, and operation A has a directed edge pointing to the projection operation project(in_table, column_names_2) but no other directed edges pointing to other nodes, then these two projection operations are merged into project(in_table, column_names_2). The directed edge emitted by operation A is adjusted to point to all the nodes that the latter projection operation originally pointed to, and the directed edge emitted by the merged projection operation is adjusted to point to operation A.

[0322] The rule for changing the merge-after-filter to filter-after-merge is as follows: If there is a directed edge from a certain merge operation in a directed acyclic graph to a fifth selection operation and no other directed edges pointing to other nodes, and all the field names involved in the fifth selection operation that do not appear in the nested sub-tables are either the left table field names of the merge operation or the right table field names of the merge operation, then the specific directed edge originally pointing to the merge operation is adjusted to point to the fifth selection operation. The directed edge emitted by the merge operation is adjusted to point to all the nodes that the fifth selection operation originally pointed to, and the directed edge emitted by the fifth selection operation is adjusted to point to the merge operation, where the merge operation is any one of the inner union merge operation, full outer union merge operation, left outer union merge operation, and right outer union merge operation.

[0323] The above-mentioned specific directed edge is one of the two directed edges originally pointing to the merge operation. Specifically, if all the field names involved in the fifth selection operation that do not appear in the nested sub-tables are the left table field names of the merge operation, then this specific directed edge is the directed edge originally pointing to the merge operation for transmitting the left table data structure; if all the field names involved in the fifth selection operation that do not appear in the nested sub-tables are the right table field names of the merge operation, then this specific directed edge is the directed edge originally pointing to the merge operation for transmitting the right table data structure.

[0324] The following gives an example of changing the merged filtering to the filtered merging rule:

[0325] If there is a directed edge from the join operation join(in_table_1, key_1, in_table_2, key_2, device) to the select operation select(in_table, nested_tables, cmp_column_pairs, cmp_functions) but no other directed edges to other nodes, and all the field names involved in cmp_column_pairs that do not appear in nested_tables are either in in_table_1.column_names or in in_table_2.column_names, where join is one of inner_join, full_outer_join, left_outer_join, and right_outer_join, then the specific directed edge originally pointing to the join operation join is adjusted to point to the select operation select, the directed edges emitted by the join operation join are adjusted to all the nodes originally pointed to by the select operation select, and the directed edges emitted by the select operation select are adjusted to the join operation join. There are two directed edges originally pointing to the join operation join, which are used to pass in_table_1 and in_table_2 respectively; if all the field names involved in cmp_column_pairs that do not appear in nested_tables are in in_table_1.column_names, then this specific directed edge is the one passing in_table_1, otherwise this specific directed edge is the one passing in_table_2.

[0326] In the embodiment of the present application, after obtaining the execution directed acyclic graph through the above optimization rules, the input two-dimensional tensor parameters of the passive nodes in the execution directed acyclic graph can be further determined, where the passive node refers to a node without any directed edges pointing to it. Specifically, the input string type field of the passive node can be obtained by querying the above given database, and then the prefix tree of the database can be queried using the input string type field to obtain the sequence numbers corresponding to all values of the input string type field. The input two-dimensional tensor parameters can be obtained by replacing the field values in the string type field with the sequence numbers. In addition, the Python code of the tensor processing function for the data operation corresponding to each node of the execution directed acyclic graph, the Python code of the basic tensor processing function, and the identifier of the user-specified heterogeneous accelerator applicable to the AI computing framework will be used to obtain the Python code of the tensor processing function corresponding to the type of data operation of each node of the execution directed acyclic graph. After obtaining the Python code of the tensor processing function for each node in the execution directed acyclic graph, according to the hierarchical structure of each node in the execution directed acyclic graph, the input two-dimensional tensor parameters and the Python code of the tensor processing function for the data operation corresponding to all nodes in the execution directed acyclic graph are merged to obtain the execution code, and the Python parser is called to run the execution code to obtain the output two-dimensional tensor, and numerical replacement needs to be performed on the output two-dimensional tensor, that is, the field value sequence numbers of all string type fields in the output two-dimensional tensor are replaced with the original field values. The query of the original field values needs to use the string array in the bidirectional mapping relationship data structure of the database. After replacing the field value sequence numbers in the output two-dimensional tensor with the field values, the answer set of the SQL query statement can be obtained to complete the query.

[0327] Compared with the prior art, the present application has three key innovations. The first is that the database query method provided by the present invention can convert SQL query statements into a sequence of relational operation operations, effectively covering various basic relational algebra operations such as union, difference, Cartesian product, projection, and selection that are semantically equivalent to SQL, and is applicable to various types of SQL query statements. The second is to pre-define a tensor processing function for data operations using Python code to adapt to the user-specified AI computing framework and the heterogeneous accelerators applicable to the AI computing framework, so that the database query method provided by the present invention can execute SQL query statements in the AI computing framework and heterogeneous accelerators, thereby improving the execution efficiency of SQL query statements. The third is that the data query method provided by the present invention also involves converting the sequence of relational operation operations into a directed acyclic graph, and performing an optimization adjustment to maintain the SQL query statement on the directed acyclic graph to obtain an execution directed acyclic graph, so that the SQL query statement can be translated into Python code that is executed in the specified AI computing framework and heterogeneous accelerators through the finally generated execution directed acyclic graph, thereby enabling the efficient execution of SQL query statements using the existing AI computing framework.

[0328] Figure 3 shows the internal structure diagram of a computer device in an embodiment. The computer device can specifically be a terminal or a server. The computer device uses an AI computing framework, and the AI computing framework has a heterogeneous accelerator (not shown in the figure). As Figure 3 shown, the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and can also store a computer program. When the computer program is executed by the processor, the processor can implement each step in the above method embodiment. The internal memory can also store a computer program. When the computer program is executed by the processor, the processor can execute each step in the above method embodiment. Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0329] In one embodiment, a computer device is proposed, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps in the above database query method for adapting to the AI computing framework.

[0330] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, causes the processor to execute the steps in the above database query method for adapting to an AI computing framework.

[0331] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiment methods can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0332] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0333] The above-described embodiments merely represent several implementation manners of this application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application should be subject to the appended claims.

Claims

1. A database query method adapted to an AI computing framework, characterized in that, The method includes: Before answering any SQL query statement in a given database, constructing a bidirectional mapping relationship data structure between all the values of all string-type fields included in the database and serial numbers; Obtaining the SQL query statement to be answered, and converting the SQL query statement into an abstract syntax tree; Using the bidirectional mapping relationship data structure, replacing the field values of the string value nodes in the abstract syntax tree with field value serial numbers to obtain a target abstract syntax tree after replacement; Traversing all the nodes in the target abstract syntax tree, and generating an operation sequence by using the data operations corresponding to all the nodes in the target abstract syntax tree, where the data operations correspond to tensor processing functions defined using Python code; Generating an execution directed acyclic graph according to the operation sequence, determining the input two-dimensional tensor parameters of the source-free nodes in the execution directed acyclic graph, and obtaining the Python code of the tensor processing function corresponding to the data operation of each node in the execution directed acyclic graph according to the user-specified AI computing framework and the user-specified heterogeneous accelerator applicable to the AI computing framework, where the source-free nodes refer to the nodes that have no directed edges pointing to them; Merging the Python code of the tensor processing functions corresponding to the data operations of all the nodes in the execution directed acyclic graph according to the hierarchical structure of the execution directed acyclic graph to obtain execution code; Invoking a Python parser to run the execution code to obtain an output two-dimensional tensor; According to the bidirectional mapping relationship data structure, replacing the field value serial numbers in the output two-dimensional tensor with field values to obtain the answer set of the SQL query statement.

2. The method according to claim 1, wherein The generating an execution directed acyclic graph according to the operation sequence includes: Converting the operation sequence into a directed acyclic graph according to the principle that each data operation corresponds to a node, and there is a directed edge from a node A to another node B if and only if the output tensor of the tensor processing function corresponding to node A is an input parameter of the tensor processing function corresponding to node B; Optimizing the nodes of the directed acyclic graph according to a preset optimization rule for maintaining query semantics to obtain an execution directed acyclic graph.

3. The method according to claim 2, wherein The optimization rules include: If there is a directed edge from a certain first selection operation in the directed acyclic graph to a second selection operation and there are no other directed edges pointing to other nodes, then merging the first selection operation and the second selection operation into a new selection operation.

4. The method according to claim 2, wherein The optimization rules include: If there is a directed edge from a certain first projection operation in the directed acyclic graph to a second projection operation and there are no other directed edges pointing to other nodes, then merging the first projection operation and the second projection operation into a new projection operation.

5. The method according to claim 2, wherein The optimization rules include: If there is a directed edge from a certain third selection operation in the directed acyclic graph to a third projection operation and no other directed edges point to other nodes, and there is a directed edge from the third projection operation to a fourth selection operation and no other directed edges point to other nodes, then merge the third selection operation and the fourth selection operation into a new selection operation, adjust the directed edges emitted by the new selection operation to point to the third projection operation, and adjust the directed edges emitted by the third projection operation to point to all the nodes that the fourth selection operation originally pointed to.

6. The method according to claim 2, wherein The optimization rules include: If there is a directed edge from a certain fourth projection operation in the directed acyclic graph to the next operation and no other directed edges point to other nodes, there is a directed edge from the next operation to a fifth projection operation and no other directed edges point to other nodes, and the next operation does not involve the fields in the input parameters of the fourth projection operation and the fifth projection operation, then merge the fourth projection operation and the fifth projection operation into a new projection operation, adjust the directed edges emitted by the new projection operation to point to the next operation, and adjust the directed edges emitted by the next operation to point to all the nodes that the fifth projection operation originally pointed to.

7. The method according to claim 2, characterized in that, The optimization rules include: If there is a directed edge from a certain merge class operation in the directed acyclic graph to a fifth selection operation and no other directed edges point to other nodes, and all the field names that appear in the fifth selection operation and are not in the nested sub-tables are the left table field names of the merge class operation, or all are the right table field names of the merge class operation, then adjust the specific directed edge that originally pointed to the merge class operation to point to the fifth selection operation, adjust the directed edges emitted by the merge class operation to point to all the nodes that the fifth selection operation originally pointed to, and adjust the directed edges emitted by the fifth selection operation to point to the merge class operation. The merge class operation is any one of the inner union merge operation, full outer union merge operation, left outer union merge operation, and right outer union merge operation. The specific directed edge is one of the two directed edges that originally pointed to the merge class operation. If all the field names that appear in the fifth selection operation and are not in the nested sub-tables are the left table field names of the merge class operation, then the specific directed edge is the directed edge that originally pointed to the merge class operation and is used to transfer the left table data structure. If all the field names that appear in the fifth selection operation and are not in the nested sub-tables are the right table field names of the merge class operation, then the specific directed edge is the directed edge that originally pointed to the merge class operation and is used to transfer the right table data structure.

8. The method according to claim 1, wherein Constructing the bidirectional mapping relationship data structure includes: For all fields of string type in the database, construct a string array and a prefix tree of different field values in all the fields. The string array is used to map integer sequence numbers to strings, and the prefix tree is used to map field values to the integer sequence numbers in the string array.

9. The method according to claim 1, characterized in that The corresponding tensor processing function for the data operation needs to call a basic tensor processing function, which is defined using the interface functions provided in the AI computing framework specified by the user. If the parameters of the basic tensor processing function include a running device parameter, the running device parameter is set to the identifier of the heterogeneous accelerator specified by the user.

10. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, which when executed by the processor causes the processor to execute the steps of the method according to any one of claims 1 to 9.

Images