Data query method and apparatus, and storage medium and computer program product

Abstract

The present disclosure relates to the technical field of databases. Provided are a data query method and apparatus, and a storage medium and a computer program product. The method is applied to a database, wherein the database comprises an SQL sub-database and a graph sub-database, the database is configured with a plurality of operator APIs, and the plurality of operator APIs include an operator API that can be invoked by an SQL operator, and an operator API that can be invoked by a graph operator. The method comprises: upon receiving a data query instruction, generating an execution plan, the execution plan comprising at least one operator to be executed; and if the operator to be executed is a graph operator, directly invoking an operator API corresponding to the operator to be executed, in order to execute an operation on a graph sub-database, thereby eliminating the need to indirectly implement queries on the graph sub-database by means of invoking an SQL operator, and thus effectively improving the data query efficiency.

Description

Data retrieval methods, devices, storage media, and computer program products

[0001] This disclosure claims priority to Chinese Patent Application No. 202411545122.9, filed on October 31, 2024, entitled “Method, Apparatus, Storage Medium and Computer Program Product for Data Query”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of database technology, and in particular to a method, apparatus, storage medium, and computer program product for data querying. Background Technology

[0003] With the advent of the big data era, more and more businesses rely on database queries. Structured Query Language (SQL) databases are among the earliest developed databases, used to store relational data, and their applications are now widespread. However, as businesses become increasingly diversified, relational data can no longer meet the needs of some business scenarios, leading to the emergence of graph data. However, building a graph database independently requires significant manpower and time; therefore, some vendors build graph databases on top of SQL databases.

[0004] When the data query command is to query an SQL database, the above solution can achieve the database query using pre-packaged SQL operators. However, when the data query command is to query a graph database, the above solution can only indirectly achieve the database query by having the graph operator call the pre-packaged SQL operators, which results in lower query efficiency. Summary of the Invention

[0005] This disclosure provides a method, apparatus, storage medium, and computer program product for data querying, which can improve the efficiency of data querying. The technical solution includes:

[0006] Firstly, a data query method is provided. The method is applied to a database, which includes an SQL sub-database and a graph sub-database. The database is configured with multiple operators, including SQL operators and graph operators. The function of each operator is implemented by calling at least one operation application interface (operator API). The database is also configured with multiple operator APIs, among which there are operator APIs that can be called by SQL operators and operator APIs that can be called by graph operators.

[0007] The processing steps of this method include: receiving a data query instruction; then, generating an execution plan based on the data query instruction, wherein the execution plan includes at least one operator to be executed; for each operator to be executed, if the operator to be executed is an SQL operator, then the operator API corresponding to the operator to be executed is directly called to perform operations on the SQL sub-database; if the operator to be executed is a graph operator, then the operator API corresponding to the operator to be executed is directly called to perform operations on the graph sub-database.

[0008] In the above scheme, the database is configured with an operator API that can be called by SQL operators, and also with an operator API that can be called by graph operators. In this way, the graph operators in the execution plan can also directly call the operator API to implement the corresponding functions, eliminating the step of the graph operators calling SQL operators and improving data query efficiency.

[0009] In one possible implementation, the operators to be executed in the execution plan include at least one SQL operator and at least one graph operator.

[0010] In the above solution, a single data query command generates an execution plan that includes both SQL and graph operators. This means that users only need to input one query command to perform operations on both the SQL and graph sub-databases. In contrast, related technologies require two separate query commands to perform operations on the SQL and graph sub-databases respectively. Compared to related technologies, the above solution is more user-friendly and improves the efficiency of data retrieval.

[0011] In one possible implementation, there exists an operator API among multiple operator APIs that can be called by both SQL operators and graph operators.

[0012] In the above scheme, the database is configured with an operator API that can be called by both SQL and graph operators; that is, an operator API shared by both SQL and graph operators. Therefore, compared to schemes where the operator API cannot be shared by either SQL or graph operators, the above scheme has a smaller total number of operator APIs, reducing maintenance costs and saving storage space.

[0013] In one possible implementation, the execution plan includes a first operator to be executed and a second operator to be executed. The first operator to be executed is an SQL operator, and the second operator to be executed is a graph operator. The first operator to be executed performs operations on the SQL sub-database by calling a first operator API, and the second operator to be executed performs operations on the graph sub-database by calling the first operator API.

[0014] In one possible implementation, a first data query instruction is received, and a first execution plan is generated based on the first data query instruction. The first execution plan includes a first operator to be executed, which is an SQL operator. The first operator to be executed calls a first operator API to perform an operation on the SQL sub-database.

[0015] Receive a second data query instruction, generate a second execution plan based on the second data query instruction, the second execution plan includes a second operator to be executed, the second operator to be executed is a graph operator, the second operator to be executed calls the first operator API mentioned above to perform operations on the graph sub-database.

[0016] In one possible implementation, multiple operator APIs include at least one operator API derived from function wrappers based on the actor model.

[0017] The actor model is a model used in software development that can be understood as the working mode of low-level functions. When combined with a processor of a specific architecture, the actor model can effectively improve processing efficiency. For example, a specific architecture could be a Non-Uniform Memory Access (NUMA) architecture. Thus, in a database server employing a specific architecture, combining the actor model can improve the overall efficiency of data retrieval.

[0018] In one possible implementation, the process of performing operations on the SQL sub-database can be as follows:

[0019] Perform operations on the SQL sub-database using at least one coroutine.

[0020] The process of performing operations on the graph sub-database can be as follows:

[0021] Perform operations on the graph sub-database using at least one coroutine.

[0022] Coroutines, also known as microthreads, are a lighter-weight execution unit than threads. A single thread can run a large number of coroutines, allowing coroutines to execute these data query tasks with greater concurrency when there are a large number of data query tasks at the same time.

[0023] In one possible implementation, the process of generating an execution plan based on a data query instruction can be as follows:

[0024] Based on the specified fields in the data query instruction, the sub-database to be operated on is determined, where the sub-database to be operated on is an SQL sub-database and / or a graph sub-database. The optimizer corresponding to the sub-database to be operated on is then used to generate an execution plan for the data query instruction.

[0025] In the above scheme, multiple optimizers can be configured. If the sub-database to be operated on is determined to be an SQL sub-database, the SQL optimizer generates the execution plan corresponding to the data query command. If the sub-database to be operated on is determined to be a graph sub-database, the graph optimizer generates the execution plan corresponding to the data query command. If the sub-database to be operated on is determined to be both an SQL sub-database and a graph sub-database, the SQL+graph optimizer generates the execution plan corresponding to the data query command. This allows users to input data query commands that simultaneously include queries for both SQL and graph sub-databases, making it more user-friendly and improving the efficiency of data retrieval.

[0026] In one possible implementation, the method may also include the following processing:

[0027] For all operators except the last one to be executed, if invalid data is found in the result data after the operation corresponding to the other operators is completed, the invalid data is deleted from the cache. Invalid data is data that does not participate in the operation corresponding to the next operator to be executed.

[0028] In this way, timely deletion of invalid data from the buffer can reduce buffer occupancy.

[0029] Secondly, a data query apparatus is provided, which is deployed in a database including a Structured Query Language (SQL) sub-database and a graph sub-database. The database is configured with multiple operators, including SQL operators and graph operators. The function of each operator is implemented by calling at least one operator application programming interface (API). The database is also configured with multiple operator APIs, including operator APIs that can be called by SQL operators and operator APIs that can be called by graph operators. The apparatus includes at least one module for implementing the method provided by the first aspect or any possible implementation of the first aspect.

[0030] Thirdly, a computing device cluster is provided, including at least one computing device, each computing device including a processor and memory;

[0031] The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform a data query method as provided in the first aspect or any possible implementation thereof.

[0032] Fourthly, a computer-readable storage medium is provided, including computer program instructions that, when executed by a cluster of computing devices, enable the cluster of computing devices to perform a data query method as provided in the first aspect or any possible implementation thereof.

[0033] Fifthly, a computer program product containing instructions is provided, which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform a data query method as provided in the first aspect or any possible implementation thereof. Attached Figure Description

[0034] Figure 1 is a schematic diagram of a data query process provided in an embodiment of this disclosure;

[0035] Figure 2 is a schematic diagram of a data query process provided in an embodiment of this disclosure;

[0036] Figure 3 is a schematic flowchart of a data query method provided in an embodiment of this disclosure;

[0037] Figure 4 is a schematic diagram of a data query process provided in an embodiment of this disclosure;

[0038] Figure 5 is a schematic diagram of an execution plan generation provided in an embodiment of this disclosure;

[0039] Figure 6 is a schematic diagram of a data query instruction provided in an embodiment of this disclosure;

[0040] Figure 7 is a schematic diagram showing the correspondence between a data query command and an upper-level API provided in an embodiment of this disclosure;

[0041] Figure 8 is a schematic diagram illustrating the correspondence between an operator and an operator API provided in an embodiment of this disclosure;

[0042] Figure 9 is a schematic diagram illustrating the execution of an actor model provided in an embodiment of this disclosure;

[0043] Figure 10 is a schematic diagram of the structure of a data query device provided in an embodiment of this disclosure;

[0044] Figure 11 is a schematic diagram of a computing device provided in an embodiment of this disclosure;

[0045] Figure 12 is a schematic diagram of a computing device cluster provided in an embodiment of this disclosure;

[0046] Figure 13 is a schematic diagram of a computing device cluster provided in an embodiment of this disclosure. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0048] The concepts involved in this disclosure are explained below:

[0049] Graph database

[0050] Graph databases are used to store, manage, and query graph data. Graph data consists of a large number of vertices and edges. Vertices can be used to represent people, objects, knowledge, etc. Edges in graph data are used to connect two vertices and represent the relationship between the two vertices. For example, edges can represent friendships or colleague relationships between users, or connections between devices, and so on.

[0051] Structured Query Language (SQL) database

[0052] SQL databases are used to store, manage, and query relational data. Relational data is stored in tabular form, with each table consisting of rows and columns. For example, the relationship between users and ages can be stored as relational data in a tabular format in an SQL database.

[0053] graph operator

[0054] The graph operator is a function used to implement graph sub-database operations. The function includes statements that call the underlying operation application programming interface (operator API).

[0055] SQL operators

[0056] SQL operators are functions used to implement SQL sub-database operations. These functions include statements that call the underlying operator API.

[0057] Operator API.

[0058] operator API

[0059] The operator API is used to directly manipulate the database to implement the functionality of corresponding graph operators or SQL operators.

[0060] Coroutine

[0061] Coroutines, also known as microthreads, are a lighter-weight execution unit than threads. By performing database operations using coroutines, concurrency can be improved.

[0062] actor model

[0063] The actor model is a model in the field of software development that can be understood as the working mode of low-level functions. The actor model, combined with a specific processor architecture, can effectively improve processing efficiency. For example, a specific architecture could be a Non-Uniform Memory Access (NUMA) architecture.

[0064] SQL databases are an early type of database used to store relational data. However, with increasingly diversified business needs, relational data is no longer sufficient in some scenarios, leading to the emergence of graph databases. However, building a graph database independently requires significant manpower and time. Therefore, related technologies often build graph databases on top of SQL databases. As shown in Figure 1, the database includes an SQL sub-database and a graph sub-database. Upon receiving a data query command, an execution plan is generated, containing at least one operator, such as operator 1, operator 2, operator 3, etc. If the query command targets the SQL database, all operators in the generated execution plan are SQL operators, which can directly call the underlying operator API to operate on the SQL sub-database. Conversely, if the query command targets the graph sub-database, all operators in the generated execution plan are graph operators. These graph operators must first call the corresponding SQL operator, which then calls the underlying operator API to operate on the graph sub-database. This results in lower query efficiency for the graph sub-database.

[0065] This disclosure provides a data query method. The database using this method is configured with multiple operator APIs, including those that can be called by SQL operators and those that can be called by graph operators. Based on this, as shown in Figure 2, after receiving a data query instruction, a corresponding execution plan is generated. The execution plan includes at least one operator to be executed. Even if the operator to be executed is a graph operator, the corresponding operator API can be directly called to perform operations on the graph sub-database, eliminating the need to call SQL operators and improving data query efficiency.

[0066] The present disclosure provides a data query method whose processing flow can be shown in Figure 3, including the following steps:

[0067] Step 301: Receive data query instruction.

[0068] When using a database, users can send data query commands to the database through a client. These commands can be written manually by the user or generated by artificial intelligence. A data query command mainly consists of an operation keyword and a data source field. The operation keyword indicates what operation to perform on the database, while the data source field indicates the source of the data to be manipulated, which can be a specified table or a specified graph.

[0069] For example, as shown in Figure 4, the SQL sub-database stores the correspondence between users and their birth years in a table named `table_name1`. The table has two attributes: user ID and birth year. The graph sub-database stores the friend relationships between users in a graph named `graph_name1`. The nodes of the graph are user IDs, and the edges represent friend relationships. A user's query requirement is: among user P1's friends, select those born in 1990. Based on this query requirement, the user can send the following data query command to the database through the client:

[0070] EXPAND(n:person{name:'P1'})-[friend]-(set1)FROM graph_name1 WHERE year='1990'FROM table_name1

[0071] As shown in Figure 5, EXPAND is the operation keyword, and the two FROM fields are the data source fields, namely "graph_name1" and "table_name1". This data query instruction can be understood as querying the neighbor nodes of P1 (i.e., P1's friends) from graph_name1, and then querying the friends of P1 whose birth year is 1990 from table_name1.

[0072] In the examples above, the data source (either the graph name or the table name) in the data source field directly identifies whether the data originates from the SQL sub-database or the graph sub-database. For instance, a table name containing "table" indicates that `table_name1` is a table stored in the SQL sub-database, while a graph name containing "graph" indicates that `graph_name1` is a graph stored in the graph sub-database. However, in some examples, the graph name and table name in the data source field cannot directly identify whether the data originates from the SQL sub-database or the graph sub-database. In such cases, the data query instruction can also include a sub-database indicator field to indicate whether the data originates from the SQL sub-database or the graph sub-database.

[0073] For example, as shown in Figure 6, the data query command is as follows:

[0074] EXPAND(n:person{name:'P1'})-[friend]-(set1)FROM GRAPH name1 WHERE year='1990'FROM SQL name2

[0075] In the data query instruction shown in Figure 6, the data source fields "name1" and "name2" after the two FROM statements do not have "table" indicating a table, nor "graph" indicating a graph. In this case, the data query instruction includes a sub-database indicator field. As shown in Figure 6, the database indicator field can be located between the FROM and the data source fields. For example, the database indicator field between the first FROM and the data source field "name1" is "GRAPH", and the database indicator field between the second FROM and the data source field "name2" is "SQL". Here, "GRAPH" indicates that the data named name1 is stored in the graph sub-database, and "SQL" indicates that the data named name2 is stored in the SQL sub-database.

[0076] Step 302: Generate an execution plan based on the data query command.

[0077] The execution plan includes at least one operator to be executed.

[0078] Upon receiving a data query instruction, the system determines the sub-database to be operated on based on the specified fields in the instruction. This sub-database can be an SQL sub-database and / or a graph sub-database. Then, the optimizer corresponding to the sub-database generates an execution plan for the data query instruction.

[0079] For example, as shown in Figure 5, the optimizer layer includes an SQL optimizer, a graph optimizer, and an SQL+graph optimizer. If the sub-database to be operated on is determined to be an SQL sub-database, the SQL optimizer generates the execution plan corresponding to the data query instruction. If the sub-database to be operated on is determined to be a graph sub-database, the graph optimizer generates the execution plan corresponding to the data query instruction. If the sub-database to be operated on is determined to be both an SQL sub-database and a graph sub-database, the SQL+graph optimizer generates the execution plan corresponding to the data query instruction. In Figure 5, if the sub-database to be operated on is determined to be both an SQL sub-database and a graph sub-database, then the SQL+graph optimizer generates the execution plan corresponding to the data query instruction.

[0080] In this embodiment of the disclosure, the execution plan generated by the SQL+graph optimizer includes at least one SQL operator and at least one graph operator. As shown in Figures 4 and 5, the execution plan generated by the SQL+graph optimizer includes operator 1 and operator 2. Operator 1 is a graph operator, whose function is to query the friends of P1 in the graph_name1 of the graph sub-database. Operator 2 is an SQL operator, whose function is to filter people born in 1990 from (P4, P6, P2) in the table_name1 of the SQL sub-database.

[0081] The basic methods for generating execution plans using the optimizer are explained below:

[0082] The optimizer estimates the cost of each database operation involved in a data query command based on database statistics. These statistics include the number of rows in tables and indexes, the cardinality of indexed columns, and data distribution histograms. These statistics are collected periodically by the database team or manually updated by relevant personnel. Costs can include disk read / write operations, computational load, and so on. Furthermore, based on the data query command and the cost of each database operation, the optimizer generates the execution plan with the lowest possible cost, which becomes the execution plan for the corresponding data query command.

[0083] There are several methods for determining the sub-database to be operated on. Several of these methods are illustrated below:

[0084] Method 1:

[0085] Specify the data source field. The sub-database to be operated on can be determined based on the data source field. This method can be further divided into the following two cases.

[0086] 1. The characters in the data source field directly identify whether the data originates from an SQL sub-database or a graph sub-database. If the data source field includes the first character, the sub-database to be operated on is determined to be an SQL sub-database. If the data source field includes the second character, the sub-database to be operated on is determined to be a graph sub-database. The first and second characters differ, and their specific content is related to the table names and graph names stored in the database, which can be configured by relevant personnel.

[0087] For example, as shown in Figure 5, the first character is "table" and the second character is "graph". The data query request shown in Figure 5 includes two data source fields. One data source field is "table_name1", which contains the first character "table", and the other data source field is "graph_name1", which contains the second character "graph". Therefore, the sub-databases to be operated on are determined to be the SQL sub-database and the graph sub-database.

[0088] 2. The characters in the "Data Source" field cannot directly identify whether the data originates from the SQL sub-database or the graph sub-database. In this case, the correspondence between the SQL sub-database and the stored table names, as well as the correspondence between the graph sub-database and the stored graph names, can be recorded. See Table 1 below:

[0089] Table 1

[0090] By querying Table 1 based on the data source fields in the data query command, the sub-database to be operated on can be determined. For example, if the data query command includes a data source field "name1", querying Table 1 will determine that the sub-database to be operated on is the "graph" sub-database. As another example, if the data query command includes a data source field "name2", querying Table 1 will determine that the sub-database to be operated on is the "SQL" sub-database. Furthermore, if the data query command includes two data source fields, "name1" and "name2", querying Table 1 based on each data source field will determine whether the sub-database to be operated on is the "graph" sub-database or the "SQL" sub-database.

[0091] Method 2:

[0092] The specified field is the sub-database type indicator field. The sub-database to be operated on can be determined based on the sub-database type indicator field. Specifically, if the sub-database type indicator field is an SQL sub-database indicator, then the sub-database to be operated on is an SQL sub-database. If the sub-database type indicator field is a graph sub-database indicator, then the sub-database to be operated on is a graph sub-database. If the sub-database type indicator field is both an SQL sub-database indicator and a graph sub-database indicator, then the sub-database to be operated on is both an SQL sub-database and a graph sub-database.

[0093] For example, as shown in Figure 6, if the sub-database type indicator field is "SQL" for SQL sub-database and "graph" for graph sub-database, then the sub-databases to be operated on are determined to be SQL sub-database and graph sub-database.

[0094] In one example, after receiving a data query instruction, the corresponding upper-level API can be called first, and then the optimizer can generate an execution plan.

[0095] Specifically, the corresponding upper-level API can be called based on the operation keywords included in the data query command.

[0096] The correspondence between operation keywords and upper-level APIs is shown in Table 2:

[0097] Table 2

[0098] In this embodiment of the disclosure, if the first data query instruction is to perform a first operation on the SQL sub-database and the second data query instruction is to perform a second operation on the graph sub-database, and the first and second operations are similar operations, and the operation keywords in the first and second data query instructions are the same, then these two data query instructions can call the same upper-level API.

[0099] For example, as shown in Figure 7, data query instruction 1 is used to query the data of node A in graph1, and data query instruction 2 is used to query the data of row N in table1. These two data query instructions have the same operation keyword, such as "SCAN", so data query instruction 1 and data query instruction 2 can call the same upper-level API1.

[0100] Step 303: For each operator to be executed, if the operator is an SQL operator, the corresponding operator API is directly called to perform operations on the SQL sub-database. If the operator is a graph operator, the corresponding operator API is directly called to perform operations on the graph sub-database.

[0101] The database can be configured with operator APIs that can be called by both SQL operators and graph operators, as well as operator APIs that can only be called by SQL operators and operator APIs that can only be called by graph operators.

[0102] For example, as shown in Figure 8, SQL operator 1 can call operator API 1, SQL operator 2 can call operator API 2 and operator API 3, operator API 4 can be called by SQL operator 3 and also by graph operator 2, graph operator 1 can call operator API 5, and graph operator 3 can call operator API 6 and operator API 7. Furthermore, the same operator API can be called by multiple SQL operators, the same operator API can be called by multiple graph operators, and the same operator API can be called by both multiple SQL operators and multiple graph operators.

[0103] The following table 3 provides examples of several operators and their corresponding operator API calls:

[0104] Table 3

[0105] The NodeByIDScan, Expand, and Filter operators are graph operators, while the Hash join operator is an SQL operator. Implementing the NodeByIDScan operator requires calling the `map()` operator API. Implementing the Expand operator requires calling `map()` and `shuffle()` operator APIs. Implementing the Filter operator requires calling `map()` operator APIs. Implementing the Hash join operator requires calling `shuffle()`, `barrier()`, and `map()` operator APIs.

[0106] The same operator API can correspond to multiple user-defined functions (UDFs). When the same operator API is called by different operators, different user-defined functions can be executed to achieve the corresponding functions.

[0107] For example, in implementing the NodeByIDScan operator, the operator API called is map(), and the custom function f1 of map() is executed to map the specified node identifier (ID) to the node data.

[0108] For the Expand operator mentioned above, when implementing the function of the Expand operator, the operator API is first called map(), and the custom function f2 of map() is executed to query the data of the next-hop neighbor node of the specified node. Then, shuffle() is called and the custom function h1 is executed to distribute the queried data to different computing nodes to achieve load balancing.

[0109] For the Filter operator mentioned above, when implementing the functionality of the Filter operator, the operator API called is map(), and the custom function f3 of map() is executed to filter the data of nodes that meet the filtering conditions among the neighboring nodes of the specified node.

[0110] In one example, at least one operator API in the database is written by inheriting from the operator base class (operatorbase). The operator base class defines member variables and member methods, and the operator API inherits the member variables and member methods of the operator base class, while also defining its own member variables and member methods.

[0111] In one example, to improve the execution efficiency of the operator API, at least one operator API in the database can be encapsulated from a function based on the actor model. The execution process when this encapsulated operator API is called is explained below:

[0112] As shown in Figure 9, when calling an operator API, a client reference for the actor is first created, such as actor_ref in the figure. Then, actor_ref sends a signal to the actor corresponding to the operator API. This signal is used to instruct the actor to perform what operation. Specifically, the operation can include the following four types:

[0113] Operation 1: Set up the proxy for the next actor.

[0114] If the target operator calling the operator API is not the last operator to be executed in the execution plan, and the operator API is not the last operator API called by the target operator, then the above signal may instruct the actor to execute operation one. If the target operator calling the operator API is the last operator to be executed in the execution plan, and the operator API is the last operator API called by the target operator, then the above signal may not instruct the actor to execute operation one.

[0115] Step 2: Set a custom function.

[0116] Based on the correspondence between the operator to be executed and the user-defined function, the user-defined function required to implement the target operator can be determined. The aforementioned signal can instruct the actor to perform operation two, setting the user-defined function corresponding to the target operator as the user-defined function to be executed.

[0117] For example, for the NodeByIDScan operator in Table 3, the above signal can instruct the actor to perform operation two and set the custom function f1 as the custom function to be executed.

[0118] Operation 3: Statistical analysis of received data.

[0119] If the operator API is not the first operator API called by the first execution operator in the execution plan, the above signal can instruct the actor to perform operation three, and count whether all data sent by the previous operator API has been received.

[0120] Step 4: Manipulate the data according to the set custom function.

[0121] If implementing the functionality of the target operator requires a custom function, the aforementioned signal can instruct the actor to execute operation four. If operation three is required, operation four will be executed after operation three is completed. Operation four executes the custom function set in operation two to perform data manipulation.

[0122] In one example, to improve concurrency, when calling the operator API to perform operations on the SQL sub-database, coroutines can be used to implement these operations. Similarly, when calling the operator API to perform operations on the graph sub-database, coroutines can be used to implement these operations.

[0123] When coroutines are implemented together with the actor model described above, each coroutine can run one actor to perform database operations.

[0124] In one example, to reduce cache usage, invalid intermediate data generated during the execution of the execution plan can be deleted from the cache. The specific process can be as follows:

[0125] For all operators in the execution plan except the last one, if invalid data is found in the result data after the operation corresponding to that operator is completed, the invalid data is deleted from the cache. Invalid data refers to data that does not participate in the operations corresponding to subsequent operators of that operator.

[0126] For example, as shown in Figure 4, the execution plan includes operator 1 and operator 2. After operator 1 is executed, the result data (P4, P6, P2) is obtained and stored in the cache. Since P6 does not have a corresponding birth year recorded in table_name1, the birth year corresponding to P6 cannot be found during the execution of operator 2. P6 will not participate in the operations corresponding to other pending operators and is therefore invalid data. Thus, P6 can be deleted from the buffer.

[0127] Based on the same technical concept, this disclosure provides a data manipulation apparatus deployed in a database, which includes an SQL sub-database and a graph sub-database. The database is configured with multiple operators, including SQL operators and graph operators. Each operator's function is implemented by calling at least one operator API. The database also contains multiple operator APIs, including those that can be called by SQL operators and those that can be called by graph operators. As shown in Figure 10, the apparatus includes a receiving module 1010, a generating module 1020, and an execution module 1030, wherein:

[0128] The receiving module 1010 is used to receive data query instructions; specifically, it can implement the processing function of step 301 above, as well as other implicit steps.

[0129] The generation module 1020 is used to generate an execution plan based on the data query instruction, wherein the execution plan includes at least one operator to be executed; specifically, it can implement the processing function of step 302 above, as well as other implicit steps.

[0130] The execution module 1030 is configured to, for each operator to be executed, directly call the operator API corresponding to the operator to execute if the operator to be executed is an SQL operator, to perform operations on the SQL sub-database; and directly call the operator API corresponding to the operator to execute to perform operations on the graph sub-database if the operator to be executed is a graph operator. Specifically, it can implement the processing function of step 303 above, as well as other implicit steps.

[0131] In one possible implementation, the operators to be executed in the execution plan include at least one SQL operator and at least one graph operator.

[0132] In one possible implementation, among the plurality of operator APIs, there exists an operator API that can be called by both the SQL operator and the graph operator.

[0133] In one possible implementation, the plurality of operator APIs includes at least one operator API derived from encapsulating functions based on the actor model.

[0134] In one possible implementation, the execution module 1030 is configured to:

[0135] The operation on the SQL sub-database is performed through at least one coroutine;

[0136] Alternatively, the execution module is configured to:

[0137] Operations on the graph sub-database are performed through at least one coroutine.

[0138] In one possible implementation, the generation module 1020 is configured to:

[0139] Based on the specified fields in the data query instruction, the sub-database to be operated on is determined, wherein the sub-database to be operated on is the SQL sub-database and / or the graph sub-database;

[0140] The optimizer corresponding to the sub-database to be operated on generates the execution plan corresponding to the data query instruction.

[0141] In one possible implementation, the execution module 1030 is further configured to:

[0142] For all operators to be executed except the last one, if invalid data is found in the result data after the operation corresponding to the other operators is completed, the invalid data is deleted from the cache. The invalid data is data that does not participate in the operation corresponding to the next operator to be executed.

[0143] This disclosure provides a data query method. The database using this method is configured with multiple operator APIs, including those that can be called by SQL operators and those that can be called by graph operators. Based on this, upon receiving a data query instruction, a corresponding execution plan is generated. The execution plan includes at least one operator to be executed. Even if the operator to be executed is a graph operator, the corresponding operator API can be directly called to perform operations on the graph sub-database, eliminating the need to call SQL operators and improving data query efficiency.

[0144] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0145] The receiving module 1010, generating module 1020, and executing module 1030 can all be implemented in software or in hardware. For example, the implementation of the executing module 1030 will be described below. Similarly, the implementation of the receiving module 1010 and generating module 1020 can refer to the implementation of the executing module 1030.

[0146] As an example of a software functional unit, the execution module 1030 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, the execution module 1030 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0147] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0148] As an example of a hardware functional unit, the execution module 1030 may include at least one computing device, such as a server. Alternatively, the execution module 1030 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0149] The execution module 1030 includes multiple computing devices that can be distributed within the same region or in different regions. Similarly, the execution module 1030 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the execution module 1030 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0150] It should be noted that, in other embodiments, the receiving module 1010, the generating module 1020, and the execution module 1030 can be used for any step in the data query method. The steps implemented by the receiving module 1010, the generating module 1020, and the execution module 1030 can be specified as needed. The receiving module 1010, the generating module 1020, and the execution module 1030 respectively implement different steps in the data query method to realize all the functions of the data query device.

[0151] This disclosure also provides a computing device 100. As shown in FIG11, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that this disclosure does not limit the number of processors and memories in the computing device 100.

[0152] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 11, but this does not imply that there is only one bus or one type of bus. Bus 102 can include pathways for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, communication interface 108).

[0153] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0154] The memory 106 may include volatile memory, such as random access memory (RAM). The memory 106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0155] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the aforementioned receiving module 1010, generating module 1020, and execution module 1030, thereby realizing the data operation method. That is, the memory 106 stores instructions for the data operation method.

[0156] Alternatively, the memory 106 stores executable code, which the processor 104 executes to implement the functions of the aforementioned receiving module 1010, generating module 1020, and execution module 1030, thereby realizing the data query method. In other words, the memory 106 stores instructions for the data query method.

[0157] The communication interface 108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 100 and other devices or communication networks.

[0158] This disclosure also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0159] As shown in Figure 12, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for data manipulation methods.

[0160] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for methods of data manipulation. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for methods of data manipulation.

[0161] It should be noted that the memory 106 in different computing devices 100 within the computing device cluster can store different instructions, each used for a portion of the functions of the data operation device. That is, the instructions stored in the memory 106 of different computing devices 100 can implement the functions of one or more of the aforementioned receiving module 1010, generating module 1020, and execution module 1030.

[0162] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), and can be a Transmission Control Protocol (TCP) network or a Remote Direct Memory Access (RDMA) network. Figure 13 illustrates one possible implementation. As shown in Figure 13, two computing devices 100A and 100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 106 in computing device 100A stores instructions for the functions of the receiving module 1010 and the generating module 1020. Simultaneously, the memory 106 in computing device 100B stores instructions for executing the functions of the execution module 1030.

[0163] The connection method between the computing device clusters shown in Figure 13 can be considered in light of the fact that the data operation method provided in this disclosure requires a large amount of data storage, so the function implemented by the execution module 1030 is to be executed by the computing device 100B.

[0164] It should be understood that the functions of computing device 100A shown in Figure 13 can also be performed by multiple computing devices 100. Similarly, the functions of computing device 100B can also be performed by multiple computing devices 100.

[0165] This disclosure also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 12 and 13. The difference is that the memory 106 of one or more computing devices 100 in this computing device cluster can store the same instructions for data manipulation methods.

[0166] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for a data query method. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for the data query method.

[0167] This disclosure also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a data query method.

[0168] This disclosure also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disk (DVD)), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform a data query method.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for data querying, characterized in that, The method is applied to a database, which includes a Structured Query Language (SQL) sub-database and a graph sub-database. The database is configured with multiple operators, including SQL operators and graph operators. The function of each operator is implemented by calling at least one operator application programming interface (API). The database is also configured with multiple operator APIs, including those that can be called by SQL operators and those that can be called by graph operators. The method includes: Receive data query instructions; Based on the data query instruction, an execution plan is generated, wherein the execution plan includes at least one operator to be executed; For each operator to be executed, if the operator to be executed is an SQL operator, the operator API corresponding to the operator to be executed is directly called to perform the operation on the SQL sub-database; if the operator to be executed is a graph operator, the operator API corresponding to the operator to be executed is directly called to perform the operation on the graph sub-database.

2. The method according to claim 1, characterized in that, The execution plan includes at least one SQL operator and at least one graph operator.

3. The method according to claim 1 or 2, characterized in that, Among the multiple operator APIs, there is an operator API that can be called by both SQL operators and graph operators.

4. The method according to claims 1-3, characterized in that, The plurality of operator APIs includes at least one operator API that is a wrapper around a function based on an actor model.

5. The method according to any one of claims 1-4, characterized in that, The operation on the SQL sub-database includes: The operation on the SQL sub-database is performed through at least one coroutine; Alternatively, performing operations on the graph sub-database includes: Operations on the graph sub-database are performed through at least one coroutine.

6. The method according to any one of claims 1-5, characterized in that, The step of generating an execution plan based on the data query instruction includes: Based on the specified fields in the data query instruction, the sub-database to be operated on is determined, wherein the sub-database to be operated on is the SQL sub-database and / or the graph sub-database; The optimizer corresponding to the sub-database to be operated on generates the execution plan corresponding to the data query instruction.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: For all operators to be executed except the last one, if invalid data is found in the result data after the operation corresponding to the other operators is completed, the invalid data is deleted from the cache. The invalid data is data that does not participate in the operation corresponding to the next operator to be executed.

8. A data query device, characterized in that, The device is deployed on a database, which includes a Structured Query Language (SQL) sub-database and a graph sub-database. The database is configured with multiple operators, including SQL operators and graph operators. Each operator's function is implemented by calling at least one operator application programming interface (API). The database also contains multiple operator APIs, including those that can be called by SQL operators and those that can be called by graph operators. The device includes: The receiving module is used to receive data query commands; A generation module is used to generate an execution plan based on the data query instruction, wherein the execution plan includes at least one operator to be executed; The execution module is used to, for each operator to be executed, if the operator to be executed is an SQL operator, directly call the operator API corresponding to the operator to execute to perform the operation on the SQL sub-database; if the operator to be executed is a graph operator, directly call the operator API corresponding to the operator to execute to perform the operation on the graph sub-database.

9. The apparatus according to claim 8, characterized in that, The execution plan includes at least one SQL operator and at least one graph operator.

10. The apparatus according to claim 8 or 9, characterized in that, Among the multiple operator APIs, there is an operator API that can be called by both SQL operators and graph operators.

11. The apparatus according to claims 8-10, characterized in that, The plurality of operator APIs includes at least one operator API that is a wrapper around a function based on an actor model.

12. The apparatus according to any one of claims 8-11, characterized in that, The execution module is used for: The operation on the SQL sub-database is performed through at least one coroutine; Alternatively, the execution module is configured to: Operations on the graph sub-database are performed through at least one coroutine.

13. The apparatus according to any one of claims 8-12, characterized in that, The generation module is used for: Based on the specified fields in the data query instruction, the sub-database to be operated on is determined, wherein the sub-database to be operated on is the SQL sub-database and / or the graph sub-database; The optimizer corresponding to the sub-database to be operated on generates the execution plan corresponding to the data query instruction.

14. The apparatus according to any one of claims 8-13, characterized in that, The execution module is further configured to: For all operators to be executed except the last one, if invalid data is found in the result data after the operation corresponding to the other operators is completed, the invalid data is deleted from the cache. The invalid data is data that does not participate in the operation corresponding to the next operator to be executed.

15. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a cluster of computing devices, enable the cluster of computing devices to perform the data query method as described in any one of claims 1-7.

17. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the data query method as described in any one of claims 1-7.

Images