Data query method, computer device and storage medium

Abstract

The present invention discloses a data query method, computer device, and storage medium, including: executing a matching data query task in response to SQL, wherein the query task includes a root node operator, a leaf node operator, and an intermediate node operator, and establishing a coroutine message channel between operators with a parent-child dependency relationship; obtaining intermediate result data through calculation by the leaf node operator and caching it in the channel between the operator and the parent operator on which it depends; obtaining new intermediate result data through calculation by the intermediate node operator based on the intermediate result data obtained from the channel between the operator and the child operator on which it depends, and caching it in the channel between the operator and the parent operator on which it depends; and obtaining a final calculation result through calculation by the root node operator based on the intermediate result data obtained from the channel between the operator and the child operator on which it depends. Without the need for interface method calls and thread switching overhead, operators can actively produce data without passive waiting, thereby improving data query efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a data query method, a computer device, and a storage medium. Background Art

[0002] The executor is a crucial component of any database. It is responsible for executing Structured Query Language (SQL) queries by invoking the storage engine's interface based on the execution plan generated by the query optimizer. The Volcano model is the most classic executor model, currently adopted by all mainstream relational databases.

[0003] Figure 1 This is a structural diagram of a volcano model provided by the prior art. Figure 1 As shown in the figure, in the volcano model, each operator provides three method interfaces: open, next, and close. The query plan tree consists of individual operators. The query system starts by calling the next interface at the root node operator. After recursively calling the next interface, it reaches the leaf node operator, which then pushes data from the physical table. Each operator treats the output of the intermediate node operator (i.e., the input of the current operator) as a logical table. A single call to the next interface returns only the number of rows in the table.

[0004] The logical definitions of operators in the Volcano model are simple and clear, and operators interact through a unified interface. Algebraic calculations are independent of each other, resulting in low coupling and making it easy for the query optimizer to assemble various operators into a query plan tree. However, the next interface must be called once to retrieve each row of data, and each data transfer between operators also requires a next interface call. The fluidity of data between operators results in many redundant control operations. Furthermore, the next call is a virtual function call, which the compiler cannot inline. Each virtual function call requires a virtual function table lookup, which results in more branch instructions and ultimately leads to low CPU execution efficiency.

[0005] The prior art proposes a batch processing model based on the volcano model. Figure 2 This is a structural diagram of a batch processing model provided by the prior art. Figure 2As shown, the batch model is similar to the volcano model. Each operator provides interfaces such as "next." However, the data transferred is no longer a single row, but rather a collection of rows, known as a chunk. Typically, the chunk capacity is 1024. While the batch model reduces calls to the "next" interface to a certain extent, improving control efficiency and enhancing code locality, it also introduces the challenge of balancing function call overhead with memory allocation. A larger chunk capacity reduces function call overhead but increases corresponding memory allocation overhead. For example, for online transaction processing (OLTP), a chunk capacity of 1024 is clearly wasteful.

[0006] Furthermore, both the existing volcano model and the batch processing model require the parent node operator to pull data from the child node operator, leaving the child node operator in a passive waiting state until the next call to the parent node operator's next interface. Furthermore, when implementing a complete data query service, the execution threads of each operator must constantly switch, resulting in low data query efficiency and high CPU overhead. Summary of the Invention

[0007] The embodiments of the present invention provide a data query method, a computer device, and a storage medium, which do not require interface method calls and thread switching overhead, can actively generate data, and improve data query efficiency.

[0008] In a first aspect, an embodiment of the present invention provides a data query method, the method comprising:

[0009] In response to the target structured query language SQL, trigger the execution of a data query task that matches the target SQL. The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependency relationships.

[0010] The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on.

[0011] By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator;

[0012] The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0013] In a second aspect, an embodiment of the present invention further provides a computer device, including a processor and a memory, wherein the memory is configured to store instructions, and when the instructions are executed, the processor performs the following operations:

[0014] In response to the target structured query language SQL, trigger the execution of a data query task that matches the target SQL. The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependency relationships.

[0015] The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on.

[0016] By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator;

[0017] The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0018] In a third aspect, an embodiment of the present invention further provides a storage medium for storing instructions for executing:

[0019] In response to the target structured query language SQL, trigger the execution of a data query task that matches the target SQL. The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependency relationships.

[0020] The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on.

[0021] By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator;

[0022] The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0023] The technical solution of the embodiment of the present invention triggers the execution of a data query task that matches the target SQL in response to the target SQL, wherein the data query task includes a root node operator, a leaf node operator, and an intermediate node operator, and a coroutine message channel is pre-established between two operators with a parent-child dependency relationship; the leaf node operator calculates and obtains intermediate result data, and caches the intermediate result data in the coroutine message channel between the parent operator on which it depends; the intermediate node operator calculates and obtains new intermediate result data based on the intermediate result data obtained from the coroutine message channel between the child operator on which it depends, and caches the new intermediate result data in the coroutine message channel between the parent operator on which it depends; the root node operator calculates and obtains the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the child operator on which it depends. This solves the problem of inefficiency caused by multiple calls to the interface during data query, realizes that the leaf node operator drives the upper-level operator to use the data, avoids the overhead caused by the next interface call, reduces thread switching, reduces CPU overhead, and thus improves the efficiency of data query. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a structural diagram of a volcano model provided by the prior art;

[0025] Figure 2 It is a structural diagram of a batch processing model provided by the prior art;

[0026] Figure 3a This is a flow chart of a data query method provided by Example 1 of the present invention;

[0027] Figure 3b This is a schematic diagram of a data query task structure provided by the first embodiment of the present invention;

[0028] Figure 3c This is a schematic diagram of a coroutine message channel provided by the first embodiment of the present invention;

[0029] Figure 4a This is a flow chart of a data query method provided by Embodiment 2 of the present invention;

[0030] Figure 4b This is a schematic diagram of a structure for enabling multi-core computing capabilities provided by the second embodiment of the present invention;

[0031] Figure 4c Schematic diagram of a preemptive scheduling model provided by an embodiment of the present invention;

[0032] Figure 4d This is a schematic diagram of a structure for controlling a data query task through a control system provided by the second embodiment of the present invention;

[0033] Figure 4e This is a structural diagram of a coroutine reuse mechanism provided by the second embodiment of the present invention;

[0034] Figure 5 This is a structural diagram of a data query device provided by Embodiment 3 of the present invention;

[0035] Figure 6 This is a structural diagram of a computer device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0037] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0038] The term "data query task" used in this article can be a task in the database component that completes the execution of SQL statements through the execution plan generated by the executor.

[0039] The term "root node operator" used in this article can be the top-level operator of a data query task; "leaf node operator" can be the bottom-level operator of a data query task; "intermediate node operator" can be the operator between the top-level operator and the bottom-level operator in a data query task; "parent-child dependency" can represent the relationship between a lower-level node operator and the upper-level node operator to which it is connected. Specifically, the data output by the lower-level node operator can be used as the input of the upper-level node operator to which it is connected.

[0040] The term "coroutine message channel" used in this article can be a lightweight thread in user mode, which can be a transformation of the traditional linear dependency calling method (such as the volcano model or the batch model) into a message-based coroutine concurrent calling model (Communicating Sequential Processes, CSP). Specifically, the coroutine message channel can be implemented through the Golang coroutine pipeline connection, and operators connected through the coroutine message channel can transfer data in a message manner. Golang implements concurrent communication based on message passing, which is called Golang channel. Among them, message passing can be direct communication between threads, and different threads achieve the purpose of interaction by explicitly sending messages.

[0041] Example 1

[0042] Figure 3a This is a flow chart of a data query method provided by the first embodiment of the present invention. This embodiment is applicable to the case of performing data query based on SQL. The method can be executed by a data query device, which can be implemented by software and / or hardware. The device can be integrated into a computer, such as Figure 3a As shown, the method specifically includes:

[0043] Step 110: In response to the target SQL, trigger the execution of a data query task matching the target SQL.

[0044] The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependencies. In this embodiment of the present invention, the data query task can be specifically understood as abstracting each operation in relational algebra into an operator, constructing the entire SQL statement into an operator tree, and transferring data from leaf node operators to root node operators from bottom to top via coroutine message channels.

[0045] Figure 3b This is a schematic diagram of a data query task structure provided by the first embodiment of the present invention. Figure 3b As shown in the figure, in a data query task, a node operator retrieves data from a leaf node operator through one or more intermediate node operators to determine the final calculation result. Sequentially connected operators can be considered to have a parent-child dependency relationship. Specifically, an upper-level operator can serve as the parent operator of a connected lower-level operator, and a lower-level operator can serve as the child operator of a connected upper-level operator. Operators with parent-child dependencies are connected via channels.

[0046] Specifically, Figure 3c This is a schematic diagram of a coroutine message channel provided by the first embodiment of the present invention. Figure 3cAs shown in the figure, lower-level operators (such as the child operator) can be understood as data producers, and upper-level operators (such as the parent operator) can be understood as data consumers. The producer and consumer are connected through a channel, and each is a coroutine. This coroutine can be implemented at the Golang language level and is called a goroutine.

[0047] In the embodiments of the present invention, upper-layer calculations can be pushed down to operators that produce data through goroutine and channel. Leaf node operators drive upper-layer operators to consume data. There is no need to call the next method interface or the corresponding thread overhead. Instead, the channel-based switching overhead of the coroutine is used. The coroutine switching overhead of Golang is about 100 nanoseconds, which is 30 times the performance of thread switching. This improves CPU execution efficiency and data query efficiency, and is more suitable for high-concurrency distributed database execution scenarios.

[0048] Step 120: Calculate the intermediate result data through the leaf node operator and cache the intermediate result data in the coroutine message channel between the operator and the parent operator on which it depends.

[0049] Among them, Figure 3c As shown, the leaf node operator can actively calculate the intermediate result data according to the query task corresponding to the SQL language, without having to wait for the next interface call of the upper-level operator and then calculate and generate the intermediate result data like the volcano model in the prior art. In the Go language, there are two types of channels: unbuffered and buffered. In the embodiment of the present invention, a buffered channel can be used to cache the intermediate result data. The buffered channel does not require that the goroutines must complete sending and receiving at the same time. When the channel contains a buffer area, the receiving action will be blocked when the buffer area is empty; the sending action will be blocked when the buffer area is full.

[0050] Step 130: Through the intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the dependent parent operator.

[0051] In an embodiment of the present invention, data can be produced by lower-level node operators from the bottom up and pushed to the channel. When the upper-level node operators need data, they directly obtain the data in the channel and perform calculations to obtain new intermediate data, which is then pushed upward until it is pushed to the root node operator for final calculation.

[0052] Step 140: The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0053] The technical solution of the embodiment of the present invention triggers the execution of a data query task matching the target SQL in response to the target SQL, wherein the data query task includes a root node operator, a leaf node operator and an intermediate node operator, and a coroutine message channel is pre-established between any two operators with a parent-child dependency relationship; the leaf node operator calculates and obtains intermediate result data, and caches the intermediate result data in the coroutine message channel between the operator and the dependent parent operator; the intermediate node operator calculates and obtains new intermediate result data based on the intermediate result data obtained from the coroutine message channel between the operator and the dependent child operator, and caches the new intermediate result data in the coroutine message channel between the operator and the dependent parent operator; the root node operator calculates and obtains the final calculation result matching the data query task based on the intermediate result data obtained from the coroutine message channel between the operator and the dependent child operator, thereby solving the problem in the prior art of requiring an interface call when performing data query through the volcano model, and achieving the goal of eliminating the need for interface method calls and thread switching overhead. The operator can actively produce data without passive waiting, thereby improving the efficiency of data query.

[0054] Example 2

[0055] Figure 4a This is a flow chart of a data query method provided by Example 2 of the present invention. The implementation method in this embodiment may be a refinement of the technical solution of the above embodiment, and may be combined with one or more of the above implementation methods, such as Figure 4a As shown, the method specifically includes:

[0056] Step 210: In response to the target SQL, trigger the execution of a data query task matching the target SQL.

[0057] The data query task includes root node operators, leaf node operators, and intermediate node operators. A coroutine message channel is pre-established between operators with parent-child dependencies. The coroutine message channel includes at least one buffer.

[0058] Step 220: Calculate the intermediate result data through the leaf node operator, and when it is determined that the cache is not full, cache the intermediate result data in the cache of the coroutine message channel between the parent operator on which it depends.

[0059] Among them, when the cache storage is not full, the leaf node operator can store data in the cache area of ​​the coroutine message channel. When the cache area is full, it can suspend data production and wait for the upper-level node operator to consume the data before continuing to produce data.

[0060] Step 230: When the intermediate node operator determines that the cache storage is not empty, the intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent child operator according to the preset data reading rules, and new intermediate result data is calculated. When it is determined that the cache storage is not full, the new intermediate result data is cached in the cache area of ​​the coroutine message channel between the dependent parent operator.

[0061] Among them, when the cache area of ​​the coroutine message channel between the intermediate node operator and the dependent child operator is not empty, the upper-level node can obtain data from the cache area for calculation; and when the cache area of ​​the coroutine message channel between the intermediate node operator and the dependent parent operator is not full, the new intermediate result data of the calculation is cached in the cache area.

[0062] In an optional implementation of an embodiment of the present invention, when it is determined that the cache area storage is not empty, obtaining intermediate result data from the cache area of ​​the coroutine message channel between the dependent sub-operator according to a preset data reading rule, and calculating to obtain new intermediate result data, including: when it is determined that the cache area storage is not empty, starting at least one Golang coroutine that is the same as the intermediate node operator according to the preset data reading rule, using a parallel execution method to obtain intermediate result data from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rule, and calculating to obtain new intermediate result data;

[0063] When the amount of data being queried is large, multiple identical consumer goroutines can be started in the operator to receive and process the data, thereby fully utilizing the multi-core computing power. Figure 4b This is a schematic diagram of a structure for starting multi-core computing capability provided by the second embodiment of the present invention. Figure 4b As shown, the coroutine used in this embodiment of the present invention is a user-mode thread that is completely controlled by the program. The overhead of context switching is minimal, and millions of coroutines can be created without exhausting system resources. Therefore, when querying large amounts of data, Golang coroutines can be added at will. For the same operator, the coroutines launched can be identical, and intermediate result data can be obtained and calculated through parallel execution.

[0064] Step 240: When the root node operator determines that the cache storage is not empty, the intermediate result data is obtained from the cache of the coroutine message channel between the dependent sub-operator according to the preset data reading rules, and the final calculation result that matches the data query task is calculated.

[0065] In an optional implementation of an embodiment of the present invention, when it is determined that the cache storage is not empty, intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules, and the final calculation result that matches the data query task is calculated, including: when it is determined that the cache storage is not empty, at least one Golang coroutine that is the same as the root node operator is started according to the preset data reading rules, and the intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules in a parallel execution manner, and the final calculation result that matches the data query task is calculated.

[0066] Among them, by starting at least one Golang coroutine that is the same as the root node operator, the speed of data acquisition can be accelerated without causing system burden.

[0067] On the basis of the above implementation, optionally, new intermediate result data is calculated by an intermediate node operator based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent sub-operator, including: when there are at least two intermediate node operators with the same operator purpose, the intermediate node operator with a faster execution rate is used to obtain data, and the new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the same sub-operator on which it depends.

[0068] SQL is a declarative language, allowing databases to generate reasonable execution plans based on user-written queries. However, in some scenarios, the query optimizer cannot accurately determine which operator solution is most efficient. Traditional models can only select a specific algorithm based on experience to generate the final query plan tree and then execute the query according to that tree.

[0069] In the embodiment of the present invention, it is possible to determine which operator has higher execution efficiency through a preemptive scheduling model based on the coroutine message channel. Figure 4c This is a schematic diagram of the structure of a preemptive scheduling model provided by an embodiment of the present invention. Figure 4c As shown in the figure, the filter operator and the join operator have the same purpose. When obtaining data from the lower-level scan operator at the same time, the filter operator can obtain downstream data first due to its faster execution speed, while the join operator cannot obtain data. This allows the data query task to use the operator with faster execution efficiency to execute the task, saving data query time.

[0070] On the basis of the above implementation, optionally, the root node operator is used to calculate the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the dependent sub-operator, including: when there are at least two root node operators with the same operator purpose, the root node operator with a faster execution rate is used to obtain data, and the final calculation result that matches the data query task is calculated based on the intermediate result data obtained from the coroutine message channel between the same sub-operator on which it depends.

[0071] Based on the above implementation, optionally, the method further includes: obtaining a control signal of a control system that matches the data query task, wherein a coroutine message channel is pre-established between the control system and the data query task; according to the control signal, triggering the leaf node operator of the data query task to stop calculating to obtain intermediate result data, so as to stop the data query task.

[0072] in, Figure 4d This is a schematic diagram of a structure of controlling a data query task through a control system provided by the second embodiment of the present invention. Figure 4d As shown, a system-level channel can be established between the data query task and the control system, allowing all operators to monitor the system-level channel, respond in real time, and report their status, facilitating unified management. Specifically, the control system can directly control the leaf node operators of the data query task, thereby controlling the data query task, simplifying the control process of the data query task.

[0073] In an embodiment of the present invention, the control signal includes: a memory control signal, a processor control signal, and a process shutdown signal. Specifically, the memory control signal (memory controller) can decide whether to suspend a data query task based on the actual memory situation. For example, when memory resources are tight, the memory control signal can be used to control the leaf node operator of SQL task 1 to stop the data query task. The processor control signal (CPU controller) can decide whether to suspend a data query task based on the load of the actual CPU execution situation. For example, when processor resources are tight, the processor control signal can be used to control the leaf node operator of SQL task 2 to stop the data query task. The process shutdown signal (kill signal) can decide whether to stop the data query task based on the specific situation of the data query business. For example, when you want to close SQL task 3, you can directly stop the data query task through the process shutdown signal.

[0074] Based on the above implementation, optionally, the method further includes: storing the root node operator, leaf node operator and intermediate node operator included in the data query task matching the target SQL, as well as the coroutine message channel between each operator with parent-child dependency relationship, into the object pool.

[0075] Among them, in an embodiment of the present invention, data query tasks can be stored, such as in an object pool (sync.Pool). Through the coroutine reuse mechanism, objects that have been allocated but temporarily different can be stored in the object pool and then extracted from the object pool when needed, which can reduce the overhead of garbage collection. Specifically, by storing the root node operator, leaf node operator and intermediate node operator of the data query task in the object pool, as well as the coroutine message channel between operators with parent-child dependencies, and storing them in the object pool, the coroutine-based executor can be reused within or across tasks, further reducing the overall coroutine communication overhead.

[0076] Specifically, Figure 4e This is a schematic diagram of the structure of a coroutine reuse mechanism provided by the second embodiment of the present invention. Figure 4e As shown, SQL task 1 can be stored in the object pool. When SQL task 4 performs task execution, SQL task 4 needs to use the same intermediate node operator 1 as in SQL task 1. There is no need to recreate the intermediate node operator 1. The intermediate node operator 1 of SQL task 1 can be directly reused, thereby realizing the reuse of coroutines within or across tasks based on the coroutine executor, further reducing the overall coroutine communication overhead.

[0077] The technical solution of the embodiment of the present invention triggers the execution of a data query task matching the target SQL in response to the target SQL; calculates and obtains intermediate result data through the leaf node operator, and caches the intermediate result data in the cache of the coroutine message channel between the dependent parent operator when it is determined that the cache is not full; obtains the intermediate result data from the cache of the coroutine message channel between the dependent child operator according to the preset data reading rules when it is determined that the cache is not empty, calculates and obtains new intermediate result data, and caches the new intermediate result data in the cache of the coroutine message channel between the dependent parent operator when it is determined that the cache is not full; and obtains the intermediate result data from the cache of the coroutine message channel between the dependent child operator according to the preset data reading rules when it is determined that the cache is not empty, and calculates and obtains the final calculation result matching the data query task. This solves the problem of interface calls required when performing data queries through the volcano model in the prior art, achieves the goal of eliminating the need for interface method calls and thread switching overhead, and allows operators to actively produce data without passive waiting, thereby improving data query efficiency.

[0078] Example 3

[0079] Figure 5 This is a structural diagram of a data query device provided by the third embodiment of the present invention. Figure 5 The device includes: a data query task trigger execution module 310, an intermediate result data cache module 320, a new intermediate result data cache module 330 and a final calculation result calculation module 340.

[0080] The data query task triggering execution module 310 is used to trigger the execution of the data query task that matches the target SQL in response to the target SQL. The data query task includes a root node operator, a leaf node operator, and an intermediate node operator. A coroutine message channel is pre-established between any two operators with a parent-child dependency relationship.

[0081] The intermediate result data caching module 320 is used to calculate the intermediate result data through the leaf node operator and cache the intermediate result data in the coroutine message channel between the parent operator on which it depends;

[0082] A new intermediate result data caching module 330 is used to calculate new intermediate result data through the intermediate node operator based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and cache the new intermediate result data in the coroutine message channel between the intermediate node operator and the dependent parent operator;

[0083] The final calculation result calculation module 340 is used to calculate the final calculation result matching the data query task through the root node operator based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0084] Optionally, the coroutine message channel includes at least one buffer area;

[0085] The intermediate result data cache module 320 includes:

[0086] The intermediate result data cache unit is used to cache the intermediate result data in the cache area of ​​the coroutine message channel between the dependent parent operators according to the preset cache rules when it is determined that the cache area is not full.

[0087] The new intermediate result data cache module 330 includes:

[0088] The new intermediate result data cache unit is used to cache the new intermediate result data into the cache area of ​​the coroutine message channel between the dependent parent operators according to the preset cache rules when it is determined that the cache area is not full.

[0089] Optionally, the new intermediate result data cache module 330 includes:

[0090] A new intermediate result number calculation unit is used to obtain intermediate result data from the cache of the coroutine message channel between the dependent sub-operator according to the preset data reading rules when determining that the cache storage is not empty, and calculate new intermediate result data;

[0091] The final calculation result calculation module 340 includes:

[0092] The final calculation result calculation unit is used to obtain the intermediate result data from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules when it is determined that the cache area storage is not empty, and calculate the final calculation result that matches the data query task.

[0093] Optional, new intermediate result calculation unit, specifically used for:

[0094] When it is determined that the buffer area is not empty, at least one Golang coroutine that is the same as the intermediate node operator is started according to the preset data reading rules, and the intermediate result data is obtained from the buffer area of ​​the coroutine message channel between the dependent sub-operator in a parallel execution manner according to the preset data reading rules, and new intermediate result data is calculated;

[0095] Optionally, the final calculation result calculation unit is specifically used to:

[0096] When it is determined that the cache storage is not empty, at least one Golang coroutine that is the same as the root node operator is started according to the preset data reading rules, and the intermediate result data is obtained from the cache of the coroutine message channel between the dependent sub-operators according to the preset data reading rules using a parallel execution method, and the final calculation result that matches the data query task is calculated.

[0097] Optionally, the new intermediate result data cache module 330 includes:

[0098] A new intermediate result data calculation unit is used to, when there are at least two intermediate node operators with the same operator purpose, obtain new intermediate result data by using the intermediate node operator with a faster execution rate through data acquisition, based on the intermediate result data obtained from the coroutine message channel between the operator and the same dependent sub-operator;

[0099] The final calculation result calculation module 340 includes:

[0100] Another final calculation result calculation unit is used to obtain the root node operator with a faster execution rate through data when there are at least two root node operators with the same operator purpose, and calculate the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the same sub-operator it depends on.

[0101] Optionally, the device further includes:

[0102] A control signal acquisition module is used to obtain the control signal of the control system that matches the data query task, wherein a coroutine message channel is pre-established between the control system and the data query task;

[0103] The data query task stop module is used to trigger the leaf node operator of the data query task to stop calculating and obtain intermediate result data according to the control signal, so as to stop the data query task.

[0104] Optionally, the control signal includes: a memory control signal, a processor control signal, and a process shutdown signal.

[0105] Optionally, the device further includes:

[0106] The task storage module is used to store the root node operator, leaf node operator, and intermediate node operator included in the data query task that matches the target SQL, as well as the coroutine message channel between operators with parent-child dependencies, in the object pool.

[0107] The data query device provided by the embodiment of the present invention can execute the data query method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0108] Example 4

[0109] Figure 6 This is a schematic diagram of the structure of a computer device provided by the fourth embodiment of the present invention. Figure 6 As shown, the device includes:

[0110] one or more processors 610, Figure 6 A processor 610 is taken as an example;

[0111] Memory 620;

[0112] The device may further include an input device 630 and an output device 640 .

[0113] The processor 610, memory 620, input device 630 and output device 640 in the device may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0114] The memory 620 is a non-transitory computer-readable storage medium that can be used to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to a data query method in an embodiment of the present invention (for example, the attached Figure 5The data query task triggers the execution module 310, the intermediate result data caching module 320, the new intermediate result data caching module 330, and the final calculation result calculation module 340 shown in FIG. 1 . The processor 610 executes the software programs, instructions, and modules stored in the memory 620 to execute various functional applications and data processing of the computer device, that is, to implement a data query method of the above method embodiment, namely:

[0115] In response to the target structured query language SQL, trigger the execution of a data query task that matches the target SQL. The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependency relationships.

[0116] The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on.

[0117] By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator;

[0118] The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0119] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 620 may optionally include a memory remotely located relative to the processor 610, and these remote memories may be connected to the terminal device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The input device 630 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device. The output device 640 may include a display device such as a display screen.

[0121] Example 5

[0122] A fifth embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a data query method according to an embodiment of the present invention is implemented, namely:

[0123] In response to the target structured query language SQL, trigger the execution of a data query task that matches the target SQL. The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependency relationships.

[0124] The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on.

[0125] By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator;

[0126] The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator.

[0127] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0128] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0129] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0130] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0131] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data query method, characterized in that: include: In response to the target structured query language SQL, trigger the execution of a data query task that matches the target SQL. The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependency relationships. The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on. By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator; The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator; The coroutine message channel includes at least one buffer area; Caching the intermediate result data in a coroutine message channel between the dependent parent operator, or caching the new intermediate result data in a coroutine message channel between the dependent parent operator, including: When it is determined that the buffer area is not full, the intermediate result data or the new intermediate result data is cached in the buffer area of ​​the coroutine message channel between the dependent parent operators according to the preset cache rule; Calculating new intermediate result data based on the intermediate result data obtained from the coroutine message channel between the dependent sub-operator, or calculating the final calculation result matching the data query task based on the intermediate result data obtained from the coroutine message channel between the dependent sub-operator, including: When it is determined that the cache area storage is not empty, the intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules, and new intermediate result data or the final calculation result matching the data query task is calculated.

2. The method according to claim 1, characterized in that When it is determined that the buffer area is not empty, the intermediate result data is obtained from the buffer area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rule, and new intermediate result data or a final calculation result matching the data query task is calculated, including: When it is determined that the buffer area is not empty, at least one Golang coroutine identical to the intermediate node operator is started according to the preset data reading rules, and intermediate result data is obtained from the buffer area of ​​the coroutine message channel between the dependent sub-operator in a parallel execution manner according to the preset data reading rules, and new intermediate result data is calculated; or When it is determined that the cache area storage is not empty, at least one Golang coroutine that is the same as the root node operator is started according to the preset data reading rules, and the intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules using a parallel execution method, and the final calculation result that matches the data query task is calculated.

3. The method according to claim 1, characterized in that Through the intermediate node operator, based on the intermediate result data obtained from the coroutine message channel between the dependent sub-operator, new intermediate result data is calculated, including: When there are at least two intermediate node operators with the same operator purpose, the intermediate node operator with a faster execution rate obtains new intermediate result data based on the intermediate result data obtained from the coroutine message channel between it and the same sub-operator it depends on. The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator, including: When there are at least two root node operators with the same operator purpose, the root node operator with a faster execution rate is used to obtain the intermediate result data obtained from the coroutine message channel between the same sub-operator it depends on, and calculates the final calculation result that matches the data query task.

4. The method according to claim 1, wherein Also includes: Acquiring a control signal of a control system that matches the data query task, wherein a coroutine message channel is pre-established between the control system and the data query task; According to the control signal, the leaf node operator of the data query task is triggered to stop calculating to obtain intermediate result data, so as to stop the data query task.

5. The method according to claim 4, characterized in that The control signals include: memory control signals, processor control signals and process shutdown signals.

6. The method according to claim 1, characterized in that Also includes: The root node operator, leaf node operator, and intermediate node operator included in the data query task matching the target SQL, as well as the coroutine message channels between operators with parent-child dependencies, are stored in the object pool.

7. A computer device comprising a processor and a memory, wherein the memory is configured to store instructions that, when executed, cause the processor to perform the following operations: In response to the target structured query language SQL, trigger the execution of a data query task matching the target SQL, wherein, The data query task includes root node operators, leaf node operators, and intermediate node operators. Coroutine message channels are pre-established between operators with parent-child dependencies. The leaf node operator calculates the intermediate result data and caches the intermediate result data in the coroutine message channel between the parent operator and the operator it depends on. By using an intermediate node operator, new intermediate result data is calculated based on the intermediate result data obtained from the coroutine message channel between the intermediate node operator and the dependent child operator, and the new intermediate result data is cached in the coroutine message channel between the intermediate node operator and the dependent parent operator; The root node operator calculates the final calculation result that matches the data query task based on the intermediate result data obtained from the coroutine message channel between the root node operator and the dependent sub-operator; The coroutine message channel includes at least one buffer area; the processor is configured to cache intermediate result data in the following manner: When it is determined that the buffer area is not full, the intermediate result data or the new intermediate result data is cached in the buffer area of ​​the coroutine message channel between the dependent parent operators according to the preset cache rule; The processor is configured to calculate new intermediate result data or a final calculation result matching the data query task in the following manner: When it is determined that the cache area storage is not empty, the intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules, and new intermediate result data or the final calculation result matching the data query task is calculated.

8. The computer device according to claim 7, wherein: The processor is configured to calculate new intermediate result data or a final calculation result matching the data query task in the following manner: When it is determined that the buffer area is not empty, at least one Golang coroutine identical to the intermediate node operator is started according to the preset data reading rules, and intermediate result data is obtained from the buffer area of ​​the coroutine message channel between the dependent sub-operator in a parallel execution manner according to the preset data reading rules, and new intermediate result data is calculated; or When it is determined that the cache area storage is not empty, at least one Golang coroutine that is the same as the root node operator is started according to the preset data reading rules, and the intermediate result data is obtained from the cache area of ​​the coroutine message channel between the dependent sub-operator according to the preset data reading rules using a parallel execution method, and the final calculation result that matches the data query task is calculated.

9. The computer device according to claim 7, wherein: The processor is configured to calculate new intermediate result data in the following manner: When there are at least two intermediate node operators with the same operator purpose, the intermediate node operator with a faster execution rate obtains new intermediate result data based on the intermediate result data obtained from the coroutine message channel between it and the same sub-operator it depends on. The processor is configured to calculate and obtain a final calculation result matching the data query task in the following manner: When there are at least two root node operators with the same operator purpose, the root node operator with a faster execution rate is used to obtain the intermediate result data obtained from the coroutine message channel between the same sub-operator it depends on, and calculates the final calculation result that matches the data query task.

10. The computer device according to claim 7, wherein: The processor is configured to stop the data query task in the following manner: Acquiring a control signal of a control system that matches the data query task, wherein a coroutine message channel is pre-established between the control system and the data query task; According to the control signal, the leaf node operator of the data query task is triggered to stop calculating to obtain intermediate result data, so as to stop the data query task.

11. The computer device according to claim 10, wherein: The control signals include: memory control signals, processor control signals and process shutdown signals.

12. The computer device according to claim 7, wherein: The processor is configured to store data query tasks in the following manner: The root node operator, leaf node operator, and intermediate node operator included in the data query task matching the target SQL, as well as the coroutine message channels between operators with parent-child dependencies, are stored in the object pool.

13. A storage medium, wherein the storage medium is used to store instructions, wherein the instructions are used to execute the data query method according to any one of claims 1 to 6.

Images