Database-based data processing method, device, medium and electronic equipment

Abstract

The present disclosure relates to a database-based data processing method, device, medium and electronic equipment, the method comprising: receiving a query request task to be executed, and determining a plurality of coroutine tasks corresponding to the query request task, the coroutine tasks being initially added to a global task queue shared by a plurality of threads, and then being distributed from the global task queue to a local task queue of a corresponding thread; in each thread, determining a target coroutine task to be executed according to time information of each coroutine task in the local task queue of the thread; interrupting the target coroutine task and adding the target coroutine task to the global task queue when the target coroutine task is not executed and the target coroutine task is after the current time slice of the execution of the thread; determining a new target coroutine task of the thread according to the global task queue and the local task queue of the thread, and executing the new target coroutine task in the next time slice.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically, to a database-based data processing method, apparatus, medium, and electronic device. Background Technology

[0002] In the current database architecture, after a SQL request is processed by the Query Optimizer and Planner, an execution plan is generated, which is then handed over to the Execution Engine for specific calculations. In related technologies, the execution engine mainly consists of multiple operators (e.g., Join operators, Shuffle operators) and executors that drive the data flow within these operators. The execution flow of the operators within the executors has a significant impact on the execution of the SQL statement. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, this disclosure provides a database-based data processing method, the method comprising:

[0005] Receive a query request task to be executed and determine multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread.

[0006] In each thread, the target coroutine task to be executed is determined based on the time information of each coroutine task in the thread's local task queue.

[0007] If the target coroutine task has not been completed and the target coroutine task has finished executing the current time slice of the thread, the target coroutine task is interrupted and added to the global task queue.

[0008] Based on the global task queue and the thread's local task queue, a new target coroutine task for the thread is determined, and the new target coroutine task is executed in the next time slice.

[0009] Secondly, this disclosure provides a database-based data processing apparatus, the apparatus comprising:

[0010] The receiving module is used to receive a query request task to be executed and determine the multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread.

[0011] The first determining module is used to determine the target coroutine task to be executed in each thread based on the time information of each coroutine task in the thread's local task queue.

[0012] The processing module is configured to interrupt the target coroutine task before it has been completed and after the current time slice of the thread's execution, and add the target coroutine task to the global task queue.

[0013] The second determining module is used to determine the new target coroutine task of the thread based on the global task queue and the thread's local task queue, and execute the new target coroutine task in the next time slice.

[0014] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0015] Fourthly, this disclosure provides an electronic device, comprising:

[0016] A storage device on which computer programs are stored;

[0017] A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0018] In the above technical solution, multiple coroutine tasks corresponding to the query request task to be executed are identified. These coroutine tasks can then be scheduled and executed on existing threads. A two-level scheduling system, using a global task queue and a thread-local task queue, is implemented. Each coroutine task within a thread is interrupted after completing a time slice to allow for the execution of other coroutine tasks. This supports parallel execution of multiple coroutine tasks, providing an accurate and reasonable execution flow for the query request task, and improving the concurrency and efficiency of task execution. Furthermore, user-space coroutines can be used to simulate threads to implement the task execution flow, making it suitable for scenarios with fewer threads, achieving resource reuse, and further broadening the application scenarios of this data processing method. In addition, this technical solution improves task execution efficiency while avoiding additional resource consumption and overhead, thus ensuring the utilization rate of system resources.

[0019] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] Figure 1 This is a flowchart illustrating how the executor executes the operator using the volcano model.

[0022] Figure 2 This is a schematic diagram of the executor executing operators using the Push model;

[0023] Figure 3 This is a schematic diagram of the process by which the actuator executes the operator using a DAG pipeline model;

[0024] Figure 4 This is a schematic diagram of the process by which the actuator executes the operator using a segmented pipeline model;

[0025] Figure 5 This is a flowchart of a database-based data processing method provided according to one embodiment of the present disclosure;

[0026] Figure 6 This is a schematic diagram of segmentation in a database-based data processing method according to one embodiment of the present disclosure;

[0027] Figure 7 This is a schematic diagram of a global task queue and a local task queue provided according to one embodiment of the present disclosure;

[0028] Figure 8 This is a block diagram of a database-based data processing apparatus provided according to one embodiment of the present disclosure;

[0029] Figure 9 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0037] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0038] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0039] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0040] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0041] The applicant's research found that, in related technologies, actuators typically handle the transfer of operators in the following ways:

[0042] The first approach, the Volcano Model (Pull Model), abstracts each operator as an iterator in its execution engine. Each iterator implements a Next method. Each call to the Next method returns the data obtained by that operator. The execution engine obtains the complete query result by continuously calling the Next method at the root node of the SQL query's computation tree. For example... Figure 1 As shown, operator A needs to go through 4 steps to get a row of data from downstream operator B: S1: call B.Next(), S2: call C.Next(), S3: C.Next() returns, S4: B.Next() returns.

[0043] The second approach is the Push model. In the Push model, each operator only implements its own data transformation method, and the data flow is driven from the bottom up by external callers, such as... Figure 2 S1, S2, and S3 shown in the diagram all call work() to drive the data flow between different operators, but there is no mutual calling relationship between the operators.

[0044] The third approach is the pipeline model. In the DAG (Directed Acyclic Graph) pipeline model, operators are abstracted as a DAG, and each thread can execute the task of any node (i.e., operator) on the DAG. For example... Figure 3 As shown, threads C1, C2, and C3 can execute S1 (calling A.Work()), S2 (calling D.Work()), and S3 (calling C.Work()) respectively, achieving parallel execution. In the segmented pipeline model, it is based on the DAG pipeline model, further segmented so that each operator in a segment is executed by the same thread, and each thread can only execute the operators on its corresponding segment. Data between segments is exchanged through a message queue. For example... Figure 4 The diagram shows three segments P1-P3. Thread C3 can only execute operator C, thread C2 can only execute operator D, and thread C1 can execute either operator A or operator B. Figure 4The execution logic shown allows threads C1, C2, and C3 to execute S1: calling A.Work(), S2: calling D.Work(), and S3: calling C.Work(), respectively.

[0045] However, using the methods described above, the Volcano and Push models cannot execute the corresponding physical plans in parallel, making them unsuitable for processing complex SQL requests with large data volumes. In the DAG pipeline model, each thread may attempt to execute any operator on the DAG, potentially leading to multiple threads attempting to execute the same operator, which introduces a locking mechanism. This can result in significant lock contention overhead in systems with large amounts of data or a large number of cores. Furthermore, if two threads execute related upstream and downstream operators simultaneously, the downstream operator cannot continue executing the upstream operator after completion; instead, the currently processed data must be transferred to another thread executing the upstream operator. This cross-thread data exchange disrupts CPU cache locality. The segmented pipeline model struggles to control system thread and CPU resource usage and isolate requests from resource usage. Simultaneously, a large number of system-level threads puts pressure on the operating system's task scheduling, slowing down the overall SQL request execution speed. Based on this, this disclosure provides the following embodiments.

[0046] Figure 5 The diagram shown is a flowchart of a database-based data processing method according to an embodiment of this disclosure. Figure 5 As shown, the method may include:

[0047] In step 11, a query request task to be executed is received, and multiple coroutine tasks corresponding to the query request task are determined. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread.

[0048] A coroutine is a program component that can be logically understood as a lightweight thread or an interruptible function with its own runtime context. Coroutine tasks run on top of threads. After a coroutine task completes or is interrupted, it can choose to voluntarily yield so that another coroutine task can run on the current thread. In this embodiment, a query request task to be executed can be divided into multiple coroutine tasks corresponding to that query request task. After all the multiple coroutine tasks have completed execution, the query request task is considered complete.

[0049] In this embodiment, the coroutine tasks obtained from the division can be scheduled and executed through a two-level queue. Each thread can have its own local task queue, which can be used to store the coroutine tasks that the thread needs to execute. In addition, there is a global task queue shared by the multiple threads, which is used to distribute and schedule coroutine tasks to the local task queues of each thread.

[0050] In step 12, in each thread, the target coroutine task to be executed is determined based on the time information of each coroutine task in the thread's local task queue.

[0051] The time information of the coroutine task can be the distribution time when the coroutine task is distributed to the local task queue. When determining the target coroutine task to be executed, the coroutine task with the earliest time information in the local task queue can be determined as the target coroutine task. This allows the thread to execute the coroutine tasks in the local task queue according to the FIFO (First Input First Output) rule.

[0052] In step 13, if the target coroutine task has not been completed and the target coroutine task has finished executing the current time slice of the thread, the target coroutine task is interrupted and added to the global task queue.

[0053] The duration of the time slice can be set according to the actual application scenario, and this disclosure does not limit it. After the target coroutine task is determined, it can be run on the current thread. If the target coroutine task has not been completed after one time slice, it needs to be interrupted and added to the global task queue so that it can be rescheduled in a subsequent time slice and thus complete.

[0054] In this embodiment, multiple threads share a global task queue. That is, any coroutine tasks that have not yet completed in any thread are added back to the same global task queue after an interruption and rescheduled until the coroutine task is completed. If the target coroutine task completes within the current time slice, it can be completed without needing to be added to the global task queue.

[0055] In step 14, a new target coroutine task for the thread is determined based on the global task queue and the thread's local task queue, and the new target coroutine task is executed in the next time slice.

[0056] The thread's local task queue contains the coroutine tasks that the thread needs to execute, and the global task queue contains coroutine tasks that need to be scheduled to multiple threads. A new target coroutine task can be determined from these multiple coroutine tasks that need to be executed, so that other coroutine tasks can be run on the same thread, thus running multiple coroutine tasks on one thread.

[0057] In the above technical solution, multiple coroutine tasks corresponding to the query request task to be executed are identified. These coroutine tasks can then be scheduled and executed on existing threads. A two-level scheduling system, using a global task queue and a thread-local task queue, is implemented. Each coroutine task within a thread is interrupted after completing a time slice to allow for the execution of other coroutine tasks. This supports parallel execution of multiple coroutine tasks, providing an accurate and reasonable execution flow for the query request task, and improving the concurrency and efficiency of task execution. Furthermore, user-space coroutines can be used to simulate threads to implement the task execution flow, making it suitable for scenarios with fewer threads, achieving resource reuse, and further broadening the application scenarios of this data processing method. In addition, this technical solution improves task execution efficiency while avoiding additional resource consumption and overhead, thus ensuring the utilization rate of system resources.

[0058] In one possible embodiment, an exemplary implementation of determining the multiple coroutine tasks corresponding to the query request task may include:

[0059] The query request task is parsed to determine the multiple operators corresponding to the query request task.

[0060] The query request task can be an SQL request task used to query from a corresponding database. For example, the query request task can be parsed to determine the operators it contains. The execution of an SQL statement can be divided into different execution steps, each typically containing one or more SQL operators. SQL operators represent the specific actions taken in each step of the SQL statement execution process, and may include, but are not limited to, Select, From, Join, Group by, and Having. For example, multiple operators corresponding to the query request task can be determined based on SQL operator parsing methods existing in the field. Figure 6 As shown, the operator corresponding to the query request task can include the operator AH.

[0061] For each operator, if there are multiple input operators, then the operator and the input operators are assigned to different coroutine tasks;

[0062] If the operator has only one input operator, then the operator and the input operator are assigned to the same coroutine task.

[0063] like Figure 6As shown, the arrows between operators indicate their execution flow. Taking operator A as an example, where operator A's input operator is operator B (meaning operator A has only one input operator), operators A and B can be assigned to the same coroutine task. Operator B's input operators are operators C and D (meaning operator B has multiple input operators), so operators B, C, and D can be assigned to different coroutine tasks. Similarly, operator AH can be divided into 7 coroutine tasks, such as... Figure 6 The coroutine tasks C1-C7 are shown in the diagram.

[0064] Therefore, the above technical solution can parse the query request task to generate multiple coroutine tasks. This can either divide the query request task into segments to enable parallel execution to a certain extent, or run multiple coroutine tasks in a smaller number of threads, thereby improving the execution efficiency of the query request task.

[0065] In one possible implementation, the target coroutine task can be added to the global task queue in the following manner:

[0066] Determine the cumulative execution time of the target coroutine task and the CPU resource quota corresponding to the target coroutine task.

[0067] The cumulative execution time can be determined by the number of scheduling attempts and time slices of the target coroutine task. For example, the product of the number of scheduling attempts and the time slices can be used as the cumulative execution time. The CPU resource quota can be determined based on the priority of the requester in the query request task corresponding to the coroutine task. Quotas corresponding to different priorities can be pre-set, so the corresponding quota for the target coroutine task can be determined based on the priority of its corresponding query request task. This quota can be the number of CPU cores used; higher-priority coroutine tasks use more CPU cores.

[0068] The execution parameters of the target coroutine task are determined based on the cumulative execution time and the quota.

[0069] The cumulative execution time can be used to represent the number of times the coroutine task has been scheduled. A longer cumulative execution time indicates that the coroutine task has been executed multiple times, thus its priority should be lower, and its execution parameters should be smaller. Conversely, a larger quota indicates that more resources are available for executing the coroutine task, and its corresponding execution parameters should be smaller. As an example, the ratio of the cumulative execution time to the quota can be determined as the execution parameter. This allows for a comprehensive consideration of both the coroutine task's own execution time and its configuration, ensuring fairness in the coroutine task scheduling process and guaranteeing the smoothness and rationality of task execution.

[0070] Then, the target coroutine task can be added to the global task queue according to the execution parameters.

[0071] The execution parameters can be used to represent the execution priority of coroutine tasks. Coroutine tasks in the global task queue can be arranged in ascending order of execution parameters. When adding a target coroutine task to the global task queue, it can be inserted based on its execution parameters, ensuring that the execution parameters of the coroutine tasks in the global task queue remain in ascending order after insertion. For example... Figure 7 The diagram shown is a schematic representation of a global task queue and a local task queue provided according to an embodiment of the present disclosure.

[0072] As an example, the global task queue can be implemented using a red-black tree structure. A red-black tree is a self-balancing binary search tree that maintains its balance during insertion and deletion operations, thereby improving search performance, facilitating the fast and accurate insertion of target coroutine tasks, and ensuring the accuracy of the global task queue's sorting. The red-black tree structure can be implemented using methods commonly used in this field, which will not be elaborated upon here.

[0073] Therefore, through the above technical solution, the scheduling priority of the target coroutine task can be considered based on the cumulative execution time and CPU resource quota, which can ensure the accuracy and rationality of the execution parameters to a certain extent. This provides accurate data support for subsequent scheduling of coroutine tasks in the global task queue based on the execution parameters, ensures the fairness of coroutine task scheduling, and thus improves the efficiency of task execution.

[0074] In one possible embodiment, the implementation of determining the new target coroutine task of the thread based on the global task queue and the thread's local task queue may include:

[0075] If a coroutine task exists in the local task queue of the thread, the coroutine task with the earliest dispatch time in the local task queue shall be used as the new target coroutine task.

[0076] If the coroutine task in the local task queue of the thread is empty, then a coroutine task is requested from the global task queue and added to the local task queue, and the new target coroutine task is determined from the local task queue after the coroutine task is requested.

[0077] In this embodiment, each thread has its own local task queue. When consuming tasks, the thread can prioritize consuming the coroutine tasks in its local task queue. That is, if a coroutine task exists in the thread's local task queue, a new target coroutine task can be directly determined from its local task queue. After consuming the coroutine tasks in the local task queue, the thread then consumes tasks from the global coroutine task queue. In other words, if the coroutine tasks in the thread's local task queue are empty, a new coroutine task can be requested from the global queue and executed locally.

[0078] In one possible embodiment, the coroutine tasks in the global task queue are sorted according to their execution priority from high to low. As mentioned above, this execution priority can be represented by execution parameters. The smaller the execution parameter, the higher the execution priority. That is, the coroutine tasks in the global task queue can be sorted in ascending order of their execution parameters, with priority given to coroutine tasks with smaller execution parameters.

[0079] Accordingly, the implementation of requesting coroutine tasks from the global task queue to the local task queue may include:

[0080] Send a coroutine task request to the global task queue, wherein the coroutine task request includes the number of requests.

[0081] Each thread can have its own scheduler to schedule coroutine tasks in its local task queue. When a thread's local task queue is empty, a coroutine task request can be sent to the global task queue to distribute the coroutine task from the global task queue to the local thread. For example, the global task queue can have a corresponding global scheduler to schedule the coroutine tasks in it. As an example, for a newly received query request task, its corresponding coroutine task can first be added to the global task queue. Then, it can be randomly distributed from the global task queue to the local task queues of multiple threads for execution. Afterward, it can be further added to the global task queue for subsequent distribution based on the methods described above.

[0082] Coroutine tasks distributed from the global task queue in response to the coroutine task request are added to the local task queue, wherein the distributed coroutine tasks are the requested number of coroutine tasks selected sequentially based on the order in the global task queue.

[0083] When the global scheduler receives a coroutine task request, it can obtain the number of requests, i.e., the number of coroutine tasks requested by the thread. Then, it can select the top M coroutine tasks from the global task queue, sorted by priority, and distribute them to the local task queue, where M is the number of requests. Since coroutine tasks in the global task queue are sorted by execution priority, when a thread requests a coroutine task in its local task queue, it is prioritized to distribute coroutine tasks with higher execution priority, ensuring both priority and efficiency in task execution.

[0084] Specifically, after distributing the first M coroutine tasks from the global task queue to the local task queue, these M coroutine tasks can be removed from the global task queue to facilitate the subsequent distribution of other coroutine tasks in the global task queue.

[0085] Therefore, through the above technical solution, the scheduling of coroutine tasks can be realized through the global task queue and the local task queue of the thread, so as to limit the CPU resource consumption of the executor of the segmented process line model driven by coroutines, improve the efficiency and concurrency of task execution, and improve the utilization of CPU resources.

[0086] Based on the same inventive concept, this disclosure also provides a database-based data processing apparatus, such as... Figure 8 As shown, the device 10 includes:

[0087] The receiving module 100 is used to receive a query request task to be executed and determine multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread.

[0088] The first determining module 200 is used to determine the target coroutine task to be executed in each thread based on the time information of each coroutine task in the local task queue of the thread.

[0089] The processing module 300 is configured to interrupt the target coroutine task before it has been completed and after the current time slice of the thread's execution, and add the target coroutine task to the global task queue.

[0090] The second determining module 400 is used to determine a new target coroutine task for the thread based on the global task queue and the thread's local task queue, and to execute the new target coroutine task in the next time slice.

[0091] Optionally, the target coroutine task can be added to the global task queue via an adding module, the adding module comprising:

[0092] The first determining submodule determines the cumulative execution time of the target coroutine task and the CPU resource quota corresponding to the target coroutine task;

[0093] The second determining submodule is used to determine the execution parameters of the target coroutine task based on the cumulative execution time and the quota;

[0094] Add a submodule to add the target coroutine task to the global task queue according to the execution parameters.

[0095] Optionally, the second determining submodule is further configured to:

[0096] The ratio of the cumulative execution time to the quota is determined as the execution parameter.

[0097] Optionally, the second determining module includes:

[0098] The third determining submodule is used to select the coroutine task with the earliest dispatch time in the local task queue as the new target coroutine task if there is a coroutine task in the local task queue of the thread.

[0099] The fourth determination submodule is used to request a coroutine task from the global task queue to the local task queue if the coroutine task in the local task queue of the thread is empty, and to determine the new target coroutine task from the local task queue after requesting the coroutine task.

[0100] Optionally, the coroutine tasks in the global task queue are sorted from high to low according to their execution priority;

[0101] The fourth determining submodule includes:

[0102] The sending submodule is used to send coroutine task requests to the global task queue, wherein the coroutine task request includes the number of requests;

[0103] A processing submodule is configured to add coroutine tasks distributed from the global task queue in response to the coroutine task request to the local task queue, wherein the distributed coroutine tasks are the number of coroutine tasks requested in order of sorting in the global task queue.

[0104] Optionally, the receiving module is further configured to:

[0105] The query request task is parsed to determine multiple operators corresponding to the query request task;

[0106] For each operator, if there are multiple input operators, then the operator and the input operators are assigned to different coroutine tasks;

[0107] If the operator has only one input operator, then the operator and the input operator are assigned to the same coroutine task.

[0108] Optionally, the global task queue is implemented using a red-black tree structure.

[0109] The following is for reference. Figure 9 The diagram illustrates a structural schematic of an electronic device (terminal device or server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0110] like Figure 9 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0111] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0112] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0113] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0114] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0115] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0116] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a query request task to be executed, and determine multiple coroutine tasks corresponding to the query request task, wherein the coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread; in each thread, determine a target coroutine task to be executed based on the time information of each coroutine task in the thread's local task queue; interrupt the target coroutine task before it is completed and after the current time slice of the thread's execution, and add the target coroutine task to the global task queue; determine a new target coroutine task for the thread based on the global task queue and the thread's local task queue, and execute the new target coroutine task in the next time slice.

[0117] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not necessarily limiting in certain circumstances; for example, a receiving module can also be described as "a module that receives a query request task to be executed and determines the multiple coroutine tasks corresponding to the query request task."

[0120] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0121] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0122] According to one or more embodiments of this disclosure, Example 1 provides a database-based data processing method, wherein the method includes:

[0123] Receive a query request task to be executed and determine multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread.

[0124] In each thread, the target coroutine task to be executed is determined based on the time information of each coroutine task in the thread's local task queue.

[0125] If the target coroutine task has not been completed and the target coroutine task has finished executing the current time slice of the thread, the target coroutine task is interrupted and added to the global task queue.

[0126] Based on the global task queue and the thread's local task queue, a new target coroutine task for the thread is determined, and the new target coroutine task is executed in the next time slice.

[0127] According to one or more embodiments of this disclosure, Example 2 provides the method of Example 1, wherein the target coroutine task is added to the global task queue in the following manner:

[0128] Determine the cumulative execution time of the target coroutine task and the CPU resource quota corresponding to the target coroutine task;

[0129] The execution parameters of the target coroutine task are determined based on the cumulative execution time and the quota.

[0130] The target coroutine task is added to the global task queue according to the execution parameters.

[0131] According to one or more embodiments of this disclosure, Example 3 provides the method of Example 2, wherein determining the execution parameters of the target coroutine task based on the cumulative execution time and the quota includes:

[0132] The ratio of the cumulative execution time to the quota is determined as the execution parameter.

[0133] According to one or more embodiments of this disclosure, Example 4 provides the method of Example 1, wherein determining a new target coroutine task for the thread based on the global task queue and the thread's local task queue includes:

[0134] If a coroutine task exists in the local task queue of the thread, the coroutine task with the earliest dispatch time in the local task queue shall be used as the new target coroutine task.

[0135] If the coroutine task in the local task queue of the thread is empty, then a coroutine task is requested from the global task queue and added to the local task queue, and the new target coroutine task is determined from the local task queue after the coroutine task is requested.

[0136] According to one or more embodiments of this disclosure, Example 5 provides the method of Example 4, wherein the coroutine tasks in the global task queue are sorted in descending order of execution priority;

[0137] The step of requesting a coroutine task from the global task queue to the local task queue includes:

[0138] Send a coroutine task request to the global task queue, wherein the coroutine task request includes the number of requests;

[0139] Coroutine tasks distributed from the global task queue in response to the coroutine task request are added to the local task queue, wherein the distributed coroutine tasks are the requested number of coroutine tasks selected sequentially based on the order in the global task queue.

[0140] According to one or more embodiments of this disclosure, Example 6 provides the method of Example 1, wherein determining the plurality of coroutine tasks corresponding to the query request task includes:

[0141] The query request task is parsed to determine multiple operators corresponding to the query request task;

[0142] For each operator, if there are multiple input operators, then the operator and the input operators are assigned to different coroutine tasks;

[0143] If the operator has only one input operator, then the operator and the input operator are assigned to the same coroutine task.

[0144] According to one or more embodiments of this disclosure, Example 7 provides a method of any of Examples 1-6, wherein the global task queue is implemented in a red-black tree structure.

[0145] According to one or more embodiments of this disclosure, Example 8 provides a database-based data processing apparatus, wherein the apparatus includes:

[0146] The receiving module is used to receive a query request task to be executed and determine the multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread.

[0147] The first determining module is used to determine the target coroutine task to be executed in each thread based on the time information of each coroutine task in the thread's local task queue.

[0148] The processing module is configured to interrupt the target coroutine task before it has been completed and after the current time slice of the thread's execution, and add the target coroutine task to the global task queue.

[0149] The second determining module is used to determine the new target coroutine task of the thread based on the global task queue and the thread's local task queue, and execute the new target coroutine task in the next time slice.

[0150] According to one or more embodiments of the present disclosure, Example 9 provides a computer-readable medium having a computer program stored thereon that, when executed by a processing device, implements the steps of the method described in any one of Examples 1-7.

[0151] According to one or more embodiments of this disclosure, Example 10 provides an electronic device, including:

[0152] A storage device on which computer programs are stored;

[0153] A processing device for executing the computer program in the storage device to implement the steps of any one of the methods in Examples 1-7.

[0154] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0155] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0156] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A database-based data processing method, characterized in that, The method includes: Receive a query request task to be executed and determine multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread. The coroutine tasks in the global task queue are sorted from high to low execution priority and distributed from high to low execution priority. In each thread, the target coroutine task to be executed is determined based on the time information of each coroutine task in the thread's local task queue. If the target coroutine task has not been completed and the target coroutine task has finished executing the current time slice of the thread, the target coroutine task is interrupted and added to the global task queue. Based on the global task queue and the thread's local task queue, a new target coroutine task for the thread is determined, and the new target coroutine task is executed in the next time slice.

2. The method according to claim 1, characterized in that, The target coroutine task can be added to the global task queue in the following way: Determine the cumulative execution time of the target coroutine task and the CPU resource quota corresponding to the target coroutine task; The execution parameters of the target coroutine task are determined based on the cumulative execution time and the quota. The target coroutine task is added to the global task queue according to the execution parameters.

3. The method according to claim 2, characterized in that, The step of determining the execution parameters of the target coroutine task based on the cumulative execution time and the quota includes: The ratio of the cumulative execution time to the quota is determined as the execution parameter.

4. The method according to claim 1, characterized in that, The step of determining the new target coroutine task for the thread based on the global task queue and the thread's local task queue includes: If a coroutine task exists in the local task queue of the thread, the coroutine task with the earliest dispatch time in the local task queue shall be used as the new target coroutine task. If the coroutine task in the local task queue of the thread is empty, then a coroutine task is requested from the global task queue and added to the local task queue, and the new target coroutine task is determined from the local task queue after the coroutine task is requested.

5. The method according to claim 4, characterized in that, The step of requesting a coroutine task from the global task queue to the local task queue includes: Send a coroutine task request to the global task queue, wherein the coroutine task request includes the number of requests; Coroutine tasks distributed from the global task queue in response to the coroutine task request are added to the local task queue, wherein the distributed coroutine tasks are the requested number of coroutine tasks selected sequentially based on the order in the global task queue.

6. The method according to claim 1, characterized in that, The determination of the multiple coroutine tasks corresponding to the query request task includes: The query request task is parsed to determine multiple operators corresponding to the query request task; For each operator, if there are multiple input operators, then the operator and the input operators are assigned to different coroutine tasks; If the operator has only one input operator, then the operator and the input operator are assigned to the same coroutine task.

7. The method according to any one of claims 1-6, characterized in that, The global task queue is implemented using a red-black tree structure.

8. A database-based data processing device, characterized in that, The device includes: The receiving module is used to receive a query request task to be executed and determine multiple coroutine tasks corresponding to the query request task. The coroutine tasks are initially added to a global task queue shared by multiple threads, and then distributed from the global task queue to the local task queue of the corresponding thread. The coroutine tasks in the global task queue are sorted from high to low execution priority and distributed from high to low execution priority. The first determining module is used to determine the target coroutine task to be executed in each thread based on the time information of each coroutine task in the thread's local task queue. The processing module is configured to interrupt the target coroutine task before it has been completed and after the current time slice of the thread's execution, and add the target coroutine task to the global task queue. The second determining module is used to determine the new target coroutine task of the thread based on the global task queue and the thread's local task queue, and execute the new target coroutine task in the next time slice.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by the processing device, the program implements the steps of the method described in any one of claims 1-7.

10. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-7.

Images