Method, device, equipment, medium and product for multi-accelerator card pipeline parallel execution

Abstract

The application discloses a kind of multi-acceleration card pipeline parallel execution method, device, equipment, medium and product.The method is applied to distributed system in database, comprising: by coordinator, distributed plan is divided into multiple fragment logical plans and is transmitted to corresponding executor;By executor, fragment logical plan is received, fragment logical plan is converted into local multi-card physical operator tree, re-partitioning exchange operator is inserted in local multi-card physical operator tree and is optimized, and the operator tree after optimization is obtained;By executor, pipeline blocking point in the operator tree after optimization is identified, and the operator tree after optimization is divided according to pipeline blocking point, and the pipeline task is obtained;By the computing card in executor, corresponding pipeline task is executed, and the fragment execution result is obtained, and all fragment execution results in executor are transmitted to coordinator;By coordinator, the fragment execution result transmitted by all executors is received, and the final execution result is obtained.It can improve the resource utilization of computing card.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to methods, apparatus, equipment, media, and products for parallel execution of multi-accelerator card pipelines. Background Technology

[0002] Database systems typically translate Structured Query Language (SQL) into execution plans. To improve execution efficiency, databases pipeline multiple operators capable of processing data blocks sequentially; when an operator is encountered that requires waiting for complete input before continuing, such as global aggregation, sorting, or repartitioning and swapping, a pipeline bottleneck is formed.

[0003] Heterogeneous accelerated databases or query execution acceleration systems typically execute operators in columnar data blocks. In multi-device systems, query plans can be divided into multiple data segments or partitions, processed by different devices. When joins or aggregations require key-based merging, repartitioning and swapping are necessary, sending data to the target device according to hash, range, broadcast, or aggregation methods. However, when splitting query plans, resource utilization is often insufficient due to overly coarse pipeline splitting granularity or over-reliance on single-device execution. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, medium, and product for parallel execution of multi-accelerator card pipelines to solve the problem of insufficient resource utilization.

[0005] According to one aspect of the present invention, a method for parallel execution of multi-accelerator card pipelines is provided, comprising: The coordinator divides the distributed plan into multiple fragment logical plans, and transmits the fragment logical plans to the corresponding executors. The executor receives the fragment logic plan, converts the fragment logic plan into a local multi-card physical operator tree, inserts a repartitioning and swapping operator into the local multi-card physical operator tree and optimizes it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning. The actuator identifies pipeline blockage points in the optimized operator tree, and the optimized operator tree is segmented according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning and swapping operator. The corresponding pipeline task is executed through the computing card in the executor to obtain the fragment execution result, and all the fragment execution results in the executor are transmitted to the coordinator. The coordinator receives the fragment execution results and obtains the final execution result based on the fragment execution results transmitted by all the executors.

[0006] According to another aspect of the present invention, a multi-accelerator card pipelined parallel execution apparatus is provided, comprising: The fragment logic plan partitioning module is used to divide the distributed plan into multiple fragment logic plans through the coordinator, and transmit the fragment logic plans to the corresponding executors; The local multi-card physical operator tree optimization module is used to receive the fragment logic plan through the executor, convert the fragment logic plan into a local multi-card physical operator tree, insert a repartitioning and swapping operator into the local multi-card physical operator tree and optimize it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning. The optimized operator tree segmentation module is used to identify pipeline blockage points in the optimized operator tree through the executor, and segment the optimized operator tree according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning exchange operator. The pipeline task execution module is used to execute the corresponding pipeline task through the computing card in the executor, obtain the fragment execution result, and transmit all the fragment execution results in the executor to the coordinator; The fragment execution result receiving module is used to receive the fragment execution result through the coordinator and obtain the final execution result based on the fragment execution results transmitted by all the executors.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method described in any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the method described in any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the method described in any embodiment of the present invention.

[0010] The technical solution of this invention involves dividing a distributed plan into multiple fragment logical plans through a coordinator, and transmitting these fragment logical plans to the corresponding executors. The executors receive the fragment logical plans, convert them into local multi-card physical operator trees, insert repartitioning and swapping operators into these local multi-card physical operator trees, and optimize them to obtain an optimized operator tree. The repartitioning and swapping operators include operators indicating data repartitioning. The executors identify pipeline bottlenecks in the optimized operator tree, and segment the optimized operator tree according to these bottlenecks to obtain pipeline tasks. The pipeline bottlenecks include the repartitioning and swapping operators. The compute cards in the executors execute the corresponding pipeline tasks to obtain fragment execution results, and all fragment execution results in the executors are transmitted to the coordinator. The coordinator receives the fragment execution results and, based on all the fragment execution results transmitted by the executors, obtains the final execution result. In the technical solution of this invention embodiment, in a distributed system, the coordinator uses the executor's process as the scheduling unit for stages, fragment logic plans, communication, isolation, and recovery. After the fragment logic plan arrives at the executor, the CPU within the executor uniformly manages all local computing cards and generates, binds, and dispatches pipeline tasks. This delineates the boundaries of responsibility between cross-process distributed scheduling and multi-card scheduling within the executor, enabling the system to retain the isolation, recoverability, and cluster scalability of a multi-process architecture while automatically utilizing multiple heterogeneous acceleration devices locally in a low-overhead manner within a single executor process. By refining the optimized operator tree into multiple fine-grained pipeline tasks that can be executed in parallel across heterogeneous computing cards based on pipeline bottlenecks within the corresponding executor, it ensures that the pipeline task will not wait for the upstream data at the pipeline bottleneck to be calculated before continuing execution. This avoids insufficient resource utilization caused by single computing card execution or overly coarse pipeline task granularity, thereby improving the resource utilization of computing cards and thus improving query efficiency.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1This is a flowchart of a multi-accelerator card pipeline parallel execution method provided in Embodiment 1 of the present invention; Figure 2 This is an execution flowchart of a pipeline task provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of a directed acyclic graph (DAG) for a pipeline provided in an embodiment of the present invention. Figure 4 This is a flowchart of a method for an executor to execute a segment logic plan according to an embodiment of the present invention; Figure 5 This is a flowchart of a multi-accelerator card pipeline parallel execution method provided in Embodiment 2 of the present invention; Figure 6 This is a flowchart of a method for exchanging task data using a repartitioning exchange operator, provided in an embodiment of the present invention. Figure 7 This is a schematic diagram of the structure of an actuator provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a distributed system provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a multi-accelerator card pipeline parallel execution device provided in Embodiment 3 of the present invention; Figure 10 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0016] Example 1 Figure 1This is a flowchart of a multi-accelerator card pipelined parallel execution method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where a distributed system is used for querying in a database. This method can be executed by a multi-accelerator card pipelined parallel execution device, which can be implemented in hardware and / or software and can be configured in an electronic device. For example, the electronic device can be a computer, workstation, or server. Figure 1 As shown, this method is applied to a distributed system, which includes a coordinator and multiple executors. Each executor includes multiple computing cards, including: S110. The distributed plan is divided into multiple fragment logical plans by the coordinator, and the fragment logical plans are transmitted to the corresponding executors.

[0017] In this embodiment, the coordinator can be a node used for scheduling in a distributed system. The coordinator includes at least one Central Processing Unit (CPU) for executing the method provided by this invention. The coordinator can schedule multiple executors. An executor can be a node in the distributed system actually used to execute queries. An executor includes at least one CPU. An executor can include multiple computing cards. The coordinator and executors can be different processes within the same electronic device. The architecture of the computing cards is not limited here; an executor can include multiple heterogeneous computing cards. Query execution is accelerated through computing cards. The distributed plan can be a plan obtained after parsing the query instructions. The distributed plan can be a plan that can be split and distributed to the corresponding executors for processing. The distributed plan can include multiple boundaries indicating data exchange. The query instructions can be instructions in Structured Query Language (SQL). The fragmented logical plan can be a plan obtained after the distributed plan is divided by the coordinator. The fragmented logical plan can include the semantics of the distributed plan. Each fragmented logical plan can be executed by a corresponding executor.

[0018] For example, the coordinator and the executor exchange fragment logic plans, metadata, control information on the completion or failure of fragment logic plan execution, and the results generated by the executor in executing the fragment logic plan. Data exchange between executors can be completed directly through inter-process communication or network data channels. The coordinator is responsible for channel and metadata coordination, but does not act as a necessary transit point for data payloads between executors.

[0019] Specifically, the coordinator obtains the SQL instruction to perform the query, parses the instruction, generates a distributed plan, and divides the distributed plan into multiple stages based on the boundaries for data exchange indicated in the distributed plan, as well as the fragment logical plan within each stage. The coordinator then sends the fragment logical plan to the corresponding executor for execution.

[0020] S120. The executor receives the fragment logic plan, converts the fragment logic plan into a local multi-card physical operator tree, inserts a repartitioning and swapping operator into the local multi-card physical operator tree and optimizes it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning.

[0021] In this embodiment, the local multi-card physical operator tree can be the physical plan obtained after the executor transforms the fragment logic plan. The local multi-card physical operator tree can include multiple operators, and topological relationships can exist between these operators. These topological relationships indicate the data flow direction between operators. The local multi-card physical operator tree can be a tree structure, with each operator as a node in the tree structure. The repartitioning and switching operator can be an operator that indicates data repartitioning. Through data repartitioning, data can be mapped from one logical region to another, thereby completing the transfer of data between computing cards or within a computing card. The optimized operator tree can be a local multi-card physical operator tree optimized based on the repartitioning and switching operator, the semantics of the operator, and the distribution of data. The optimized operator tree preserves the semantics of the fragment logic plan.

[0022] Specifically, the executor receives the fragment logic plan. The CPU process within the executor physicalizes the fragment logic plan, converting it into a local multi-GPU physical operator tree composed of multiple ordered nodes while preserving its semantics. The local multi-GPU physical operator tree is then rewritten based on the semantics of each operator and the distribution of the input data. Re-partitioning operators are inserted at locations requiring cross-GPU data re-partitioning, resulting in an optimized operator tree.

[0023] S130. The executor identifies pipeline blockage points in the optimized operator tree, and the optimized operator tree is segmented according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning exchange operator.

[0024] In this embodiment, a pipeline bottleneck can be a node in the optimized operator tree that blocks streaming processing between operators due to waiting for data input or data redistribution. Pipeline bottlenecks include, but are not limited to, repartitioning and swapping, result collection, unsplit aggregation, and boundaries of special connections. A pipeline task can be formed by dividing the optimized operator tree according to the pipeline bottleneck and then generating pipeline tasks based on the operators obtained from the division. During execution, a pipeline task can be executed streaming without waiting for data input. A pipeline task can include the operator to be executed and the identifier of the corresponding computation card. The identifier of the computation card can be determined when the pipeline task is generated and written into the context of the pipeline task.

[0025] Specifically, the CPU process in the executor identifies pipeline bottlenecks in the optimized operator tree. Based on the dependencies and data flow between the input and output data of nodes at these bottlenecks, it determines the pipeline task and the corresponding compute card for execution. The identifier of the corresponding compute card is written into the context of the pipeline task, and the pipeline task is submitted to the corresponding compute card. The dependency relationship can be one where upstream nodes produce output data, which is then consumed as input data by downstream nodes.

[0026] S140. The corresponding pipeline task is executed through the computing card in the executor to obtain the fragment execution result, and all the fragment execution results in the executor are transmitted to the coordinator.

[0027] In this embodiment, the fragment execution result can be the execution result obtained after the computing card in the executor executes the corresponding pipeline task.

[0028] Specifically, the compute card in the executor executes the corresponding pipelined tasks and obtains the corresponding fragment execution results. The executor collects the fragment execution results. After the fragment execution results of all pipelined tasks in the executor have been collected, or after the amount of collected fragment execution results meets certain conditions, all collected fragment execution results are transmitted to the coordinator. The collected fragment execution results can be stored in corresponding locations in memory; the specific storage location of the fragment execution results is not limited in this invention.

[0029] For example, Figure 2This is a flowchart of the execution of a pipelined task provided by an embodiment of the present invention. The executor binds to the corresponding compute card (denoted as the execution compute card) and the explicit Compute Unified Device Architecture (CUDA) stream in the compute card that executes the pipelined task by identifying it. The compute card reads the data source of the pipelined task from the context of the pipelined task, executes all operators in the order of the operators in the pipelined task (i.e., operator one, operator two, etc.), obtains the fragment execution result, writes the fragment execution result to the terminal operator (i.e., the downstream operator in the pipelined task) or stores the fragment execution result as output data in a specific location in memory, and then performs a synchronization operation or triggers a completion callback on the explicit CUDA stream bound to the pipelined task to confirm that all computation and copy operations in the explicit CUDA stream are ready, and finally ends the task and releases or marks the resources in the compute card as reusable.

[0030] S150. The coordinator receives the segment execution results and obtains the final execution result based on the segment execution results transmitted by all the executors.

[0031] In this embodiment, the final execution result can be the output of the coordinator in response to the instruction indicating the query. The final execution result can be obtained by processing the execution results of all received fragments.

[0032] Specifically, the coordinator receives the fragment execution results transmitted by all executors, performs processing such as merging, reorganizing and / or aggregating the fragment execution results, and obtains the final execution result.

[0033] The technical solution of this invention involves dividing a distributed plan into multiple fragment logical plans through a coordinator, and transmitting these fragment logical plans to the corresponding executors. The executors receive the fragment logical plans, convert them into local multi-card physical operator trees, insert repartitioning and swapping operators into these local multi-card physical operator trees, and optimize them to obtain an optimized operator tree. The repartitioning and swapping operators include operators indicating data repartitioning. The executors identify pipeline bottlenecks in the optimized operator tree, and segment the optimized operator tree according to these bottlenecks to obtain pipeline tasks. The pipeline bottlenecks include the repartitioning and swapping operators. The compute cards in the executors execute the corresponding pipeline tasks to obtain fragment execution results, and all fragment execution results in the executors are transmitted to the coordinator. The coordinator receives the fragment execution results and, based on all the fragment execution results transmitted by the executors, obtains the final execution result. In the technical solution of this invention embodiment, in a distributed system, the coordinator uses the executor's process as the scheduling unit for stages, fragment logic plans, communication, isolation, and recovery. After the fragment logic plan arrives at the executor, the CPU within the executor uniformly manages all local computing cards and generates, binds, and dispatches pipeline tasks. This delineates the boundaries of responsibility between cross-process distributed scheduling and multi-card scheduling within the executor, enabling the system to retain the isolation, recoverability, and cluster scalability of a multi-process architecture while automatically utilizing multiple heterogeneous acceleration devices locally in a low-overhead manner within a single executor process. By refining the optimized operator tree into multiple fine-grained pipeline tasks that can be executed in parallel across heterogeneous computing cards based on pipeline bottlenecks within the corresponding executor, it ensures that the pipeline task will not wait for the upstream data at the pipeline bottleneck to be calculated before continuing execution. This avoids insufficient resource utilization caused by single computing card execution or overly coarse pipeline task granularity, thereby improving the resource utilization of computing cards and thus improving query efficiency.

[0034] In one embodiment, the step of dividing the distributed plan into multiple fragment logical plans through the coordinator includes: S1101. Obtain the query statement through the coordinator.

[0035] In this embodiment, the query statement can be a statement that instructs the query to retrieve content from a specific location in the database. The query statement can be an SQL statement. The query statement can be contained within an instruction indicating the query.

[0036] Specifically, the coordinator can receive query statements through a specific data interface or through a specific data channel.

[0037] S1102. The coordinator parses the query statement to generate a distributed plan, the distributed plan including exchange boundaries across the executors.

[0038] In this embodiment, the exchange boundary can be a boundary indicating where data exchange takes place. The exchange boundary can be represented as a logical node in a distributed plan. In the prior art, when executing a distributed plan to the exchange boundary, data needs to be transmitted across logical nodes. Logical nodes downstream of the exchange boundary in the distributed plan must wait for the logical nodes upstream of the exchange boundary to complete execution and obtain the corresponding data before they can start. This invention, by dividing the distributed plan according to the exchange boundary, can decompose the distributed plan into multiple clearly defined sub-plans executed by the executor, thus improving the utilization efficiency of the executor.

[0039] Specifically, the coordinator parses the query statement to obtain a distributed plan, which includes multiple logical nodes, including exchange boundaries that indicate data exchange.

[0040] S1103. The distributed plan is divided by the exchange boundary to obtain a fragmented logical plan.

[0041] Specifically, the distributed plan is divided according to the exchange boundary to obtain multiple stages of the distributed plan, as well as the corresponding fragment logic plan for each stage.

[0042] In one embodiment, transmitting the fragment logic plan to the corresponding executor includes: A1. Generate a stage-directed acyclic graph based on the exchange boundary, wherein the stage-directed acyclic graph is used to indicate the stage dependencies between stages of the distributed plan.

[0043] In this embodiment, a staged directed acyclic graph (DAG) can be used to indicate the stage dependencies between stages of a distributed plan. Stage dependencies indicate the relationship between data produced by an upstream stage and consumed by the corresponding downstream stage.

[0044] Specifically, based on the various stages in the distributed plan indicated by the exchange boundary, each stage is treated as a node. The topology between the nodes is determined according to the stage dependencies between the stages, and then a stage-directed acyclic graph is generated based on each node and its corresponding topology.

[0045] A2. Based on the directed acyclic graph of the stage, determine the target executor corresponding to the segment logic plan, and write the identifier of the target executor into the segment logic plan.

[0046] In this embodiment, the target executor can be the executor corresponding to the execution fragment logic plan. The coordinator can determine the target executor through an identifier in the context of the fragment logic plan. The identifier can be a unique identifier for the target executor.

[0047] Specifically, based on the stage dependencies indicated by the directed acyclic graph, the corresponding target executor is determined for each fragment logic plan in each stage, and the identifier of the target executor is written into the context of the fragment logic plan.

[0048] A3. Transmit the fragment logic plan to the target executor.

[0049] Specifically, the coordinator identifies the target executor by recognizing the identifier in the context of the fragment logic plan and then transmits the fragment logic plan to the target executor.

[0050] In one embodiment, the local multi-GPU physical operator tree further includes an aggregation operator and a connection operator, and the insertion and optimization of the repartitioning swap operator within the local multi-GPU physical operator tree includes: S1201. Identify the aggregation operators in the local multi-card physical operator tree, optimize the aggregation mode of the aggregation operators, the aggregation operators include operators that indicate the aggregation of data, and the optimized aggregation mode includes local aggregation, repartitioning by grouping key and merge aggregation.

[0051] In this embodiment, the aggregation operator can be an operator that instructs the aggregation operation to be performed. The aggregation mode can be the execution mode of the aggregation operator. Optimized aggregation modes include, but are not limited to, partial aggregation, repartitioning by grouping key, and merge aggregation. Partial aggregation can be a mode that aggregates a portion of the input data of the aggregation operator. Repartitioning by grouping key can be a mode that maps the input data of the aggregation operator to a specific logical region according to the indication of the grouping key in the data. The grouping key can be a column or expression that groups the input data by equality. Merge aggregation can be a mode that aggregates similar input data of the aggregation operator. Similar input data can be input data with the same grouping key.

[0052] Specifically, identifying aggregation operators in the local multi-card physical operator tree can be achieved by changing the type of aggregation operator or inserting new operators before and / or after the aggregation operator to change the aggregation mode, reduce the amount of data that needs to be aggregated as indicated by the aggregation operator, and optimize the aggregation mode by selectively aggregating data through changing the aggregation mode. The optimized aggregation mode can include any one of the following: local aggregation, repartitioning by grouping key, and merge aggregation.

[0053] For example, optimizing the aggregation mode of the aggregation operator can be achieved by inserting a new operator before the aggregation operator, so that the aggregation mode indicated by the aggregation operator is optimized from the original aggregation of all data to the partial aggregation of input data with specific grouping keys after classification according to the grouping key; or the aggregation operator can be rewritten as a repartitioning and swapping operator that repartitions data according to the grouping key, etc.

[0054] S1202. Identify the connection operators in the local multi-card physical operator tree, and optimize the data connection strategy indicated by the connection operators. The connection operators include operators that indicate data connection, and the data connection strategy includes strategies that indicate broadcasting and / or hash repartitioning.

[0055] In this embodiment, the join operator can be an operator used to instruct data to be joined. For example, the join operator can instruct input data to be joined according to a join key. The join key can be a column or expression that joins input data by equality. The data join strategy can instruct the join operator on how the input data is joined. The input data of the join operator can include input data from both sides. The input data from each side can be used to join or be joined, or it can be used to build or insert into a hash table.

[0056] Specifically, identifying join operators in the local multi-card physical operator tree can optimize data join strategies by considering the existing distribution of input data, join keys, and the scale of the construction side (i.e., the amount of data in the input data used to build the hash table). This can be achieved by reusing existing broadcast or hash partitioning layouts, or by inserting broadcast exchanges or bilateral hash repartitioning exchanges. Specifically, if one side of the input data already exists as a copy on the compute card, reusing existing broadcasts allows for direct hash table construction based on the copy, eliminating the need for broadcast exchanges. Reusing existing hash partitioning layouts means that both sides of the input data have been hashed and partitioned into the same logical region and mapped to the same compute card according to the join key. Optimizing the data join strategy by inserting broadcast exchanges can be achieved by inserting broadcast exchange operators after the join operators. Optimizing the data join strategy by hash repartitioning can be achieved by inserting hash repartitioning operators after the join operators.

[0057] S1203. Identify the location where data needs to be transferred across the computing card, and insert a repartitioning exchange operator at the location.

[0058] In this embodiment, the location can be a location where data transfer across computing cards is required. The computing card can be a computing card within the same actuator or a computing card within different actuators.

[0059] Specifically, a repartitioning swap operator is inserted at the location where data transfer needs to be performed across computing cards.

[0060] In one embodiment, the step of segmenting the optimized operator tree according to the pipeline bottleneck to obtain the pipeline task includes: S1301. The optimized operator tree is divided according to the pipeline blockage point to obtain the pipeline.

[0061] In this embodiment, the pipeline can be multiple stream-executable operators obtained by splitting the optimized operator tree according to the pipeline choke point. The pipeline retains the semantics of the corresponding operators in the optimized operator tree. The pipeline choke point can be the terminal operator of the pipeline. The pipeline can execute continuously.

[0062] Specifically, by splitting the optimized operator tree based on the pipeline blockage point, the resulting pipeline includes the corresponding operators and the topological structure between operators, which can preserve the corresponding semantics in the optimized operator tree.

[0063] S1302. Generate a directed acyclic graph of the pipeline based on the pipeline blockage point and the pipeline. The directed acyclic graph of the pipeline is used to indicate the pipeline dependencies between the pipelines.

[0064] In this embodiment, a pipelined directed acyclic graph (DAG) can describe the data flow between pipelines. A pipelined DAG can include multiple nodes representing pipelines. Pipeline dependencies can be the relationship where upstream nodes' output data is consumed as input data by downstream nodes.

[0065] Specifically, a directed acyclic graph of the pipeline is generated based on the pipeline blockage points and the pipeline dependencies indicated by the pipeline.

[0066] For example, Figure 3This is a schematic diagram of a directed acyclic graph (DAG) for pipelines provided in an embodiment of the present invention. Each pipeline task (i.e., pipeline one, pipeline two, pipeline three, and pipeline four) includes a data source (i.e., input data), several intermediate operators, a terminal operator (i.e., the last operator in the pipeline task) or output data storage, dependencies, and repartition metadata. The optimized operator tree is segmented according to the semantics of the operators (i.e., the semantics of pipeline blocking points). Repartition swaps, result collectors, unsplit aggregations, and special join boundaries are considered pipeline blocking points. The arrows between pipelines in the diagram represent the output data generated by the upstream pipeline at the terminal operator or boundary data storage, as well as the execution dependencies formed by this output. This data is the boundary output of the upstream pipeline and also the intermediate data of the entire query. For example, the output data segment of the upstream pipeline one (i.e., the output data) is input to the downstream pipeline two. After the upstream pipeline two is repartitioned and swapped according to the grouping key, the partitioned data (i.e., the ready batch) is input to the downstream pipeline three. Finally, the output data of the upstream pipeline three is input to the downstream pipeline four, and the pipeline four collects the output results of the pipeline three.

[0067] S1303. Generate the pipeline task based on the directed acyclic graph of the pipeline and the pipeline.

[0068] Specifically, the computation card to execute the pipeline task is determined based on the directed acyclic graph of the pipeline, and the pipeline task is generated based on the operators in the pipeline and the identifiers of the computation card.

[0069] In one embodiment, generating the pipeline task based on the directed acyclic graph of the pipeline and the pipeline includes: B1. Based on the pipeline's directed acyclic graph, determine the execution computation card corresponding to the pipeline task.

[0070] In this embodiment, the execution computing card can be a computing card used to execute the corresponding pipeline task.

[0071] Specifically, the target computing card is determined based on the logical storage area of ​​the input data indicated by the directed acyclic graph of the pipeline, the partition affiliation of the repartition, the batch of input data, and / or the input location of the input data.

[0072] B2. Generate the pipeline task based on the semantics of the operators in the pipeline and the input data, combined with the identifier of the execution computing card.

[0073] Specifically, based on the semantics of the operators in the pipeline and the input data, corresponding pipeline tasks are generated, and the identifier of the target compute card is added to the context of the pipeline task to obtain the final pipeline task. By determining the target compute card and writing it into the context of the pipeline task when creating the pipeline task, the pipeline task is submitted to the corresponding executor. Before execution, the execution thread of the target compute card sets the context of the target compute card and binds the memory resources and explicit CUDA streams. The data segments and synchronization objects of the input data retain the ownership of the target compute card, ensuring that the pipeline task, data segments, execution stream (i.e., explicit CUDA stream), synchronization objects, and the memory resources of the target compute card have consistent ownership of the target compute card, avoiding context or resource mismatch in cross-compute card execution.

[0074] In one example Figure 4 This is a flowchart illustrating a method for an executor to execute a fragment logic plan, as provided in an embodiment of the present invention. First, the executor receives the fragment logic plan and, through a CPU process within the executor, inputs the fragment logic plan into a physical plan converter (i.e., a converter that converts the logic plan into a physical plan) to convert it into a local multi-GPU physical operator tree. Second, the local multi-GPU physical operator tree is input into a parallel repartition optimizer, where optimization is performed. This optimization includes optimizing aggregation and connection operators, as well as optimizing the insertion of repartition swapping operators into the local multi-GPU physical operator tree, resulting in an optimized operator tree. Finally, the optimized operator tree is input into a pipeline. The pipeline builder generates a directed acyclic graph (DAG) for the pipeline by partitioning and optimizing the operator tree based on pipeline bottlenecks. Then, in the pipeline scheduler, based on the pipeline dependencies indicated by the DAG, and according to the logical storage area of ​​the input data, the batch size of the input data, the partition affiliation of the repartition, and / or the input position of the input data, it determines the execution compute card and generates pipeline tasks, which are then transferred to the execution compute card. Finally, the pipeline tasks are executed in the execution compute card to obtain fragment execution results, which are stored in a designated storage area. By dividing the distributed plan into multiple semantically preserved fragment logical plans, and then further dividing the fragment logical plans into multiple semantically preserved pipeline tasks, the pipeline execution efficiency and the semantic correctness of SQL operators are simultaneously guaranteed in a multi-heterogeneous compute card environment, especially for operators with blocking or repartitioning requirements (i.e., operators with pipeline bottlenecks), such as aggregation, join, sorting, first few items, and window functions.

[0075] Example 2 Figure 5This is a flowchart illustrating a multi-accelerator card pipeline parallel execution method according to Embodiment 2 of the present invention. This embodiment is an optimization based on any of the above embodiments, and mainly includes a detailed description of the process of executing the corresponding pipeline task through the computing card in the executor. It should be noted that technical details not described in detail in this embodiment can be found in any of the above embodiments. Figure 5 As shown, the method includes: S210. The distributed plan is divided into multiple fragment logical plans by the coordinator, and the fragment logical plans are transmitted to the corresponding executors.

[0076] S220. The executor receives the fragment logic plan, converts the fragment logic plan into a local multi-card physical operator tree, inserts a repartitioning and swapping operator into the local multi-card physical operator tree and optimizes it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning.

[0077] S230. The executor identifies pipeline blockage points in the optimized operator tree, and the optimized operator tree is segmented according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning exchange operator.

[0078] S240. The actuator determines the execution computing card to execute the pipeline task according to the instruction of the pipeline task.

[0079] Specifically, the executor determines the execution compute card for the pipelined task based on the identifier of the execution compute card in the context of the pipelined task.

[0080] S250. Using the execution computing card, the pipeline task is executed in the order of the operators in the pipeline task and task data is generated. When the repartitioning and swapping operator in the pipeline task is executed, the task data is assigned to the logical partition corresponding to the execution computing card. The logical partition includes a memory partition for storing data.

[0081] In this embodiment, the task data can be data generated during the execution of a pipelined task. Task data generated by upstream operators in a pipelined task can serve as input data for downstream operators. A logical partition can be a logical storage area used to store the task data. Logical partitions can be indicated by partition keys within the task data.

[0082] Specifically, the execution computing card executes the pipeline tasks in the order of the operators in the pipeline task. When it executes the repartitioning and swapping operator, it will input the task data of the repartitioning and swapping operator and assign it to the corresponding logical partition according to the partition key.

[0083] S260. Using the execution computing card, according to the partition affiliation mapping between the logical partition and the target computing card, the task data in the logical partition is mapped to the target computing card, the pipeline task is executed again, fragment execution results are obtained, and all fragment execution results in the executor are transmitted to the coordinator.

[0084] In this embodiment, the target computing card can be the destination computing card after the task data has been redistributed by a repartitioning and swapping operator. The target computing card can be indicated according to the repartitioning and swapping operator. The pipelined task executed on the target computing card can be a pipelined task downstream of the pipelined task executed on the execution computing card. The partition attribution mapping can be a mapping used to indicate the target computing card corresponding to the logical partition.

[0085] For example, partition attribution mapping can be indicated by a mapping table stored in the executor. The mapping table can include logical partition numbers and their corresponding target computing cards. When determining the target computing card corresponding to a logical partition based on the partition attribution mapping, the mapping table can be searched based on the logical partition number to determine the corresponding target computing card. Specifically, the search can be performed using the hash value of the logical partition number.

[0086] Specifically, the execution computing card queues or maps task data to the target computing card in batches according to the partition affiliation mapping of the logical partition.

[0087] S270. The coordinator receives the segment execution results and obtains the final execution result based on the segment execution results transmitted by all the executors.

[0088] The technical solution of this invention involves dividing a distributed plan into multiple fragment logical plans through a coordinator, and transmitting these fragment logical plans to corresponding executors. Each executor receives a fragment logical plan, converts it into a local multi-GPU physical operator tree, inserts a repartitioning and swapping operator into the local multi-GPU physical operator tree, and optimizes it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator indicating data repartitioning. The executor identifies pipeline bottlenecks in the optimized operator tree, segments the optimized operator tree according to these bottlenecks, and obtains pipeline tasks. The pipeline bottlenecks include the repartitioning and swapping operator. Finally, the executor determines the execution of the pipeline task based on the instructions of the pipeline task. The execution computing card executes the pipelined tasks in the order of the operators and generates task data. When the repartitioning operator in the pipelined task is reached, the task data is allocated to the logical partition corresponding to the execution computing card. The logical partition includes a memory partition for storing data. The execution computing card maps the task data in the logical partition to the target computing card according to the partition affiliation mapping between the logical partition and the target computing card, and continues to execute the pipelined tasks to obtain fragment execution results. All fragment execution results in the executor are transmitted to the coordinator. The coordinator receives the fragment execution results and obtains the final execution result based on all the fragment execution results transmitted by the executors. The technical solution of this invention determines the ownership of task data by logical partitioning and maps the task data in the logical partition to the target computing card. This ensures that pipeline tasks, task data, the stream executing pipeline tasks, synchronization objects, and the memory resources of computing cards have consistent computing card ownership when multiple threads simultaneously schedule multiple computing cards within the same executor process, thus avoiding context or resource mismatch during cross-computing card execution.

[0089] In one embodiment, mapping the task data in the logical partition to the target computing card includes: S2601. If the target computing card and the execution computing card are the same computing card, then the task data in the logical partition is written into the partition queue.

[0090] In this embodiment, the partition queue can be a queue allocated by the executor to each computing card for caching data.

[0091] Specifically, if the target computing card and the execution computing card are the same computing card, the mapping of task data in the logical partition to the target computing card is completed by writing the task data into the partition queue of the target computing card.

[0092] S2602. If the target computing card and the execution computing card are not the same computing card, then copy the task data in the logical partition and transfer the copied task data to the target computing card.

[0093] Specifically, if the target computing card and the execution computing card are not the same computing card, the task data in the logical partition is asynchronously copied through the corresponding data channel, and the copied task data is transferred to the target computing card.

[0094] In one embodiment, mapping the task data in the logical partition to the target computing card includes: Determine the number of rows of task data in the logical partition. If the number of rows is greater than a row count threshold, then map the task data in the logical partition to the target computing card.

[0095] In this embodiment, the row count threshold can be the upper limit of the number of rows of task data during the batch mapping process. The row count threshold can be set based on experience, and this invention does not impose any restrictions on it.

[0096] Specifically, task data in logical partitions can be mapped to target computing cards in batches. The number of rows of task data to be loaded into the logical partition is calculated. When the number of rows exceeds the threshold, the task data is mapped to the target computing card.

[0097] For example, if all operators upstream of the repartitioning swap operator have been executed, and the number of rows of task data in the logical partition is not greater than the row number threshold, then the task data will still be mapped to the target computing card.

[0098] For example, each batch of task data also includes a ready event. A ready event can be an event indicating that a batch of task data has completed copying. When each batch of task data includes a ready event, it means that the batch of task data has been copied, and at this time, an asynchronous thread can be notified to transfer the batch of task data to the target compute card. Only when the batch of task data includes a ready event does the executor notify the target compute card, and only then can the target compute card read the copied task data. By determining task data ownership by logical partitioning, directly enqueuing task data on the same compute card, copying only on different compute cards, forming different batches of task data by logical partitioning, using ready events, and notifying downstream consumer tasks through ready events, the overhead of intermediate result materialization, inter-device data transfer, and global synchronization in multi-heterogeneous compute card query execution is reduced.

[0099] In one embodiment, the computing card includes a non-pipelined repartitioning mode and a pipelining repartitioning mode, the executor further includes a pipelining scheduler, and the method further includes: C1. In the non-pipelined repartitioning mode, after the upstream pipeline task in the execution computing card completes the mapping of all the task data, the target computing card reads the task data according to the logical partition. The upstream pipeline includes the pipeline task before the repartitioning exchange operator.

[0100] In this embodiment, the non-pipelined repartitioning mode can be a mode that waits for all upstream pipeline tasks to complete and obtain the corresponding task data before executing downstream pipeline tasks. The pipeline scheduler can be a unit within the executor used to generate and allocate pipeline tasks. Upstream pipeline tasks can be pipeline tasks executed on the compute card. The output data of upstream pipeline tasks can be mapped to the target compute card based on the partition affiliation after being assigned to the corresponding logical partition, and then consumed by downstream pipeline tasks on the target compute card.

[0101] Specifically, in non-pipeline repartitioning mode, task data is mapped to the target computing card only after all upstream pipeline tasks have been completed.

[0102] C2. In the pipelined repartitioning mode, when the task data in the logical partition of the execution computing card reaches the scheduling condition, a consumption task is created for the target computing card through the pipeline scheduler. The consumption task includes a pipeline task that consumes the task data.

[0103] In this embodiment, the pipeline repartitioning mode can be a mode that consumes task data in real time for each batch that meets the scheduling conditions. The pipeline scheduler can be a logical unit within the CPU thread of the executor used to implement specific functions. The scheduling conditions can be conditions for determining whether a batch of task data can be consumed. For example, scheduling conditions can include ready batch events or the completion of upstream pipeline tasks. The consuming task can be a pipeline task in the target compute card that consumes task data. The consuming task can be a pipeline task located downstream of the upstream pipeline task.

[0104] Specifically, in the pipelined repartitioning mode, if a batch of task data meets the scheduling conditions, the pipeline scheduler creates a consumption task for the target compute card to consume the task data. This allows the upstream pipeline task to produce task data, the cross-compute card transfer of task data, and the downstream consumption task data to overlap and execute, improving the resource utilization of the compute card and thus improving query efficiency.

[0105] For example, Figure 6This is a flowchart illustrating a method for exchanging task data using a repartitioning exchange operator, as provided in an embodiment of the present invention. Pipeline tasks on the computing card are executed sequentially. When a repartitioning exchange operator (such as an operator indicating hash partitioning, broadcasting, and / or aggregation) is executed, the task data is partitioned into the corresponding logical partition according to the partition key. The target computing card for the task data is determined based on the partition affiliation mapping. When the target computing card and the executing computing card are the same computing card, the task data is mapped through a partition queue. When the target computing card and the executing computing card are not the same computing card, the task data is copied through an asynchronous thread and transmitted to the target computing card. During the mapping process, the task data can be divided into different batches according to the number of rows, and mapped in batches. For the last batch of task data generated by the pipelined task, it can be mapped to the target computing card before reaching the row threshold. Each batch of copied task data also includes a ready event. The target computing card can only read the task data when the ready event indicates that the copying is complete. The consuming task on the target computing card reads and consumes the task data. The executing computing card can include non-pipelined repartitioning mode and pipelining repartitioning mode.

[0106] In one example Figure 7 This is a schematic diagram of an executor provided in an embodiment of the present invention. The executor includes a CPU process, which can be used to process fragment logic plans, invoke the pipeline scheduler, and / or manage queues of pipeline tasks. The executor manages multiple compute cards. Each compute card includes a resource pool for CUDA streams and a resource pool for video memory. The multiple compute cards in the executor are connected via interconnect channels, Peripheral Component Interconnect Express (PCIe), and / or Pinned Host Memory. The executor can perform cross-executor communication at the granularity of stage or fragment logic plans, as well as intra-executor data scheduling at the granularity of data, logical partitions, or batches, realizing multi-process distributed scheduling and intra-process multi-threaded multi-compute card collaborative execution.

[0107] In one example Figure 8This is a schematic diagram of a distributed system provided by an embodiment of the present invention. The distributed system includes a master-slave structure, in which a coordinator manages and schedules multiple executors. In the coordinator's process, a distributed plan is obtained by parsing query statements, the distributed plan is divided according to exchange boundaries to obtain stages and fragment query plans under each stage, a stage-directed acyclic graph (DAG) is generated based on the exchange boundaries, the target executor corresponding to the fragment logical plan is determined based on the stage DAG, the fragment logical plan is transmitted to the target executor, and the fragment execution results transmitted by the executor are received. The final execution result is determined based on the fragment execution results. In the executor's process, the executor manages multiple computing cards uniformly and designates a computing card (denoted as the master computing card) as the endpoint for external communication. A pipeline scheduler generates pipeline tasks and distributes the pipeline tasks to the corresponding computing cards for execution. During execution, data repartitioning is performed to complete data exchange, and the fragment execution results obtained from executing the pipeline tasks are cached or transmitted to the coordinator. The coordinator and executor communicate by exchanging fragment logic plans, metadata, control messages indicating completion or failure of fragment logic plan execution, and the results generated by the executor executing fragment logic plans. The executor is responsible for the unified management of all local computing cards and generates, binds, and dispatches pipeline tasks through the pipeline scheduler. This delineates the boundaries of responsibility between cross-process distributed scheduling and multi-computing card scheduling within the executor, enabling the distributed system to retain the isolation, recoverability, and cluster scalability of the multi-process architecture while automatically utilizing multiple local heterogeneous acceleration devices (i.e., heterogeneous computing cards) in a low-overhead manner within a single executor process.

[0108] The present invention will be described by way of example below: Database systems typically translate SQL queries into logical and physical execution plans. The physical plan consists of operators such as scan, filter, projection, join, aggregation, sort, and result collection. To improve execution efficiency, databases pipe together multiple operators capable of processing data blocks sequentially; when an operator is encountered that requires waiting for complete input before continuing, such as global aggregation, sorting, or repartitioning and swapping, a pipeline bottleneck occurs.

[0109] Heterogeneous accelerated databases or accelerated query execution systems typically execute operators in columnar data blocks. Multiple heterogeneous acceleration devices can be graphics processing units (GPUs), AI accelerator cards, data processors, field-programmable gate arrays (FPGAs), or other dedicated computing devices. In multi-device systems, input data can be divided into multiple segments or partitions and processed by different devices; when joins or aggregations require key-based merging, repartitioning and swapping are performed to send data to the target device according to hash, range, broadcast, or aggregation methods.

[0110] When running on heterogeneous accelerated devices, device memory, execution flow, synchronization objects, and memory allocation resources typically have device context attributes. If multiple threads within the same process operate on multiple devices simultaneously without a unified maintenance of the device ownership relationships of tasks, data, and resource objects, it is easy to use resources of other devices in the wrong device context.

[0111] In existing technologies, multi-device queries are typically executed synchronously in a coarse-grained manner, either by stage or by operator. The steps usually include: converting the SQL query into a physical operator tree (i.e., a tree structure containing physical operators); splitting the scan input into multiple acceleration devices (i.e., computing cards); each acceleration device executing a chain of operators (i.e., an ordered chain structure of multiple operators) on its local input; when encountering operators requiring global data, such as joins, aggregations, or sorts, all upstream results are materialized first, and after all acceleration devices have completed the upstream stage, inter-device repartitioning or aggregation is performed uniformly; downstream stages wait for all repartitioning to complete before starting; finally, the main acceleration device or CPU collects and returns the results. Existing technologies can achieve basic multi-device parallelism, but their execution model is primarily based on stage boundaries (i.e., exchange boundaries), resulting in a relatively coarse-grained pipeline.

[0112] Because existing technologies typically wait for all upstream acceleration devices (i.e., compute cards) to complete at blocking operators and / or repartitioning operators (i.e., pipeline bottlenecks), downstream devices may remain idle even when some devices (i.e., compute cards) have already generated consumable data. This is because the scheduling unit is too coarse, and data readiness events do not directly drive downstream task generation, resulting in insufficient overlap between computation, data transport, and downstream consumption. Since existing solutions tend to fully materialize intermediate results before unified repartitioning, intermediate data accumulates in device GPU memory or local storage. This increases storage pressure, easily triggering overflows or memory shortages, and causes concentrated inter-device data transport, resulting in bandwidth spikes and synchronization waits. Furthermore, existing technologies expose each acceleration device as an independent distributed scheduling object, significantly increasing the number of cross-process tasks, communication connections, scheduling metadata, and fault handling complexity. On the other hand, an extreme approach places all acceleration devices in a single process for unified operation, weakening the process-level isolation and recovery capabilities required for distributed execution. Therefore, existing solutions struggle to achieve a good balance between multi-process distributed execution and intra-process multi-device parallelism. Furthermore, with the introduction of multiple acceleration devices and execution threads within the executor process, data objects, execution flows, synchronization objects, and device memory resources may belong to different devices. If task scheduling only focuses on parallelism without explicitly maintaining the device affiliation of these objects, inconsistencies between device context and resource context may occur, leading to cross-device execution failures or the need for additional synchronization.

[0113] The purpose of this invention is to establish a two-level execution model: an outer-layer multi-process stage scheduling (i.e., a coordinator scheduling multiple executors) and an inner-layer multi-threaded multi-heterogeneous acceleration device pipeline scheduling (i.e., executors managing multiple computing cards). The cross-process layer (i.e., the coordinator scheduling multiple executors) is responsible for task assignment, data exchange, and result collection among executors; the inner executor layer is responsible for fine-grained scheduling of multiple local heterogeneous acceleration devices, execution flows, data segments, partitions, and pipeline tasks.

[0114] This invention also improves the correctness and stability of multi-device pipelined execution by using a unified cross-device execution context consistency mechanism, which ensures that task switching, data flow, and resource access are all based on clear device affiliation.

[0115] The technical effects achievable by this invention include: improving the utilization rate of multiple heterogeneous acceleration devices, enabling overlapping of scanning, computation, repartitioning write, repartitioning read, and downstream consumption; reducing global wait points and shortening end-to-end query latency; reducing the peak storage of devices caused by the centralized materialization of intermediate results; scheduling tasks according to data location or partition affiliation, reducing unnecessary data copying between devices; employing strategies such as local computation, repartitioning, merging, or broadcasting for operators such as aggregation, joining, and sorting while maintaining the correctness of SQL semantics; balancing process-level isolation, cluster scalability, scheduling overhead, and the utilization rate of multiple heterogeneous acceleration devices within the executor; and avoiding mismatches in device context, execution flow, synchronization objects, or memory resources during multi-threaded, multi-device execution, thereby improving the stability of cross-device pipelined execution.

[0116] In one embodiment, a distributed system includes: a coordinator first parses the SQL (i.e., the query statement) and generates a distributed plan containing cross-executor exchange boundaries, then segments the query into stages and query fragments (i.e., fragment logical plans) according to the exchange boundaries, thereby forming a directed acyclic dependency graph (i.e., a stage-based directed acyclic graph); a stage scheduler (i.e., a segment element in the coordinator used to schedule the fragment logical plan) then distributes the query fragments according to this dependency graph. This stage-based directed acyclic graph describes the stage dependencies between executors. The coordinator and executors exchange query fragments, exchange metadata, completion or failure control information, and final results; data exchange between executors can be directly completed through inter-process communication or network data channels, with the coordinator responsible for channel and metadata coordination, but not acting as a necessary intermediary for data payloads.

[0117] In one embodiment, the executor executes a fragment logic plan, including: after receiving a query fragment logic plan sent by the coordinator, the executor converts the logic plan (i.e., fragment logic plan) into a local multi-accelerator card physical operator tree (i.e., local multi-card physical operator tree) by a logic plan to multi-accelerator card physical plan converter (i.e., physical plan converter). A parallel repartition optimizer rewrites the entire physical operator tree based on operator semantics and input data distribution, and inserts repartition exchange operators at locations requiring cross-device redistribution (i.e., inserting repartition exchange operators within the local multi-card physical operator tree and optimizing them to obtain an optimized operator tree). For aggregation operators, they can be rewritten as local aggregation, repartition by grouping key, and merge aggregation, with a final projection added if necessary; for join operators, existing broadcast or hash partition layouts can be reused, or broadcast exchanges or bilateral hash repartition exchanges can be inserted, based on the existing distribution of the input data, the join key, and the build-side scale. The pipeline builder segments the physical operator tree at boundaries such as repartitioning, swapping, and blocking operators, and establishes pipeline dependencies. The pipeline scheduler then generates runtime tasks based on data segment location, partition ownership, ready batches, and dependencies, determines the target accelerator card (i.e., the execution accelerator card), and dispatches it for execution. Task outputs are written to the repartitioning data storage, output data storage, or result collection module.

[0118] In one embodiment, the pipeline is a directed acyclic graph (DAG) comprising multiple pipelines (i.e., pipelines corresponding to pipeline tasks). Each pipeline includes a data source, several intermediate operators, terminal operators or output data storage, dependencies, and repartition metadata. The pipeline builder segments the physical operator tree based on operator semantics. Repartition swaps, result collectors, unsplit aggregations, and special join boundaries serve as pipeline blocking points. The output data generated by the upstream pipeline at the terminal operator or boundary data storage, along with the execution dependencies formed by this output, constitutes the boundary output of the upstream pipeline and is also the intermediate data for the entire query.

[0119] In one embodiment, the execution method of the repartitioning swap operator includes: a parallel repartitioning optimizer inserting and configuring the repartitioning swap operator during the planning phase; during query execution, after the upstream production task (i.e., the pipeline task located upstream of the swap operator) reaches the swap operator, it first divides the input rows into logical partitions according to the partition key, and then determines the target accelerator card for each logical partition according to the partition affiliation mapping. Data on the same card directly enters the target partition queue (i.e., the partition queue), while data on different cards is asynchronously copied through the corresponding data channels. The repartitioned data storage accumulates data rows (i.e., the number of task data rows) by partition (i.e., logical partition) to form schedulable batches (i.e., determining the number of task data rows in the logical partition; if the number of rows is greater than the row count threshold, the task data in the logical partition is mapped to the target computing card). At the end of the upstream process, any batches that have not reached the threshold are refreshed (i.e., if all operators upstream of the repartitioning exchange operator have completed execution, and the number of task data rows in the logical partition is not greater than the row count threshold, the task data is still mapped to the target computing card). If asynchronous replication is not yet complete, the batch carries a ready event, and the consuming task waits for this event before reading (i.e., only when the batch of task data includes a ready event does the executor notify the target computing card, and the replicated task data can be read by the target computing card). In non-pipeline repartitioning mode, the downstream pipeline (i.e., the consumption task) waits for the upstream repartitioning (i.e., the data repartitioning of the upstream pipeline task) to be completed before reading data by partition; in pipeline repartitioning mode, when the batch reaches the scheduling conditions, the data storage notifies the pipeline scheduler to create a downstream consumption task on the target accelerator card, so that upstream production, cross-accelerator card transfer and downstream consumption are executed in an overlapping manner.

[0120] In one embodiment, this invention pertains to database execution methods and system architectures, and does not involve specific circuit implementations. During multi-device execution within the executor, this invention further maintains cross-device execution context consistency. When creating a pipelined task, the pipeline scheduler binds a target acceleration device (i.e., an execution acceleration card). Specifically, the target device (i.e., the execution acceleration card) is determined based on the device where the input data segment (i.e., the input data) is located, the partition affiliation of the repartition, or the dependent input location, and the identifier of the target device is written into the task context. The task is then submitted to the executor of the corresponding target device. Before executing the task, the device execution thread sets the target device context and binds the device's memory resources and explicit execution flow. Each data segment, execution flow, synchronization object, and device memory resource retains a clear device affiliation. When the data read by the task is located on another device, or when cross-device data transfer occurs during execution, the system completes data migration and corresponding resource binding based on device affiliation before executing subsequent operators. This avoids misusing the execution flow, synchronization object, or memory resources of another device within a single device context. Through the above-described general-to-specific structure, this invention controls cross-executor communication at the stage / fragment granularity and controls internal executor computation at the data segment, partition, or ready batch granularity, thereby achieving the coordination of multi-process distributed scheduling and intra-process multi-threaded multi-accelerator card execution.

[0121] In one embodiment, the present invention includes a two-level execution architecture of coordinator and executor in a distributed system. The executor's process serves as the unit for scheduling, communication, isolation, and recovery of stages or fragments (i.e., fragment logical plans), without exposing each acceleration device as an independent distributed scheduling object. After receiving a query fragment, the inner layer of the executor performs multi-threaded pipeline scheduling and task execution on multiple heterogeneous acceleration devices within the same process. This achieves a balance between the isolation, recoverability, and cluster scalability of the multi-process architecture and the low overhead of in-process multi-threaded scheduling and the utilization rate of local multiple devices.

[0122] In one embodiment, the present invention includes a cross-device execution context consistency mechanism for pipelined execution of multiple heterogeneous accelerated devices within an executor process, including: establishing clear device ownership relationships for pipeline tasks, data segments, execution flows, synchronization objects, and device memory resources, and uniformly maintaining device context and resource context when data flows across devices or tasks are switched to execute on different devices, thereby avoiding mismatches in device context, execution flow, synchronization objects, or memory resources during multi-threaded and multi-device execution.

[0123] Example 3 Figure 9 This is a schematic diagram of a multi-accelerator card pipeline parallel execution device provided in Embodiment 3 of the present invention. Figure 9 As shown, the device includes: The fragment logic plan partitioning module 910 is used to partition the distributed plan into multiple fragment logic plans through the coordinator, and transmit the fragment logic plans to the corresponding executors; The local multi-card physical operator tree optimization module 920 is used to receive the fragment logic plan through the executor, convert the fragment logic plan into a local multi-card physical operator tree, insert a repartitioning and swapping operator into the local multi-card physical operator tree and optimize it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning. The optimized operator tree segmentation module 930 is used to identify pipeline blockage points in the optimized operator tree through the executor, and segment the optimized operator tree according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning exchange operator. The pipeline task execution module 940 is used to execute the corresponding pipeline task through the computing card in the executor, obtain the fragment execution result, and transmit all the fragment execution results in the executor to the coordinator; The fragment execution result receiving module 950 is used to receive the fragment execution result through the coordinator and obtain the final execution result based on the fragment execution results transmitted by all the executors.

[0124] In this embodiment of the invention, the fragment logic plan partitioning module divides the distributed plan into multiple fragment logic plans through the coordinator and transmits the fragment logic plans to the corresponding executors. The local multi-card physical operator tree optimization module receives the fragment logic plans through the executors, converts the fragment logic plans into local multi-card physical operator trees, inserts repartitioning and swapping operators into the local multi-card physical operator trees, and optimizes them to obtain an optimized operator tree. The repartitioning and swapping operators include operators that indicate data repartitioning. The optimized operator tree segmentation module, through the executors... The system identifies pipeline bottlenecks in the optimized operator tree, segments the optimized operator tree according to the bottlenecks, and obtains pipeline tasks. The pipeline bottlenecks include the repartitioning and swapping operator. The pipeline task execution module executes the corresponding pipeline task through the computing card in the executor, obtains fragment execution results, and transmits all fragment execution results in the executor to the coordinator. The fragment execution result receiving module receives the fragment execution results through the coordinator and obtains the final execution result based on all the fragment execution results transmitted by the executors. In the technical solution of this invention embodiment, in a distributed system, the coordinator uses the executor's process as the scheduling unit for stages, fragment logic plans, communication, isolation, and recovery. After the fragment logic plan arrives at the executor, the CPU within the executor uniformly manages all local computing cards and generates, binds, and dispatches pipeline tasks. This delineates the boundaries of responsibility between cross-process distributed scheduling and multi-card scheduling within the executor, enabling the system to retain the isolation, recoverability, and cluster scalability of a multi-process architecture while automatically utilizing multiple heterogeneous acceleration devices locally in a low-overhead manner within a single executor process. By refining the optimized operator tree into multiple fine-grained pipeline tasks that can be executed in parallel across heterogeneous computing cards based on pipeline bottlenecks within the corresponding executor, it ensures that the pipeline task will not wait for the upstream data at the pipeline bottleneck to be calculated before continuing execution. This avoids insufficient resource utilization caused by single computing card execution or overly coarse pipeline task granularity, thereby improving the resource utilization of computing cards and thus improving query efficiency.

[0125] In one embodiment, the fragment logic plan partitioning module 910 is specifically used for: The query statement is obtained through the coordinator; The coordinator parses the query statement to generate a distributed plan, which includes exchange boundaries across the executors. The distributed plan is divided by the exchange boundary to obtain a fragmented logical plan.

[0126] In one embodiment, the fragment logic plan partitioning module 910 is specifically used for: A phase-directed acyclic graph is generated based on the exchange boundary, and the phase-directed acyclic graph is used to indicate the phase dependencies between phases of the distributed plan; Based on the stage directed acyclic graph, determine the target executor corresponding to the segment logic plan, and write the identifier of the target executor into the segment logic plan; The fragment logic plan is transmitted to the target executor.

[0127] In one embodiment, the local multi-GPU physical operator tree further includes an aggregation operator and a connection operator. The local multi-GPU physical operator tree optimization module 920 is specifically used for: Identify the aggregation operators in the local multi-card physical operator tree, optimize the aggregation mode of the aggregation operators, and the optimized aggregation operators include operators that indicate data aggregation, and the aggregation modes include local aggregation, repartitioning by grouping key, and merge aggregation; Identify the connection operators in the local multi-card physical operator tree, optimize the data connection strategy indicated by the connection operators, the connection operators include operators that indicate data connection, and the data connection strategy includes strategies that indicate broadcasting and / or hash repartitioning; Identify the locations where data transfer across the computing card is required, and insert a repartitioning exchange operator at those locations.

[0128] In one embodiment, the optimized operator tree segmentation module 930 further includes: The segmentation unit is used to segment the optimized operator tree according to the pipeline blockage point to obtain the pipeline; A pipeline directed acyclic graph generation unit is used to generate a pipeline directed acyclic graph based on the pipeline blockage point and the pipeline. The pipeline directed acyclic graph is used to indicate the pipeline dependencies between pipelines. The pipeline includes the pipelines that can be flow-executed. The pipeline task generation unit is used to generate the pipeline task based on the directed acyclic graph of the pipeline and the pipeline.

[0129] In one embodiment, the pipeline task generation unit is specifically used for: Based on the pipeline's directed acyclic graph, determine the execution computing card corresponding to the pipeline task; The pipeline task is generated based on the semantics of the operators in the pipeline and the input data, combined with the identifier of the execution computing card.

[0130] In one embodiment, the pipeline task execution module 940 further includes: An execution computing card determination unit is used to determine, through the actuator, the execution computing card to execute the pipeline task according to the instruction of the pipeline task; The allocation unit is used to execute the pipeline task and generate task data according to the order of operators in the pipeline task through the execution computing card. When the repartitioning and swapping operator in the pipeline task is executed, the task data is allocated to the logical partition corresponding to the execution computing card. The logical partition includes a memory partition for storing data. The mapping unit is used to map the task data in the logical partition to the target computing card through the execution computing card, according to the partition affiliation mapping between the logical partition and the target computing card.

[0131] In one embodiment, the mapping unit is specifically used for: If the target computing card and the execution computing card are the same computing card, then the task data in the logical partition is written into the partition queue; If the target computing card and the execution computing card are not the same computing card, then the task data in the logical partition is copied, and the copied task data is transferred to the target computing card.

[0132] In one embodiment, the mapping unit is specifically used for: Determine the number of rows of task data in the logical partition. If the number of rows is greater than a row count threshold, then map the task data in the logical partition to the target computing card.

[0133] In one embodiment, the computing card includes a non-pipelined repartitioning mode and a pipelining repartitioning mode, and the actuator further includes a pipelining scheduler. The device also includes: The waiting module is used in the non-pipeline repartitioning mode to wait for the upstream pipeline tasks in the execution computing card to complete the mapping of all the task data, and then read the task data through the target computing card according to the logical partition. The upstream pipeline includes the pipeline tasks executed in the execution computing card. A creation module is used to create a consumption task for the target computing card through the pipeline scheduler when the task data in the logical partition of the execution computing card reaches the scheduling condition in the pipeline repartition mode. The consumption task includes a pipeline task that consumes the task data.

[0134] The multi-accelerator card pipeline parallel execution device provided in this embodiment of the invention can execute the multi-accelerator card pipeline parallel execution method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0135] Example 4 Figure 10A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0136] like Figure 10 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the methods proposed in this invention.

[0139] In some embodiments, the method proposed in this invention can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the method proposed in this invention by any other suitable means (e.g., by means of firmware).

[0140] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this invention, a computer-readable storage medium stores computer instructions that are used to cause a processor to execute and implement the method provided by this invention.

[0143] The present invention also provides a computer program product comprising a computer program that, when executed by a processor, implements the method provided according to embodiments of the present invention.

[0144] Computer-readable storage media can be tangible media that may contain or store computer programs for use by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable storage media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user, such as a cathode ray tube (CRT) or a liquid crystal display (LCD); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0146] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0147] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0148] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0149] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for parallel execution of a multi-accelerator card pipeline, characterized in that, Applied to a distributed system, the distributed system including a coordinator and multiple executors, the executors including multiple computing cards, the method includes: The coordinator divides the distributed plan into multiple fragment logical plans, and transmits the fragment logical plans to the corresponding executors. The executor receives the fragment logic plan, converts the fragment logic plan into a local multi-card physical operator tree, inserts a repartitioning and swapping operator into the local multi-card physical operator tree and optimizes it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning. The actuator identifies pipeline blockage points in the optimized operator tree, and the optimized operator tree is segmented according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning and swapping operator. The corresponding pipeline task is executed through the computing card in the executor to obtain the fragment execution result, and all the fragment execution results in the executor are transmitted to the coordinator. The coordinator receives the fragment execution results and obtains the final execution result based on the fragment execution results transmitted by all the executors.

2. The method according to claim 1, characterized in that, The process of dividing the distributed plan into multiple fragment logical plans through the coordinator includes: The query statement is obtained through the coordinator; The coordinator parses the query statement to generate a distributed plan, which includes exchange boundaries across the executors. The distributed plan is divided by the exchange boundary to obtain a fragmented logical plan.

3. The method according to claim 2, characterized in that, The step of transmitting the fragment logic plan to the corresponding executor includes: A phase-directed acyclic graph is generated based on the exchange boundary, and the phase-directed acyclic graph is used to indicate the phase dependencies between phases of the distributed plan; Based on the stage directed acyclic graph, determine the target executor corresponding to the segment logic plan, and write the identifier of the target executor into the segment logic plan; The fragment logic plan is transmitted to the target executor.

4. The method according to claim 1, characterized in that, The local multi-GPU physical operator tree further includes aggregation operators and connection operators. The insertion and optimization of the repartitioning swap operator within the local multi-GPU physical operator tree includes: Identify the aggregation operators in the local multi-card physical operator tree, optimize the aggregation mode of the aggregation operators, and the optimized aggregation operators include operators that indicate data aggregation, and the aggregation modes include local aggregation, repartitioning by grouping key, and merge aggregation; Identify the connection operators in the local multi-card physical operator tree, optimize the data connection strategy indicated by the connection operators, the connection operators include operators that indicate data connection, and the data connection strategy includes strategies that indicate broadcasting and / or hash repartitioning; Identify the locations where data transfer across the computing card is required, and insert a repartitioning exchange operator at those locations.

5. The method according to claim 1, characterized in that, The step of dividing the optimized operator tree according to the pipeline blockage point to obtain the pipeline task includes: The optimized operator tree is divided according to the pipeline blockage point to obtain the pipeline; A directed acyclic graph (DAG) is generated based on the pipeline blockage points and the pipelines. The DAG is used to indicate the pipeline dependencies between the pipelines. The pipeline task is generated based on the directed acyclic graph of the pipeline and the pipeline.

6. The method according to claim 5, characterized in that, The step of generating the pipeline task based on the directed acyclic graph of the pipeline and the pipeline includes: Based on the pipeline's directed acyclic graph, determine the execution computing card corresponding to the pipeline task; The pipeline task is generated based on the semantics of the operators in the pipeline and the input data, combined with the identifier of the execution computing card.

7. The method according to claim 1, characterized in that, The execution of the corresponding pipeline task via the computing card in the executor includes: The actuator determines the execution computing card to execute the pipeline task according to the instructions of the pipeline task; The execution computing card executes the pipeline task and generates task data according to the order of the operators in the pipeline task. When the repartitioning and swapping operator in the pipeline task is executed, the task data is assigned to the logical partition corresponding to the execution computing card. The logical partition includes a memory partition for storing data. The execution computing card maps the task data in the logical partition to the target computing card according to the partition affiliation mapping between the logical partition and the target computing card.

8. The method according to claim 7, characterized in that, The step of mapping the task data in the logical partition to the target computing card includes: If the target computing card and the execution computing card are the same computing card, then the task data in the logical partition is written into the partition queue; If the target computing card and the execution computing card are not the same computing card, then the task data in the logical partition is copied, and the copied task data is transferred to the target computing card.

9. The method according to claim 7, characterized in that, The step of mapping the task data in the logical partition to the target computing card includes: Determine the number of rows of task data in the logical partition. If the number of rows is greater than a row count threshold, then map the task data in the logical partition to the target computing card.

10. The method according to any one of claims 7-9, characterized in that, The computing card includes a non-pipelined repartitioning mode and a pipelining repartitioning mode, the executor further includes a pipelining scheduler, and the method further includes: In the non-pipeline repartitioning mode, after the upstream pipeline task in the execution computing card completes the mapping of all the task data, the target computing card reads the task data according to the logical partition. The upstream pipeline includes the pipeline tasks executed in the execution computing card. In the pipelined repartitioning mode, when the task data in the logical partition of the execution computing card reaches the scheduling condition, the pipeline scheduler creates a consumption task for the target computing card. The consumption task includes a pipeline task that consumes the task data.

11. A multi-accelerator card pipeline parallel execution device, characterized in that, include: The fragment logic plan partitioning module is used to divide the distributed plan into multiple fragment logic plans through the coordinator, and transmit the fragment logic plans to the corresponding executors; The local multi-card physical operator tree optimization module is used to receive the fragment logic plan through the executor, convert the fragment logic plan into a local multi-card physical operator tree, insert a repartitioning and swapping operator into the local multi-card physical operator tree and optimize it to obtain an optimized operator tree. The repartitioning and swapping operator includes an operator that indicates data repartitioning. The optimized operator tree segmentation module is used to identify pipeline blockage points in the optimized operator tree through the executor, and segment the optimized operator tree according to the pipeline blockage points to obtain pipeline tasks. The pipeline blockage points include the repartitioning exchange operator. The pipeline task execution module is used to execute the corresponding pipeline task through the computing card in the executor, obtain the fragment execution result, and transmit all the fragment execution results in the executor to the coordinator; The fragment execution result receiving module is used to receive the fragment execution result through the coordinator and obtain the final execution result based on the fragment execution results transmitted by all the executors.

12. An electronic device, characterized in that, The actuator includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-10.

Images