Data processing method and device for distributed database, equipment, storage medium and program product

By allocating shared memory blocks to each worker process in the distributed database and writing remote data access information, the problems of low data transmission and parallel execution efficiency are solved, thereby improving the overall performance of query execution.

CN121542356APending Publication Date: 2026-02-17CHINA TELECOM CLOUD TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202610065329.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-01-19
Publication Date
2026-02-17

AI Technical Summary

Technical Problem

In distributed databases, when the data volume is large and there are many parallel processes, the low efficiency of data transmission and parallel execution leads to a decline in the overall performance of query execution.

Method used

By allocating a shared memory block to each worker process and writing the distribution information for remote data access into the shared memory, worker processes are allowed to send data directly to the target node, avoiding aggregation through collection and merging operators.

Benefits of technology

It improves the efficiency of data transmission and parallel execution, enhances the overall performance of query execution, and eliminates the single-point bottleneck of the collection and merging operator.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121542356A_ABST
    Figure CN121542356A_ABST
Patent Text Reader

Abstract

The invention relates to a data processing method and device for a distributed database, equipment, a storage medium and a program product, and relates to the technical field of databases. According to the invention, the efficiency of data transmission and parallel execution can be improved. The method comprises the steps that a shared memory is determined, the shared memory comprises shared memory blocks corresponding to each working process of a collection and combination operator, and distribution information of remote data access is further written into the shared memory; and writing the data into the shared memory block according to the distribution information of the shared memory, so that the network module accessed by the remote data sends the data to the target node.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a data processing method, apparatus, device, storage medium, and program product for a distributed database. Background Technology

[0002] In distributed databases, parallel execution within a node is primarily coordinated by the Gather operator. Its main function is to collect and merge the query results from multiple parallel worker processes, and then pass the merged result to the upper-level query plan node. If the upper-level execution plan node of the Gather operator is a Remote Data Access (RDA) operator, then after the parallel worker processes obtain the data, they need to send the data to the Gather operator for aggregation. The Gather operator then aggregates the data and passes it to the RDA operator, which then interacts with other nodes via the network.

[0003] However, when dealing with large amounts of data and numerous parallel processes, current technologies are prone to inefficient data transmission and parallel execution. Summary of the Invention

[0004] Therefore, it is necessary to provide a data processing method, apparatus, device, storage medium, and program product for a distributed database to address the aforementioned technical problems.

[0005] Firstly, this application provides a data processing method for a distributed database, applied to the workflow of collecting and merging operators, including:

[0006] The shared memory is determined; the shared memory includes a shared memory block corresponding to each worker process of the collection and merging operator; the shared memory also contains distribution information for remote data access.

[0007] Data is written into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send the data to the target node.

[0008] In one embodiment, before determining the shared memory, the method further includes: identifying whether it is in a parallel transmission mode; if it is in a parallel transmission mode, then performing the step of determining the shared memory.

[0009] In one embodiment, identifying whether it is in parallel transmission mode includes: identifying whether it is in parallel transmission mode based on whether the optimizer has a parallel transmission flag bit set.

[0010] In one embodiment, the shared memory is allocated by the remote data access receiving module to each of the worker processes of the collection and merging operator when it is detected that the data is in a parallel transmission mode.

[0011] In one embodiment, the receiving module of the remote data access is further configured to scan the shared memory block allocated to each of the working processes of the collection and merging operator to obtain the data, and to send the data to the target node through the network module of the remote data access.

[0012] In one embodiment, the distribution information of the remote data access is written into the shared memory by the remote data access sending module when it detects that it is in parallel sending mode.

[0013] Secondly, this application also provides a data processing apparatus for a distributed database, applied to the process of collecting and merging operators, including:

[0014] A determination module is used to determine shared memory; the shared memory includes shared memory blocks corresponding to each worker process of the collection and merging operator; the shared memory also contains distribution information for remote data access;

[0015] The writing module is used to write data into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send the data to the target node.

[0016] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0017] Determine the shared memory; the shared memory includes a shared memory block corresponding to each worker process of the collection and merging operator; the shared memory also contains distribution information for remote data access; data is written into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send the data to the target node.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0019] Determine the shared memory; the shared memory includes a shared memory block corresponding to each worker process of the collection and merging operator; the shared memory also contains distribution information for remote data access; data is written into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send the data to the target node.

[0020] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0021] Determine the shared memory; the shared memory includes a shared memory block corresponding to each worker process of the collection and merging operator; the shared memory also contains distribution information for remote data access; data is written into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send the data to the target node.

[0022] The aforementioned distributed database data processing method, apparatus, device, storage medium, and program product define shared memory, which includes shared memory blocks corresponding to each worker process of the collection and merging operator. The shared memory also contains distribution information for remote data access. Data is written to the shared memory blocks according to the distribution information, so that the network module for remote data access can send the data to the target node. This scheme allows multiple parallel worker processes of the collection and merging operator to write data to their respective shared memory blocks, and then the network module for remote data access sends the data from the shared memory blocks corresponding to each worker process to the target node. This eliminates the need for aggregation through the collection and merging operator, improving data transmission and parallel execution efficiency, and enhancing the overall performance of query execution. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is an application environment diagram of a distributed database data processing method in one embodiment;

[0025] Figure 2 This is a flowchart illustrating a data processing method for a distributed database in one embodiment.

[0026] Figure 3 This is a flowchart illustrating a data processing method for a distributed database in another embodiment;

[0027] Figure 4 This is a structural block diagram of a data processing device for a distributed database in one embodiment;

[0028] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various objects, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the second object. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the solutions, or any combination of multiple solutions.

[0031] Explanation of relevant terminology in the embodiments of this application:

[0032] RDA: Remote Data Access, is a database operator that enables data redistribution in a distributed database. It can be divided into a data sender and a data receiver.

[0033] The Gather operator is a data management system / distributed database operator used for parallel queries. Its main function is to collect and merge the query results from multiple parallel worker processes, and then pass the merged result to the upper-level query plan node.

[0034] Parallel transmission: When the database is executed in parallel, data from different parallel worker processes can be transmitted simultaneously.

[0035] In distributed databases, parallel execution within a node is primarily coordinated by the collect-merge operator. If the execution plan node above the collect-merge operator is a remote data access operator, then after the parallel worker processes acquire the data, they first need to send the data to the collect-merge operator for aggregation. After aggregation, the collect-merge operator then passes the data to the remote data access operator, which then interacts with other nodes via the network. When the data volume is large and there are many parallel worker processes, the collect-merge process can become a single point of bottleneck, leading to reduced efficiency in data transmission and parallel execution, thus affecting the overall performance of query execution.

[0036] In response, this application provides a data processing method for distributed databases that eliminates the need for aggregation through collection and merging operators, thereby removing the single-point bottleneck of collection and merging operators. This makes data interaction between nodes more efficient under parallel processing, improves the efficiency of data transmission and parallel execution, and enhances the overall performance of query execution.

[0037] The data processing method for distributed databases provided in this application can be applied to, for example... Figure 1 The application environment shown includes: an optimizer, a remote data access receiving module, a remote data access sending module, a collection and merging operator module, multiple parallel worker processes (worker processes 1, 2, ...), and shared memory. The optimizer, a component of the database management system, converts query statements into corresponding execution plans. The remote data access receiving module, during remote data access execution, can divide the remote data access operator into a sending module and a receiving module. This module is responsible for receiving data from its own node and other nodes, acting as a consumer of data interaction. The remote data access sending module, also divided into sending and receiving modules, is responsible for acquiring data from the execution shards under the operator and then sending the data to other nodes locally or over the network, acting as a producer of data interaction. In this embodiment, data is sent by the worker processes; the remote data access sending module does not need to actually send data. The collection and merging operator module collects and merges the query results from multiple parallel worker processes and then passes the merged result to the upper-level query plan node. In this embodiment, the data for the query structure of multiple parallel worker processes is sent automatically by the worker processes themselves, without the need for collection and merging by a collection and merging operator. In this embodiment, the worker processes can execute assigned tasks to acquire data and directly send the data to the receiving module of the remote data access. In this embodiment, shared memory can serve as a medium for data interaction between the multiple parallel worker processes and the receiving module of the remote data access; the multiple parallel worker processes can directly write data to the shared memory, and the network module of the remote data access can scan the shared memory and send the data out.

[0038] In one exemplary embodiment, such as Figure 2 As shown, a data processing method for a distributed database is provided, which can be applied to... Figure 1 The process of collecting and merging operators in a process may include the following steps:

[0039] Step S201: Determine shared memory.

[0040] In this step, for each worker process of the collection and merging operator, a shared memory is determined. This shared memory may include a shared memory block corresponding to each worker process of the collection and merging operator, and it also contains distribution information for remote data access. Each worker process can correspond to a different shared memory block; that is, each worker process can be allocated a separate shared memory block, and the data of each worker process does not interfere with each other. The distribution information for remote data access may include information such as distribution columns and the distribution of remote data access operator nodes.

[0041] Step S202: Write data into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send data to the target node.

[0042] In this step, for each worker process of the collection and merging operator, the worker process can obtain the distribution information of remote data access written to shared memory and determine the shared memory block used to store its data. Based on the distribution information of shared memory, it writes the data to the shared memory block. Thus, the network module for remote data access can scan all shared memory blocks and send the data stored therein to the corresponding target node. The target node is the node to which the data needs to be sent. The collection and merging operator can wait for the worker processes to finish and does not need to receive data from the aggregation worker processes.

[0043] The distributed database data processing method of this embodiment determines shared memory, which includes shared memory blocks corresponding to each worker process of the collection and merging operator. The shared memory also contains distribution information for remote data access. Data is written to the shared memory blocks according to the distribution information, so that the network module for remote data access can send the data to the target node. This scheme allows multiple parallel worker processes of the collection and merging operator to write data to their respective shared memory blocks, and then the network module for remote data access sends the data in the shared memory blocks corresponding to each worker process to the target node. This eliminates the need for aggregation through the collection and merging operator, improving the efficiency of data transmission and parallel execution, and enhancing the overall performance of query execution.

[0044] In an exemplary embodiment, before determining the shared memory in step S201, the following steps may also be included:

[0045] Identify whether it is in parallel transmission mode; if it is in parallel transmission mode, then perform the step of determining shared memory.

[0046] In this embodiment, after each worker process of the collection and merging operator is established, the worker process can identify whether it is currently in parallel transmission mode. This parallel transmission mode represents a scenario of Remote Data Access and Collection & Merging (RDA+Gather). If the worker process identifies that it is currently in parallel transmission mode, it can execute the aforementioned step of determining shared memory. Otherwise, it can maintain the original workflow.

[0047] The solution in this embodiment can introduce a workflow that does not require data aggregation through collection and merging operators by recognizing the parallel sending mode, based on the original workflow of the working process.

[0048] In an exemplary embodiment, the above-described identification of whether it is in parallel transmission mode may include:

[0049] Whether the optimizer is in parallel transmission mode can be identified by whether it has a parallel transmission flag set.

[0050] In this embodiment, the optimizer can identify the remote data access plus collection and merging scenario during the execution plan generation stage. Then, by setting a parallel transmission flag, the receiving module, sending module, collection and merging operator module, and worker processes can easily identify whether they are in parallel transmission mode based on the parallel transmission flag. Specifically, the optimizer can set the parallel transmission flag when a remote data access plus collection and merging scenario is identified, and not set the parallel transmission flag when a scenario other than remote data access plus collection and merging is identified.

[0051] In one exemplary embodiment, shared memory is allocated by the receiving module of remote data access for each worker process collecting the merging operator when it is detected that it is in parallel transmission mode.

[0052] In this embodiment, the optimizer can identify remote data access and collection / merging scenarios during the execution plan generation phase and set a parallel transmission flag. Therefore, the remote data access receiving module can identify that it is in parallel transmission mode based on the parallel transmission flag set by the optimizer. During initialization, it can allocate shared memory for multiple parallel worker processes, allocating a separate shared memory block for each worker process within the shared memory, ensuring that the data of each worker process does not interfere with each other.

[0053] In one exemplary embodiment, the receiving module of the remote data access is further configured to scan the shared memory blocks allocated to each working process of the collection and merging operator to obtain data, and to send the data to the target node through the network module of the remote data access.

[0054] In this embodiment, the remote data access receiving module can scan the shared memory blocks allocated to each worker process of the collection and merging operator to directly obtain data from each worker process. Therefore, the remote data access receiving module can also send this data to the corresponding target node through the remote data access network module.

[0055] In one exemplary embodiment, the distribution information of remote data access is written to shared memory by the remote data access sending module when it detects that it is in parallel sending mode.

[0056] In this embodiment, the optimizer can identify remote data access and collection / merging scenarios during the execution plan generation phase and set a parallel transmission flag. Thus, the remote data access sending module can identify that it is in parallel transmission mode based on the parallel transmission flag set by the optimizer. Then, the remote data access sending module can attach to the shared memory allocated by the remote data access receiving module and create relevant execution handles for the parallel executors, writing distribution information such as the distribution columns and RDA operator node distributions into the shared memory for use by subsequent multiple parallel worker processes.

[0057] In one exemplary embodiment, such as Figure 3 As shown, a data processing method for a distributed database is also provided, which may include the following steps:

[0058] In step S301, when the remote data access receiving module detects that it is in parallel transmission mode, it allocates shared memory for each working process of the collection and merging operator.

[0059] In step S302, when the remote data access sending module detects that it is in parallel sending mode, it writes the distribution information of remote data access to the shared memory.

[0060] Step S303: When the worker process of the collection and merging operator identifies that it is in parallel sending mode, it determines the shared memory.

[0061] Step S304: The working process of the collection merging operator writes data into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send data to the target node.

[0062] In this embodiment, during the optimizer's execution plan generation phase, a scenario involving remote data access and collection / merging is identified, and a parallel transmission flag is set. If the remote data access receiving module identifies that it is currently in parallel transmission mode based on the parallel transmission flag, it allocates shared memory for the multiple parallel worker processes of the collection / merging operator during initialization, assigning separate shared memory to each worker process to ensure that the data of each worker process does not interfere with each other. If the remote data access sending module identifies that it is currently in parallel transmission mode based on the parallel transmission flag, it attaches to the shared memory allocated by the receiving module and creates relevant execution handles for the parallel executors, writing distribution information such as the distribution column and RDA operator node distribution into the shared memory for use by subsequent worker processes. For the collection / merging operator, if it identifies that it is currently in parallel transmission mode based on the parallel transmission flag, it only needs to wait for the worker processes to finish and does not need to receive data from the aggregation worker processes. After the worker processes for collecting and merging operators are established, if they identify that they are currently in parallel transmission mode based on the parallel transmission flag, they determine the shared memory. They can then retrieve the distribution information of remote data access written by the remote data access sending module and identify the shared memory blocks used to store data. During data acquisition, they can write data to the shared memory blocks of the target nodes based on the distribution information. The remote data access network module can then send the data from the shared memory blocks to the corresponding target nodes. The remote data access receiving module can scan the shared memory blocks allocated to each worker process to directly obtain data from each worker process.

[0063] This embodiment's solution allows the data acquisition process of the collection and merging operator to directly send the acquired data to the receiving process of the remote data access operator, eliminating the need for aggregation by the collection and merging operator. This significantly simplifies the data transmission process in scenarios involving remote data access and collection and merging, improving execution efficiency and data transmission performance, and effectively enhancing the efficiency of parallel data interaction within nodes in a distributed environment. Compared to current technologies, this embodiment's solution solves the single-point bottleneck problem of the original collection and merging operator, greatly improving data interaction performance. Previously, the collection and merging operator needed to receive and aggregate data from the aggregation process one by one. Even with high parallelism of the processes, overall performance was still limited by the collection and merging operator. This embodiment cleverly utilizes the shared memory of remote data access to complete data interaction. Data reception and transmission still utilize the original capabilities of remote data access, but while the original remote data access's interactive data producer was the remote data access sending module, this embodiment's solution uses parallel processes. The data acquisition and transmission of these parallel processes do not interfere with each other and can be easily expanded into multiple producers according to configuration, improving the overall parallelism of the parallel execution chain.

[0064] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0065] Based on the same inventive concept, this application also provides a distributed database data processing apparatus for implementing the data processing method of the distributed database described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more distributed database data processing apparatus embodiments provided below can be found in the limitations of the distributed database data processing method described above, and will not be repeated here.

[0066] In one exemplary embodiment, such as Figure 4 As shown, a data processing device for a distributed database is provided, which can be applied to the workflow of collecting and merging operators. The device 400 may include:

[0067] The determination module 401 is used to determine the shared memory; the shared memory includes a shared memory block corresponding to each working process of the collection and merging operator; the shared memory also contains distribution information for remote data access;

[0068] The writing module 402 is used to write data into the shared memory block according to the distribution information of the shared memory, so that the network module for remote data access can send the data to the target node.

[0069] In an exemplary embodiment, the determining module 401 is further configured to identify whether it is in a parallel transmission mode; if it is in a parallel transmission mode, the step of determining shared memory is performed.

[0070] In an exemplary embodiment, the determining module 401 is used to identify whether it is in parallel transmission mode based on whether the optimizer has a parallel transmission flag bit set.

[0071] In one exemplary embodiment, the shared memory is allocated by the remote data access receiving module to each of the worker processes of the collection and merging operator when it is detected that the data is in parallel transmission mode.

[0072] In an exemplary embodiment, the remote data access receiving module is further configured to scan the shared memory block allocated to each of the worker processes of the collection and merging operator to obtain the data, and to send the data to the target node through the remote data access network module.

[0073] In an exemplary embodiment, the distribution information of the remote data access is written into the shared memory by the remote data access sending module when it detects that it is in parallel sending mode.

[0074] Each module in the aforementioned distributed database data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0075] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The database stores data. The I / O interfaces allow the processor to exchange information with external devices. The communication interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a distributed database data processing method.

[0076] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0078] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0079] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0080] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0081] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0082] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0083] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method of a distributed database, characterized by, The method is applied to a work process of a reduction and aggregation operator, and comprises the following steps: determining shared memory; the shared memory comprises a shared memory block corresponding to each work process of the reduction and aggregation operator; the shared memory is also written with distribution information of remote data access; writing data into the shared memory block according to the distribution information of the shared memory, so that a network module of remote data access sends the data to a target node.

2. The method of claim 1, wherein, Before the step of determining shared memory, the method further comprises the following steps: identifying whether in a parallel sending mode; if in the parallel sending mode, performing the step of determining shared memory.

3. The method of claim 2, wherein, The step of identifying whether in the parallel sending mode comprises the following step: identifying whether in the parallel sending mode according to whether a parallel sending flag bit is set in an optimizer.

4. The method of claim 1, wherein, The shared memory is allocated by a receiving module of remote data access for each work process of the reduction and aggregation operator when it is identified that the parallel sending mode is in.

5. The method of claim 4, wherein, The receiving module of remote data access is also used to scan the shared memory block allocated to each work process of the reduction and aggregation operator to obtain the data, and send the data to a target node through the network module of remote data access.

6. The method according to any one of claims 1 to 5, characterized in that, The distribution information of remote data access is written into the shared memory by a sending module of remote data access when it is identified that the parallel sending mode is in.

7. A data processing apparatus of a distributed database, characterized by, The device is applied to a work process of a reduction and aggregation operator, and comprises the following modules: a determining module, configured to determine shared memory; the shared memory comprises a shared memory block corresponding to each work process of the reduction and aggregation operator; the shared memory is also written with distribution information of remote data access; a writing module, configured to write data into the shared memory block according to the distribution information of the shared memory, so that a network module of remote data access sends the data to a target node. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The processor executes the computer program to realize the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 6. The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Memory page migration method and device and related equipment

    CN120670339A

  • Distributed storage method and system of real-time database, electronic equipment and storage medium

    CN121009140A

  • Method for realing sharing internal stored data base and internal stored data base system

    CN1740978A

  • Repartitioning data in a distributed computing system

    US20180046398A1