Data processing method, device and system

Abstract

The present application discloses a data processing method, device and system, which belongs to the field of computer technology. By deploying DPUs in computing nodes and storage nodes respectively, and establishing a communication link between the DPU in the computing node and the DPU in the storage node. Multiple business processes in the computing node only need to interact with the DPU in the computing node, and then the DPU in the computing node interacts with the DPU in the storage node through the communication link, so that data can be written in the storage node or read from the storage node. Only one communication link needs to be established between a single computing node and a storage node through the DPU, which solves the problem that the data transmission performance of the computing node and the storage node is reduced due to the excessive number of links, thereby improving the data processing efficiency.

Description

[0001] This application is a divisional application. The application number of the original application is 202210981958.8, and the original application date is August 16, 2022. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data processing method, device and system. Background Art

[0003] The MapReduce (MR) model is a big data distributed processing model used for parallel computing of large data sets. MapReduce runs the job by dividing it into tasks. Tasks are divided into two types: map tasks and reduce tasks. In a distributed situation, data is distributed on different nodes. Therefore, after the data on different nodes are processed by the map task, the temporary data is written to the local disk. Subsequently, the node where the reduce task is located reads the data remotely to the local disk to complete the data processing. This data exchange process is called shuffle.

[0004] With the trend of big data storage and computing separation and cloudification, the industry has proposed a remote shuffle architecture to reduce hardware costs and improve resource utilization. The remote shuffle architecture includes computing nodes and remote shuffle service (RSS) nodes. In the remote shuffle architecture, disks are uniformly deployed on RSS nodes, and storage services are provided by RSS nodes. The computing node stores temporary data generated during the shuffle process in the local disk of the RSS node. In specific implementation, the MR process runs on the computing node, and the MR process and the RSS process on the RSS node transmit data through the transmission control protocol (TCP) link or remote direct memory access (RDMA) queue pairs (QP) (collectively referred to as communication links), and the RSS process interacts with the local disk deployed on the RSS node to complete shuffle reading and / or shuffle writing. Among them, the MR process is a business process. The running of an MR process on a computing node indicates that the computing node executes a map task or a reduce task.

[0005] In the big data processing scenario, a computing node will execute multiple map tasks and / or reduce tasks at the same time, that is, the computing node will run multiple MR processes at the same time. Since each MR process needs to establish a communication link with the RSS process separately, there will be a large number of communication links between the computing node and the RSS node. Too many links will cause a sharp drop in data transmission performance, thereby affecting data processing efficiency. Summary of the invention

[0006] The present application provides a data processing method, device and system, which can improve the data processing efficiency under the remote shuffle architecture.

[0007] In a first aspect, a data processing method is provided, which is applied to a computing node. The computing node includes a first data processing unit (DPU). A communication link is established between the first DPU and a second DPU in a storage node. Memory address allocation information is stored in the first DPU. The memory address allocation information is used to indicate a virtual memory address corresponding to a physical memory allocated to the computing node by the storage node. The method includes: the first DPU receives a memory allocation request from a first business process in the computing node. The first DPU determines a virtual memory address range allocated to the first business process in a virtual memory address corresponding to the physical memory allocated to the computing node by the storage node according to the memory address allocation information. When the first DPU receives a data storage request from the first business process, the first DPU sends a write data request to the second DPU through a communication link. The data storage request includes first data and a first memory address. The first memory address belongs to the virtual memory address range. The write data request includes the first data and the first memory address. The write data request is used to request the second DPU to write the first data into the physical memory corresponding to the first memory address.

[0008] In the present application, DPUs are deployed in computing nodes and storage nodes respectively, and a communication link is established between the DPU in the computing node and the DPU in the storage node. The DPU in the storage node synchronizes memory address allocation information to the DPU in the computing node through the communication link, so that the DPU in the computing node pre-stores the virtual memory address corresponding to the physical memory assigned to the computing node by the storage node. In this way, the computing node can implement remote memory direct mapping through the DPU to achieve direct access to the physical memory of the storage node. In specific implementation, the DPU in the computing node can allocate available memory in the storage node for the business process in the computing node according to the memory address allocation information. For the business process, the business process only needs to interact with the DPU in the computing node to obtain available memory, and further can write data to the available memory in the storage node through the DPU in the computing node. Different business processes in the computing nodes do not need to establish communication links with the storage nodes separately. Only one communication link needs to be established between a single computing node and the storage node through the DPU, which greatly reduces the number of links between the computing nodes and the storage nodes. It can alleviate the problem of excessive memory noise caused by too many links in the computing nodes or storage nodes, thereby ensuring the data transmission efficiency between the computing nodes and the storage nodes, thereby improving the data processing efficiency, and improving the reliability and stability of data processing.

[0009] Optionally, after the first DPU sends a write data request to the second DPU via the communication link, when the first DPU receives a write data response from the second DPU via the communication link, and the write data response indicates that the first data has been written successfully, the first DPU sends a data storage response to the first business process, and the data storage response indicates that the first data has been stored successfully. The first business process stores the first metadata corresponding to the first data in the metadata service device. The first metadata includes a first memory address. The metadata service device may be the computing node, or may be other devices independent of the computing node and the storage node.

[0010] Optionally, when the first DPU receives a data read request from a second business process in the computing node, the first DPU sends a data read request to the second DPU via a communication link. The data read request includes a second memory address. The second memory address is a virtual memory address corresponding to a physical memory in the storage node. The data read request includes the second memory address. The data read request is used to request the second DPU to read the second data from the physical memory corresponding to the second memory address.

[0011] In this application, when a business process needs to read data from a storage node, the business process only needs to pull data from the corresponding memory in the storage node through the DPU in the computing node. Different business processes in the computing node do not need to establish communication links with the storage node separately. A single computing node and a storage node only need to establish a communication link through the DPU, which greatly reduces the number of links between the computing node and the storage node, and can alleviate the problem of excessive memory noise caused by too many links in the computing node or storage node, thereby ensuring the data transmission efficiency between the computing node and the storage node, thereby improving the data processing efficiency, and improving the reliability and stability of data processing.

[0012] Optionally, after the first DPU sends a read data request to the second DPU via the communication link, when the first DPU receives a read data response including the second data from the second DPU via the communication link, the first DPU writes the second data into the target cache of the computing node. The second service process reads the second data from the target cache.

[0013] Optionally, the read data request also includes a cache address of the target cache, and the read data request is used to request the second DPU to write the data read from the physical memory corresponding to the second memory address into the target cache. Correspondingly, the read data response also includes the cache address of the target cache, and the read data response is used to instruct the first DPU to write the second data into the target cache.

[0014] Optionally, the first business process is a map process. Optionally, the second business process is a reduce process.

[0015] Optionally, after the computing node is powered on, the first DPU establishes a communication link with the second DPU. Then the first DPU receives the memory address allocation information sent by the second DPU. The communication link may be a long connection.

[0016] Optionally, the first DPU includes a first RDMA network card, and the second DPU includes a second RDMA network card. The communication link between the first DPU and the second DPU is an RDMA link established between the first RDMA network card and the second RDMA network card.

[0017] In a second aspect, a data processing method is provided, which is applied to a storage node. The storage node includes a first DPU. A communication link is established between the first DPU and a second DPU in a computing node. The method includes: the first DPU receives a write data request from the second DPU through a communication link, and the write data request includes first data and a first memory address. The first memory address is a virtual memory address corresponding to the physical memory assigned by the storage node to the computing node. The first DPU writes the first data into the physical memory corresponding to the first memory address. The first DPU sends a write data response to the second DPU through the communication link, and the write data response indicates that the first data has been written successfully.

[0018] Optionally, the first DPU receives a read data request from the second DPU via a communication link, the read data request including a second memory address, the second memory address being a virtual memory address corresponding to a physical memory in the storage node. The first DPU reads the second data from the physical memory corresponding to the second memory address. The first DPU sends a read data response to the second DPU via the communication link, the read data response including the second data.

[0019] Optionally, the read data request also includes a cache address of the target cache, and the read data request is used to request the first DPU to write the data read from the physical memory corresponding to the second memory address into the target cache. Correspondingly, the read data response also includes the cache address of the target cache, and the read data response is used to instruct the second DPU to write the second data into the target cache.

[0020] Optionally, the first DPU obtains a virtual memory address corresponding to the physical memory of the storage node. After the first DPU establishes a communication link with the second DPU, the first DPU sends memory address allocation information to the second DPU, where the memory address allocation information includes a virtual memory address corresponding to the physical memory allocated by the storage node to the computing node.

[0021] Optionally, the storage node includes multiple memories, and an implementation method for the first DPU to obtain the virtual memory address corresponding to the physical memory of the storage node includes: the first DPU uniformly addresses the multiple memories to obtain the virtual memory address corresponding to the physical memory of the storage node.

[0022] Optionally, the first DPU includes a first RDMA network card, and the second DPU includes a second RDMA network card. The communication link between the first DPU and the second DPU is an RDMA link established between the first RDMA network card and the second RDMA network card.

[0023] Optionally, the storage node includes a solid-state drive. The first DPU is connected to the solid-state drive via a memory access interface. The memory access interface includes a cache and a memory control chip. The cache is used to store the data to be stored written by the first DPU to the solid-state drive, and the memory control chip is used to write the data to be stored in the cache to the solid-state drive. The memory control chip is also used to write the data to be read read by the first DPU from the solid-state drive into the cache, so that the first DPU can read the data to be read from the cache.

[0024] In this application, the DPU in the storage node can read and write data directly from the solid-state drive through the memory access interface without going through the OS file system, which can improve disk access efficiency and thus improve data processing efficiency.

[0025] Optionally, the memory access interface is integrated on the solid state drive.

[0026] In a third aspect, a computing node is provided. The computing node includes a DPU, and the DPU includes multiple functional modules, and the multiple functional modules interact with each other to implement the method in the first aspect and its various embodiments. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations.

[0027] In a fourth aspect, a storage node is provided. The storage node includes a DPU, and the DPU includes multiple functional modules, and the multiple functional modules interact with each other to implement the method in the above second aspect and its various embodiments. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations.

[0028] In a fifth aspect, a computing node is provided, including: a DPU and a central processing unit (CPU). The CPU is used to call a computer program including program instructions to implement the operation of a business process. The DPU is used to call a computer program including program instructions to implement the actions performed by the first DPU in the first aspect and its embodiments.

[0029] In a sixth aspect, a storage node is provided, comprising: a DPU and a memory. The DPU is used to call a computer program including program instructions, read data from the memory and / or write data to the memory, so as to implement the actions performed by the first DPU in the second aspect and its respective embodiments.

[0030] In a seventh aspect, a data processing system is provided, comprising: the computing node as described in the third aspect or the fifth aspect and the storage node as described in the fourth aspect or the sixth aspect. The data processing system can be applied to a remote shuffle architecture.

[0031] The computing node includes a first DPU and a CPU. The storage node includes a second DPU and a memory. A communication link is established between the first DPU and the second PDU. The first DPU stores memory address allocation information, which is used to indicate the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node. The physical memory in the storage node belongs to the memory.

[0032] In the process of computing nodes writing data to storage nodes (shuffle writing process):

[0033] The CPU is used to send a memory allocation request from a first business process to the first DPU, and the first business process is any map process running in the CPU. The first DPU is used to determine the virtual memory address range allocated to the first business process in the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node according to the memory address allocation information. The CPU is also used to send a data storage request from the first business process to the first DPU, and the data storage request includes the first data and the first memory address, and the first memory address belongs to the virtual memory address range allocated by the first DPU to the first business process. The first DPU is used to send a write data request to the second DPU through the communication link between the first DPU and the second PDU, and the write data request includes the first data and the first memory address. The second DPU is used to write the first data into the physical memory corresponding to the first memory address. The second DPU is also used to send a write data response to the first DPU through the communication link between the first DPU and the second PDU, and the write data response indicates that the first data has been written successfully.

[0034] In the process of computing nodes reading data from storage nodes (shuffle reading process):

[0035] The CPU is used to send a data read request from a second business process to the first DPU, where the data read request includes a second memory address, where the second memory address is a virtual memory address corresponding to a physical memory in a storage node, and the second business process is any reduce process running in the CPU. The first DPU is used to send a read data request to the second DPU via a communication link between the first DPU and the second PDU, where the read data request includes the second memory address. The second DPU is used to read the second data from the physical memory corresponding to the second memory address. The second DPU is also used to send a read data response to the first DPU via a communication link between the first DPU and the second PDU, where the read data response includes the second data.

[0036] In an eighth aspect, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the method in the above-mentioned first aspect and its various embodiments or the method in the above-mentioned second aspect and its various embodiments is implemented.

[0037] In a ninth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method in the above-mentioned first aspect and its various embodiments or the method in the above-mentioned second aspect and its various embodiments.

[0038] In the tenth aspect, a chip is provided, which includes a programmable logic circuit and / or program instructions. When the chip is running, it implements the method in the above-mentioned first aspect and its various embodiments or the method in the above-mentioned second aspect and its various embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of a shuffle process provided by an embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of the architecture of a local disk shuffle in the related art;

[0041] Figure 3 This is a schematic diagram of a remote shuffle architecture in the related technology;

[0042] Figure 4 This is a data interaction diagram under a remote shuffle architecture in related technologies;

[0043] Figure 5 It is a structural diagram of a data processing system provided in an embodiment of the present application;

[0044] Figure 6 This is a schematic diagram of data interaction under a remote shuffle architecture provided in an embodiment of the present application;

[0045] Figure 7 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0046] Figure 8 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0047] Fig. 9 is a schematic diagram of the structure of a computing node provided in an embodiment of the present application;

[0048] Fig.10 is a schematic diagram of the structure of a storage node provided in an embodiment of the present application;

[0049] Fig.11 is a block diagram of a computing node provided in an embodiment of the present application;

[0050] Fig.12 This is a block diagram of a storage node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] The MapReduce model is the most classic and important processing model in big data distributed processing, and has derived distributed big data processing frameworks such as Hadoop, Spark, Flink, and Tez. The current software implementation of MapReduce specifies a map function and a concurrent reduce function. The map function is used to map a set of key-value pairs into a new set of key-value pairs. The reduce function is used to ensure that all key-value pairs with a mapping relationship share the same key group.

[0053] Shuffle is a stage of task processing in the MapReduce framework, which refers to the process of exchanging data from the node where the map task is located to the node where the reduce task is located through processing according to certain rules. In other words, Shuffle describes the process of data from the output of the map task to the input of the reduce task. Shuffle is the bridge between map and reduce. Since in a distributed situation, the reduce task needs to cross nodes to pull the output results of the map task on other nodes, the output of the map must go through the shuffle link to be used in the reduce. The shuffle process will consume network resources and memory and disk input / output (IO) consumption.

[0054] Usually, shuffle is divided into two parts: data preparation in the map phase and data copy processing in the reduce phase. Data preparation in the map phase, i.e. shuffle on the map side, is also called shuffle write. Data copy processing in the reduce phase, i.e. shuffle on the reduce side, is also called shuffle read.

[0055] The shuffle process on the map side includes: partitioning, spilling, and merging. Partitioning means that after the data is input, the data slices (split) read by the map task are written to the corresponding memory buffer. Among them, each map task has a corresponding memory buffer to store the output data of the map task. Spilling means that when the memory buffer is almost full, the data in the memory buffer is stored in the disk as a temporary file. Merging means that after the entire map task is completed, all temporary files generated by this map task on the disk are merged to generate the final output file, that is, the output result of the map task.

[0056] The shuffle process on the Reduce side includes: copying and merging. Copying means pulling the output results of the map task from the node where each map task is currently working. Merging means sorting and merging the data pulled from the nodes where each map task is located, and finally forming a file as the input file of the reduce task.

[0057] For example, Figure 1 This is a schematic diagram of a shuffle process provided in an embodiment of the present application. Figure 1 In this paper, Hadoop MapReduce shuffle is taken as an example. The shuffle process is presented in the form of data flow. Figure 1As shown, the Hadoop distributed file system (HDFS) includes 4 data shards, namely data shards 1 to 4, which are input data. The Map side runs 4 map tasks, which are respectively recorded as map1 to 4. The Reduce side runs 3 reduce tasks, which are respectively recorded as reduce1 to 3. In the Map phase, each map task processes a data shard. Among them, map1 processes data shard 1, map2 processes data shard 2, map3 processes data shard 3, and map4 processes data shard 4. Each map task generates an output file through shuffle (partitioning, overflowing, and merging), and each output file includes 3 parts of data, which are processed by 3 reduce tasks respectively. The output file generated by Map1 includes data A1, data B1, and data C1, the output file generated by map2 includes data A2, data B2, and data C2, the output file generated by map3 includes data A3, data B3, and data C3, and the output file generated by map4 includes data A4, data B4, and data C4. In the Reduce phase, each reduce task pulls the corresponding part of the data from the output files generated by the four map tasks through shuffle (copying and merging), and merges the four copies of data pulled from the output files generated by the four map tasks as the input files of the corresponding reduce tasks. Among them, the input files obtained by reduce1 include data A1~A4, the input files obtained by reduce2 include data B1~B4, and the input files obtained by reduce3 include data C1~C4. Then, each reduce task performs inductive calculations on the obtained input files and outputs the final processing results to HDFS. Among them, the processing result output by reduce1 is data part 1, the output result of reduce2 is data part 2, and the output result of reduce3 is data part 3.

[0058] In the related art, the MapReduce model is deployed in multiple computing nodes to achieve distributed processing of big data. A computing node can be, for example, a server or a server cluster consisting of multiple servers. In the traditional implementation, a local disk is usually deployed in the computing node, and the computing node overflows the temporary data generated during the shuffle process to the local disk. For example, Figure 2 This is a schematic diagram of the architecture of a local disk shuffle in the related art. Figure 2As shown, the MapReduce model in the computing node stores the generated temporary data to the local disk through shuffle writing, and remotely reads the data in the local disk of other computing nodes to the local through shuffle reading.

[0059] Currently, there are two forms of local disks deployed in computing nodes. One form is to deploy more than ten hard disk drives (HDDs) in a computing node. This form requires a large number of disk slots and occupies a large amount of space. The other form is to deploy two terabyte (TB) non-volatile memory express (NVMe) solid state drives (SSDs) in a computing node. This form has a higher hardware cost.

[0060] With the trend of big data storage and computing separation and cloudification, the industry has proposed a remote shuffle architecture to reduce hardware costs and improve resource utilization. In the remote shuffle architecture, disks are uniformly deployed on RSS nodes, so that there is no need or only a small number of local disks can be deployed on computing nodes. The computing node stores the temporary data generated during the shuffle process on the local disk of the RSS node. For example, Figure 3 This is a schematic diagram of a remote shuffle architecture in the related technology. Figure 3 As shown in the figure, the remote shuffle architecture includes multiple computing nodes and RSS node clusters. The RSS node cluster includes one or more RSS nodes. The MapReduce model in the computing node stores the generated temporary data in the local disk of the RSS node through shuffle write, and reads the data in the local disk of the RSS node remotely to the local through shuffle read.

[0061] In the remote shuffle architecture, since computing nodes do not need to deploy local disks or can deploy a small number of local disks, the size of computing nodes can be greatly reduced, so high-density deployment of computing nodes can be achieved, thereby achieving high-density computing. In addition, existing distributed computing frameworks (such as Spark, which needs to rely on local disks to store shuffle data) greatly limit the cloud-native deployment mode. Using the remote shuffle architecture can effectively reduce the dependence on local disks, support multiple deployment modes of the cluster, improve resource utilization, and help the cloud-native architecture.

[0062] Since all shuffle reads and shuffle writes will go through the network in the remote shuffle architecture, the efficiency of data transmission is the key bottleneck. In related technologies, when a computing node needs to execute a map task or a reduce task, the computing node will run an MR process and establish a TCP link or RDMA QP between the running MR process and the RSS process running on the RSS node, so that the MR process and the RSS process can transmit data through the TCP link or RDMAQP, and the RSS process interacts with the local disk deployed on the RSS node to complete the shuffle read or shuffle write of the computing node. For example, Figure 4 This is a schematic diagram of data interaction under a remote shuffle architecture in related technologies. Figure 4 As shown, the remote shuffle architecture includes a computing node cluster and an RSS node cluster. The computing node cluster includes n computing nodes, denoted as computing nodes 1 to n, where n is a positive integer. The RSS node cluster includes m RSS nodes, denoted as RSS nodes 1 to m, where m is a positive integer. A communication link (TCP link or RDMA QP) is established between each MR process on each computing node and the RSS process on the RSS node. The RSS process on the RSS node interacts with the local disk deployed on the RSS node to write data to the local disk or read data from the local disk.

[0063] In the big data processing scenario, a computing node will execute multiple map tasks and / or reduce tasks at the same time, that is, the computing node will run multiple MR processes at the same time. Since each MR process needs to establish a communication link with the RSS process separately, there will be a large number of communication links between the computing node and the RSS node. Too many links will cause a sharp decline in data transmission performance. Therefore, the data transmission reliability between the computing node and the RSS node is currently low. Especially for RSS nodes, since an RSS node needs to provide storage services to multiple computing nodes at the same time, there will be a large number of communication links on the RSS node, resulting in excessive memory noise of the RSS node, which seriously affects the data transmission performance of the RSS node. In addition, every time the computing node wants to execute a new map task or reduce task, it needs to establish a new communication link with the RSS process in the RSS node. On the one hand, the remote shuffle data processing efficiency is low, and on the other hand, the overhead of frequent link establishment is large, which will affect the processing performance of the computing node and the RSS node. Therefore, the data processing efficiency under the current remote shuffle architecture is low, and the data processing stability and reliability are also low.

[0064] Based on this, an embodiment of the present application provides a technical solution. The present application deploys DPUs in computing nodes and storage nodes respectively, and establishes a communication link between the DPU in the computing node and the DPU in the storage node. The DPU in the storage node synchronizes memory address allocation information to the DPU in the computing node through the communication link, so that the DPU in the computing node pre-stores the virtual memory address corresponding to the physical memory assigned to the computing node by the storage node. In this way, the computing node can implement remote memory direct mapping through the DPU to achieve direct access to the physical memory of the storage node. In specific implementation, the DPU in the computing node can allocate available memory in the storage node for the business process in the computing node according to the memory address allocation information. For the business process, when performing shuffle write, the business process only needs to interact with the DPU in the computing node to obtain available memory, and further write data to the available memory in the storage node through the DPU in the computing node. When performing shuffle read, the business process only needs to pull data from the corresponding memory in the storage node through the DPU in the computing node. In the technical solution of the present application, different business processes in the computing nodes do not need to establish communication links with the storage nodes respectively. Only one communication link needs to be established between a single computing node and a storage node through the DPU. Compared with the above-mentioned existing remote shuffle architecture implementation scheme, the number of links between the computing nodes and the storage nodes is greatly reduced, which can alleviate the problem of excessive memory noise caused by too many links in the computing nodes or storage nodes, thereby ensuring the data transmission efficiency between the computing nodes and the storage nodes, and then improving the data processing efficiency, and improving the reliability and stability of data processing. In addition, since the number of links between a single computing node and a storage node is reduced, the storage node can support the provision of storage services for more computing nodes while ensuring data transmission performance, thereby expanding the scale of the system.

[0065] The following is a detailed introduction to the technical solution of the present application from multiple perspectives including system structure, method flow, software device, hardware device, etc.

[0066] For example, Figure 5 Schematic diagram of a data processing system provided in an embodiment of the present application. Figure 5 As shown, the data processing system includes a computing node cluster and a storage node cluster. The computing node cluster includes one or more computing nodes 510 ( Figure 5 Three computing nodes 510 are shown in FIG. 5 , but are not limited to three computing nodes 510). The computing nodes 510 can communicate with each other. The storage node cluster includes one or more storage nodes 520 ( Figure 5 Three storage nodes 520 are shown in the figure, but are not limited to three storage nodes 520).

[0067] The computing node 510 is a computing device, such as a server, a desktop computer, or a storage array controller. Figure 5 As shown, a computing node 510 includes at least a processor 511, a memory 512 and a DPU 513. A storage node 520 includes a DPU 521 and one or more memories 522. Among them, the DPU has the versatility and programmability of a central processing unit (CPU), but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU has a network card function. The DPU may physically include a network card, or the DPU may also be integrated with a network card, that is, the DPU may not physically include a separate network card, and the network card function is implemented by the DPU. In the embodiment of the present application, the DPU physically includes a network card as an example for illustration, but it is not used to limit the structural form of the DPU.

[0068] In the embodiment of the present application, a communication link is established between the DPU 513 in the computing node 510 and the DPU 521 in the storage device 520, which is used to realize the communication between the computing node 510 and the storage device 520. Optionally, the DPU 513 includes an RDMA network card, and the DPU 521 includes an RDMA network card. The DPU 513 and the DPU 521 establish an RDMA link through the RDMA network card, and the RDMA link can be implemented by a network interface card (NIC) or a bus. Alternatively, the communication link between the DPU 513 and the DPU 521 can also be a TCP link, and the embodiment of the present application does not limit the link establishment method adopted between the DPUs.

[0069] In the data processing system provided in the embodiment of the present application, the DPU 513 in the computing node 510 and the DPU 521 in the storage node 520 can be fully interconnected, so that any computing node 510 in the computing node cluster can access any storage node 520 in the storage node cluster.

[0070] The hardware functions of the computing node 510 are introduced below.

[0071] Processor 511 is a CPU, which is used to process data access requests from outside computing node 510, or requests generated inside computing node 510. For example, when processor 511 receives write data requests sent by users, it temporarily saves the data in these write data requests in memory 512. When the total amount of data in memory 512 reaches a certain threshold, processor 511 sends the data stored in memory 512 to storage node 520 through DPU 513 for persistent storage. Processor 511 reads and writes data by running business processes. Business processes that can be run on processor 511 include but are not limited to map processes and reduce processes. For example, processor 511 can write data by running a map process, and can also read data by running a reduce process. In addition, processor 511 is also used to calculate or process data, such as metadata management, deduplication, data compression, virtualized storage space, and address conversion. Figure 5 Only one CPU 511 is shown in the figure. In actual applications, there are often multiple CPUs 511, wherein one CPU 511 has one or more CPU cores. The embodiment of the present application does not limit the number of CPUs and the number of CPU cores.

[0072] Memory 512 refers to an internal memory that directly exchanges data with processor 511. It can read and write data at any time, and the speed is very fast. It serves as a temporary data storage for the operating system or other running programs. Memory 512 includes at least two types of memories. For example, memory 512 can be either a random access memory or a read-only memory (ROM). For example, the random access memory is a dynamic random access memory (DRAM) or a storage class memory (SCM). DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memory. Storage class memory can provide faster read and write speeds than hard disks, but the access speed is slower than DRAM, and the cost is cheaper than DRAM. However, DRAM and SCM are only exemplary descriptions in the embodiments of the present application, and memory can also include other random access memories, such as static random access memory (SRAM). As for the read-only memory, for example, it can be a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), etc. In addition, the memory 512 can also be a dual in-line memory module or a dual-line memory module (DIMM), that is, a module composed of a dynamic random access memory (DRAM), or a solid-state drive (SSD). In actual applications, multiple memories 512 and different types of memories 512 can be configured in the computing node 510. The embodiment of the present application does not limit the number and type of memory 512. In addition, the memory 512 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 512 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0073] Alternatively, see Figure 5, DPU 513 includes a processing unit and a network card. The processing unit may be a CPU. The processing unit includes a cache and a remote memory management unit (RMMU). The cache may be, for example, a DRAM. RMMU is used to store memory address allocation information from a storage node 520 in a storage node cluster, the memory address allocation information indicating a virtual memory address corresponding to a physical memory allocated by the storage node 520 to the computing node 510. RMMU is also used to allocate available memory in the storage node 520 to a business process running on the processor 511 according to the memory address allocation information, so that the business process can write data, where the business process may be a map process. RMMU is also used to pull data from the storage node 520 for a business process running on the processor 511 to complete the data reading task of the business process, where the business process may be a reduce process. In summary, RMMU can communicate with DPU 521 in the storage node 520 through a network card to complete the data reading and writing tasks of the business process running on the processor 511. RMMU can be further subdivided into a memory channel and a memory allocation module. The memory allocation module is used to perform the data processing functions of the RMMU, including but not limited to storing memory address allocation information and allocating memory to business processes. The memory channel is used to perform the data transmission functions of the RMMU, including but not limited to transmitting the data to be written and read of the business process.

[0074] The network card in DPU 513 is used to communicate with storage node 520. For example, when the total amount of data in memory 512 reaches a certain threshold, computing node 510 can send a request to storage node 520 through the network card in DPU 513 to store the data persistently. In addition, computing node 510 can also include a bus for communication between components inside computing node 510. Figure 5 The main function of the computing node 510 in the example is computing services. When storing data, remote storage can be used to achieve persistent storage. Therefore, it has less local storage than a conventional server, thereby saving cost and space. However, this does not mean that the computing node 510 cannot have local storage. In actual implementation, the computing node 510 can also have a small number of hard disks built in or externally connected.

[0075] The hardware functions of the storage node 520 are introduced below.

[0076] A storage node 520 may have a memory pool, which is composed of one or more memories 522 in the storage node 520, that is, the physical memory of the memory pool of a storage node 520 comes from the various memories 522 contained in the storage node 520. The memory 522 is used to store data and can be a disk or other types of storage media. Optionally, a storage node 520 may include multiple different types of memories, including but not limited to DRAM, phase change memory (PCM), and SSD.

[0077] Alternatively, see Figure 5 , DPU 521 includes a memory access controller and a network card. The memory access controller can be a CPU. The memory access controller has data reading and writing, address conversion and other computing functions. The memory access controller is used to uniformly address the memory included in the memory pool. After the uniform addressing, each space segment of the memory pool has a unique global address in the entire storage node cluster. After the physical space is allocated to a space segment of the memory pool, the global address of the space has its corresponding physical address, which indicates which storage node and memory the space represented by the global address is actually located on, and the offset in the memory, that is, the location of the physical space. That is, the global address allocated to a space segment of the memory pool is a virtual memory address, which corresponds to the physical address of the physical memory. For example, the global address of a space segment of the memory pool can be represented by a 64-bit address, which consists of 4 parts, represented as [hostId, media type, numaId, blockIndex]. Among them, hostId is the identifier of the storage node, occupying 0-15 bits of the global address. media type is the media type, which occupies bits 16-18 of the global address. numaId is the memory bar identifier, which occupies bits 19-22 of the global address. blockIndex is the block index, which occupies bits 23-63 of the global address.

[0078] The memory access controller is used to write data to or read data from the memory 522 according to the read / write data request sent by the computing node 510. In the process of reading and writing data, the memory access controller needs to convert the virtual memory address carried in the read / write data request into a physical address that the memory 522 can recognize.

[0079] Alternatively, see Figure 5, the memory 522 in the storage node 520 includes but is not limited to DRAM, PCM and SSD. DPU521 and SDD can be connected through a memory access interface. The memory access interface includes a cache and a memory control chip. The cache in the memory access interface is used to store the data to be stored written by DPU 521 to the SSD, and the memory control chip is used to write the data to be stored in the cache to the SSD. The memory control chip is also used to write the data to be read read by DPU 521 from the SSD into the cache, so that DPU 521 can read the data to be read from the cache. Optionally, the memory access interface can be integrated on the SSD. Alternatively, the memory access interface can also be used as an independent device to connect the SSD and DPU 521.

[0080] In the related art, the RSS node needs to interact with the SSD through the RSS process to complete the reading and writing of data, and the RSS process needs to go through the operating system (OS) file system to access the SSD. Due to the limitations of the OS file system, the disk access efficiency is low. In the embodiment of the present application, the DPU in the storage node can directly read and write data from the SSD through the memory access interface without going through the OS file system, which can improve the disk access efficiency and thus improve the data processing efficiency.

[0081] The network card in the DPU 521 is used to communicate with the computing node 510. For example, after the memory access controller of the DPU 521 completes the unified addressing of the memory included in the memory pool in the storage node 520, the memory address allocation information can be sent to the computing node 510 through the network card in the DPU 521. In addition, the storage node 520 may also include a bus for communication between the components inside the storage node 520.

[0082] Optionally, the communication link between the DPU 513 of the computing node 510 and the DPU 521 of the storage node 520 is a long connection, that is, the communication link between the DPU 513 in the computing node 510 and the DPU 521 in the storage node 520 exists for a long time. In this way, there is no need to frequently establish a link between the computing node 510 and the storage node 520, which can save processing resources of the computing node and the storage node. After the computing node 510 and the storage node 520 are powered on, full interconnection and memory address mapping are first achieved through the DPU.

[0083] For the storage node 520, after the storage node 520 is powered on, the DPU 521 in the storage node 520 obtains the virtual memory address corresponding to the physical memory of the storage node 520. If the storage node 520 includes multiple memories, the DPU 521 can uniformly address the multiple memories to obtain the virtual memory address corresponding to the physical memory of the storage node 520. In the case where the DPU 521 includes a network card and a memory access controller, the memory access controller can specifically uniformly address the multiple memories, and then register the memory with the network card, so that the network card can directly read and write data from the physical memory.

[0084] For the computing node 510, after the computing node 510 is powered on, the DPU 513 in the computing node 510 establishes a communication link with the DPU 521 in the storage node 520. Then, the DPU 521 in the storage node 520 can send memory address allocation information to the DPU 513 in the computing node 510, and the memory address allocation information includes the virtual memory address corresponding to the physical memory allocated by the storage node 520 to the computing node 510. Optionally, when the computing node 510 needs to write data to the storage node 520, it can apply for available memory from the storage node 520 through a pre-established communication link, and then the storage node 520 allocates memory to the computing node 510 through the communication link. Alternatively, after the DPU 513 in the computing node 510 establishes a communication link with the DPU 521 in the storage node 520, the storage node 520 can also directly allocate memory to the computing node 510 through the communication link. After receiving the memory address allocation information from the storage node 520, the DPU 513 in the computing node 510 stores the memory address allocation information and initializes the memory mapping relationship. The memory mapping relationship includes, but is not limited to, which storage node the memory address of the available memory allocated by the computing node 510 belongs to, the starting addresses and memory sizes corresponding to each type of memory in the storage node to which the available memory belongs, and the access protocol, port number, and transmission protocol used by the computing node 510 when accessing the storage node 520 to which the available memory belongs.

[0085] The DPU 521 in a storage node 520 usually establishes a communication link with the DPU 513 in multiple computing nodes 510. The storage node 520 allocates different memories to different computing nodes 510, that is, the physical memories allocated to different computing nodes 510 are completely different.

[0086] After the DPU 513 in the computing node 510 establishes a communication link with the DPU 521 in the storage node 520 and the DPU 513 in the computing node 510 receives the memory address allocation information from the storage node 520, the computing node 510 can implement the data reading and writing function in the storage node 520 through the DPU 513. For example, Figure 6 Schematic diagram of data interaction under a remote shuffle architecture provided by an embodiment of the present application. Figure 6 As shown, the remote shuffle architecture includes a computing node cluster and a storage node cluster. The computing node cluster includes n computing nodes, denoted as computing nodes 1 to n, where n is a positive integer. The storage node cluster includes m storage nodes, denoted as storage nodes 1 to m, where m is a positive integer. Each MR process on the computing node is connected to the DPU in the computing node. The DPUs in the n computing nodes and the DPUs in the m storage nodes have established communication links. Each MR process on the computing node interacts with the DPU in the computing node, the DPU in the computing node transmits data with the DPU in the storage node through the communication link, and the DPU in the storage node interacts with the memory on the storage node, so that the MR process on the computing node can write data to the memory on the storage node or read data from the memory on the storage node.

[0087] Combination Figure 4 and Figure 6 By comparison, it can be seen that the remote shuffle architecture implementation in the present application greatly reduces the number of links between computing nodes and storage nodes compared to the remote shuffle architecture implementation in the related art, and can alleviate the problem of excessive memory noise caused by too many links in computing nodes or storage nodes, thereby ensuring the data transmission efficiency between computing nodes and storage nodes, thereby improving data processing efficiency, and improving the reliability and stability of data processing.

[0088] The following is an example of the method flow of the embodiment of the present application.

[0089] The data processing method provided in the embodiment of the present application includes two parts: writing data and reading data. The present application describes the implementation process of writing data and the implementation process of reading data through the following two embodiments. For the sake of example, in the implementation process of writing data ( Figure 7 In the method shown in FIG. 1 , a data processing system including a computing node 1 and a storage node 1 is used as an example for explanation. The computing node 1 includes a DPU 1, the storage node 1 includes a DPU 2, and a communication link 1 is established between the DPU 1 and the DPU 2. Figure 8In the method shown in the figure, a data processing system including a computing node 2 and a storage node 2 is used as an example for description, wherein the computing node 2 includes a DPU 3, the storage node 2 includes a DPU 4, and a communication link 2 is established between DPU 3 and DPU 4. In practical applications, any computing node in the data processing system can have the functions of computing node 1 and / or computing node 2. Similarly, any storage node in the data processing system can have the functions of storage node 1 and / or storage node 2.

[0090] In one embodiment of the present application, the implementation process of a computing node writing data to a storage node is shown. For example, Figure 7 is a flow chart of a data processing method provided in an embodiment of the present application. The method can be applied to Figure 5 The data processing system shown in FIG. 1 can be a computing node 1. Figure 5 Any computing node 510 in the data processing system shown in the figure, the storage node 1 can be as follows Figure 5 Any storage node 520 in the data processing system shown. Figure 7 As shown, the method includes:

[0091] Step 701: Business process 1 sends a memory allocation request to DPU1.

[0092] Business process 1 is a business process running on computing node 1. Business process 1 may be, for example, a map process. Optionally, the memory allocation request includes an indication of a memory size to request DPU1 to allocate a memory space of a corresponding size to business process 1.

[0093] Step 702: DPU1 determines the virtual memory address range allocated to business process 1 in the virtual memory address corresponding to the physical memory allocated by storage node 1 to computing node 1 according to the memory address allocation information.

[0094] The memory address allocation information is used to indicate the virtual memory address corresponding to the physical memory allocated by storage node 1 to computing node 1. In one possible case, the virtual memory address corresponding to the physical memory allocated by storage node 1 to computing node 1 is a continuous virtual memory address segment, and the memory address allocation information may include the starting address and the ending address of the virtual memory address segment, for example, expressed as <starting address, ending address>; or may include the starting address and the address length of the virtual memory address segment, for example, expressed as <starting address, address length>; or may include the ending address and the address length of the virtual memory address segment, for example, expressed as <ending address, address length>. In another possible situation, the virtual memory address corresponding to the physical memory allocated by storage node 1 to computing node 1 includes multiple discontinuous virtual memory address segments. The memory address allocation information may include the starting address and ending address of each virtual memory address segment, for example, expressed as {<starting address 1, ending address 1>; <starting address 2, ending address 2>; ...}; or may include the starting address and address length of each virtual memory address segment, for example, expressed as {<starting address 1, address length 1>; <starting address 2, address length 2>; ...}; or may include the ending address and address length of each virtual memory address segment, for example, expressed as {<ending address 1, address length 1>; <ending address 2, address length 2>; ...}.

[0095] Optionally, if the memory allocation request includes an indication of the memory size, DPU1 may allocate a memory space of a corresponding size to business process 1, and provide business process 1 with a virtual memory address range corresponding to the allocated memory space.

[0096] Step 703: DPU1 sends the virtual memory address range to business process 1.

[0097] Step 704: Business process 1 sends a data storage request to DPU1, where the data storage request includes data 1 and memory address 1.

[0098] Memory address 1 belongs to the virtual memory address range assigned by DPU1 to business process 1. The data storage request is used to request to write data 1 into the physical memory corresponding to memory address 1. Business process 1 sends a data storage request to DPU1, that is, business process 1 initiates the shuffle write process.

[0099] Step 705 : DPU1 sends a write data request to DPU2 via communication link 1 , where the write data request includes data 1 and memory address 1 .

[0100] The write data request is used to request DPU2 to write data 1 into the physical memory corresponding to memory address 1.

[0101] Step 706: DPU2 writes data 1 into the physical memory corresponding to memory address 1.

[0102] After DPU2 receives the write data request from DPU1 through communication link 1, it determines the physical memory address corresponding to memory address 1 based on the memory address 1 carried in the write data request, and then writes the data 1 carried in the write data request into the physical memory corresponding to the physical memory address.

[0103] Optionally, the memory in the storage node 1 includes an SSD, and the DPU2 is connected to the SSD through a memory access interface, and the memory access interface includes a cache and a memory control chip. If the physical memory corresponding to the memory address 1 belongs to the SSD, the implementation process of DPU2 writing data 1 into the physical memory corresponding to the memory address 1 includes: DPU2 first writes data 1 into the cache of the memory access interface, and then the memory control chip writes the data 1 in the cache into the physical space corresponding to the memory address 1 in the SSD. In this way, the DPU in the storage node can write data directly to the SSD through the memory access interface without going through the OS file system, which can improve the disk access efficiency and thus improve the data processing efficiency.

[0104] Optionally, after DPU2 successfully writes data 1 into the physical memory corresponding to memory address 1, it may perform the following step 707. If DPU2 fails to successfully write data 1 into the physical memory corresponding to memory address 1, DPU2 may send a write data failure response to DPU1 through communication link 1 to indicate that data 1 has not been written successfully. Alternatively, DPU2 may not send any response to the write data request to DPU1. After DPU1 fails to receive a response to the write data request from DPU2 through communication link 1 within a certain period of time after sending the write data request, it determines that data 1 has failed to be written. After determining that data 1 has failed to be written, DPU1 may further send a data storage failure response to business process 1 so that business process 1 can rewrite the data.

[0105] Step 707: DPU2 sends a write data response to DPU1 via communication link 1, where the write data response indicates that data 1 has been written successfully.

[0106] Optionally, after DPU1 receives a write data response from DPU2 via communication link 1 indicating that data 1 has been written successfully, the following step 708 may be executed.

[0107] Step 708: DPU1 sends a data storage response to business process 1, where the data storage response indicates that data 1 has been stored successfully.

[0108] Optionally, after receiving the data storage response indicating that the data 1 has been stored successfully, the business process 1 may execute the following step 709 .

[0109] Step 709 : Business process 1 stores metadata 1 corresponding to data 1 in the metadata service device, where metadata 1 includes memory address 1 .

[0110] Optionally, the metadata service device may be a computing node 1, and a storage space dedicated to storing metadata may be allocated in the computing node 1, and the business process 1 may write metadata 1 corresponding to the data 1 in the storage space of the computing node 1. Alternatively, the metadata service device may also be another device independent of the computing node 1 and the storage node 1. After the business process 1 writes the metadata 1 corresponding to the data 1 in the metadata service device, other business processes in the computing node 1 or business processes in other computing nodes may read the data 1 from the storage node 1 by pulling the metadata 1 from the metadata service device.

[0111] In an embodiment of the present application, a DPU is deployed in a computing node and a storage node respectively, and a communication link is established between the DPU in the computing node and the DPU in the storage node. The DPU in the storage node synchronizes memory address allocation information to the DPU in the computing node through the communication link, so that the DPU in the computing node pre-stores the virtual memory address corresponding to the physical memory assigned to the computing node by the storage node. In this way, the computing node can implement remote memory direct mapping through the DPU to achieve direct access to the physical memory of the storage node. In specific implementation, the DPU in the computing node can allocate available memory in the storage node for the business process in the computing node according to the memory address allocation information. For the business process, the business process only needs to interact with the DPU in the computing node to obtain available memory, and can further write data to the available memory in the storage node through the DPU in the computing node. Different business processes in the computing nodes do not need to establish communication links with the storage nodes separately. Only one communication link needs to be established between a single computing node and the storage node through the DPU, which greatly reduces the number of links between the computing nodes and the storage nodes. It can alleviate the problem of excessive memory noise caused by too many links in the computing nodes or storage nodes, thereby ensuring the data transmission efficiency between the computing nodes and the storage nodes, thereby improving the data processing efficiency, and improving the reliability and stability of data processing.

[0112] In another embodiment of the present application, the implementation process of a computing node reading data from a storage node is shown. For example, Figure 8 is a flow chart of another data processing method provided in an embodiment of the present application. The method can be applied to Figure 5 The data processing system shown in FIG. 1 can be a computing node 2. Figure 5 Any computing node 510 in the data processing system shown in the figure, the storage node 2 can be as follows Figure 5 Any storage node 520 in the data processing system shown. Figure 8 As shown, the method includes:

[0113] Step 801, business process 2 sends a data read request to DPU3, and the data read request includes memory address 2.

[0114] Business process 2 is a business process running on computing node 2. Business process 2 may be, for example, a reduce process. Memory address 2 is a virtual memory address corresponding to the physical memory in storage node 2. The data read request is used to request to read data from the physical memory corresponding to memory address 2. For ease of explanation, the data in the physical memory corresponding to memory address 2 is referred to as data 2 in this application.

[0115] Optionally, business process 2 obtains metadata corresponding to the data to be read from the metadata service device, and the metadata includes a memory address corresponding to the physical memory used to store the data to be read. Then business process 2 can send a data read request carrying the memory address corresponding to the data to be read to DPU3. Business process 2 sends a data read request to DPU3, that is, business process 2 initiates a shuffle read process.

[0116] Step 802 , DPU3 sends a read data request to DPU4 via communication link 2 , where the read data request includes memory address 2 .

[0117] The read data request is used to request DPU4 to read data from the physical memory corresponding to memory address 2.

[0118] Optionally, the read data request also includes a cache address of a target cache. The read data request is used to request DPU4 to write the data read from the physical memory corresponding to memory address 2 into the target cache. The target cache may be a high-speed cache in DPU3, which is accessible to the business process in computing node 2. The target cache may be a cache allocated to business process 2.

[0119] Step 803 , DPU4 reads data 2 from the physical memory corresponding to memory address 2.

[0120] After DPU4 receives the read data request from DPU3 through communication link 2, it determines the physical memory address corresponding to memory address 2 according to memory address 2 carried in the read data request, and then reads data from the physical memory corresponding to the physical memory address.

[0121] Optionally, the memory in the storage node 2 includes an SSD, and the DPU4 is connected to the SSD through a memory access interface, and the memory access interface includes a cache and a memory control chip. If the physical memory corresponding to the memory address 2 belongs to the SSD, the implementation process of the DPU4 reading the data 2 from the physical memory corresponding to the memory address 2 includes: the DPU4 sends a read data instruction to the memory control chip, and the read data instruction includes the memory address 2. Then the memory control chip reads the data 2 from the physical space corresponding to the memory address 2 in the SSD, and writes the read data 2 to the cache of the memory access interface. DPU4 then reads the data 2 from the cache. In this way, the DPU in the storage node can read data directly from the SSD through the memory access interface without going through the OS file system, which can improve the disk access efficiency and thus improve the data processing efficiency.

[0122] Optionally, after DPU4 reads data from the physical memory corresponding to memory address 2, it may perform the following step 804. If DPU4 does not read data from the physical memory corresponding to memory address 2, DPU4 may send a data read failure response to DPU3 through communication link 2 to indicate that the data read failed, or DPU4 may not send any response to the data read request to DPU3. After DPU3 does not receive a response to the data read request from DPU4 through communication link 2 within a certain period of time after sending the data read request, it determines that the data read failed. After determining that the data read failed, DPU3 may further send a data read failure response to business process 2 so that business process 2 can reread the data.

[0123] Step 804 , DPU4 sends a read data response to DPU3 via communication link 2 , where the read data response includes data 2 .

[0124] Optionally, when the read data request received by DPU4 carries the cache address of the target cache, the read data response sent by DPU4 for the read data request also includes the cache address of the target cache, and the read data response is used to instruct DPU3 to write data 2 into the target cache.

[0125] Optionally, after DPU3 receives a read data response carrying data 2 from DPU3 via communication link 2, the following steps 805 may be performed.

[0126] Step 805 , DPU3 writes data 2 into the target cache of computing node 2 .

[0127] Optionally, the target cache may be the cache indicated by the cache address carried in the read data response. Alternatively, the target cache may also be any cache in the computing node 2 for access by the service process, such as a high-speed cache in the DPU 3.

[0128] Step 806: Business process 2 reads data 2 from the target cache.

[0129] In an embodiment of the present application, a DPU is deployed in a computing node and a storage node respectively, and a communication link is established between the DPU in the computing node and the DPU in the storage node. The computing node can implement remote memory direct mapping through the DPU to achieve direct access to the physical memory of the storage node. For the business process running on the computing node, when the business process needs to read data from the storage node, the business process only needs to pull data from the corresponding memory in the storage node through the DPU in the computing node. Different business processes in the computing node do not need to establish communication links with the storage node respectively. Only one communication link needs to be established between a single computing node and a storage node through the DPU, which greatly reduces the number of links between the computing node and the storage node, and can alleviate the problem of excessive memory noise caused by too many links in the computing node or storage node, thereby ensuring the data transmission efficiency between the computing node and the storage node, thereby improving the data processing efficiency, and improving the reliability and stability of data processing.

[0130] The order of the steps of the data processing method provided in the embodiment of the present application can be adjusted appropriately, and the steps can be increased or decreased accordingly. Any person skilled in the art who can easily think of a change within the technical scope disclosed in the present application should be included in the protection scope of the present application.

[0131] The following is an example of a virtual device according to an embodiment of the present application.

[0132] Fig. 9 Schematic diagram of a computing node provided in an embodiment of the present application. Fig. 9 As shown, the computing node 900 includes a first DPU 901 and a CPU 902. A communication link is established between the first DPU 901 and a second DPU in the storage node. The first DPU 901 stores memory address allocation information. The memory address allocation information is used to indicate the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node. Fig. 9 As shown, the first DPU 901 includes: a first communication module 9011 , a processing module 9012 , and a second communication module 9013 .

[0133] The first communication module 9011 is used to receive a memory allocation request from a first service process in the CPU.

[0134] The processing module 9012 is used to determine, according to the memory address allocation information, a virtual memory address range allocated to the first business process in the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node.

[0135] The second communication module 9013 is used to send a write data request to the second DPU through the communication link when the first communication module receives a data storage request from the first business process. The data storage request includes the first data and the first memory address, and the first memory address belongs to the virtual memory address range. The write data request includes the first data and the first memory address, and the write data request is used to request the second DPU to write the first data into the physical memory corresponding to the first memory address.

[0136] Optionally, the first communication module 9011 is further configured to send a data storage response to the first business process when the second communication module 9013 receives a write data response from the second DPU through the communication link, and the write data response indicates that the first data has been successfully written, and the data storage response indicates that the first data has been successfully stored. The CPU 902 is configured to store first metadata corresponding to the first data in the metadata service device through the first business process, and the first metadata includes a first memory address.

[0137] Optionally, the second communication module 9013 is further used to send a data read request to the second DPU through the communication link when the first DPU 901 receives a data read request from the second business process in the CPU 902 through the first communication module 9011. The data read request includes a second memory address, and the second memory address is a virtual memory address corresponding to the physical memory in the storage node. The read data request includes the second memory address, and the read data request is used to request the second DPU to read the second data from the physical memory corresponding to the second memory address.

[0138] Optionally, the processing module 9012 is further configured to write the second data into the target cache of the computing node when the second communication module 9013 receives a read data response including the second data from the second DPU via the communication link. The CPU 902 is configured to read the second data from the target cache via the second service process.

[0139] Optionally, the read data request also includes a cache address of the target cache, and the read data request is used to request the second DPU to write the data read from the physical memory corresponding to the second memory address into the target cache. Correspondingly, the read data response also includes the cache address of the target cache, and the read data response is used to instruct the first DPU to write the second data into the target cache.

[0140] Optionally, the first business process is a map process, and the second business process is a reduce process.

[0141] Optionally, the second communication module 9013 is further used to establish a communication link with the second DPU after the computing node is powered on. The second communication module 9013 is further used to receive memory address allocation information sent by the second DPU.

[0142] Optionally, the first DPU includes a first RDMA network card, and the second communication module 9013 is integrated in the first RDMA network card. The second DPU includes a second RDMA network card, and the communication link is an RDMA link established between the first RDMA network card and the second RDMA network card.

[0143] Fig.10 Schematic diagram of a storage node provided by an embodiment of the present application. Fig.10 As shown, the storage node 1000 includes a first DPU 1001 and a memory 1002. A communication link is established between the first DPU 1001 and a second DPU in the computing node. Fig.10 As shown, the first DPU 1001 includes: a communication module 10011 and a processing module 10012 .

[0144] The communication module 10011 is used to receive a write data request from the second DPU through a communication link, where the write data request includes first data and a first memory address, where the first memory address is a virtual memory address corresponding to the physical memory allocated by the storage node to the computing node.

[0145] The processing module 10012 is used to write the first data into the physical memory corresponding to the first memory address.

[0146] The communication module is also used to send a write data response to the second DPU through the communication link, and the write data response indicates that the first data has been written successfully.

[0147] Optionally, the communication module 10011 is further used to receive a read data request from a second DPU via a communication link, the read data request including a second memory address, the second memory address being a virtual memory address corresponding to a physical memory in the storage node. The processing module 10012 is further used to read the second data from the physical memory corresponding to the second memory address. The communication module 10011 is further used to send a read data response to the second DPU via a communication link, the read data response including the second data.

[0148] Optionally, the read data request also includes a cache address of a target cache, and the read data request is used to request the first DPU to write the data read from the physical memory corresponding to the second memory address into the target cache. The read data response also includes a cache address of the target cache, and the read data response is used to instruct the second DPU to write the second data into the target cache.

[0149] Optionally, the processing module 10012 is further used to obtain a virtual memory address corresponding to the physical memory of the storage node. The communication module 10011 is further used to send memory address allocation information to the second DPU after establishing a communication link with the second DPU, where the memory address allocation information includes a virtual memory address corresponding to the physical memory allocated by the storage node to the computing node.

[0150] Optionally, the storage node 1000 includes multiple memories 1002. The processing module 10012 is used to uniformly address the multiple memories to obtain a virtual memory address corresponding to the physical memory of the storage node.

[0151] Optionally, the first DPU includes a first RDMA network card, and the communication module 10011 is integrated in the first RDMA network card. The second DPU includes a second RDMA network card, and the communication link is an RDMA link established between the first RDMA network card and the second RDMA network card.

[0152] Optionally, the memory 1002 includes a solid-state drive. The first DPU 1001 is connected to the solid-state drive through a memory access interface. The memory access interface includes a cache and a memory control chip. The cache is used to store the data to be stored written by the first DPU 1001 to the solid-state drive, and the memory control chip is used to write the data to be stored in the cache to the solid-state drive. The memory control chip is also used to write the data to be read read by the first DPU 1001 from the solid-state drive into the cache, so that the first DPU 1001 can read the data to be read from the cache.

[0153] Optionally, the memory access interface is integrated on the solid state drive.

[0154] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0155] The following is an illustration of the hardware structure involved in the embodiments of the present application.

[0156] For example, Fig.11 is a block diagram of a computing node provided in an embodiment of the present application. Fig.11 As shown, the computing node 1100 includes: a DPU 1101 and a CPU 1102. Optionally, the computing node 1100 also includes a memory 1103.

[0157] The memory 1103 is used to store a computer program, which includes program instructions. For example, the memory 1103 can store an operating system and an application unit required for at least one function. The operating system can be a real-time operating system (RTX), LINUX, UNIX, WINDOWS, or OS X.

[0158] The CPU 1102 is used to call the computer program stored in the memory 1103 to implement the operation of the business process. The business process can be the above Figure 7 The business process 1 or the above method shown Figure 8 Business process 2 in the method shown. Optionally, the business processes running on the CPU 1102 include but are not limited to map processes and reduce processes. The CPU 1102 includes one or more processing cores, and the CPU 1102 executes various functional applications and data processing by running computer programs.

[0159] DPU 1101 is used to call a computer program including program instructions to implement the above Figure 7 The method shown in DPU1 and / or above Figure 8 The actions performed by DPU3 in the method shown. DPU 1101 includes a communication interface 1104. There may be multiple communication interfaces 1104, and the communication interface 1104 is used to communicate with other devices. For example, in the embodiment of the present application, the communication interface 1104 is used to communicate with a storage node or other computing nodes.

[0160] Optionally, the computing node 1100 further includes a communication bus 1105. The DPU 1101 and the CPU 1102, and the CPU 1102 and the memory 1103 are connected via the communication bus 1105, respectively.

[0161] For example, Fig.12 is a block diagram of a storage node provided in an embodiment of the present application. Fig.12 As shown, the storage node 1200 includes: a DPU 1201 and a memory 1202 .

[0162] The memory 1202 is used to store data. Optionally, the memory 1202 includes but is not limited to DRAM, PCM and SSD.

[0163] DPU 1201 is used to call a computer program containing program instructions to implement the above Figure 7 The method shown in DPU2 and / or above Figure 8The actions performed by DPU4 in the method shown. DPU 1201 includes a communication interface 1203. There may be multiple communication interfaces 1203, and the communication interface 1203 is used to communicate with other devices. For example, in the embodiment of the present application, the communication interface 1203 is used to communicate with one or more computing nodes.

[0164] Optionally, the storage node 1200 further includes a communication bus 1204 . The DPU 1201 and the memory 1202 are connected via the communication bus 1204 .

[0165] The system structure of the embodiment of the present application is described below.

[0166] The present application embodiment provides a data processing system. It includes: computing nodes and storage nodes. The computing nodes can be as follows: Fig. 9 or Fig.11 The storage nodes can be Fig.10 or Fig.12 Optionally, the data processing system includes a computing node cluster and a storage node cluster. The computing node cluster includes one or more computing nodes, and the storage node cluster includes one or more storage nodes. The data processing system can be, for example, Figure 5 As shown, the data processing system can be applied to a remote shuffle architecture.

[0167] The computing node includes a first DPU and a CPU. The storage node includes a second DPU and a memory. A communication link is established between the first DPU and the second PDU. The first DPU stores memory address allocation information, which is used to indicate the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node. The physical memory in the storage node belongs to the memory.

[0168] In the process of computing nodes writing data to storage nodes (shuffle writing process):

[0169] The CPU is used to send a memory allocation request from a first business process to the first DPU, and the first business process is any map process running in the CPU. The first DPU is used to determine the virtual memory address range allocated to the first business process in the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node according to the memory address allocation information. The CPU is also used to send a data storage request from the first business process to the first DPU, and the data storage request includes the first data and the first memory address, and the first memory address belongs to the virtual memory address range allocated by the first DPU to the first business process. The first DPU is used to send a write data request to the second DPU through the communication link between the first DPU and the second PDU, and the write data request includes the first data and the first memory address. The second DPU is used to write the first data into the physical memory corresponding to the first memory address. The second DPU is also used to send a write data response to the first DPU through the communication link between the first DPU and the second PDU, and the write data response indicates that the first data has been written successfully.

[0170] In the process of computing nodes reading data from storage nodes (shuffle reading process):

[0171] The CPU is used to send a data read request from a second business process to the first DPU, where the data read request includes a second memory address, where the second memory address is a virtual memory address corresponding to a physical memory in a storage node, and the second business process is any reduce process running in the CPU. The first DPU is used to send a read data request to the second DPU via a communication link between the first DPU and the second PDU, where the read data request includes the second memory address. The second DPU is used to read the second data from the physical memory corresponding to the second memory address. The second DPU is also used to send a read data response to the first DPU via a communication link between the first DPU and the second PDU, where the read data response includes the second data.

[0172] An embodiment of the present application further provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed by a processor, the actions performed by the computing node or the actions performed by the storage node in the above method embodiment are implemented.

[0173] An embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the computer program implements the actions performed by the computing node or the storage node in the above method embodiment.

[0174] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0175] In the embodiments of the present application, the terms “first”, “second” and “third” are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0176] The term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0177] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the memory address allocation information, first data and second data involved in this application are all obtained with full authorization.

[0178] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: A first data processing unit DPU of a computing node receives a memory allocation request from a first service process in the computing node; The first DPU determines, according to the memory address allocation information, a virtual memory address range allocated to the first business process in the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node, wherein the memory address allocation information is used to indicate the virtual memory address corresponding to the physical memory allocated by the storage node to the computing node; When the first DPU receives a data storage request from the first business process, the first DPU sends a write data request to the second DPU of the storage node, the data storage request includes first data and a first memory address, the first memory address belongs to the virtual memory address range, the write data request includes the first data and the first memory address, and the write data request is used to request the second DPU to write the first data into the physical memory corresponding to the first memory address.

2. The method according to claim 1, characterized in that After the first DPU sends a write data request to the second DPU of the storage node, the method further includes: When the first DPU receives a write data response from the second DPU, and the write data response indicates that the first data has been written successfully, the first DPU sends a data storage response to the first service process, and the data storage response indicates that the first data has been stored successfully; The first business process stores first metadata corresponding to the first data in a metadata service device, where the first metadata includes the first memory address.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: When the first DPU receives a data read request from a second business process in the computing node, the first DPU sends a data read request to the second DPU, the data read request includes a second memory address, the second memory address is a virtual memory address corresponding to the physical memory in the storage node, the read data request includes the second memory address, and the read data request is used to request the second DPU to read second data from the physical memory corresponding to the second memory address.

4. The method according to claim 3, characterized in that After the first DPU sends a read data request to the second DPU, the method further includes: When the first DPU receives a read data response including the second data from the second DPU, the first DPU writes the second data into a target cache of the computing node; The second service process reads the second data from the target cache.

5. The method according to claim 4, characterized in that The read data request also includes a cache address of the target cache, and the read data request is used to request the second DPU to write the data read from the physical memory corresponding to the second memory address into the target cache; The read data response also includes a cache address of the target cache, and the read data response is used to instruct the first DPU to write the second data into the target cache.

6. The method according to any one of claims 3 to 5, characterized in that: The first business process is a mapping process, and the second business process is an induction process.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: After the computing node is powered on, the first DPU establishes a communication link with the second DPU; The first DPU receives the memory address allocation information sent by the second DPU.

8. The method according to any one of claims 1 to 7, characterized in that: The first DPU includes a first remote direct memory access RDMA network card, the second DPU includes a second RDMA network card, and the communication link established between the first DPU and the second DPU is an RDMA link established between the first RDMA network card and the second RDMA network card.

9. A data processing method, characterized in that: The method comprises: A first data processing unit DPU of the storage node receives a write data request from a second DPU of the computing node, wherein the write data request includes first data and a first memory address, where the first memory address is a virtual memory address corresponding to a physical memory allocated by the storage node to the computing node; The first DPU writes the first data into the physical memory corresponding to the first memory address; The first DPU sends a write data response to the second DPU, where the write data response indicates that the first data has been written successfully.

10. The method according to claim 9, characterized in that The method further comprises: The first DPU receives a read data request from the second DPU, where the read data request includes a second memory address, where the second memory address is a virtual memory address corresponding to a physical memory in the storage node; The first DPU reads second data from the physical memory corresponding to the second memory address; The first DPU sends a read data response to the second DPU, where the read data response includes the second data.

11. The method according to claim 10, characterized in that The read data request also includes a cache address of a target cache, and the read data request is used to request the first DPU to write data read from the physical memory corresponding to the second memory address into the target cache; The read data response also includes a cache address of the target cache, and the read data response is used to instruct the second DPU to write the second data into the target cache.

12. The method according to any one of claims 9 to 11, characterized in that: The method further comprises: The first DPU obtains a virtual memory address corresponding to the physical memory of the storage node; After the first DPU establishes a communication link with the second DPU, the first DPU sends memory address allocation information to the second DPU, where the memory address allocation information includes a virtual memory address corresponding to the physical memory allocated by the storage node to the computing node.

13. The method according to claim 12, characterized in that The storage node includes a plurality of memories, and the first DPU obtains a virtual memory address corresponding to a physical memory of the storage node, including: The first DPU uniformly addresses the multiple memories to obtain a virtual memory address corresponding to the physical memory of the storage node.

14. The method according to any one of claims 9 to 13, characterized in that: The storage node includes a solid-state hard disk, the first DPU is connected to the solid-state hard disk via a memory access interface, and the memory access interface includes a cache and a memory control chip; The cache is used to store the data to be stored written by the first DPU into the solid state drive, and the memory control chip is used to write the data to be stored in the cache into the solid state drive; The memory control chip is also used to write the to-be-read data read by the first DPU from the solid-state hard disk into the cache, so that the first DPU can read the to-be-read data from the cache.

15. The method according to claim 14, characterized in that The memory access interface is integrated on the solid state drive.

16. A computing node, characterized in that: The computing node includes a first data processing unit DPU and a central processing unit CPU, and the first DPU includes: A first communication module, used to receive a memory allocation request from a first service process in the CPU; a processing module, configured to determine, according to memory address allocation information, a virtual memory address range allocated to the first business process in a virtual memory address corresponding to a physical memory allocated by a storage node to the computing node, wherein the memory address allocation information is used to indicate a virtual memory address corresponding to a physical memory allocated by the storage node to the computing node; A second communication module is used to send a write data request to the second DPU of the storage node when the first communication module receives a data storage request from the first business process, the data storage request includes first data and a first memory address, the first memory address belongs to the virtual memory address range, the write data request includes the first data and the first memory address, and the write data request is used to request the second DPU to write the first data into the physical memory corresponding to the first memory address.

17. The computing node according to claim 16, characterized in that: The first communication module is further configured to send a data storage response to the first service process when the second communication module receives a data write response from the second DPU, and the data write response indicates that the first data has been written successfully, wherein the data storage response indicates that the first data has been stored successfully; The CPU is used to store first metadata corresponding to the first data in a metadata service device through the first business process, where the first metadata includes the first memory address.

18. The computing node according to claim 16 or 17, characterized in that: The second communication module is also used to send a read data request to the second DPU when the first DPU receives a data read request from the second business process in the CPU through the first communication module. The data read request includes a second memory address, and the second memory address is a virtual memory address corresponding to the physical memory in the storage node. The read data request includes the second memory address, and the read data request is used to request the second DPU to read the second data from the physical memory corresponding to the second memory address.

19. The computing node according to claim 18, characterized in that: The processing module is further configured to write the second data into the target cache of the computing node when the second communication module receives a read data response including the second data from the second DPU; The CPU is configured to read the second data from the target cache through the second service process.

20. The computing node according to any one of claims 16 to 19, characterized in that: The second communication module is further used to establish a communication link with the second DPU after the computing node is powered on; The second communication module is also used to receive the memory address allocation information sent by the second DPU.

21. A storage node, characterized in that: The storage node includes a first data processing unit DPU, and the first DPU includes: A communication module, configured to receive a write data request from a second DPU of a computing node, wherein the write data request includes first data and a first memory address, where the first memory address is a virtual memory address corresponding to a physical memory allocated by the storage node to the computing node; A processing module, configured to write the first data into a physical memory corresponding to the first memory address; The communication module is further used to send a write data response to the second DPU, where the write data response indicates that the first data has been written successfully.

22. The storage node according to claim 21, characterized in that: The communication module is further used to receive a read data request from the second DPU, where the read data request includes a second memory address, where the second memory address is a virtual memory address corresponding to the physical memory in the storage node; The processing module is further used to read second data from the physical memory corresponding to the second memory address; The communication module is further used to send a read data response to the second DPU, where the read data response includes the second data.

23. The storage node according to claim 21 or 22, characterized in that: The processing module is further used to obtain a virtual memory address corresponding to the physical memory of the storage node; The communication module is further used to send memory address allocation information to the second DPU after establishing a communication link with the second DPU, where the memory address allocation information includes a virtual memory address corresponding to the physical memory allocated by the storage node to the computing node.

24. The storage node according to claim 23, characterized in that: The storage node includes a plurality of memories. The processing module is used to uniformly address the multiple memories to obtain the virtual memory address corresponding to the physical memory of the storage node.

25. The storage node according to any one of claims 21 to 24, characterized in that: The storage node includes a solid-state hard disk, the first DPU is connected to the solid-state hard disk via a memory access interface, and the memory access interface includes a cache and a memory control chip; The cache is used to store the data to be stored written by the first DPU into the solid state drive, and the memory control chip is used to write the data to be stored in the cache into the solid state drive; The memory control chip is also used to write the to-be-read data read by the first DPU from the solid-state hard disk into the cache, so that the first DPU can read the to-be-read data from the cache.

26. A computing node, characterized in that: include: Data processing unit DPU and central processing unit CPU; The CPU is used to call a computer program including program instructions to implement the operation of the business process; The DPU is used to call a computer program containing program instructions to implement the actions performed by the first DPU in the data processing method according to any one of claims 1 to 8.

27. A storage node, characterized in that: include: Data processing unit DPU and memory; The DPU is used to call a computer program containing program instructions, read data from the memory and / or write data to the memory, so as to implement the data processing method as described in any one of claims 9 to 15.

28. A data processing system, characterized in that: include: A computing node and a storage node, wherein the computing node includes a first data processing unit DPU and a central processing unit CPU, and the storage node includes a second DPU and a memory, wherein the first DPU stores memory address allocation information, and the memory address allocation information is used to indicate a virtual memory address corresponding to a physical memory allocated by the storage node to the computing node, and the physical memory in the storage node belongs to the memory; The CPU is used to send a memory allocation request from a first service process to the first DPU, where the first service process is any mapping process running in the CPU; The first DPU is used to determine, according to the memory address allocation information, a virtual memory address range allocated to the first business process in a virtual memory address corresponding to the physical memory allocated by the storage node to the computing node; The CPU is further configured to send a data storage request from the first service process to the first DPU, where the data storage request includes first data and a first memory address, and the first memory address belongs to the virtual memory address range; The first DPU is used to send a write data request to the second DPU, where the write data request includes the first data and the first memory address; The second DPU is used to write the first data into the physical memory corresponding to the first memory address; The second DPU is further used to send a write data response to the first DPU, where the write data response indicates that the first data has been written successfully.

29. The system according to claim 28, characterized in that The CPU is further used to send a data read request from a second business process to the first DPU, where the data read request includes a second memory address, where the second memory address is a virtual memory address corresponding to the physical memory in the storage node, and the second business process is any induction process running in the CPU; The first DPU is used to send a read data request to the second DPU, where the read data request includes the second memory address; The second DPU is used to read second data from the physical memory corresponding to the second memory address; The second DPU is further configured to send a read data response to the first DPU, where the read data response includes the second data.

30. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by the processor, the data processing method according to any one of claims 1 to 15 is implemented.

Images