A data processing method and a cluster of computing devices

Abstract

This application discloses a data processing method and a computing device cluster, relating to the field of computer technology. The method includes: a destination device receiving a first transmission request from a source device, the first transmission request including a first command and first data; the destination device writing the first command into a first memory area, the first memory area being a region for storing the command; the destination device, upon determining that it has received the first data, allocating a second memory area for the first data, the second memory area being a region for storing the data; and the destination device, upon successfully allocating the second memory area, writing the first data into the second memory area according to the first command. The destination device can pre-allocate the first memory area for storing the command and in real-time allocate the second memory area for storing the data, thereby ensuring that the destination device can receive commands from the source device at any time while reducing memory overhead.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method and a cluster of computing devices. Background Technology

[0002] Remote direct memory access (RDMA) is a technology that allows a source device to access data in the memory of a destination device by bypassing the processor of the destination device. In other words, it can transfer data directly from the memory of one device to the memory of another device without the intervention of the processors of both devices.

[0003] Currently, when a source device and a destination device establish an RDMA connection, they can communicate through queue pairs (QPs). A QP can include a send queue (SQ), a receive queue (RQ), and a completion queue (CQ). An RQ can include one or more receive queue elements (RQEs), and an RQE can include one or more scatter-gather elements (SGEs). An SGE can be used to indicate the address, length, local key (lkey), and other information of a memory region, so that commands and / or data received by the destination device can be correctly stored in the destination device's memory.

[0004] To ensure that the destination device can receive commands and data from the source device at any time, the destination device's memory needs to have sufficient RQE and memory areas. As the number of source devices communicating with the destination device via QP increases, the amount of RQE and memory areas required by the destination device also increases. Therefore, in large-scale communication scenarios, the memory overhead for communication between source and destination devices via QP is significant. Summary of the Invention

[0005] This application provides a data processing method and a computing device cluster to reduce the memory overhead of communication between the source device and the destination device via QP.

[0006] In a first aspect, embodiments of this application provide a data processing method. This method is applied to a destination device (or components within the destination device, such as a processor, network interface card, or memory). The destination device and the source device can be computing devices (e.g., servers, desktop computers, laptops, and mobile devices), storage devices (e.g., storage arrays), or other devices supporting RDMA. The method includes: receiving a first transmission request from the source device, the first transmission request including a first command and first data; writing the first command into a first memory area, the first memory area being an area for storing the command; if it is determined that the first data has been received, allocating a second memory area for the first data, the second memory area being an area for storing the data; and if the allocation of the second memory area is successful, writing the first data into the second memory area according to the first command.

[0007] In this embodiment, the destination device may request a first memory area for storing commands in advance, but not a second memory area for storing data. Instead, after determining that the first data from the source device has been received, the destination device requests a second memory area for storing the first data in real time. This reduces memory overhead while ensuring that the destination device can receive commands from the source device at any time.

[0008] In one possible implementation, if the request for the second memory area fails, the first data is written to a third memory area or the first data is discarded, the third memory area being a region used to store erroneous data.

[0009] In this implementation, if the destination device fails to request a second memory area for storing data in real time, it indicates that the destination device may not have enough memory resources. Therefore, in order to avoid the destination device replying to the source device with "receiver not ready" (RNR), which would cause the communication connection between the source device and the destination device to be broken and the first data sent by the source device to be unable to be received by the destination device, thus requiring the re-establishment of the RDMA connection between the source device and the destination device, the destination device can either write the first data to a third memory area for storing erroneous data or discard the first data.

[0010] In one possible implementation, allocating a second memory area for the first data includes: determining a first receive queue element in a first receive queue corresponding to the first data, the first receive queue element being used to indicate a virtual address; and allocating a second memory area for the first data.

[0011] In this implementation, after the destination device determines that it has received the first data from the source device, it can use the address indicated by the first receive queue element corresponding to the first data to determine whether it needs to request a second memory area for storing the first data in real time. For example, if the first receive queue element corresponding to the first data indicates a virtual address but not an address for a memory area for storing the data, it means that the destination device has not requested a memory area for storing the data in advance. Therefore, after receiving the first data from the source device, the destination device needs to request a second memory area for storing the first data in real time, thereby improving memory management efficiency.

[0012] In one possible implementation, the method further includes: when the first data is written to the second memory area, updating the virtual address indicated by the first receive queue element to the address of the second memory area; or, when the first data is written to the third memory area or the first data is discarded, updating the virtual address indicated by the first receive queue element to the address of the third memory area or an invalid address, wherein the third memory area is an area used to store erroneous data.

[0013] In this implementation, after the destination device requests a second memory area for storing the first data in real time, it can update the address indicated by the first receiving queue element corresponding to the first data according to the request result. If the request is successful, the virtual address indicated by the first receiving queue element is updated to the address of the second memory area for storing the data. If the request fails, the virtual address indicated by the first receiving queue element is updated to the address of the third memory area for storing erroneous data or an invalid address, thereby improving memory management efficiency.

[0014] In one possible implementation, the method further includes: releasing the second memory area when the first data is written to the second memory area; or, sending a first transmission response to the source device when the first data is written to the third memory area or the first data is discarded, the first transmission response indicating that the first data was not successfully received and the reason for the failure to receive the first data.

[0015] In this implementation, if the destination device has successfully received the first data, it can release the second memory area storing the first data, thereby improving memory utilization. Alternatively, if the destination device has failed to receive the first data, it can indicate to the source device the reason for the failure, so that the source device can process the first data according to the reason, such as retransmitting the first data or handling errors in the first data, thereby ensuring communication reliability as much as possible.

[0016] In one possible implementation, the method further includes: receiving a second transmission request from the source device, the second transmission request including a second command and second data; writing the second command into a fourth memory area, the fourth memory area being an area for storing commands; and, if it is determined that the second data has been received, writing the second data into a third memory area or discarding the second data, the third memory area being an area for storing error data.

[0017] In this implementation, the destination device can pre-allocate memory areas for storing commands and memory areas for storing data. If pre-allocating memory areas for storing data fails, it indicates that the destination device may not have sufficient memory resources. To avoid the destination device responding with an RNR from the source device, which would cause the communication connection between the source and destination devices to be broken and thus require re-establishing the RDMA connection, the second data sent by the source device cannot be received by the destination device. The destination device can either write the second data to a third memory area used to store erroneous data or discard the second data.

[0018] In one possible implementation, writing the second data to a third memory area or discarding the second data includes: determining that the second data corresponds to a second receive queue element in a first receive queue, the second receive queue element being used to indicate the address of the third memory area or an invalid address; writing the second data to the third memory area or discarding the second data.

[0019] In this implementation, after the destination device determines that it has received the second data from the source device, it can use the address indicated by the second receive queue element corresponding to the second data to determine whether it needs to request a second memory area for storing the second data in real time. For example, if the second receive queue element corresponding to the second data indicates the address of a third memory area for storing erroneous data or an invalid address, but does not indicate the address of a memory area for storing data, it means that the destination device has already requested a memory area for storing data but the request failed. Therefore, after receiving the second data from the source device, the destination device does not need to request a second memory area for storing the second data in real time, but instead writes the second data into the third memory area for storing erroneous data or discards the second data, thereby improving memory management efficiency.

[0020] Secondly, this application also provides a data processing apparatus having the function of implementing the methods described in the first aspect and various possible implementations of the first aspect. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions. In one possible implementation, the apparatus includes a transmitting module, a receiving module, and a processing module. These modules can implement the functions of the method examples described in the first aspect and various possible implementations of the first aspect, as detailed in the method examples, which will not be repeated here.

[0021] Thirdly, this application also provides a computing device cluster, including at least one computing device, each computing device comprising a memory and a processor. The memory is used to store program instructions and / or data. The memory is coupled to the processor, and when the processor executes the program instructions stored in the memory, it can implement the methods described in the first aspect and various possible implementations of the first aspect. Each computing device also includes a communication interface for communicating with other computing devices, such as receiving first data.

[0022] Fourthly, this application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect and various possible implementations of the first aspect.

[0023] Fifthly, this application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the first aspect and various possible implementations of the first aspect.

[0024] In a sixth aspect, this application also provides a computing chip connected to a memory, the chip being used to read and execute software programs stored in the memory, and to execute the methods described in the first aspect and various possible implementations of the first aspect.

[0025] For the description of the beneficial effects of any of the second to sixth aspects above, please refer to the description of the technical effects in any possible implementation of the first aspect above, and this application will not repeat it here. Attached Figure Description

[0026] Figure 1 A schematic diagram of the structure of a system provided in this application embodiment;

[0027] Figure 2 This is a schematic diagram of the structure of a device provided in an embodiment of this application;

[0028] Figure 3AThis is a schematic diagram illustrating a communication process between a source device and a destination device via QP, provided as an embodiment of this application.

[0029] Figure 3B This is a schematic diagram illustrating a process where multiple source devices and one destination device communicate via QP, as provided in this embodiment.

[0030] Figure 3C This is a schematic diagram of the command memory area and data memory area in the memory of a destination device provided in this embodiment;

[0031] Figure 3D This embodiment provides a schematic diagram of a process for transmitting commands and data between a source device and a destination device through work requests stored in a QP.

[0032] Figure 3E This is a schematic diagram of a shared receiving queue provided in this embodiment;

[0033] Figure 4 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0034] Figure 5A This is a schematic diagram of a first receiving queue provided in this embodiment;

[0035] Figure 5B This is a flowchart illustrating another data processing method provided in this embodiment;

[0036] Figure 5C This is a flowchart illustrating another data processing method provided in this embodiment;

[0037] Figure 5D This is a flowchart illustrating another data processing method provided in this embodiment;

[0038] Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application.

[0042] Figure 1 This is a schematic diagram of the structure of a system provided in an embodiment of this application. For example... Figure 1 As shown, the system may include devices 11-14 and network 15, and devices 11-14 can communicate with each other through network 15.

[0043] Among them, devices 11-14 can be computing devices (e.g., servers, desktop computers, laptops, and mobile devices), storage devices (e.g., storage arrays), or other devices that support remote direct memory access (RDMA).

[0044] Network 15 may include one or more switches and / or routers. Network 15 may enable communication connections between devices 11-14 through protocols that support RDMA, such as the Infiniband (IB) protocol, RDMA over converged Ethernet (RoCE) protocol, and Internet Wide Area RDMA protocol (iWARP).

[0045] Understandable. Figure 1 The number of devices in the system shown does not constitute a limitation on this application. Figure 1 This explanation uses a system consisting of four devices as an example. For ease of description, Figure 1 The system shown can be categorized as follows: the device sending the work request (WR) is called the source device, and the device receiving the work request is called the destination device. Work requests include read requests, write requests, send requests, and receive requests.

[0046] Figure 2 This is a schematic diagram of the structure of a device provided in this embodiment. Figure 2 As shown, the device may include a processor, memory, and a network interface card (NIC). It can be understood that the device may be... Figure 1 Any one of the devices.

[0047] In this system, the processor, memory, and network interface card (NIC) are connected via a bus. For example, the Peripheral Component Interconnect Express (PCIe) standard. In addition to the data bus, the bus can also include power buses, control buses, and status signal buses.

[0048] The processor can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), etc. For ease of description, the following embodiments use a CPU as an example.

[0049] A processor can run one or more business applications, for example, Figure 2 The diagram shows Business Application 1 and Business Application 2. These business applications can include database applications, high-performance computing (HPC) applications, etc. Business applications can generate work requests. Optionally, business applications can generate work requests by calling interfaces provided by the middleware layer. For example, in a high-performance computing scenario, a business application can generate different parallel computing requests, such as non-blocking communication requests and set operation requests, by calling the application programming interface (API) provided by the message passing interface (MPI) for various parallel computing technologies. As another example, in a storage scenario, a business application can generate data transmission requests by calling the exploitation network (xNet) communication component.

[0050] Memory can include read-only memory and random access memory (RAM), providing instructions and data to the processor. Memory can be volatile or non-volatile, or a combination of both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0051] Memory can store work requests generated by business applications, as well as commands and data transmitted by these applications. Optionally, memory can utilize interconnect context memory (ICM) to store work requests generated by business applications. Interconnect context memory can be a storage area within memory. Interconnect context memory can store queue pairs (QPs) (e.g., ...). Figure 2 The queue pair shown (queue pair 1 and queue pair 2) can be used to store work requests generated by business applications. Additionally, the interconnect context memory can also store queue pair contexts (QPCs) (e.g., Figure 2 The queue pair context 1 and queue pair context 2 shown) and the memory key context (MKC) (e.g., Figure 2 The memory key context shown is 1).

[0052] When a communication connection (such as an RDMA connection) is established between the source device and the destination device, the processor of the source device allocates one or more QPs in the memory of the source device for each service application running on the processor, and the processor of the destination device allocates one or more QPs in the memory of the source device for each service application running on the processor.

[0053] Each QP can include a send queue (SQ), a receive queue (RQ), and a completion queue (CQ).

[0054] SQ is used to store work requests sent by business applications running on the processor of the source device to the destination device. For example, an SQ may include one or more work queue entries (WQEs). Each WQE is used to store one work request sent by a business application to the destination device on the processor of the source device. WQEs in an SQ can also be called send queue elements (SQEs) or S-WQEs.

[0055] RQ is used to store work requests sent by the destination device to a service application running on the processor of the source device. For example, an RQ may include one or more WQEs, each WQE storing a work request sent by the destination device to a service application running on the processor of the source device. The WQE in the RQ can also be called a receive queue element (RQE) or R-WQE.

[0056] A Completion Queue (CQ) is used to store notifications returned by the network interface card (NIC) of the source device to the application running on the processor of the source device after completing a work request. These notifications include flags indicating whether the work request was executed correctly and error codes. Alternatively, it can store notifications returned by the NIC of the destination device to the application running on the processor of the destination device after completing a work request. For example, a CQ may include one or more completion queue elements (WQEs). Each WQE stores completion information for a work request sent from the application running on the processor of the source device to the destination device, or for a work request sent from the destination device to the application running on the processor of the source device. WQEs in a CQ can also be called completion queue elements (CQEs) or C-WQEs.

[0057] A network interface card (NIC) can also be called a host channel adapter (HCA). NICs can be RDMA-enabled. For example, the NIC of the source device can execute a send request, sending data stored in the source device's memory to the NIC of the destination device over the network. The NIC of the destination device can execute a receive request, receiving the same data from the source device's NIC over the network and storing it in the destination device's memory.

[0058] The above briefly introduces the systems and devices applicable to the embodiments of this application. The following describes the relevant technical solutions involved in the embodiments of this application.

[0059] 1) RDMA

[0060] RDMA is a technique where a source device bypasses the processor of the destination device to access data in the destination device's memory. In other words, it can directly transfer data from the memory of one device to the memory of another without the intervention of the processors of either device. RDMA supports two types of transfer primitives: one-sided and two-sided. One-sided primitives allow the source device to directly manipulate the memory of the destination device without the active participation of the destination device, reducing communication latency and improving communication efficiency. Examples of one-sided primitives are read and write operations. Two-sided primitives require cooperation between the source and destination devices. The source device initiates a send request, and the destination device needs to prepare the memory area in advance. Examples of two-sided primitives are send and receive operations.

[0061] When the source device and the destination device establish an RDMA connection, the source device and the destination device can communicate through QP (including SQ, RQ and CQ).

[0062] For example, Figure 3A This embodiment provides a schematic diagram of a communication process between a source device and a destination device via QP. For example... Figure 3AAs shown, the source and destination devices can communicate via QP1 and QP2. QP1 includes SQ1, RQ1, and CQ1, while QP2 includes SQ2, RQ2, and CQ2. When SQ1 is associated with RQ2, the service application running on the source device's processor can generate a send request. The source device's processor stores this send request in an SQE of SQ1 within the source device's memory, and stores the completion information of the send request in a CQE of CQ1 within the source device's memory. Similarly, the service application running on the destination device's processor can generate a receive request. The destination device's processor stores this receive request in an RQE of RQ2 within the destination device's memory, and stores the completion information of the receive request in a CQE of CQ2 within the destination device's memory.

[0063] Furthermore, head and tail pointers can be used to indicate the storage status of SQE in SQ, RQE in RQ, and CQE in CQ. For example, as Figure 3A As shown, in the source device's memory, the head pointer of SQ1 points to the address of SQE12, and the tail pointer points to the address of SQE11. At this time, only SQE12 in the source device's memory contains the work request. When a service application running on the source device's processor generates a send request, the source device's processor stores the send request in SQE13 of SQ1 in the source device's memory. At this time, the head pointer of SQ1 in the source device's memory will move down one storage unit to point to SQE13. When the source device's network card retrieves the work request stored in SQE12 from SQ1 in the source device's memory, the tail pointer of SQ1 will move down one storage unit to point to SQE12.

[0064] It is understood that multiple source devices can communicate with one destination device through QP, or one source device can communicate with multiple destination devices through QP, or multiple source devices can communicate with multiple destination devices through QP. This application does not limit this.

[0065] For example, Figure 3B This is a schematic diagram illustrating a communication process between multiple source devices and one destination device via QP, as provided in this embodiment. Figure 3BAs shown, devices 1, 2, and 3 are source devices, and device 4 is the destination device. Devices 1 and 4 can communicate via QP11 and QP12. QP11 includes SQ11, RQ11, and CQ11, and QP12 includes SQ12, RQ12, and CQ12. Devices 2 and 4 can communicate via QP21 and QP22. QP21 includes SQ21, RQ21, and CQ21, and QP22 includes SQ22, RQ22, and CQ22. Devices 3 and 4 can communicate via QP31 and QP32. QP31 includes SQ31, RQ31, and CQ31, and QP32 includes SQ32, RQ32, and CQ32.

[0066] It is understandable that source and destination devices can transmit commands and data through work requests (including read requests, write requests, send requests, and receive requests) stored in QP. Specifically, source and destination devices can store commands and data in different memory areas. The memory area used to store commands in both the source and destination devices can be called the command memory area, and the memory area used to store data in both devices can be called the data memory area.

[0067] For example, Figure 3C This is a schematic diagram of the command memory area and data memory area in the memory of a destination device provided in this embodiment. Figure 3C As shown, the RQ in the memory of the destination device can include multiple RQEs, such as RQE1-RQEn. Each RQE can include multiple scatter-gather elements (SGEs), and each SGE can indicate the address, size, local key (lkey), and other information of a memory area to ensure that the commands and / or data received by the destination device are correctly stored in the memory of the destination device. For example, RQE1 includes SGE1-SGE4. SGE1 indicates the address 1, size 1, and local key 1 of memory area 1; SGE2 indicates the address 2, size 2, and local key 2 of memory area 2; SGE3 indicates the address 3, size 3, and local key 3 of memory area 3; and SGE4 indicates the address 4, size 4, and local key 4 of memory area 4. Memory area 1 is the command memory area used to store commands, and memory areas 2-4 are data memory areas used to store data.

[0068] It is understandable that the destination device can indicate the size of the memory area indicated by the RQE in the destination device's memory to the source device. The source device can then determine whether the size of the transmitted command and data is smaller than the size of the memory area indicated by the RQE in the destination device. If it is smaller, the source device can transmit the command and data together to the destination device through bilateral primitives, thereby reducing communication latency. If it is greater than or equal to the size of the RQE, the source device can first transmit the command to the destination device through bilateral primitives, and then transmit the data to the destination device through unilateral primitives.

[0069] For example, Figure 3D This is a schematic diagram of the process by which a source device and a destination device transmit commands and data through work requests stored in a QP, as provided in this embodiment. As shown in 3D (1), the source device sends commands and data together to the destination device through bilateral primitives.

[0070] Specifically, the application running on the processor of the source device can generate a send request. The processor of the source device can store this send request in the SQE of the SQ (Short Query) in the memory of the source device. The network interface card (NIC) of the source device can execute the send request and send the commands and data stored in the memory of the source device to the NIC of the destination device. After executing the send request, the NIC of the source device can also send the completion information of the send request to the application running on the processor of the source device. The processor of the source device can also store the completion information of the send request in the CQE of the CQ (Cross Query) in the memory of the source device. The processor of the source device can also receive a response from the processor of the destination device, which indicates the processing result (e.g., success or failure) of the commands and data from the source device.

[0071] A service application running on the processor of the destination device can generate a receive request. The processor of the destination device can store this receive request in the RQE of the RQ in the destination device's memory. The network interface card (NIC) of the destination device can execute the receive request to receive commands and data from the source device and store the commands and data in the destination device's memory. After the network interface card of the destination device completes the receive request, the processor of the destination device can also store the completion information of the receive request in the CQE of the CQ in the destination device's memory. The processor of the destination device can also send a response to the processor of the source device, which indicates the result of the destination device's processing of the commands and data from the source device (e.g., success, failure).

[0072] As shown in 3D (2), the source device first sends the command to the destination device through bilateral primitives, and then moves the data to the destination device through unilateral primitives.

[0073] Specifically, the application running on the processor of the source device can generate send and write requests. The processor of the source device can store these send and write requests in the SQE of the SQ in the memory of the source device. The network interface card (NIC) of the source device can execute the send request to send the commands stored in the memory of the source device to the NIC of the destination device, and execute the write request to move the data stored in the memory of the source device to the memory of the destination device. After executing the send and write requests, or after executing the send and write requests and receiving a response from the destination device, the NIC of the source device can also send completion information of the send and write requests to the application running on the processor of the source device. The processor of the source device can also store the completion information of the send and write requests in the CQE of the CQ in the memory of the source device. The processor of the source device can also receive a response from the processor of the destination device, which indicates the processing result (e.g., success or failure) of the commands and data sent by the source device.

[0074] A service application running on the processor of the destination device can generate a receive request. The processor of the destination device can store this receive request in the RQE of the RQ in the destination device's memory. The network interface card (NIC) of the destination device can receive commands from the source device by executing the receive request and store the commands in the destination device's memory. After executing the receive request, the NIC of the destination device can also send completion information of the receive request to the service application running on the processor of the destination device. The processor of the destination device can also store the completion information of the receive request in the CQE of the CQ in the destination device's memory. The processor of the destination device can also send a response to the processor of the source device, which indicates the result (e.g., success, failure) of the processing of commands and data from the source device by the destination device.

[0075] Understandably, to ensure the destination device can receive commands and data from the source device at any time, the destination device's memory needs to prepare sufficient RQE and memory areas. As the number of source devices communicating with the destination device via QP increases, the amount of RQE and memory areas required by the destination device also increases. In other words, in large-scale communication scenarios, if an RDMA connection is established between the source and destination devices, the memory overhead for communication via QP between the source and destination devices is significant.

[0076] 2) Shared Receive Queue (SRQ)

[0077] SRQ is an optimization mechanism in RDMA that allows multiple QPs to share a common RQ instead of each QP having its own independent RQ, thereby reducing memory overhead and improving resource utilization in many-to-one or many-to-many communication scenarios.

[0078] For example, Figure 3E This is a schematic diagram of an SRQ provided in this embodiment. For example... Figure 3E As shown, devices 1, 2, and 3 are source devices, and device 4 is the destination device. Devices 1 and 4 can communicate via QP11 and QP12, devices 2 and 4 can communicate via QP21 and QP22, and devices 3 and 4 can communicate via QP31 and QP32. Specifically, QP11 includes SQ11, RQ11, and CQ11; QP12 includes SQ12, SRQ, and CQ12; QP21 includes SQ21, RQ21, and CQ21; QP22 includes SQ22, SRQ, and CQ22; QP31 includes SQ31, RQ31, and CQ31; and QP32 includes SQ32, SRQ, and CQ32. In other words, QP12, QP22 and QP32 share SRQ. When commands and data from network cards of device 1, device 2 or device 3 arrive at network card of device 4, network card of device 4 can obtain RQE from SRQ and write the commands and data to the memory area indicated by RQE.

[0079] It is understandable that when multiple business applications share a single SRQ, the data transmitted by different business applications may interfere with each other, leading to performance degradation and quality of service (QoS) issues.

[0080] Currently, when source and destination devices communicate via QPs, whether multiple QPs share a single SRQ or each QP has its own dedicated RQ, several problems exist in practical applications. For example, the ratio of commands to data transmitted between the source and destination devices is uncertain. If the destination device's memory pre-prepares the command and data memory areas indicated by the RQE in the RQ or SRQ, the data memory area in the destination device's memory will be idle when only commands are transmitted, resulting in wasted memory resources. If the destination device's memory only pre-prepares the command memory area indicated by the RQE in the RQ or SRQ, when the source and destination devices transmit commands and data, the source device needs to first send commands to the destination device using bilateral primitives and then use unilateral primitives to move data to the destination device, increasing the round-trip time (RTT) and affecting communication latency. Furthermore, the number of RQEs and their indicated memory areas prepared in the destination device's memory cannot be too small. If the number of RQEs and the memory areas they indicate is too small, when the source device transmits commands and data to the destination device, the destination device may fail to successfully acquire the RQEs and the memory areas they indicate. In this case, the destination device will reply to the source device that the receiver is not ready (RNR), causing the RDMA connection between the source device and the destination device to be disconnected. The commands and data sent by the source device may not be received by the destination device, which will then require the re-establishment of the RDMA connection between the source device and the destination device.

[0081] In view of this, embodiments of this application provide a data processing method for reducing the memory overhead of communication between the source device and the destination device via QP.

[0082] In the embodiments of this application, "when," "if," and "if" all refer to the device taking corresponding actions under certain objective circumstances, and are not time-limited, nor do they require the device to perform a judgment action, nor do they imply any other limitations. Unless otherwise specified, "if" and "if" can be substituted, and "when" and "in the case of" can be substituted. "When" and "if" / "if" can be substituted.

[0083] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0084] In this application embodiment, the number of nouns, unless otherwise specified, refers to "singular nouns or plural nouns," that is, "one or more." "At least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " can indicate that the related objects before and after are in an "or" relationship. For example, A / B means: A or B. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c means: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0085] In this application, the ordinal numbers such as "first" and "second" are used to distinguish multiple objects, and are not used to limit the size, content, order, timing, priority, or importance of the multiple objects. For a technical feature, the technical features within that technical feature are distinguished by "A", "B", "C", and "D", and there is no sequential or size order among the technical features described by "A", "B", "C", and "D".

[0086] The technical solutions provided by the embodiments of this application are described in detail below with reference to the accompanying drawings. For example, Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. This data processing method can be applied to... Figures 1-2 The shown or the Figures 1-2 Systems and devices with similar functional structures are not specifically limited in the embodiments of this application. The specific flow of the data processing method is described below.

[0087] S401. The source device sends a first transmission request to the destination device, and the destination device receives the first transmission request from the source device, wherein the first transmission request includes a first command and first data.

[0088] In this embodiment, the first command can be a command stored in the memory of the source device, such as a send command, write command, read command, etc. The first data can be data stored in the memory of the source device.

[0089] In specific implementation, the network interface card (NIC) of the source device and the NIC of the destination device can communicate through a first queue pair and a second queue pair. It is understood that an RDMA connection can be established between the destination device and the source device, or other connections can be established that communicate through queue pairs. This application embodiment does not limit this; for ease of explanation, this application embodiment takes an RDMA connection between the destination device and the source device as an example.

[0090] The first queue pair can be a queue pair stored in the memory of the destination device. The first queue pair may include a first sending queue, a first receiving queue, and a first completion queue. The first sending queue may include one or more sending queue elements, the first receiving queue may include one or more receiving queue elements, and the first completion queue may include one or more completion queue elements. It is understood that the first receiving queue may be an independent receiving queue or a shared receiving queue; this embodiment of the application does not limit this.

[0091] The second queue pair can be a queue pair stored in the memory of the source device. The second queue pair may include a second transmit queue, a second receive queue, and a second complete queue. The second transmit queue may include one or more transmit queue elements, the second receive queue may include one or more receive queue elements, and the second complete queue may include one or more complete queue elements. It is understood that the second receive queue may be an independent receive queue or a shared receive queue; this embodiment of the application does not limit this.

[0092] For example, a service application running on the processor of the source device can generate a first send request. The processor of the source device can store the first send request in the first send queue element of the second send queue in the second queue pair in the memory of the source device. The first send request can request the network interface card (NIC) of the source device to send commands and data stored in the memory of the source device to the NIC of the destination device. The first send request can include a first command and first data. The first send queue element can be an unused send queue element from one or more send queue elements included in the second send queue. The NIC of the source device can retrieve and execute the first send request from the first send queue element, sending the first send request to the NIC of the destination device, that is, sending the first command and first data to the NIC of the destination device.

[0093] Correspondingly, the service application running on the processor of the destination device can generate a first receive request. The processor of the destination device can store the first receive request in the first receive queue element of the first receive queue in the first queue pair in the memory of the destination device. The first receive request can be used to request the network interface card (NIC) of the destination device to receive commands and data from the NIC of the source device. The first receive queue element can be one or more unused receive queue elements included in the first receive queue. The NIC of the destination device can retrieve and execute the first receive request from the first receive queue element, receiving the first send request from the NIC of the source device, that is, receiving the first command and first data from the NIC of the source device.

[0094] S402. The destination device writes the first command into the first memory area, where the first memory area is the area used to store the command.

[0095] In this embodiment, when the network interface card (NIC) of the destination device determines that it has received a first command from the NIC of the source device, the NIC of the destination device can determine that the first command corresponds to a first receive queue element in the first receive queue. When the NIC of the destination device determines that the first receive queue element indicates the address of a first memory area used to store the command, the NIC of the destination device can write the first command into the first memory area used to store the command.

[0096] In other words, before the network interface card (NIC) of the destination device receives the first command from the NIC of the source device, the application running on the processor of the destination device can request a first memory area from the memory of the destination device to store the command. If the request for the first memory area to store the command is successful, the processor of the destination device can fill the address of the first memory area to store the command into the first receive queue element in the first receive queue, thereby ensuring that the NIC of the destination device can receive commands from the NIC of the source device at any time.

[0097] It is understood that the first receiving queue element may include one or more scattered aggregation elements.

[0098] A scatter aggregate element can indicate the address, size, local key, and other information of a memory region. This memory region can be a usable memory region for storing commands, a usable memory region for storing data, a usable memory region for storing error data, or an unusable memory region. Error data can be understood as data that needs to be discarded or data that cannot be processed.

[0099] In other words, a scatter aggregate element can indicate a valid address, a default address, or an invalid address. A valid address is the address of a usable memory area for storing commands and data, and can be successfully accessed. For example, 0x1000. A default address is the address of a usable memory area for storing error data, and can be successfully accessed. For example, 0xDEADBEEF. An invalid address is the address of an unusable memory area, and cannot be successfully accessed. For example, 0xFFFFFFFFFFFFFFFF.

[0100] Alternatively, a scatter aggregate element may not indicate the address, size, local key, or other information of a memory region.

[0101] In other words, a scatter aggregate element can point to a virtual address. This virtual address is not a memory address and cannot be successfully accessed from the memory location. For example, 0x7FFE12345678.

[0102] In practice, before the network card of the destination device receives the first command and the first data from the network card of the source device, the business application running on the processor of the destination device can request a memory area for storing commands and a memory area for storing error data in advance, but does not request a memory area for storing data in advance.

[0103] If the request for a memory area to store commands is successful, the processor of the destination device can fill the address of the memory area to be used to store commands into the scattered aggregate element in the first receive queue, and fill the address of the memory area indicated by the remaining scattered aggregate element in the first receive queue as a virtual address.

[0104] For example, Figure 5A This is a schematic diagram of a first receiving queue provided in an embodiment of this application. Figure 5A As shown in (1), the first receive queue in the memory of the destination device may include multiple receive queue elements, such as receive queue element 1 to receive queue element n. A receive queue element may include multiple scatter cluster elements, such as receive queue element 1 including scatter cluster elements 1 to scatter cluster elements 4.

[0105] In this cluster, scatter cluster element 1 indicates the correct address 1, size 1, and local key 1. The correct address 1 is the address of memory region 1 used to store the command. Scatter cluster element 2 indicates virtual address 1, size 2, and local key 2. Scatter cluster element 3 indicates virtual address 2, size 3, and local key 3. Scatter cluster element 4 indicates virtual address 3, size 4, and local key 4.

[0106] It is understandable that the memory of the destination device has been prepared in advance for storing commands, such as memory area 1 indicated by the scattered aggregation element 1, but memory area for storing data has not been prepared in advance.

[0107] Alternatively, before the destination device receives the first command and the first data from the source device, the business application running on the processor of the destination device can request memory areas from the memory of the destination device in advance for storing commands, storing data, and storing error data.

[0108] If the request for a memory area to store commands is successful, the processor of the destination device can fill the address of the memory area to be used to store commands into the scatter aggregation element in the first receive queue.

[0109] If the request for a memory area to store data is successful, the processor of the destination device can fill the address of the memory area to be used for storing data into the remaining scattered aggregate element in the first receive queue. Alternatively, if the request for a memory area to store data fails, the processor of the destination device can fill the address of the memory area indicated by the remaining scattered aggregate element in the first receive queue with the address of the memory area to store erroneous data or an invalid address.

[0110] For example, such as Figure 5A As shown in (2), the first receive queue in the memory of the destination device may include multiple receive queue elements, such as receive queue element 1 to receive queue element n. A receive queue element may include multiple scatter cluster elements, such as receive queue element 1 including scatter cluster elements 1 to scatter cluster elements 4.

[0111] In this cluster, scatter cluster element 1 indicates correct address 1, size 1, and local key 1. Correct address 1 is the address of memory area 1 used to store the command. Scatter cluster element 2 indicates incorrect address 1, size 2, and local key 2. Incorrect address 1 is the address of memory area 2 used to store incorrect data. Scatter cluster element 3 indicates invalid address 1, size 3, and local key 3. Scatter cluster element 4 indicates invalid address 2, size 4, and local key 4.

[0112] It is understandable that the destination device's memory has been prepared in advance for storing commands, such as memory area 1 indicated by scatter aggregation element 1. However, the destination device's memory preparation for storing data failed, therefore the destination device's memory preparation for storing erroneous data has been changed to memory area 2 indicated by scatter aggregation element 2.

[0113] S403. When the destination device determines that it has received the first data, it requests a second memory area for the first data, wherein the second memory area is an area used to store the data.

[0114] In this embodiment, when the network interface card (NIC) of the destination device determines that it has received first data from the NIC of the source device, the NIC of the destination device can determine that the first data corresponds to a first receive queue element in a first receive queue. When the NIC of the destination device determines that the first receive queue element indicates a virtual address, the NIC of the destination device can request a second memory area for storing the first data.

[0115] In other words, before the destination device's network interface card (NIC) receives the first data from the source device's NIC, the application running on the destination device's processor does not need to request a memory area for storing data from the destination device's memory in advance. After the destination device's NIC receives the first data from the source device's NIC, the destination device's NIC can request a second memory area for storing the first data from the destination device's memory in real time, thereby reducing memory overhead by requesting the memory area for storing data in real time.

[0116] For example, such as Figure 5A As shown in (1), the network card of the destination device determines that data 1 corresponds to the receiving queue element 1. Since the receiving queue element 1 includes scattered aggregation elements 1-4, but scattered aggregation element 1 indicates the address of memory area 1 used to store the command, scattered aggregation element 2 indicates virtual address 1, scattered aggregation element 3 indicates virtual address 2, and scattered aggregation element 4 indicates virtual address 3, that is, the receiving queue element 1 only indicates the virtual address and does not indicate the address of the memory area used to store the data. Therefore, the network card of the destination device can apply for memory area 2 used to store the data for data 1.

[0117] S404. If the destination device successfully requests the second memory area, it writes the first data into the second memory area according to the first command.

[0118] In this embodiment of the application, when the network card of the destination device requests a second memory area for storing data for the first data and the request for the second memory area for storing data is successful, the network card of the destination device can write the first data into the second memory area according to the first command, and update the virtual address indicated by the first receive queue element to the address of the second memory area for storing data.

[0119] One possible implementation, such as Figure 5B As shown, after executing S404, this application may also perform the following steps.

[0120] Step A1: The destination device releases the second memory area.

[0121] It is understandable that after the network interface card (NIC) of the destination device writes the first data into the second memory area used for data storage, the NIC of the destination device can determine that the first data corresponds to the first completion queue element in the first completion queue, and store the first completion information of the first receive request used to receive the first data into the first completion queue element. The first completion queue element can be an unused completion queue element among one or more completion queue elements included in the first completion queue, and the first completion information can indicate that the first data has been successfully received.

[0122] The processor of the destination device can obtain first completion information from the first completion queue element. If the first completion information indicates that the first data has been successfully received, the processor of the destination device can release the first memory area used to store the data, thereby improving the memory utilization of the destination device. The processor of the destination device can also send a second transmission response to the processor of the source device to indicate that the first data has been successfully received.

[0123] One possible implementation, such as Figure 5C As shown, after executing S403, this application may also perform the following steps.

[0124] Step B1: If the destination device fails to request the second memory area, it writes the first data into the third memory area or discards the first data. The third memory area is used to store erroneous data.

[0125] In this embodiment of the application, if the network card of the destination device requests a second memory area for storing the first data, but the request for the second memory area for storing the data fails, the network card of the destination device can write the first data into a third memory area for storing erroneous data or discard the first data, and update the virtual address indicated by the first receive queue element to the address of the third memory area or an invalid address.

[0126] Step B2: The destination device sends a first transmission response to the source device. Correspondingly, the source device receives the first transmission response from the destination device. The first transmission response is used to indicate that the first data was not successfully received and the reason for the failure to receive the first data.

[0127] It is understandable that after the destination device's network interface card (NIC) writes the first data to the third memory area used for storing erroneous data or discards the first data, the NIC can determine that the first data corresponds to a first completion queue element in the first completion queue, and store the second completion information for the first receive request used to receive the first data into the first completion queue element. The first completion queue element can be an unused completion queue element among one or more completion queue elements included in the first completion queue, and the second completion information can indicate the failure to successfully receive the first data and the reason for the failure (e.g., the destination device may not have sufficient memory resources).

[0128] The processor of the destination device can obtain second completion information from the first completion queue element. If the second completion information indicates that the first data was not successfully received, the processor of the destination device can send a first transmission response to the processor of the source device to indicate that the first data was not successfully received and the reason for the failure to receive the first data, so that the processor of the source device can process the first data according to the reason for the failure to receive the first data, such as retransmitting the first data or performing error processing on the first data.

[0129] One possible implementation, such as Figure 5D As shown, this application may also perform the following steps.

[0130] In step C1, the source device sends a second transmission request to the destination device, and correspondingly, the destination device receives a first transmission request from the source device, wherein the second transmission request includes a second command and second data.

[0131] It is understandable that step C1 can refer to S401 above, and will not be repeated here.

[0132] It is understood that step C1 can be executed before, after, or simultaneously with S401-S404 as described above. This application embodiment does not limit this. Figure 5D Take step C1 as an example, which is executed after S401-S404 above.

[0133] In step C2, the destination device writes the second command into the fourth memory area, which is the area used to store commands.

[0134] It is understandable that step C2 can refer to S402 above, and will not be repeated here.

[0135] In step C3, if the destination device determines that it has received the second data, it writes the second data into the third memory area or discards the second data. The third memory area is an area used to store erroneous data.

[0136] It is understandable that if the network interface card (NIC) of the destination device determines that it has received the second data from the NIC of the source device, the NIC of the destination device can determine that the second data corresponds to the second receive queue element in the first receive queue. If the NIC of the destination device determines that the second receive queue element indicates the address of the third memory area used to store erroneous data or an invalid address, the NIC of the destination device can write the second data to the third memory area used to store erroneous data or discard the second data.

[0137] In other words, before the destination device's network interface card (NIC) receives the second data from the source device's NIC, the application running on the destination device's processor can request a memory area from the destination device's memory in advance to store the data. If the request for memory area fails, it indicates that the destination device may not have enough memory resources. To avoid the destination device responding with an RNR from the source device, which would cause the RDMA connection between the source and destination devices to break and the second data sent by the source device to be unreceived by the destination device, thus requiring the re-establishment of the RDMA connection between the source and destination devices, the destination device's NIC can either write the second data to a third memory area used to store erroneous data or discard the second data.

[0138] For example, such as Figure 5A As shown in (2), the network card of the destination device determines that data 2 corresponds to receive queue element 1. Since receive queue element 1 includes scattered aggregation elements 1-4, scattered aggregation element 1 indicates the address of memory area 1 used to store commands, scattered aggregation element 2 indicates the address of memory area 2 used to store error data, scattered aggregation element 3 indicates invalid address 1, and scattered aggregation element 4 indicates invalid address 2, that is, receive queue element 1 only indicates the address of memory area 2 used to store error data and invalid address, but does not indicate the address of memory area used to store data. Therefore, the network card of the destination device can write data 2 into memory area 2 used to store error data or discard data 2.

[0139] It is understood that the above embodiments of this application can be implemented individually or in combination with each other, and the embodiments of this application are not limited.

[0140] The methods provided by the embodiments of this application have been described above with reference to the accompanying drawings. The apparatus provided by the embodiments of this application will be described below with reference to the accompanying drawings.

[0141] Based on the above embodiments, this application also provides a data processing apparatus for performing the above-described... Figure 4 The method shown in the embodiment is executed by the destination device. Figure 6 As shown, the data processing device 600 includes a transmitting module 601, a receiving module 602, and a processing module 603. Specifically, in the data processing device 600, the modules are connected through a communication path.

[0142] In one example, the data processing device 600 can communicate with the source device via queue pairs.

[0143] The receiving module 602 is configured to receive a first sending request from the source device, wherein the first sending request includes a first command and first data;

[0144] Processing module 603 is used to write the first command into a first memory area, the first memory area being an area for storing commands;

[0145] Processing module 603 is configured to, upon determining that the first data has been received, request a second memory area for the first data, wherein the second memory area is an area for storing data;

[0146] The processing module 603 is used to write the first data into the second memory area according to the first command if the application for the second memory area is successful.

[0147] In one possible implementation, the processing module 603 is configured to write the first data into a third memory area or discard the first data if the application for the second memory area fails, wherein the third memory area is an area for storing erroneous data.

[0148] In one possible implementation, the processing module 603 is configured to determine the first receive queue element in the first receive queue corresponding to the first data, the first receive queue element being used to indicate a virtual address; and to request the second memory area for the first data.

[0149] In one possible implementation, the processing module 603 is configured to update the virtual address indicated by the first receive queue element to the address of the second memory area when the first data is written to the second memory area; or, when the first data is written to the third memory area or the first data is discarded, update the virtual address indicated by the first receive queue element to the address of the third memory area or an invalid address, wherein the third memory area is an area used to store erroneous data.

[0150] In one possible implementation, the processing module 603 is configured to release the second memory area when the first data is written to the second memory area; or, when the first data is written to the third memory area or the first data is discarded, send a first transmission response to the source device, the first transmission response being used to indicate that the first data was not successfully received and the reason for the failure to receive the first data.

[0151] In one possible implementation, the receiving module 602 is configured to receive a second transmission request from the source device, the second transmission request including a second command and second data; the processing module 603 is configured to write the second command into a fourth memory area, the fourth memory area being an area for storing commands; the processing module 603 is configured to, if it is determined that the second data has been received, write the second data into a third memory area or discard the second data, the third memory area being an area for storing error data.

[0152] In one possible implementation, the processing module 603 is configured to determine that the second data corresponds to a second receive queue element in the first receive queue, the second receive queue element being used to indicate the address of the third memory area or an invalid address; and to write the second data into the third memory area or discard the second data.

[0153] It is understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division. In actual implementation, there may be other division methods. The functional modules in the embodiments of this application can be integrated into one module, or each module can exist physically separately, or two or more modules can be integrated into one module. For example, the sending module 601 and the receiving module 602 can be integrated into one module. The integrated unit can be implemented in hardware or as a software functional unit.

[0154] Based on the above embodiments, this application also provides a computing device. For example... Figure 7 As shown, the computing device 700 includes a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, memory 706, and communication interface 708 communicate with each other via the bus 702. The computing device 700 can be a server, desktop computer, laptop computer, or mobile device, etc. It should be understood that this application does not limit the number of processors and memories in the computing device 700.

[0155] The 702 bus can be a PCI bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus 702 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 702 may include a path for transmitting information between various components of the computing device 700 (e.g., processor 704, memory 706, communication interface 708).

[0156] Processor 704 may include any one or more processors such as CPU, graphics processing unit (GPU), microprocessor (MP), or DSP.

[0157] The memory 706 may include volatile memory, such as random access memory (RAM). The processor 704 may also include non-volatile memory, such as ROM, flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0158] In one possible implementation, the computing device 700 performs the functions of the data processing device 600. The memory 706 stores executable program code, and the processor 704 executes this executable program code to implement the functions of the aforementioned sending module 601, receiving module 602, and processing module 603, thereby implementing the data processing method. That is, the memory 706 stores instructions for the computing device 700 to execute the data processing method provided in this application.

[0159] The communication interface 708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0160] Based on the above embodiments, this application also provides a computing device cluster. The computing device cluster includes at least one computing device. This computing device can be a server, desktop computer, laptop computer, or mobile device, etc. Taking the computing device 700 performing the functions of the data processing device 600 as an example, such as... Figure 8 As shown, the computing device cluster includes at least one computing device 700. The memory 706 of one or more computing devices 700 in the computing device cluster may store the same instructions for performing data processing methods.

[0161] In one possible implementation, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing data processing methods. In other words, a combination of one or more computing devices 700 can jointly execute instructions for executing data processing methods.

[0162] It is understood that the memory 706 in different computing devices 700 within a computing device cluster can store different instructions, each used to execute a portion of the computing device's functions. That is, the instructions stored in the memory 706 of different computing devices 700 can implement the functions of one or more modules among the sending module 601, receiving module 602, and processing module 603.

[0163] In one possible implementation, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing data processing methods. In other words, a combination of one or more computing devices 700 can jointly execute instructions for executing data processing methods.

[0164] Based on the above embodiments, this application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the data processing method described in the above method embodiments.

[0165] Based on the above embodiments, this application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the data processing method described in the above method embodiments.

[0166] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0167] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data processing method, characterized in that, Applied to a destination device, the method includes: Receive a first transmission request from the source device, the first transmission request including a first command and first data; The first command is written into the first memory area, which is a region used to store commands; Upon confirming that the first data has been received, a second memory area is allocated for the first data, the second memory area being a region used to store the data; If the application for the second memory area is successful, the first data is written into the second memory area according to the first command.

2. The method according to claim 1, characterized in that, The method further includes: If the application for the second memory area fails, the first data is written to the third memory area or the first data is discarded. The third memory area is a region used to store erroneous data.

3. The method according to claim 1 or 2, characterized in that, To allocate a second memory area for the first data, the following is included: Determine the first data to correspond to the first receive queue element in the first receive queue, where the first receive queue element is used to indicate a virtual address; Allocate a second memory area for the first data.

4. The method according to claim 3, characterized in that, The method further includes: When the first data is written to the second memory area, the virtual address indicated by the first receive queue element is updated to the address of the second memory area; or, In the event that the first data is written to the third memory area or the first data is discarded, the virtual address indicated by the first receive queue element is updated to the address of the third memory area or an invalid address, the third memory area being a region used to store erroneous data.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: If the first data is written to the second memory area, then the second memory area is released; or, In the event that the first data is written to the third memory area or the first data is discarded, a first transmission response is sent to the source device. The first transmission response is used to indicate that the first data was not successfully received and the reason for the failure to receive the first data.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Receive a second transmission request from the source device, the second transmission request including a second command and second data; The second command is written to the fourth memory area, which is a region used to store commands; If the second data is received, the second data is either written to a third memory area or discarded, the third memory area being a region for storing erroneous data.

7. The method according to claim 6, characterized in that, Writing the second data to a third memory area or discarding the second data includes: The second data is determined to correspond to a second receive queue element in the first receive queue, and the second receive queue element is used to indicate the address of the third memory area or an invalid address; Write the second data to the third memory area or discard the second data.

8. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, and each computing device includes a processor and a memory; the memory is used to store computer program instructions. The processor invokes computer program instructions stored in the memory to execute the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-7.

10. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the method as described in any one of claims 1-7.

Images