Data processing method and device

Abstract

The invention provides a data processing method and device. The method includes: obtaining the data amount of the current data to be processed, wherein the current data to be processed includes the input and output I / O data of the current remote direct data access RDMA operation; if the data amount of the current data to be processed is greater than a first predetermined threshold , then directly execute the RDMA operation of the current data to be processed on the page address corresponding to the current data to be processed; Execute the RDMA operation on the currently pending data on the page address of . Through the present invention, the problem of low data processing efficiency caused by RDMA operations in the prior art that RDMA can only transmit data through multiple interactions and multiple copies of the underlying protocol stack is solved, and then the data processing efficiency is improved. Effect.

Description

technical field

[0001] The present invention relates to the communication field, in particular, to a data processing method and device. Background technique

[0002] With the continuous expansion of user data, the rise of information technology. The bandwidth between various communication devices is increasing. A PCIe (PCI-Express) link is widely used as a mainstream high-speed transmission protocol in a node. While continuously improving the bandwidth, the PCIe protocol has gradually begun to step out of the interconnection protocol between the central processing unit (CPU, Central Processing Unit) and external devices in the node, using the non-transparent bridge (NTB, Non Transparent bridge) technology of PCIe , The PCIe protocol can support high-speed interconnection between nodes, and the address domains of interconnected nodes are isolated through NTB. After the address mapping of the NTB, the DMA engine on the local node can access the memory on the peer node by ac...

Examples

Embodiment 1

[0045] In this embodiment, a data processing method is provided, figure 1 is a flowchart of a data processing method according to an embodiment of the present invention, such as figure 1 As shown, the process includes the following steps:

[0046] Step S102, obtaining the data amount of the current data to be processed, wherein the current data to be processed includes the input and output I / O data of the current remote direct data access RDMA operation;

[0047] Step S104, if the data amount of the current data to be processed is greater than the first predetermined threshold, directly perform the RDMA operation on the current data to be processed on the page address corresponding to the current data to be processed;

[0048] Step S106, if the amount of data to be processed currently is less than or equal to the first predetermined threshold, perform an RDMA operation on the data to be processed currently on the page address in the receiving device through the pre-establishe...

Embodiment 2

[0128] In this embodiment, a data processing method is provided, Figure 7 is a flowchart of a data processing method according to an embodiment of the present invention, such as Figure 7 As shown, the process includes the following steps:

[0129] S702. When the amount of data currently to be processed acquired by the sending device is greater than a first predetermined threshold, the receiving and sending device directly performs a remote direct data access (RDMA) operation of the current data to be processed on the page address corresponding to the current data to be processed , wherein, the current data to be processed includes the input and output I / O data of the current RDMA operation;

[0130] S704. When the data amount of the current data to be processed is less than or equal to the first predetermined threshold, receive an RDMA operation of performing the current data transmission on the page address in the receiving device through the pre-established data transmiss...

Embodiment 3

[0164] In this embodiment, a data processing device is provided, Figure 8 is a schematic diagram of a data processing device according to an embodiment of the present invention, such as Figure 8 As shown, the device includes:

[0165] 1) The first acquiring unit 802 is configured to acquire the data amount of the current data to be processed, wherein the current data to be processed includes the input and output I / O data of the current remote direct data access RDMA operation;

[0166] 2) The first processing unit 804 is configured to directly perform an RDMA operation on the current data to be processed on the page address corresponding to the current data to be processed when the data amount of the current data to be processed is greater than a first predetermined threshold;

[0167] 3) The second processing unit 806 is configured to perform an RDMA operation on the current data to be processed on the page address in the receiving device through the pre-established data t...

technical field

[0001] The present invention relates to the communication field, in particular, to a data processing method and device. Background technique

[0002] With the continuous expansion of user data, the rise of information technology. The bandwidth between various communication devices is increasing. A PCIe (PCI-Express) link is widely used as a mainstream high-speed transmission protocol in a node. While continuously improving the bandwidth, the PCIe protocol has gradually begun to step out of the interconnection protocol between the central processing unit (CPU, Central Processing Unit) and external devices in the node, using the non-transparent bridge (NTB, Non Transparent bridge) technology of PCIe , The PCIe protocol can support high-speed interconnection between nodes, and the address domains of interconnected nodes are isolated through NTB. After the address mapping of the NTB, the DMA engine on the local node can access the memory on the peer node by ac...

Examples

Embodiment 1

[0045] In this embodiment, a data processing method is provided, figure 1 is a flowchart of a data processing method according to an embodiment of the present invention, such as figure 1 As shown, the process includes the following steps:

[0046] Step S102, obtaining the data amount of the current data to be processed, wherein the current data to be processed includes the input and output I / O data of the current remote direct data access RDMA operation;

[0047] Step S104, if the data amount of the current data to be processed is greater than the first predetermined threshold, directly perform the RDMA operation on the current data to be processed on the page address corresponding to the current data to be processed;

[0048] Step S106, if the amount of data to be processed currently is less than or equal to the first predetermined threshold, perform an RDMA operation on the data to be processed currently on the page address in the receiving device through the pre-establishe...

Embodiment 2

[0128] In this embodiment, a data processing method is provided, Figure 7 is a flowchart of a data processing method according to an embodiment of the present invention, such as Figure 7 As shown, the process includes the following steps:

[0129] S702. When the amount of data currently to be processed acquired by the sending device is greater than a first predetermined threshold, the receiving and sending device directly performs a remote direct data access (RDMA) operation of the current data to be processed on the page address corresponding to the current data to be processed , wherein, the current data to be processed includes the input and output I / O data of the current RDMA operation;

[0130] S704. When the data amount of the current data to be processed is less than or equal to the first predetermined threshold, receive an RDMA operation of performing the current data transmission on the page address in the receiving device through the pre-established data transmiss...

Embodiment 3

[0164] In this embodiment, a data processing device is provided, Figure 8 is a schematic diagram of a data processing device according to an embodiment of the present invention, such as Figure 8 As shown, the device includes:

[0165] 1) The first acquiring unit 802 is configured to acquire the data amount of the current data to be processed, wherein the current data to be processed includes the input and output I / O data of the current remote direct data access RDMA operation;

[0166] 2) The first processing unit 804 is configured to directly perform an RDMA operation on the current data to be processed on the page address corresponding to the current data to be processed when the data amount of the current data to be processed is greater than a first predetermined threshold;

[0167] 3) The second processing unit 806 is configured to perform an RDMA operation on the current data to be processed on the page address in the receiving device through the pre-established data t...

Images