Method, network card and server for data processing
By performing data segmentation instruction order preservation operations within the network card, the out-of-order problem in RDMA technology is solved, improving PCIe bandwidth utilization and processor speed, enhancing data transmission efficiency, and reducing processor load and data processing latency.
Patent Information
- Application Number
- CN202111223252.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-03-01
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2039-03-01
AI Technical Summary
In existing technologies, RDMA technology suffers from out-of-order issues when the processor sends data segmentation instructions to the network card, resulting in wasted PCIe bandwidth and reduced data transfer rates. This increases the system's data processing efficiency. In existing technologies, the processor needs to wait for the previous segmentation instruction to be successfully sent to the network card, which reduces data processing efficiency.
By offloading the order-keeping operation of the network interface card (NIC) to the NIC itself, the processor no longer performs the order-keeping operation of multiple segmented instructions. Instead, it directly sends multiple segmented instructions to the NIC. The NIC isolates different segmented instructions through PCIe barriers, enabling sequential processing of segmented instructions and improving PCIe bandwidth utilization and processor speed.
It improves data processing efficiency, reduces data processing latency, increases data transmission rate, and reduces processor load and the number of PCIe packets.
Smart Images

Figure CN114090495B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer, in particular to a data processing method, a network card and a server. BACKGROUND
[0002] In the remote direct memory access (RDMA) technology, the network interface card (NIC) of the sending server can first acquire a sent request entry (SQE) from a cache, and then acquire data corresponding to the SQE from a memory according to the SQE, and send the data to a receiving server. Before the NIC of the sending server acquires the SQE, a processor of the sending server first divides the SQE into multiple segments, and stores all segments of the SQE in the cache. If the length of the SQE is greater than a preset threshold, the processor can use a doorbell mechanism to inform the NIC to process the SQE. For example, the NIC uses a direct memory access (DMA) technology to acquire all segments of the SQE from the cache. If the length of the SQE is less than or equal to the preset threshold, when the processor sends all segments of the SQE stored in the cache to the NIC, out-of-order may occur.
[0003] In order to solve the out-of-order problem, in the process of dividing multiple segments by the processor in the conventional technology, each time a segment instruction is acquired, the segment instruction is issued to a cache in the processor according to the order of the segments, and then the segment instruction is sent to the NIC through a Peripheral Component Interconnect express (PCIe) message. Then, the processor sends a PCIe Barrier instruction to the cache to clear the segment instruction in the cache that has been sent to the NIC. In this way, the order between the segment instructions of the SQE is ensured through the function of the PCIe Barrier. Although the above method overcomes the out-of-order problem between all segments of the SQE, it causes waste of PCIe bandwidth. Moreover, in the above processing process, the processor needs to wait for the previous segment instruction to be successfully sent to the NIC, and then clear the cache through the Barrier function. The entire data transmission process is serial, which reduces the data transmission rate between the processor and the NIC, and further increases the processing delay of the entire data processing process, thereby affecting the data processing efficiency of the system. SUMMARY
[0004] The data processing method, the network card and the server provided by the present application solve the problem of low system data processing efficiency and high latency caused by the conventional SQE order processing method.
[0005] In a first aspect, the present application provides a data processing method, which can be applied to a network card or a communication device supporting the network card to implement the method, for example, the communication device includes a chip system, and the method can include: after receiving an i th PCIe message, storing a j th segment instruction included in the i th PCIe message to a j th storage unit of a first storage area. Wherein, the first SQE is divided into n segments according to a preset size, n is an integer, and n is greater than or equal to 1. The i th PCIe message further includes a first address segment, and the first address segment is used to indicate the segment position of the j th segment instruction in the first SQE. i is an integer, i ∈ [1, n], and j is an integer, j ∈ [1, n]. The first storage area is a storage area in the network card for storing the first SQE, and the first storage area includes m storage units, each of which is used to store a segment instruction of the first SQE, m is an integer, and n ≤ m. The technical solution is to completely offload the order preserving operation of the multiple segment instructions of the first SQE to the network card inside the sending end server to complete, and the processor of the sending end server does not need to perform the order preserving operation of the multiple segment instructions of the first SQE, and the multiple segment instructions of the first SQE can be issued at one time. Effectively improve the utilization rate of the PCIe bandwidth, and improve the running speed of the processor, reduce the time delay of the data processing process, and improve the data transmission efficiency of the entire communication system.
[0006] Further, when the n segment instructions of the first SQE are all stored to the first storage area, the first SQE can be obtained according to the instructions of the n storage units in the first storage area, and the identifier of the queue pair (QP) to which the first SQE belongs and the position identifier of the first SQE in the QP, and the data processing is performed according to the identifier of the QP to which the first SQE belongs and the position identifier of the first SQE in the QP.
[0007] In a possible implementation, when i = 1, before the network card stores the j th segment instruction to the j th storage unit of the first storage area, the method can further include: determining whether the first address segment is an address within an address range of a storage space where the first storage area is located; if the first address segment is an address within the address range of the storage space where the first storage area is located, determining that the j th segment instruction is blueframe data, and storing the j th segment instruction included in the first PCIe message to the j th storage unit of the first storage area; and if the first address segment is not an address within the address range of the storage space where the first storage area is located, determining that the j th segment instruction is not blueframe data, and obtaining the first SQE according to a non-blueframe method.
[0008] In a possible implementation, the method further includes: when j = 1, before the network card stores the jth segment of instructions to the jth storage unit of the first storage area, obtaining the total length of the first SQE included in the first segment of instructions, and comparing the total length of the first SQE with a preset threshold; if the total length of the first SQE is less than or equal to the preset threshold, determining that the first segment of instructions of the first SQE is data of a blueframe, and storing the first segment of instructions included in the first PCIe message to the first storage unit of the first storage area; if the total length of the first SQE is greater than the preset threshold, determining that the first segment of instructions of the first SQE is not data of a blueframe, and obtaining the first SQE according to a method for non-blueframe.
[0009] In a possible implementation, after the jth segment of instructions is stored to the jth storage unit of the first storage area, the method further includes: determining whether the first SQE is received completely; if the first SQE is received completely, generating an end symbol, and if the first SQE is not received completely, continuing to receive.
[0010] For example, the ith+1 PCIe message is received, and the kth segment of instructions included in the ith+1 PCIe message is stored to the kth storage unit of the first storage area. The ith+1 PCIe message further includes a second address segment, the second address segment is used to indicate a segmentation position of the kth segment of instructions in the first SQE, the kth segment of instructions is any one of other segmentation instructions except the received segmentation instructions in the first SQE, and k is an integer.
[0011] Optionally, the determination of whether the first SQE is received completely includes: determining whether the accumulated value is equal to the total length of the first SQE, if the accumulated value is equal to the total length of the first SQE, determining that the first SQE is received completely, and if the accumulated value is not equal to the total length of the first SQE, determining that the first SQE is not received completely. The accumulated value is used to indicate the length of the received segmentation instructions of the first SQE, and the initial value of the accumulated value can be 0.
[0012] In a possible implementation, the network card performs data processing according to the identifier of the QP to which the first SQE belongs and the position identifier of the first SQE in the QP, including: determining the first storage area for storing the first SQE according to the identifier of the QP to which the first SQE belongs and the position identifier of the first SQE in the QP, and then reading the n segmentation instructions of the first SQE from the starting address of the first storage area in sequence, and obtaining and sending data corresponding to the first SQE according to the n segmentation instructions of the first SQE.
[0013] In a possible implementation, if the total length of the first SQE is less than or equal to a preset threshold, the method further includes: obtaining an identifier of a QP to which the first SQE belongs and a position identifier of the first SQE in the QP, and establishing a correspondence relationship among the identifier of the QP to which the first SQE belongs, the position identifier of the first SQE in the QP, and an identifier of the first storage area, so as to obtain the first SQE according to the correspondence relationship among the identifier of the QP to which the first SQE belongs, the position identifier of the first SQE in the QP, and the identifier of the first storage area when n instructions of the first SQE are all stored in the first storage area.
[0014] In a possible implementation, when i = 1, that is, after the first PCIe packet is received, the method further includes: allocating a first storage area for the first SQE.
[0015] The execution subject of the first aspect or any possible implementation of the first aspect can be a network card of a sending server. The sending server refers to a server used for sending data. For a server, when the server is used for sending data, the server is referred to as a sending server, and when the server is used for receiving data, the server is referred to as a receiving server.
[0016] In a second aspect, the present application provides a data processing apparatus, which includes various modules for executing the method for data processing in the first aspect or any possible implementation of the first aspect.
[0017] In a third aspect, the present application provides a network card including a memory and a processor, the memory is used for storing computer execution instructions, when the network card is running, the processor executes the computer execution instructions in the memory to execute the operation steps of the method in the first aspect or any possible implementation of the first aspect by using hardware resources in the data processing apparatus.
[0018] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, when the computer program is running on a computer, the computer program causes the computer to execute the operation steps of any possible method in the first aspect or any possible implementation of the first aspect.
[0019] In a fifth aspect, the present application further provides a computer program product, when the computer program product is running on a computer, the computer program product causes the operation steps of any possible method in the first aspect or any possible implementation of the first aspect.
[0020] It can be understood that any of the above provided devices or computer readable storage medium or computer program product, etc. are used to execute the corresponding method provided above, so the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1 A structural example diagram of a communication system provided by the present application is shown in the following figure;
[0022] Figure 2 A flow chart of a data processing method provided by the present application is shown in the following figure;
[0023] Figure 3 A structural example diagram of a Blueframe buffer provided by the present application is shown in the following figure;
[0024] Figure 4 A flow chart of another data processing method provided by the present application is shown in the following figure;
[0025] Figure 5 A flow chart of still another data processing method provided by the present application is shown in the following figure;
[0026] Figure 6 A structural example diagram of a data processing device provided by the present application is shown in the following figure;
[0027] Figure 7 A structural example diagram of a network card provided by the present application is shown in the following figure;
[0028] Figure 8 A structural example diagram of a server provided by the present application is shown in the following figure. DETAILED DESCRIPTION
[0029] Figure 1 A structural example diagram of a communication system provided by the present application is shown in the following figure; Figure 1As shown, the communication system comprises a server 100 and a server 200. The server 100 and the server 200 can communicate messages through a network 300. The server sending data can be referred to as a sending-end server, and the server receiving data can be referred to as a receiving-end server. For a server, when the server is used to send data, the server can be referred to as a sending-end server, and when the server is used to receive data, the server can be referred to as a receiving-end server. The network 300 can be an Ethernet, that is, the servers can communicate through a Transmission Control Protocol / internet protocol (TCP / IP) protocol; or the network 300 can be a network based on a remote direct data access technology. The remote direct data access technology is generated to solve the time delay problem of the servers processing data in the network. According to the remote direct data access technology, the servers can quickly move data from one system to a remote system memory through a network card of the server without the participation of a processor in the server, thereby reducing the load of the processor of the server and improving the data transmission efficiency between the servers. For example, the network based on the remote direct data access technology can be an InfiniBand (IB) network, an RDMA over Converged Ethernet (RoCE) network, an iWapp network, and the like, which are not limited in the present application.
[0030] Each of the server 100 and the server 200 comprises a hardware layer and a software layer, Figure 1 Only the structure of the server 100 is shown. The hardware layer of the server 100 comprises a network card, a memory and a processor. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or any conventional processor, and the like.
[0031] It is worth noting that, Figure 1In the server hardware layer, the number of network cards, memories and processors is all 1, but the number of network cards, memories and processors in the same server does not limit the embodiments of the present application.
[0032] The network card can be a network interface card supporting RDMA technology, or a host channel adapter (HCA). The main function of the network card is to connect multiple servers to a network, so that the servers can communicate with each other through a network supporting RDMA technology. The network card can be connected to the network through an external optical fiber, cable, etc. The network card can be inserted into the PCIe slot of the computer and connected to the server through PCIe. Alternatively, the network card can be connected to the server through a certain specific (or private) bus, which is not limited by the embodiments of the present application. It can be understood that, in physical implementation, the network card can be part of the server, or a device / instrument independent of the server. For ease of description, the network card is described as the network card of the server in the following. The network card can include one or more ports, specifically ports for receiving data. The network card can include a processor, a cache, a memory and a register. The memory or cache in the network card can be implemented using a random access memory (RAM) type storage medium.
[0033] The software layer is program code running on the hardware layer. Specifically, the software layer can be divided into several layers, and the layers communicate with each other through a software interface. The software layer includes an application layer, a driver layer and an operating system layer. The application layer includes a series of program codes running application programs. The driver layer is used to realize message interaction between the hardware layer and the software layer. The driver layer includes a network card driver, etc. The operating system layer includes operating system program code and a protocol stack. The operating system can be Linux, Windows or vxWarks, etc. The protocol stack refers to a set of program codes divided according to different levels involved in the communication protocol, and processing data processing of the corresponding level.
[0034] In a possible implementation, for example, in an IB network architecture, the protocol stack can include an application layer, a transport layer, a network layer, a link layer and a physical layer. The functions of the application layer can be implemented in the software layer. The functions of the transport layer, the network layer, the link layer and the physical layer can be implemented in the network card.
[0035] In order to better understand the technical solutions provided by the embodiments of the present application, the following will briefly explain the terms and technical concepts related to the embodiments of the present application.
[0036] Before the server transmits data, the server can be pre-configured with multiple queue pairs (QPs), each of which can include a send queue (SQ) and a receive queue (RQ). Each SQ is used to store at least one SQE. Each RQ is used to store at least one receive queue entry (RQE). The SQEs and the RQEs can be collectively referred to as work request entries (WQEs).
[0037] It can be understood that the so-called SQE is instruction information required by a network card of the server to obtain data from the memory. The so-called RQE is instruction information required by the server to store data after receiving the data.
[0038] It can also be understood that each SQE and RQE is only used to transmit instruction information and does not contain data, and therefore the receiving server or the sending server needs to obtain data from other storage media (for example, the memory) according to the instruction information in the SQE or the RQE.
[0039] The receiving server and the sending server communicate based on QPs, and each server includes one or more QPs. The number of QPs in different servers can be the same or different. For example, the server 100 includes one QP11, and the server 200 includes one QP21. Figure 1 Taking the communication system as an example, when the server 100 is a sending server and the server 200 is a receiving server, the server 100 is established with a QP11 including an SQ11 and an RQ11, and the server 200 is established with a QP21 including an SQ21 and an RQ21. During the communication between the server 100 and the server 200, the correspondence between the QP11 and the QP21 is pre-established. Correspondingly, the sending server can obtain data from the memory based on the SQE in the SQ11, and the receiving server can store the received data based on the RQE in the RQ11.
[0040] It should be noted that the two QPs pre-determined by the receiving server and the sending server for communication, and the names of the SQs and the RQs in the QPs can be the same or different, which is not limited in the present application. It should also be noted that different receiving servers can be configured with different numbers of QPs, and the number of SQEs and RQEs included in each QP configured by the receiving server can also be different.
[0041] The server can also configure a complete queue (CQ) to store complete queue entries (CQE). After the network card of the server reads or writes the completion data from or to the memory, the network card can send the CQE to the processor.
[0042] It should be noted that the processor of the server is usually connected with the network card of the server through a PCIe bus. Due to the limitation of the PCIe protocol itself, the memory write is limited, that is, the processor of the server can send at most 8 bytes of data to the network card at a time. The length of each SQE is at least 64 bytes, and the lengths of SQEs corresponding to different services can be different. If the length of the SQE is relatively long, the processor of the server directly sends the SQE to the network card of the server, which can reduce the transmission efficiency and increase the processor occupancy. Therefore, the processor of the server can instruct the network card of the server to use the direct memory access technology to move large data by the doorbell method. According to the PCIe attribute limitation of the processor of the server and the network card of the server, for example, the processor of the server is limited by the storage attribute register, and the granularity of the large data can be 4K. The granularity of the large data can refer to the maximum value of reading and the maximum value of writing.
[0043] In some possible embodiments, since the length of the SQE is relatively small, if the network card still reads the SQE from the memory in the direct memory access technology mode, the data transmission delay can be increased. Therefore, when the length of the SQE is less than or equal to a preset threshold, the processor of the sending server can directly send the SQE to the network card of the sending server in the form of a segmented instruction. In the present application, the SQE whose length is less than or equal to the preset threshold can be referred to as a Blueframe. However, when the processor of the sending server sends all segments of the Blueframe to the network card, the out-of-order problem can occur. Although the out-of-order problem between all segments of the Blueframe can be overcome in the traditional technology, the PCIe bandwidth can be wasted, the rate of sending the Blueframe from the processor of the sending server to the network card of the sending server can be reduced, the processing delay of the entire data processing process can be increased, and the efficiency of the data processing of the system can be affected.
[0044] The data processing method provided in the application can completely offload the in-order operation of the plurality of segmented instructions of Blueframe to the internal network card of the sending end server to complete, the processor of the sending end server does not need to perform the in-order operation of the plurality of segmented instructions of Blueframe, and the processor of the sending end server can issue one plurality of segmented instructions of Blueframe at a time, and only needs to ensure that different Blueframes are isolated by the PCIe barrier. Thus, the plurality of segmented instructions of the same Blueframe can be continuously issued, the number of messages between the processor and the network card is effectively reduced, and the utilization rate of the PCIe bandwidth is improved. Moreover, the processor can send a plurality of segmented instructions in parallel, that is, the processor can simultaneously process the segmented instructions of different SQEs and the different segmented instructions of the same SQE, without needing to pay attention to the in-order problem of the segmented instructions of each SQE, thereby reducing the load of the processor, reducing the time delay of data processing between the processor and the network card, and improving the data processing efficiency.
[0045] Next, the data processing method provided by the embodiments of the application will be described in detail in combination with the drawings.
[0046] Figure 2 A flowchart of the data processing method provided by the embodiments of the application. Figure 2 The execution subject of the method shown can be the network card of the sending end server. The embodiments of the application can perform in-order processing on each SQE of the network card of the sending end server, and the method of performing in-order processing on the segmented instructions of each SQE is the same. For ease of description, the jth segmented instruction of one SQE is taken as an example for description. As shown in the figure, the method comprises the following steps.
[0047] S201, the processor of the sending end server divides the first SQE into n segments and stores them in the cache.
[0048] The processor of the sending end server can first divide the first SQE into n segmented instructions, and store the n segmented instructions of the first SQE in the cache, where n is an integer and n is greater than or equal to 1. For example, assuming that each segmented instruction can be 8 bytes, if the length of the SQE is 64 bytes, the processor of the sending end server can first divide the SQE into 8 segmented instructions, i.e. n = 8; if the length of the SQE is 256 bytes, the processor of the sending end server can first divide the SQE into 32 segmented instructions, i.e. n = 32. The cache can be the cache in the processor of the sending end server. For ease of description, in the following, the length of the first SQE is taken as 64 bytes and the first SQE is divided into 8 segments as an example for description.
[0049] S202, the processor of the sending end server sends the i-th PCIe message to the network card of the sending end server, and the i-th PCIe message includes the j-th segment instruction of the first SQE.
[0050] The i-th PCIe message can also include a first address segment, and the first address segment is used to indicate the segment position of the j-th segment instruction in the first SQE. Wherein, i is an integer, i ∈ [1, n], j is an integer, j ∈ [1, n]. i and j can be equal or not equal. For example, assuming that n = 8, i can take 1 to 8, and j can take 1 to 8.
[0051] S203, the network card of the sending end server stores the j-th segment instruction to the j-th storage unit of the first storage area.
[0052] Before the network card of the sending end server receives the segment instruction of the Blueframe, a buffer area for storing the Blueframe can be pre-configured in the network card. The Blueframe buffer can be realized by a segment of the storage area of the RAM, of course, it can also be realized by other storage media, which is not limited in the present application. For the convenience of description, the following embodiments of the present application take a segment of the storage area in the memory to realize the function of the Blueframe buffer as an example for further introduction.
[0053] The buffer area of the Blueframe can be divided according to the preset rule. For example, the buffer area of the Blueframe can be divided into a plurality of storage areas according to the preset threshold, and the storage capacity of each storage area is greater than or equal to the preset threshold. Each storage area can be divided into a plurality of storage units according to the size of the preset segment instruction. The storage capacity of each storage unit is greater than or equal to the size of the preset segment instruction, and one storage unit can store one or more segment instructions in order. The preset threshold can be 256 bytes, of course, it can also be other size values, which is not limited in the present application. For the convenience of description, one storage unit is used to store one segment instruction in the following embodiments of the present application.
[0054] As Figure 3As shown, the Blueframe buffer can include 128 storage regions. If the preset threshold is 256 bytes, the storage capacity of each storage region in the 128 storage regions can be 256 bytes. Assuming that the size of the preset segment instruction is 8 bytes, each storage region can further include 32 storage units, and the storage capacity of each storage unit in the 32 storage units can be 8 bytes. For ease of description, the 128 storage regions can be numbered from 0 to 127. For example, the first storage region is numbered 0, the second storage region is numbered 1, and so on, and the 128th storage region is numbered 127. Similarly, the 32 storage units can be numbered from 0 to 31. For example, the first storage unit is numbered 0, the second storage unit is numbered 1, and so on, and the 32nd storage unit is numbered 31.
[0055] The first storage region can be any idle storage region in the Blueframe buffer. The first storage region includes a plurality of storage units. Moreover, each storage unit has a unique identifier associated with the order of each segment of the SQE to be stored. For example, SQE1 is sequentially divided into three segment instructions, instruction 1, instruction 2, and instruction 3, and storage region 1 is used to store each segment instruction of SQE1, which includes three storage units: storage unit 1, storage unit 2, and storage unit 3, wherein storage unit 1 is used to store instruction 1, storage unit 2 is used to store instruction 2, and storage unit 3 is used to store instruction 3.
[0056] After the network card of the sending end server receives the i-th PCIe message sent by the processor of the sending end server, the j-th segment instruction and the first address segment are obtained by parsing the i-th PCIe message. The network card of the sending end server can determine the segment position of the j-th segment instruction in the first SQE according to the first address segment, that is, the j-th segment instruction in the first SQE, and store the j-th segment instruction to the j-th storage unit of the first storage region. For example, when j = 1, the first segment instruction is stored in the first storage unit of the first storage region. When j = 2, the second segment instruction is stored in the second storage unit of the first storage region. When j = 3, the third segment instruction is stored in the third storage unit of the first storage region. When j = 4, the fourth segment instruction is stored in the fourth storage unit of the first storage region. When j = 5, the fifth segment instruction is stored in the fifth storage unit of the first storage region. When j = 6, the sixth segment instruction is stored in the sixth storage unit of the first storage region. When j = 7, the seventh segment instruction is stored in the seventh storage unit of the first storage region. When j = 8, the eighth segment instruction is stored in the eighth storage unit of the first storage region.
[0057] S204, when the n segments of the first SQE are stored in the first storage area, the network card of the sending server acquires the first SQE according to the n storage units in the first storage area, and the identification of the QP to which the first SQE belongs and the position identification of the first SQE in the QP.
[0058] When the n segments of the first SQE are all stored in the first storage area, the network card of the sending server can generate a response (ACK). After the processor of the network card of the sending server schedules to the QP to which the first SQE belongs, the network card of the sending server can determine the first storage area storing the first SQE according to the identification of the QP to which the first SQE belongs and the position identification of the first SQE in the QP; and read the n segments of the first SQE in sequence from the start address of the first storage area.
[0059] Optionally, the network card of the sending server can also determine the first storage area storing the first SQE according to the correspondence between the identification of the QP to which the first SQE belongs, the position identification of the first SQE in the QP and the identification of the first storage area. For example, the identification of the first storage area is obtained by querying the table corresponding to the Blueframe buffer according to the identification of the QP to which the first SQE belongs and the position identification of the first SQE in the QP, and the first storage area is determined according to the identification of the first storage area, and then the Blueframe is directly taken from the first storage area in sequence for processing.
[0060] For example, the network card of the sending server can be pre-configured with a table for recording the identification of the storage area in the Blueframe buffer and the related information of the stored Blueframe, such as the correspondence between the identification of the QP to which the SQE belongs, the position identification of the SQE in the QP and the identification of the storage area. The identification of the first storage area can be the number of the storage area or the start address of the storage area. The table can be composed of logical internal registers, and records the free storage areas and the occupied storage areas. The address of the storage area can also be read according to the identification of the storage area, and whether it matches or not can be returned according to the identification of the storage area. For example, Table 1 provided by the application is an example, which includes the identification of the storage area, the identification of the QP to which the SQE belongs and the position of the SQE in the QP.
[0061] Table 1
[0062] Identification of the storage area Identification of the QP to which the SQE belongs Location of the SQE in the QP 0 qpn1 PI 1 qpn2 PI 2 qpn3 PI 3 qpn4 PI … … … 127
[0063] When the network card of the sending end server needs to write the segmented instruction into the storage area of the Blueframe buffer, the network card can query the table to obtain an idle storage area. The idle storage area can be a storage area which does not establish the correspondence between the identifier of the QP to which the SQE belongs, the position identifier of the SQE in the QP and the identifier of the storage area. If the network card of the sending end server queries the idle storage area, obtains the address of the idle storage area, and stores the jth segmented instruction of the first SQE into the jth storage unit of the idle storage area.
[0064] As a possible embodiment, if the network card of the sending end server does not query the idle storage area, it means that the Blueframe buffer is full, and the Blueframe is discarded, which is processed as a normal doorbell, that is, the network card of the sending end server needs to obtain other segmented instructions of the SQE from the cache by using the direct memory access technology. In addition, the other storage areas which have stored data can be full or not full, which is not limited in the embodiment of the application. In the embodiment of the application, the first storage area allocated by the network card of the sending end server is a storage area which does not store any data.
[0065] S205, the network card of the sending end server performs data processing according to the identifier of the QP to which the first SQE belongs and the position identifier of the first SQE in the QP.
[0066] The network card of the sending end server reads the data corresponding to the first SQE from the memory of the sending end server according to the first SQE, and sends the data to the receiving end server, that is, S206 to S209 are performed.
[0067] In a possible implementation, when the amount of data to be sent is also small, the data to be sent can be directly placed in the SQE, and the processor of the sending end server directly issues the SQE including the data to the network card of the sending end server in the form of segmented instruction, without the network card of the sending end server reading the data from the memory of the sending end server, so as to further reduce the time delay of data transmission. In the embodiment of the application, the data in the SQE can be referred to as inline data. If the first SQE includes data, the network card of the sending end server does not need to read the data from the memory of the sending end server, and directly sends the data included in the first SQE to the receiving end server after reading the first SQE from the Blueframe buffer.
[0068] S206, the network card of the sending end server sends the data to the network card of the receiving end server.
[0069] S207, the network card of the receiving end server reads the RQE corresponding to the first SQE from the memory of the receiving end server.
[0070] After the network card of the receiving end server receives the data sent by the network card of the sending end server, the data can be stored in the cache first, and then the network card of the receiving end server reads the RQE corresponding to the QP to which the first SQE belongs from the memory of the receiving end server.
[0071] After the network card of the receiving end server receives the data, the qp number in the data header can be read, and then the information (also referred to as qp context) of the corresponding QP is read according to the qp number. The QP information can exist in the memory of the processor or in the cache of the network card of the receiving end server. The address and pointer of the RQ are obtained according to the QP information, and then the RQE is read from the memory of the processor.
[0072] S208, the network card of the receiving end server stores the data in the memory indicated by the RQE.
[0073] S209, the network card of the receiving end server sends the CQE to the processor of the receiving end server.
[0074] The specific implementation of S206 to S209 can refer to the prior art, and will not be described here.
[0075] As a possible embodiment, multiple SQEs to be processed can exist in the sending end server at the same time, and the data processing process of each SQE can adopt the above operation process. The last segment of each SQE can add a barrier identifier, which is used to mark the last segment of the SQE and identify the last segment of different SQEs by the network card. In addition, the segments of multiple SQEs can be processed in parallel, and only the identifier of the belonging SQE needs to be added in each segment. At this time, the network card can identify and store the corresponding storage area according to the identifier, and store it in the corresponding storage unit, so as to further improve the data processing efficiency of the system.
[0076] The method for processing data provided by the embodiment of the application, the network card of the sending end server performs in-order processing on the out-of-order segment instructions of the Blueframe through the address of the PCIe message, directly stores the segment instructions in the corresponding storage unit of the Blueframe buffer, so that the processor of the sending end server does not need to pay attention to the in-order problem of each segment. The network card of the sending end server utilizes the idle storage area to realize the in-order processing of the segment instructions, reduces the processing process and load of the sending end server. On the other hand, the processor of the sending end server can simultaneously send multiple segment instructions of different SQEs or different segment instructions of the same SQE to the network card, thereby improving the efficiency of data processing. Moreover, in the sending process of multiple segment instructions of the same SQE, the Barrier function in the prior art is not needed to isolate each segment, thereby reducing the number of PCIe messages.
[0077] Next, in combination with Figure 4 Further introduce the data processing method provided by the embodiment of the application, the method comprises:
[0078] S401, the network card of the sending end server determines whether the i th received PCIe message is the first PCIe message of the first SQE.
[0079] If the network card of the sending end server determines that the i th received PCIe message is not the first PCIe message of the first SQE, that is, when i is not equal to 1, S402 is executed.
[0080] If the network card of the sending end server determines that the i th received PCIe message is the first PCIe message of the first SQE, that is, when i = 1, S403 is executed.
[0081] S402, the network card of the sending end server stores the j th instruction included in the i th PCIe message to the j th storage unit of the first storage area.
[0082] Specific explanation can refer to the description of S203, and the present application will not be repeated.
[0083] S403, the network card of the sending end server determines whether the j th instruction is the first segmented instruction of the first SQE.
[0084] In some embodiments, the network card of the sending end server can determine whether the low 8 of the first address segment is all 0, if the low 8 of the first address segment is all 0, it can be determined that the j th instruction is the first instruction of the first SQE, if the low 8 of the first address segment is not all 0, the j th instruction is not the first instruction of the first SQE, that is, it is any one of the second instruction to the eighth instruction of the first SQE.
[0085] It should be noted that, since the processor of the sending end server will send the network card of the sending end server other segmented instructions except the first instruction of the first SQE when the first SQE is blueframe. Therefore, the network card of the sending end server can also determine that the first SQE is blueframe, and the j th instruction is the data of blueframe.
[0086] If the network card of the sending end server determines that the j th instruction is not the first segmented instruction of the first SQE, that is, j is not equal to 1, S404 and S402 are executed.
[0087] If the network card of the sending end server determines that the j th instruction is the first segmented instruction of the first SQE, that is, j = 1, S405 is executed.
[0088] S404, the network card of the sending server allocates a first storage area.
[0089] Optionally, the network card of the sending server can record the number of the first storage area, so that after the network card of the sending server receives the PCIe message again, the network card of the sending server stores other segment instructions of the first SQE into the first storage area.
[0090] S405, the network card of the sending server determines whether the jth segment instruction is data of a blueframe.
[0091] The first segment instruction can be the content of the first 8 bytes of the first SQE. The first segment instruction can include the following contents:
[0092] 1) opcode: indicating the type of transmission instruction. For example, the opcode can include send, RDMA write, send first and sendonly. Send first indicates that the sent instruction is the first segment of the SQE. Sendonly indicates that the sent instruction is the SQE;
[0093] 2) sqe_index: indicating the position of the software generated SQE in the queue, that is, the pointer (PI);
[0094] 3) opc_mod: indicating how to process the data in the atomic (ATOMIC) operation, such as add, or, and, etc.
[0095] 4) DS: indicating the total length of the corresponding SQE;
[0096] 5) qp number: indicating the number of the QP where the SQE is located.
[0097] 6) sl: indicating the service level, that is, the scheduling priority of the corresponding QP.
[0098] The network card of the sending server can obtain the total length of the first SQE from the first segment instruction. The network card of the sending server compares the total length of the first SQE with a preset threshold. If the total length of the first SQE is less than or equal to the preset threshold, the network card of the sending server determines that the first segment instruction of the first SQE is data of a blueframe. If the total length of the first SQE is greater than the preset threshold, the network card of the sending server can determine that the first segment instruction of the first SQE is a doorbell, and obtain the first SQE according to a method of non-blueframe. For example, the direct memory access technology is needed to obtain other segment instructions of the first SQE from the cache.
[0099] If the network card of the sending server determines that the jth segment instruction is data of a blueframe, S404 and S402 are executed.
[0100] Further, if the network card of the sending server determines that the jth segment instruction is data of a blueframe, S406 to S408 can also be executed.
[0101] S406. The network card of the sending server obtains the identifier of the QP to which the first SQE belongs and the position identifier of the first SQE in the QP.
[0102] The network card of the sending server can obtain the identifier of the QP to which the first SQE belongs and the position identifier of the first SQE in the QP from the first segment instruction. The identifier of the QP to which the first SQE belongs can be the value of qp number, and the position identifier of the first SQE in the QP can be the value of sqe_index.
[0103] S407. The network card of the sending server establishes a correspondence relationship among the identifier of the QP to which the first SQE belongs, the position identifier of the first SQE in the QP, and the identifier of the first storage area.
[0104] For example, the network card of the sending server can be preconfigured with the table described in S204, which is used to record the identifier of the storage area in the Blueframe buffer and the related information of the stored Blueframe. For details, reference can be made to the description of S204, which will not be described herein.
[0105] S408. The network card of the sending server determines whether the first SQE is received completely.
[0106] In a possible implementation, the network card of the sending server can determine whether the accumulated value is equal to the total length of the first SQE to determine whether the first SQE is received completely. The accumulated value can be used to indicate the length of the received segment instruction of the first SQE. The initial value of the accumulated value can be 0. For example, the network card of the sending server can increase the accumulated value by 8 bytes after receiving 1 segment instruction, or the network card of the sending server can add 1 to the accumulated value after receiving 1 segment instruction. For example, if the network card of the sending server receives 1 segment instruction, the accumulated value is 8 bytes; if the network card of the sending server receives 2 segment instructions, the accumulated value is 16 bytes; and so on. It should be noted that the RAM implementing the Blueframe buffer can store the accumulated value. Of course, other storage media can also be used, which is not limited in the present application.
[0107] If the updated accumulated value is equal to the total length of the first SQE, it is determined that the first SQE is received completely, and S409 is executed.
[0108] If the updated accumulated value is not equal to the total length of the first SQE, it is determined that the first SQE is not received completely, and the next PCIe message is continuously received, i.e., the next segment instruction of the first SQE is received. For example, the network card of the sending server can continuously receive the i+1th PCIe message, and the i+1th PCIe message includes the kth segment instruction of the first SQE and a second address segment, where the second address segment is used to indicate the segmentation position of the kth segment instruction in the first SQE. The kth segment instruction is stored in the kth storage unit of the first storage area. The kth segment instruction is any one of the other segment instructions in the first SQE received by the network card of the sending server except the segment instruction. For example, when i=1, the first PCIe message includes the first segment instruction of the first SQE, and the kth segment instruction can be any one of the second segment instruction to the eighth segment instruction. When i=2, the first PCIe message includes the second segment instruction of the first SQE, the second PCIe message includes the third segment instruction of the first SQE, and the kth segment instruction can be any one of the first segment instruction, the fourth segment instruction to the eighth segment instruction. For each segment instruction included in the n PCIe messages, the data processing method provided in the embodiments of the present application can be used for processing, which is not described herein.
[0109] It should be noted that if the network card of the sending server determines that the first SQE is not received completely, after the other segment instructions are continuously received, it is necessary to determine whether the first SQE is received completely until the first SQE is received completely, and the end symbol is generated.
[0110] S409, the network card of the sending server generates an end symbol.
[0111] For example, the end symbol can be eop, and the end symbol can be used to separate different Blueframes.
[0112] It should be noted that after S404 and S402 are executed, the network card of the sending server receives the first segment instruction, and the first segment instruction is stored in the first storage unit of the first storage area, and S406 to S408 are executed, if the network card of the sending server determines that the jth segment instruction is not the first segment instruction of the first SQE, i.e., j is not equal to 1.
[0113] Further, in the case that the network card of the sending end server determines that the i-th PCIe message currently received is not the first PCIe message of the first SQE, i.e. i is not equal to 1, if j is equal to 1, i.e. the i-th PCIe message includes the first segment instruction, it can be understood that the network card of the sending end server has received other segment instructions except the first segment instruction of the first SQE before receiving the first segment instruction. The network card of the sending end server can also store the first segment instruction to the first storage unit of the first storage area, and perform S406 to S408.
[0114] It should be noted that the order of the steps of the data processing method provided in the present application can be appropriately adjusted, and the steps can also be increased or decreased accordingly according to the situation. For example, the order before and after S403 and S405 can be interchanged, i.e. the network card of the sending end server can first determine whether the j-th segment instruction is the data of the blueframe, and then determine whether the j-th segment instruction is the first segment instruction of the first SQE. Any person skilled in the art can easily think of a method of change within the technical range disclosed in the present application, which should be covered within the protection scope of the present application, and therefore will not be described in detail.
[0115] For example, the network card of the sending end server can determine whether the j-th segment instruction included in the first PCIe message is the data of the blueframe according to the first address segment. For example, the network card of the sending end server can include a plurality of registers, and the plurality of registers are used to store different data. In the present application, one or more registers of the plurality of registers included in the network card of the sending end server can be used as the buffer area of the blueframe, i.e. the area is used to store the blueframe. The address of the buffer area of the blueframe can be the address of the register. After the network card of the sending end server obtains the first address segment, it can first determine whether the first address segment is an address within the address range of the register, and if the first address segment is an address within the address range of the register, it can be determined that the j-th segment instruction is the data of the blueframe. Then, the network card of the sending end server can further determine whether the lower 8 bits of the first address segment are all 0, and if the lower 8 bits of the first address segment are all 0, it can be determined that the j-th segment instruction is the first segment instruction of the first SQE, and if the lower 8 bits of the first address segment are not all 0, the j-th segment instruction is not the first segment instruction of the first SQE, i.e. it is any one of the second segment instruction to the eighth segment instruction of the first SQE. If the first address segment is not an address within the address range of the register, it can be determined that the j-th segment instruction is not the data of the blueframe.
[0116] In addition, after the first SQE is received and stored to the first storage area, S204 to S209 can be performed.
[0117] To understand the data processing method provided by the embodiments of the present application, an example is given.
[0118] As shown in the figure, it is assumed that the processor of the sending server sends the 8 segment instructions of the Blueframe to the network card of the sending server in the order of 2, 4, 5, 3, 1, 8, 6, 7, i.e., the first PCIe message includes the second segment instruction, the second PCIe message includes the fourth segment instruction, the third PCIe message includes the fifth segment instruction, the fourth PCIe message includes the third segment instruction, the fifth PCIe message includes the first segment instruction, the sixth PCIe message includes the eighth segment instruction, the seventh PCIe message includes the sixth segment instruction, and the eighth PCIe message includes the seventh segment instruction. Figure 5
[0119] The order in which the network card of the sending server receives the 8 segment instructions of the Blueframe sent by the processor of the sending server is 2, 4, 5, 3, 1, 8, 6, 7.
[0120] After the network card of the sending server receives the first PCIe message sent by the processor of the sending server, the first PCIe message is parsed to obtain the second segment instruction and the first address segment. Then, the network card of the sending server can determine that the second segment instruction is located in the second segment of the Blueframe according to the first address segment. Therefore, the network card of the sending server can determine that the received second segment instruction is data of the Blueframe, and then the network card of the sending server stores the second segment instruction in the second storage unit of the first storage area. Similarly, after the network card of the sending server receives the first segment instruction, the third segment instruction to the eighth segment instruction sent by the processor of the sending server, the first segment instruction, the third segment instruction to the eighth segment instruction are respectively stored in the first storage unit, the third storage unit to the eighth storage unit of the first storage area.
[0121] In addition, the cumulative value is updated after the network card of the sending server receives each segment instruction. For example, the cumulative value can be increased by 8 bytes; or the cumulative value is added by 1 after the network card of the sending server receives each segment instruction. The initial value of the cumulative value can be 0.
[0122] It should be noted that after the network card of the sending end server receives the first segment instruction sent by the processor of the sending end server, the network card of the sending end server further obtains the identifier of the QP to which the Blueframe belongs and the location identifier of the Blueframe in the QP. Assuming that the identifier of the QP to which the Blueframe belongs is qp2 and the location identifier of the Blueframe in the QP is PI3. The network card of the sending end server establishes the correspondence among qp2, PI3 and the first storage area. In addition, the network card of the sending end server can further obtain the total length of the Blueframe and determine whether the fifth updated cumulative value is equal to the total length of the Blueframe. In the embodiment of the present application, since the Blueframe includes 8 segment instructions, the total length of the Blueframe is 64 bytes, and therefore the updated cumulative value is less than the total length of the Blueframe. The other segment instructions of the Blueframe are continuously received.
[0123] The above mainly introduces the scheme provided by the embodiments of the present application from the perspective of method. In order to realize the above functions, it contains the hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0124] The embodiments of the present application can divide the functional modules of the data processing apparatus according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module. The above integrated module can be realized in the form of hardware or software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division manner.
[0125] The above describes the method of data processing provided by the present application in combination with Figure 2 and Figure 4 , and the following describes the data processing apparatus, network card and server provided by the present application in combination with Figure 6 and Figure 8 .
[0126] Figure 6 A structural schematic diagram of a data processing apparatus 60 provided by the embodiments of the present application is shown. The apparatus 60 can be used to executeFigure 2 and Figure 4 The method of processing data shown in any of the accompanying drawings. The apparatus 60 can comprise: a receiving unit 601 and a processing unit 602. Wherein the receiving unit 601 is configured to receive an i th PCIe message, the i th PCIe message comprising a j th segment instruction of a first SQE and a first address segment. The processing unit 602 is configured to store the j th segment instruction to a j th storage unit of a first storage area. When n segment instructions of the first SQE are all stored to the first storage area, acquire the first SQE according to the instructions of the n storage units in the first storage area, and the identification of a queue pair (QP) to which the first SQE belongs and the position identification of the first SQE in the QP, and perform data processing according to the identification of the QP to which the first SQE belongs and the position identification of the first SQE in the QP. For example, in combination with Figure 2 The processing unit 602 can be configured to perform S203 to S205. For another example, in combination with Figure 4 The processing unit 601 can be configured to perform S401 to S409.
[0127] In the embodiments of the present application, further, as shown in Figure 6 The apparatus can further comprise a sending unit 603.
[0128] The sending unit 603 is configured to send data, for example, to support the communication apparatus to perform S206 in the method shown in Figure 2
[0129] It should be noted that all related contents of each step involved in the above method embodiments can be cited to the function description of the corresponding function module, which will not be repeated here.
[0130] The apparatus provided by the embodiments of the present application is used to perform the method of any of the above embodiments, and thus can achieve the same effect as the method of the above embodiments.
[0131] It should be understood that the apparatus 60 of the embodiments of the present application can be realized by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be realized by software Figure 2 and Figure 4 The method shown, the device 60 and each module thereof can also be a software module.
[0132] The explanation of the related content and the description of the beneficial effects in this embodiment can refer to the above method embodiment.
[0133] Figure 7 A hardware structure schematic diagram of a network card 70 provided in the embodiment of the present application is shown in the figure. As shown in the figure, the network card 70 comprises at least one processor 701, a communication line 702, a memory 703 and a communication interface 704. The communication line 702 can comprise a channel for transmitting information between the at least one processor 701, the memory 702 and the communication interface 704. The communication interface 704 is used for the network card 704 to communicate with other devices or apparatuses. The communication interface 704 can comprise a wired transceiver or a wireless transceiver. The wireless transceiver can comprise a communication chip. The at least one processor 701 and the communication chip can be integrated together or can be independently arranged. The memory 703 is used for storing computer execution instructions for executing the scheme of the present application and is controlled by the processor 701 to execute. The processor 701 is used for executing the computer execution instructions stored in the memory 703, thereby realizing the method of data processing provided in the above embodiment of the present application. The explanation of the related content and the description of the beneficial effects in this embodiment can refer to the above method embodiment.
[0134] Figure 8 A structure schematic diagram of a server 80 provided in the embodiment of the present application is shown in the figure. As shown in the figure, the server 80 comprises at least one processor 801, a communication line 802, a memory 803, a network card 804 and a communication interface 805. The communication interface 805 can comprise a wired transceiver or a wireless transceiver. The wireless transceiver can comprise a communication chip. The at least one processor 801 and the communication chip can be integrated together or can be independently arranged.
[0135] The processor 801 can be a general purpose CPU, and can also be other general purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general purpose processor can be a microprocessor or any conventional processor, etc. The processor 801 can also be a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling program execution of the solutions of the present application.
[0136] The communication line 802 can include a path for transmitting information between the above-mentioned components (such as the processor 801, the memory 803, the network card 804, and the communication interface 805).
[0137] The memory 803 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 803 can exist independently and be connected to the processor 801 through the communication line 802. The memory 803 can also be integrated with the processor 802. The memory 803 provided by the embodiments of the present application can generally be non-volatile. The memory 803 is used to store computer execution instructions for executing the solutions of the present application, and is controlled by the processor 801 to execute. The processor 801 is used to execute the computer execution instructions stored in the memory 803, so as to realize the method of data processing provided by the above-mentioned embodiments of the present application.
[0138] The structure of the network card 804 can refer to the above Figure 7 which will not be described here.
[0139] The communication interface 805 can be any transceiver-like device that enables the server 80 to communicate with other devices.
[0140] Optionally, the computer-executed instructions in the embodiments of the present application can also be referred to as application program codes.
[0141] As an example, the processor 801 can include one or more CPUs. As an example, the server 80 can include multiple processors. Each of the processors can be a single-CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0142] The server 80 described above can be a general-purpose device or a special-purpose device. For example, the server 80 can be an X86, ARM-based server, or other special-purpose servers such as a policy control and charging (PCC) server, etc. The embodiments of the present application do not limit the type of the server 80. ARM is an abbreviation of advanced RISC machines, and RISC is an abbreviation of reduced instruction set compute.
[0143] The embodiments of the present application also provide a communication system, which can include the server 80, where the server 80 can serve as a sending-end server. In addition, the communication system also includes a receiving-end server for receiving data sent by the sending-end server, so that the receiving-end server performs the method of data processing described above.
[0144] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer executes the computer instructions, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer readable storage medium can be any available medium that can be accessed by a computer or include one or more data storage devices that can be integrated with the medium, such as a server, data center, etc. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.
[0145] The above is only a specific embodiment of the present application. Those skilled in the art can think of changes or replacements based on the specific embodiments provided by the present application, which should be covered within the protection scope of the present application.
Claims
1. A method of data processing, characterized by, The method comprises the following steps: A network card receives a peripheral component interconnect express (PCIe) message, wherein the PCIe message comprises a jth segment instruction of a first send queue entry (SQE) and position indication information, the position indication information is used to indicate the position of the jth segment instruction in the first SQE, j is an integer, j∈[1, n], the first SQE is divided into n segments, n is an integer, n is greater than or equal to 1; When the length of the first SQE is less than or equal to a preset threshold, the network card stores the jth segment instruction into a jth storage unit of a first storage area, the first storage area is a storage area of the network card used for storing the first SQE, the first storage area comprises m storage units, m is an integer, n≤m; When the n segment instructions of the first SQE are all stored into the first storage area, the network card processes the first SQE.
2. The method of claim 1, wherein, When the network card receives a first PCIe message, the position indication information is an address segment, before the network card stores the jth segment instruction into the jth storage unit of the first storage area, the method further comprises the following steps: The network card judges whether the address segment is an address in an address range of a storage space where the first storage area is located; if the address segment is an address in the address range of the storage space where the first storage area is located, the network card determines that the jth segment instruction is blueframe data; If the address segment is not an address in the address range of the storage space where the first storage area is located, the network card determines that the jth segment instruction is not blueframe data.
3. The method of claim 1, wherein, When j=1, after the network card receives the PCIe message, the method further comprises the following steps: The network card acquires the total length of the first SQE, the first segment instruction comprises the total length of the first SQE; The network card compares the total length of the first SQE with a preset threshold; If the total length of the first SQE is less than or equal to the preset threshold, the network card determines that the first segment instruction of the first SQE is blueframe data; If the total length of the first SQE is greater than the preset threshold, the network card determines that the first segment instruction of the first SQE is not blueframe data.
4. The method of claim 3, wherein, After the network card stores the jth segment instruction into the jth storage unit of the first storage area, the method further comprises the following steps: The network card judges whether the first SQE is received completely; If the first SQE is received completely, an end symbol is generated.
5. The method according to any one of claims 3-4, characterized in that, If the total length of the first SQE is less than or equal to the preset threshold, the method further comprises the following steps: The network card acquires the identifier of the QP to which the first send queue entry SQE belongs and the position identifier of the first send queue entry SQE in the QP, and the first segment instruction includes the identifier of the QP to which the first send queue entry SQE belongs and the position identifier of the first send queue entry SQE in the QP; The network card establishes a correspondence relationship among the identifier of the QP to which the first send queue entry SQE belongs, the position identifier of the first send queue entry SQE in the QP, and the identifier of the first storage area.
6. The method of claim 5, wherein, Further comprising: The network card determines the first storage area for storing the first send queue entry SQE according to the identifier of the QP to which the first send queue entry SQE belongs and the position identifier of the first send queue entry SQE in the QP; The network card sequentially reads n segment instructions of the first send queue entry SQE from the start address of the first storage area, and acquires and sends data corresponding to the first send queue entry SQE according to the n segment instructions of the first send queue entry SQE.
7. An apparatus for data processing, characterized by Comprising: The receiving unit is configured to receive a peripheral component interconnect express (PCIe) message, the PCIe message including a jth segment instruction of a first send queue entry (SQE) and position indication information, the position indication information being used to indicate a segment position of the jth segment instruction in the first SQE, where j is an integer, j∈[1, n], the first SQE is divided into n segments according to a preset size, n is an integer, and n is greater than or equal to 1; The processing unit is configured to, in a case where a length of the first SQE is less than or equal to a preset threshold, store the jth segment instruction into a jth storage unit of a first storage area, where the first storage area is a storage area in a network card for storing the first SQE, and the first storage area includes m storage units, m is an integer, and n≤m; The processing unit is further configured to, when all the n segment instructions of the first SQE are stored into the first storage area, process the first SQE by the network card.
8. The apparatus of claim 7, wherein, The position indication information is an address segment, The processing unit is further configured to, when the received PCIe message is a first PCIe message, determine whether the address segment is an address within an address range of a storage space where the first storage area is located, determine that the jth segment instruction is data of a blue frame if the address segment is an address within the address range of the storage space where the first storage area is located, and determine that the jth segment instruction is not data of a blue frame if the address segment is not an address within the address range of the storage space where the first storage area is located.
9. The apparatus of claim 7, wherein The processing unit is further configured to, when j=1, acquire a total length of the first SQE, and the first segment instruction includes the total length of the first SQE. comparing the total length of the first send queue entry (SQE) with a preset threshold; if the total length of the first send queue entry (SQE) is less than or equal to the preset threshold, determining that the first segment instruction of the first send queue entry (SQE) is data of a blue frame; if the total length of the first send queue entry (SQE) is greater than the preset threshold, determining that the first segment instruction of the first send queue entry (SQE) is not data of a blue frame.
10. The apparatus of claim 9, wherein, the processing unit is further configured to, if the total length of the first send queue entry (SQE) is less than or equal to the preset threshold, acquire an identifier of a QP to which the first send queue entry (SQE) belongs and a position identifier of the first send queue entry (SQE) in the QP, and the first segment instruction comprises the identifier of the QP to which the first send queue entry (SQE) belongs and the position identifier of the first send queue entry (SQE) in the QP; establish a correspondence between the identifier of the QP to which the first send queue entry (SQE) belongs, the position identifier of the first send queue entry (SQE) in the QP, and an identifier of the first storage area.
11. The apparatus of claim 10, wherein, the processing unit is specifically configured to determine, according to the identifier of the QP to which the first send queue entry (SQE) belongs and the position identifier of the first send queue entry (SQE) in the QP, a first storage area in which the first send queue entry (SQE) is stored; read the n segment instructions of the first send queue entry (SQE) from a start address of the first storage area in sequence; and acquire and send data corresponding to the first send queue entry (SQE) according to the n segment instructions of the first send queue entry (SQE).
12. A server, characterized by The server comprises a processor, a memory, and a network card. The processor is configured to divide a first send queue entry (SQE) into n segment instructions according to a preset size, and send the n segment instructions of the first send queue entry (SQE) to the network card. n is an integer, and n is greater than or equal to 1. The memory is configured to store the n segment instructions of the first send queue entry (SQE) and computer execution instructions. The network card is configured to invoke the computer execution instructions to process the n segment instructions of the first send queue entry (SQE), so that when the network card is running, the computer execution instructions are executed to implement the method of any one of claims 1 to 6.
Citation Information
Patent Citations
Data processing method, network card and server
CN111641566A
Reducing number of write operations relative to delivery of out-of-order RDMA send messages
US20050132017A1