Data transmission method, apparatus, device, storage medium, and computer program product

Abstract

The application discloses a data transmission method, device, equipment, storage medium and computer program product, wherein the method comprises: a first device splits a first data buffer into multiple segments in response to a first system call of an application process; the first data buffer is used for buffering first data written by the application process; based on the multiple segments, a first mbuf and multiple second mbufs are allocated from a first memory pool; the first mbuf is used for storing message header information corresponding to a first message; the first message represents a message used for transmitting the first data; each second mbuf in the multiple second mbufs is used for indicating a segment; a first mbuf and multiple second mbufs are linked into a first linked list, and the first linked list is mapped to a network card, so that the network card sends the first message to a second device; the first linked list is used for reading the first data from the first data buffer by the network card and encapsulating the first data into the first message.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data transmission method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] In related technologies, when a device transmits data generated by an application process, it needs to perform multiple copy operations and context switches between the user space and kernel space of the operating system, resulting in high data transmission latency. Summary of the Invention

[0003] To address the related technical problems, embodiments of this application provide a data transmission method, apparatus, device, storage medium, and computer program product.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a data transmission method applied to a first device, the method comprising:

[0006] In response to a first system call from the application process, a first data buffer is divided into multiple segments; the first data buffer is used to buffer first data written by the application process; the first system call is used to instruct the sending of the first data.

[0007] Based on the multiple segments, a first mbuf and multiple second mbufs are allocated from the first memory pool; the first mbuf is used to store the header information of the first message, the first message representing a message used to transmit the first data; each of the multiple second mbufs is used to indicate one of the segments.

[0008] The first mbuf is linked with the plurality of second mbufs to form a first linked list, and the first linked list is mapped to the network card so that the network card can send the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

[0009] In the above scheme, the second mbuf includes one or more of the following:

[0010] The virtual address of the corresponding segment;

[0011] The data length of the corresponding segment.

[0012] The method in the above scheme further includes:

[0013] Add the following information corresponding to the first message to the first mbuf in sequence:

[0014] Transport layer header information, which represents the message header information encapsulated based on the transport layer protocol;

[0015] Network layer header information, which represents the message header information encapsulated based on the network layer protocol;

[0016] Link layer header information, which represents the message header information encapsulated based on the link layer protocol.

[0017] In the above scheme, the first data buffer includes the application layer header information corresponding to the first message; the application layer header information represents the message header information encapsulated based on the application layer protocol.

[0018] The application layer header information includes one or more of the following:

[0019] The first field is used to indicate the protocol type and / or protocol version corresponding to the first message;

[0020] The second field is used to indicate the service type corresponding to the first message;

[0021] The third field is used to identify the first data buffer;

[0022] The fourth field is used to indicate the memory state of the first data buffer.

[0023] In the above scheme, after the application process writes the first data into the first data buffer and before the first device responds to the application process's first system call, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or,

[0024] After the first device responds to the first system call of the application process and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or,

[0025] After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state;

[0026] The second message represents the message returned by the second device based on the first message.

[0027] In the above scheme, the second message encapsulates the same field as the third field.

[0028] In the above scheme, when the fourth field indicates that the memory state of the first data buffer is the set second memory state, the application process is allowed to perform modification and / or release operations on the first data buffer; and / or,

[0029] If the fourth field indicates that the memory state of the first data buffer is the set third memory state, the application process is not allowed to perform modification and / or release operations on the first data buffer.

[0030] In the above scheme, prior to responding to the first system call of the application process, the method further includes:

[0031] The first system call was hijacked.

[0032] This application embodiment also provides a data transmission device, applied to a first device, including:

[0033] A segmentation unit is configured to, in response to a first system call from an application process, divide a first data buffer into multiple segments; the first data buffer is configured to buffer first data written by the application process; the first system call is configured to instruct the sending of the first data.

[0034] An allocation unit is configured to allocate a first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is used to store the header information of a first message, the first message representing a message used to transmit the first data; each of the plurality of second mbufs is used to indicate one of the segments.

[0035] The linking unit is used to link the first mbuf with the plurality of second mbufs into a first linked list, and map the first linked list to the network card, so that the network card can send the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

[0036] This application also provides a first device, including: a processor and a memory for storing a computer program that can run on the processor, wherein the processor, when running the computer program, performs the steps of any of the aforementioned methods.

[0037] This application also provides a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of any of the aforementioned methods.

[0038] This application also provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of any of the aforementioned methods.

[0039] In this embodiment, the first device, in response to a first system call from an application process, divides a first data buffer into multiple segments. The first data buffer is used to buffer first data written by the application process, and the first system call is used to instruct the sending of the first data. Then, based on the multiple segments, a first mbuf and multiple second mbufs are allocated from a first memory pool. The first mbuf is used to store the header information corresponding to the first message, and the first message represents a message used to transmit the first data. Each of the multiple second mbufs is used to indicate a segment. Afterward, the first mbuf and the multiple second mbufs are linked into a first linked list, and the first linked list is mapped to the network card, so that the network card sends the first message to the second device. The first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into a first message. In the above scheme, when the application process calls the first system call, the first device links a first mbuf allocated from the first memory pool with multiple second mbufs to form a first linked list. This allows the network card to directly read the first data in the first data buffer based on the first linked list, and then encapsulate the first packet with the packet header information stored in the second mbufs and send it. Compared with related technologies, the first device avoids multiple copies and context switches between user space and kernel space during the data transmission process, thereby reducing data transmission latency. Attached Figure Description

[0040] Figure 1 A schematic diagram illustrating the implementation flow of a data transmission method provided in an embodiment of this application;

[0041] Figure 2 A schematic diagram of a first data buffer provided in an embodiment of this application;

[0042] Figure 3 A schematic diagram of the processing flow of a data transmission method provided for an application embodiment of this application;

[0043] Figure 4 A schematic diagram of memory state changes provided for an application embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the hardware structure of a first device provided in an embodiment of this application. Detailed Implementation

[0046] In related technologies, during the transmission of data generated by application processes, devices need to perform multiple copy operations and context switches between the operating system's user space and kernel space. For example, when sending data generated by an application process, the device writes the data to a buffer in user space. Then, through a system call from the application process, the operating system's context is switched from user space to kernel space, and the data is copied from user space to kernel space. Finally, through the network card driver, the data is sent from kernel space to the device receiving the data, and the operating system's context is switched back from kernel space to user space. This increases the load on the device's central processing unit (CPU), resulting in higher data transmission latency.

[0047] Based on this, in this embodiment of the application, the first device responds to a first system call from the application process by dividing the first data buffer into multiple segments. The first data buffer is used to buffer the first data written by the application process, and the first system call is used to instruct the sending of the first data. Then, based on the multiple segments, a first mbuf and multiple second mbufs are allocated from the first memory pool. The first mbuf is used to store the header information corresponding to the first message, and the first message represents the message used to transmit the first data. Each of the multiple second mbufs is used to indicate a segment. Afterward, the first mbuf and the multiple second mbufs are linked into a first linked list, and the first linked list is mapped to the network card, so that the network card sends the first message to the second device. The first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into a first message. In the above scheme, when the application process calls the first system call, the first device links a first mbuf allocated from the first memory pool with multiple second mbufs to form a first linked list. This allows the network card to directly read the first data in the first data buffer based on the first linked list, and then encapsulate the first packet with the packet header information stored in the second mbufs and send it. Compared with related technologies, the first device avoids multiple copies and context switches between user space and kernel space during the data transmission process, thereby reducing data transmission latency.

[0048] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0049] This application provides a data transmission method applied to a first device.

[0050] In practical applications, the first device can be understood as a device used for transmitting data. For example, the first device may include a client device or a server device, etc.

[0051] The first device may carry an operating system, which can provide functions such as resource management, task scheduling, user interface provision, and system service support. The operating system may include a user-mode protocol stack, which runs in the user space of the operating system. In practical applications, the data transmission method provided in this embodiment can be applied to the user-mode protocol stack of the first device.

[0052] The first device may also include a network interface card (NIC). The NIC can be understood as the hardware interface between the first device and the network. The NIC can be used to transmit data between the first device and other devices outside the first device.

[0053] See here. Figure 1 The data transmission method provided in this application includes:

[0054] Step 101: In response to the first system call of the application process, the first data buffer is divided into multiple segments.

[0055] The first data buffer is used to buffer the first data written by the application process; the first system call is used to indicate the sending of the first data.

[0056] In practical applications, an application process can be understood as an instance of an application running on the first device, that is, the process corresponding to the application.

[0057] During the execution of the application process, the first data can be generated and written into the first data buffer. Then, by calling the first system call, the first device is instructed to send the first data to other devices besides the first device.

[0058] For example, the first system call may include a write system call and / or a send system call.

[0059] In practical applications, the first data buffer can represent a data buffer used to buffer the first data. The data buffer can be understood as a memory pool or memory buffer, where data can be stored contiguously. Before an application process writes the first data to the first data buffer, the first device can allocate the first data buffer from a pre-allocated data buffer based on the application process's request.

[0060] In practical applications, the processing performed by the first device in response to the first system call of the application process can also be understood as the processing performed by the first device when the application process invokes the first system call.

[0061] In practical applications, when the first device responds to the first system call of the application process, the operating system of the first device can perform the corresponding processing only in user space, without processing in kernel space. That is, it avoids entering kernel mode, thereby avoiding interaction and context switching between user space and kernel space, and improving processing efficiency.

[0062] In one embodiment, prior to responding to a first system call from the application process, the data transmission method provided in this application further includes:

[0063] Hijacking the first system call.

[0064] In practical applications, the processing function corresponding to the first system call can be rewritten from the operating system's built-in function to a custom function. Then, when the application calls the first system call, the first device can no longer execute the operating system's built-in function, but instead execute the custom function. This custom function allows the first device to avoid entering kernel mode during processing, thereby avoiding interaction and context switching between user space and kernel space during processing, and improving processing efficiency.

[0065] The processing logic of the custom processing function can be understood by referring to the processing performed by the first device in response to the first system call in the embodiments of this application.

[0066] Here, the first device responds to the first system call of the application process by dividing the first data buffer into multiple segments. In practical applications, during the process of dividing the first data buffer into multiple segments, the first device can divide the first data buffer into multiple segments based on a first parameter, such that the data length of each segment does not exceed the data length indicated by the first parameter. The first parameter can also be called the maximum segment size or the maximum segment length.

[0067] The first parameter can be determined based on the value of the network card's Maximum Transmission Unit (MTU).

[0068] In practical applications, after the first data buffer is divided into multiple segments, the first data included in the first data buffer can be distributed in each segment. That is to say, one segment of the first data buffer can store part of the first data.

[0069] In practical applications, when the first device sends the first data, it needs to encapsulate the first data into a first message, which is then sent by the network interface card (NIC) in the first device. The data length corresponding to the first message may exceed the length of the data frame that the NIC can transmit at any time. In this case, the NIC can divide the first message into multiple segments for transmission.

[0070] In practical applications, the data in each segment of the first data buffer can be used to form a segment corresponding to the first message. Therefore, dividing the first data buffer into multiple segments can also be understood as dividing the first data buffer into multiple message segments.

[0071] In this embodiment, the first data buffer is segmented before the network card sends the first message. In this way, when the network card sends the first message, it can more efficiently divide the first message into multiple message segments for transmission, thereby improving data transmission efficiency and reducing data transmission latency.

[0072] Step 102: Based on multiple segments, allocate one first mbuf and multiple second mbufs from the first memory pool.

[0073] The first mbuf is used to store the header information of the first message, and the first message represents the message used to transmit the first data; each of the multiple second mbufs is used to indicate a segment.

[0074] In practical applications, the first memory pool and the data buffer can be two non-contiguous memory regions. The first memory pool and the data buffer can be allocated in advance through different memory requests, thereby increasing the flexibility of memory management.

[0075] In practical applications, data in the data buffer can be stored contiguously. Therefore, if message header information is written directly into the first data buffer or an adjacent data buffer after the first data is written to the first data buffer, it may corrupt the data originally stored in the data buffer. In this embodiment, the first mbuf in the first memory pool is used to store the message header information, avoiding corruption of the data in the data buffer. On this basis, the accuracy and reliability of data transmission are improved.

[0076] In practical applications, the message header information stored in the first mbuf can be used to encapsulate the message header information of the first message and / or the message header information of the segment corresponding to the first message. The first mbuf can also be called the head-mbuf.

[0077] In one embodiment, the data transmission method provided in this application further includes:

[0078] Add the following information corresponding to the first message to the first mbuf in sequence:

[0079] Transport layer header information, which represents the message header information encapsulated based on the transport layer protocol;

[0080] Network layer header information represents the message header information encapsulated based on network layer protocols.

[0081] Link layer header information represents the message header information encapsulated based on the link layer protocol.

[0082] In practical applications, the user-space protocol stack in the first device may include a network protocol stack. For example, this network protocol stack can be characterized as an Open Systems Interconnection Reference Model (OSI) network protocol stack. The network protocol stack can encapsulate transport layer header information, network layer header information, and link layer header information, and then sequentially add these packet header information to the first mbuf, which is equivalent to realizing the downward transmission of the first packet in the network protocol stack.

[0083] It should be noted that during the process of adding the header information corresponding to the first packet to the first mbuf, the first packet may not have been encapsulated yet, but the header information corresponding to each layer of the network protocol stack of the first packet has been encapsulated and added to the first mbuf in sequence. This is equivalent to simulating the process of the first packet being passed down in the network protocol stack. Therefore, it can also be regarded as realizing the downward transmission of the first packet in the network protocol stack.

[0084] For multiple segments corresponding to the first message, the field types and positions of the fields in the header information of these segments can be the same. Therefore, the header information of each segment can be encapsulated using the header information stored in a first mbuf. During the encapsulation of the header information of each segment, the header information of that segment can be encapsulated by modifying the values ​​of some fields in the header information stored in the first mbuf. In this way, the header information of multiple segments can be encapsulated using the header information stored in a first mbuf, saving memory space.

[0085] It should be noted that during the encapsulation process, modifications to the message header information stored in the first mbuf can be made after reading or copying the message header information from the first mbuf. The message header information stored in the first mbuf may remain unchanged before and after the modification.

[0086] Here, each of the multiple second mbufs is used to indicate a segment. In practical applications, the second mbuf can also be called the data-mbuf.

[0087] In one embodiment, the second mbuf includes one or more of the following:

[0088] The virtual address of the corresponding segment;

[0089] The data length of the corresponding segment.

[0090] In practical applications, the storage location of the data in the corresponding segment in the first data buffer can be determined by the second mbuf, and / or the length of the data in the corresponding segment can be determined.

[0091] Step 103: Link a first mbuf with multiple second mbufs to form a first linked list, and map the first linked list to the network card so that the network card can send the first packet to the second device.

[0092] The first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into a first message.

[0093] In practical applications, each node in the first linked list can be considered as an mbuf. The head node of the first linked list can be represented as the first mbuf, and the order of each second mbuf in the first linked list can be determined based on the order of the data corresponding to that second mbuf in the data buffer.

[0094] During the process of mapping the first linked list to the network card, the first linked list can be added to the first queue, and then the first queue can be added to the network card. After that, the network card can read the first linked list in the first queue and perform corresponding processing based on the first linked list.

[0095] In practical applications, the nodes in the first linked list are connected by pointers. Based on this, the nodes in the first linked list can be accessed from the head node. Therefore, when adding the first linked list to the first queue, only the head node of the first linked list, i.e., only the first mbuf, can be added to the first queue. The first queue can be called a descriptor queue or a send queue, and the first mbuf added to the first queue can be called an mbuf descriptor.

[0096] In practical applications, the network card can read the first data in the first data buffer into the network card's internal cache through direct memory access (DMA) based on the instruction of the second mbuf in the first linked list. The network card can also read the packet header information corresponding to the first packet based on the first mbuf in the first linked list. Then, the network card can encapsulate the first data into a first packet based on the read first data and the packet header information, and then send the first packet to the second device. The second device can be understood as other devices besides the first device.

[0097] During the process of the network card encapsulating the first packet based on the first linked list and sending it, the first device does not need to perform copy operations and context switching between user space and kernel space. Data transmission can be achieved only in user space, thereby improving data transmission efficiency and reducing data transmission latency.

[0098] In practical applications, the first device can send data without copying between user space and kernel space, which can be understood as achieving zero-copy transmission.

[0099] In this embodiment, when the application process calls the first system call, the first device links a first mbuf allocated from the first memory pool with multiple second mbufs to form a first linked list. This allows the network card to directly read the first data in the first data buffer based on the first linked list, and then encapsulate the first packet with the packet header information stored in the second mbufs and send it. Compared with related technologies, the first device avoids multiple copies and context switches between user space and kernel space during data transmission, thus enabling data transmission only in user space and reducing data transmission latency.

[0100] In practical applications, the application process in the first device can control the memory ownership of the first data, that is, the application process can modify and / or release the first data buffer where the first data is located.

[0101] The following section further explains how application processes control memory ownership.

[0102] In one embodiment, the first data buffer includes application layer header information corresponding to the first message; the application layer header information represents the message header information encapsulated based on the application layer protocol.

[0103] Application layer header information includes one or more of the following:

[0104] The first field is used to indicate the protocol type and / or protocol version corresponding to the first message;

[0105] The second field is used to indicate the service type corresponding to the first message;

[0106] The third field is used to identify the first data buffer.

[0107] The fourth field is used to indicate the memory status of the first data buffer.

[0108] In practical applications, when an application process requests a first data buffer, it can determine the required buffer length based on the amount of data in the application layer header information corresponding to the first data and the first message, and then request the first data buffer based on that buffer length. The buffer length can also be understood as the memory size of the corresponding data buffer.

[0109] The first data buffer can be divided into two regions: a first region and a second region. The first region can be used to store the application layer header information corresponding to the first message, and the second region can be used to store the first data. During the process of dividing the first data buffer into multiple segments, the second region can also be divided into multiple segments.

[0110] In practical applications, a gateway can determine the application layer header information that needs to be carried in the first message based on the application layer header information in the data buffer, and then encapsulate the determined application layer header information into the first message. For example, a gateway can encapsulate the application layer header information in the data buffer into the first message.

[0111] Here, the first field is used to indicate the protocol type and / or protocol version corresponding to the first message. In practical applications, the protocol type and / or protocol version corresponding to the first message can be used to ensure the reliability and consistency of the first message during network transmission. For example, the protocol type indicated by the first field may include at least one or more of the following: Transmission Control Protocol (TCP), Internet Protocol (UDP), and Internet Protocol (IP).

[0112] Here, the second field is used to indicate the service type corresponding to the first message. In practical applications, the service type can be understood as the service function of the first message. For example, the service type indicated by the second field can include at least one or more of the following: write request, data message, and acknowledgment message.

[0113] Here, the third field is used to identify the first data buffer. In practical applications, the first data buffer can be understood as a memory block, and the value of the third field can be equivalent to the sequence number of that memory block.

[0114] Here, the fourth field indicates the memory state of the first data buffer. In practical applications, the memory state can be used by application processes to track the lifecycle and accessibility of the memory block represented by the first data buffer. The value of the fourth field can be understood as the status bit of the memory state of the first data buffer.

[0115] For example, the memory status may include: ready, inflight, and acked, and the corresponding values ​​of the fourth field may be ready, inflight, and acked, respectively.

[0116] For example, Figure 2 The diagram of the first data buffer is given, where protocol version is equivalent to the first field, type is equivalent to the second field, sequence number is equivalent to the third field, state is equivalent to the fourth field, and data is equivalent to the first data.

[0117] In this embodiment of the application, the first data buffer includes application layer header information corresponding to the first message. Based on this, the application process can track the lifecycle and accessibility of the memory block represented by the first data buffer based on the application layer header information stored in the first data buffer, thereby realizing control over the memory ownership of the first data.

[0118] In one embodiment, after the application process writes the first data into the first data buffer and before the first device responds to the application process's first system call, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or,

[0119] After the first device responds to the first system call of the application process and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set second memory state; and / or,

[0120] After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state;

[0121] The second message represents the message returned by the second device based on the first message.

[0122] In practical applications, after the application process requests a first data buffer and writes the first data into it, the first device can set the value of the fourth field in the application layer header information of the first data buffer, so that the fourth field is used to indicate the memory state of the first data buffer as the set first memory state. For example, the set first memory state can be represented as ready, and the first device can set the value of the fourth field to ready.

[0123] After the application process invokes the first system call, when the first device responds to the first system call and processes it, the first device can set the value of the fourth field in the application layer header information of the first data buffer, so that the fourth field is used to indicate the memory state of the first data buffer as the set second memory state. For example, the set second memory state can be represented as inflight, and the first device can set the value of the fourth field to inflight.

[0124] After the first device receives the second message, it can set the value of the fourth field in the application layer header information of the first data buffer, so that the fourth field is used to indicate the memory state of the first data buffer as a set third memory state. For example, the set third memory state can be represented as acked, and the first device can set the value of the fourth field to acked.

[0125] In practical applications, the user-space protocol stack of the first device can set the value of the fourth field during the process of setting the field value.

[0126] In practical applications, the second message returned by the second device based on the first message can be used to indicate that the second device has acknowledged receiving the first message, and the second message can also be understood as an acknowledgment message.

[0127] In one embodiment, the second message encapsulates the same fields as the third field.

[0128] In practical applications, after receiving the first message, the second device can verify the data integrity of the first message and, if the verification is successful, return a second message to the first device. This second message encapsulates the same fields as the third field; that is, the second message carries the sequence number of the first data buffer.

[0129] After receiving the second message, the first device can parse the second message to determine the sequence number for the data buffer carried in the second message. Then, it can set the value of the fourth field in the data buffer corresponding to the sequence number, that is, set the value of the fourth field in the first data buffer so that the fourth field is used to indicate the memory state of the first data buffer as the set third memory state.

[0130] In one embodiment, if the fourth field indicates that the memory state of the first data buffer is a set second memory state, then application processes are allowed to perform modification and / or release operations on the first data buffer; and / or,

[0131] If the fourth field indicates that the memory state of the first data buffer is the set third memory state, the application process is not allowed to perform modification and / or release operations on the first data buffer.

[0132] In practical applications, when the fourth field indicates that the memory state of the first data buffer is the set second memory state, it can be considered that the first device has performed the first data transmission process, but has not yet received the acknowledgment message from the second device. In this case, the application process is not allowed to modify and / or release the first data buffer. This can avoid modification of the data in the first data buffer, thereby avoiding data inconsistency caused by retransmission during the transmission process and improving the reliability of data transmission.

[0133] In practical applications, when the fourth field indicates that the memory state of the first data buffer is the set third memory state, it can be considered that the first device has received the acknowledgment message from the second device, that is, the transmission of the first message has been completed. In this case, the protocol stack may not be allowed to perform modification and / or release operations on the first data buffer, but the application process may be allowed to perform modification and / or release operations on the first data buffer, thereby giving the application process ownership of the memory of the first data buffer. The application process can flexibly allocate and release memory according to its own business logic and memory usage needs.

[0134] In this embodiment, by setting the application layer header information, the memory lifecycle of the first data buffer is managed, enabling the application process to control the memory ownership of the first data and avoiding problems such as memory leaks and duplicate releases. On this basis, the stability, security and scalability of memory management are improved.

[0135] The present application will be further described in detail below with reference to application examples.

[0136] This application provides a data transmission method, applied to a first device, as illustrated in the following embodiments. Figure 3 The overall processing flow of this data transmission method can be mainly included in the following steps:

[0137] Step 1: The application process writes the first data into the first data buffer and calls the write system call.

[0138] Here, the write system call is equivalent to the first system call in this application embodiment, and the write system call may have been hijacked by the user-space protocol stack.

[0139] In practical applications, before the application process writes the first data into the first data buffer, the first device can allocate a first data buffer from the application memory pool based on the application process's request. The first data buffer can be used to buffer the first data written by the application process; that is, it is used for data caching. The application memory pool can be considered equivalent to the data buffer in this embodiment.

[0140] Step 2: The user-space protocol stack responds to the application process's write system call and divides the first data buffer into multiple segments.

[0141] Step 3: The user-space protocol stack allocates one head-mbuf and multiple data-mbufs from the mbuf pool based on multiple segments.

[0142] Here, the mbuf pool is equivalent to the first memory pool in this application embodiment, the head-mbuf is equivalent to the first mbuf in this application embodiment, and the data-mbuf is equivalent to the second mbuf in this application embodiment.

[0143] The head-mbuf can be used to store the header information corresponding to the data packet. This header information can be obtained based on the encapsulation of the network protocol stack. The data packet is equivalent to the first packet in this embodiment. Each of the multiple data-mbufs can be used to indicate a segment. Based on this, the multiple data-mbufs can be regarded as sharing the first data in the first data buffer.

[0144] Step 4: The user-space protocol stack links head-mbuf with multiple data-mbuf to form a first linked list, and adds the first linked list to the descriptor queue.

[0145] In practical applications, the head node of the first linked list can be represented as head-mbuf, and the user-space protocol stack can add head-mbuf to the descriptor queue. Adding head-mbuf to the descriptor queue by the user-space protocol stack can also be understood as the user-space driver adding the mbuf descriptor to the descriptor queue.

[0146] Step 5: The user-space protocol stack adds the descriptor queue to the network interface card.

[0147] In practical applications, the user-space protocol stack can add the descriptor queue to the network interface card (NIC) through DMA mapping.

[0148] Step 6: The network card sends data packets.

[0149] In practical applications, the network card can obtain the first linked list by reading the descriptor queue, and then, based on the indication of data-mbuf in the first linked list, read the first data in the first data buffer into the network card's internal cache through DMA. Additionally, the network card can read the packet header information corresponding to the data packet based on head-mbuf in the first linked list. After that, the network card can encapsulate the first data into a data packet based on the read first data and the packet header information, and then send the data packet to the second device.

[0150] In practical applications, see Figure 2 The first data buffer may include the application layer header information corresponding to the data packet. The application layer header information may include the following fields: protocol version, type, sequence number, and state.

[0151] In practical applications, the changes in the memory state of the first data buffer during the transmission of the first data by the first device can be found in [reference needed]. Figure 4 As shown, the changes in the memory state of the first data buffer can also be understood as changes in the value of the state field. Specifically, after the application process writes the first data into the first data buffer and before the user-space protocol stack responds to the application process's write system call, the value of the state field can be ready. After the user-space protocol stack responds to the application process's write system call and before the first device receives the acknowledgment message, the value of the state field can be inflight. After the first device receives the acknowledgment message, the value of the state field can be acked.

[0152] exist Figure 4 During the change of the state field value shown, the sequence number field value in the first data buffer can be 1, that is, the sequence number of the first data buffer is 1. Based on this, the data packet and the acknowledgment packet can carry the sequence number field, and the field value of the sequence number field can both be 1. On this basis, after the first device receives the acknowledgment packet, the user-space protocol stack can set the state field value in the data buffer with sequence number 1 to acked, that is, set the state field value of the first data buffer to acked.

[0153] If the value of the state field in the first data buffer is inflight, the application process cannot modify and / or release the first data buffer; if the value of the state field in the first data buffer is acked, the application process can modify and / or release the first data buffer.

[0154] In this application embodiment, when the application process makes a `write` system call, the first device links a head-mbuf allocated from the mbuf pool with multiple data-mbufs to form a first linked list. This allows the network card to directly read the first data from the first data buffer based on the first linked list. Then, it combines this data with the packet header information stored in the head-mbuf to encapsulate a data packet and send it. Compared to related technologies, the first device avoids multiple copies and context switches between user space and kernel space during data transmission, thus enabling data transmission solely in user space, improving data transmission efficiency and reducing latency. Furthermore, in this application embodiment, by setting the application-layer header information, the memory lifecycle of the first data buffer is managed, allowing the application process to control the memory ownership of the first data, avoiding memory leaks and duplicate releases. This improves the stability, security, and scalability of memory management.

[0155] Based on the embodiments described above, this application also provides a data transmission apparatus, applied to the first device, see [link to previous document]. Figure 5 The data transmission device includes:

[0156] The segmentation unit 51 is configured to, in response to a first system call from the application process, divide the first data buffer into multiple segments; the first data buffer is used to buffer first data written by the application process; the first system call is used to instruct the sending of the first data;

[0157] Allocation unit 52 is configured to allocate a first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is configured to store the header information of a first message, the first message representing a message used to transmit the first data; each of the plurality of second mbufs is configured to indicate one of the segments;

[0158] Link unit 53 is used to link the first mbuf with the plurality of second mbufs into a first linked list, and map the first linked list to the network card, so that the network card sends the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

[0159] In one embodiment, the second mbuf includes one or more of the following:

[0160] The virtual address of the corresponding segment;

[0161] The data length of the corresponding segment.

[0162] In one embodiment, the data transmission device further includes an encapsulation unit, the encapsulation unit being used for:

[0163] Add the following information corresponding to the first message to the first mbuf in sequence:

[0164] Transport layer header information, which represents the message header information encapsulated based on the transport layer protocol;

[0165] Network layer header information, which represents the message header information encapsulated based on the network layer protocol;

[0166] Link layer header information, which represents the message header information encapsulated based on the link layer protocol.

[0167] In one embodiment, the first data buffer includes application layer header information corresponding to the first message; the application layer header information represents message header information encapsulated based on the application layer protocol.

[0168] The application layer header information includes one or more of the following:

[0169] The first field is used to indicate the protocol type and / or protocol version corresponding to the first message;

[0170] The second field is used to indicate the service type corresponding to the first message;

[0171] The third field is used to identify the first data buffer;

[0172] The fourth field is used to indicate the memory state of the first data buffer.

[0173] In one embodiment, after the application process writes the first data into the first data buffer and before the first device responds to the application process's first system call, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or,

[0174] After the first device responds to the first system call of the application process and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or,

[0175] After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state;

[0176] The second message represents the message returned by the second device based on the first message.

[0177] In one embodiment, the second message encapsulates the same field as the third field.

[0178] In one embodiment, the data transmission device further includes a control unit, the control unit being configured to:

[0179] If the fourth field indicates that the memory state of the first data buffer is the set second memory state, the application process is allowed to perform modification and / or release operations on the first data buffer; and / or,

[0180] If the fourth field indicates that the memory state of the first data buffer is the set third memory state, the application process is not allowed to perform modification and / or release operations on the first data buffer.

[0181] In one embodiment, the control unit is further configured to:

[0182] Before the segmentation unit 51 responds to the first system call of the application process, it hijacks the first system call.

[0183] In practical applications, the segmentation unit 51, the allocation unit 52, the linking unit 53, the encapsulation unit, and the control unit can be implemented by the processor in the data transmission device.

[0184] It should be noted that the data transmission device provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the data transmission device and the data transmission method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0185] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, this application also provides a first device, referring to... Figure 6The first device includes:

[0186] Communication interface 1 enables information exchange with other devices;

[0187] Processor 2 is connected to communication interface 1 to enable information interaction with other devices and, when running a computer program, executes the methods provided by one or more technical solutions in the above embodiments. The computer program is stored in memory 3.

[0188] Specifically, the processor 2 is configured to, in response to a first system call from the application process, divide the first data buffer into multiple segments; the first data buffer is configured to buffer first data written by the application process; the first system call is configured to instruct the sending of the first data;

[0189] Based on the multiple segments, a first mbuf and multiple second mbufs are allocated from a first memory pool; the first mbuf is used to store the header information corresponding to the first message, the first message representing a message used to transmit the first data; each of the multiple second mbufs is used to indicate one of the segments; and...

[0190] The first mbuf is linked with the plurality of second mbufs to form a first linked list, and the first linked list is mapped to the network card so that the network card can send the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

[0191] In one embodiment, the second mbuf includes one or more of the following:

[0192] The virtual address of the corresponding segment;

[0193] The data length of the corresponding segment.

[0194] In one embodiment, the processor 2 is further configured to:

[0195] Add the following information corresponding to the first message to the first mbuf in sequence:

[0196] Transport layer header information, which represents the message header information encapsulated based on the transport layer protocol;

[0197] Network layer header information, which represents the message header information encapsulated based on the network layer protocol;

[0198] Link layer header information, which represents the message header information encapsulated based on the link layer protocol.

[0199] In one embodiment, the first data buffer includes application layer header information corresponding to the first message; the application layer header information represents message header information encapsulated based on the application layer protocol.

[0200] The application layer header information includes one or more of the following:

[0201] The first field is used to indicate the protocol type and / or protocol version corresponding to the first message;

[0202] The second field is used to indicate the service type corresponding to the first message;

[0203] The third field is used to identify the first data buffer;

[0204] The fourth field is used to indicate the memory state of the first data buffer.

[0205] In one embodiment, the first processor is further configured to:

[0206] After the application process writes the first data into the first data buffer, and before the first device responds to the application process's first system call, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or,

[0207] After the first device responds to the first system call of the application process and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or,

[0208] After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state;

[0209] The second message represents the message returned by the second device based on the first message.

[0210] In one embodiment, the processor 2 is further configured to:

[0211] The first system call is hijacked before the application process responds to the first system call.

[0212] It should be noted that the specific processing procedure of communication interface 1 can be understood by referring to the above method.

[0213] Of course, in practical applications, the various components in the first device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general will label all buses as Bus System 4.

[0214] The memory 3 in this embodiment is used to store various types of data to support operations in the first device. Examples of such data include any computer program used to operate on the first device.

[0215] The methods disclosed in the embodiments of this application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 2 or by instructions in the form of software. The processor 2 mentioned above may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 3. The processor 2 reads the information in the memory 3 and combines its hardware to complete the steps of the aforementioned method.

[0216] In an exemplary embodiment, the first device may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0217] It is understood that the memory 3 in the embodiments of this application can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0218] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by the processor 2 of the first device to complete the steps of the aforementioned data transmission method.

[0219] Computer-readable storage media can be FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0220] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by the processor 2 of a first device to complete the steps of the aforementioned data transmission method.

[0221] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0222] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the term "one or more" in this document refers to any combination of at least two of any one or more elements from a set of A, B, and C. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set of A, B, and C. Additionally, the term "one or more" in this document is an exemplary expression and can be replaced with any possible expressions, such as one or more, at least one, or at least one of, etc.

[0223] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0224] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data transmission method, characterized in that, Applied to a first device, the method includes: In response to a first system call from the application process, a first data buffer is divided into multiple segments; the first data buffer is used to buffer first data written by the application process; the first system call is used to instruct the sending of the first data. Based on the multiple segments, a first mbuf and multiple second mbufs are allocated from the first memory pool; the first mbuf is used to store the header information of the first message, the first message representing a message used to transmit the first data; each of the multiple second mbufs is used to indicate one of the segments. The first mbuf is linked with the plurality of second mbufs to form a first linked list, and the first linked list is mapped to the network card, so that the network card sends the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet; The step of mapping the first linked list to the network interface card (NIC) includes: adding the first linked list to a first queue, and then adding the first queue to the NIC. The network interface card (NIC) sends the first message to the second device, including: the NIC reading the first data in the first data buffer into the NIC's internal cache via direct memory access (DMA) based on the instruction of the second mbuf in the first linked list; the NIC reading the message header information corresponding to the first message based on the first mbuf in the first linked list; and the NIC encapsulating the first data into a first message based on the read first data and the message header information, and then sending the first message to the second device.

2. The method according to claim 1, characterized in that, The second mbuf includes one or more of the following: The virtual address of the corresponding segment; The data length of the corresponding segment.

3. The method according to claim 1, characterized in that, The method further includes: Add the following information corresponding to the first message to the first mbuf in sequence: Transport layer header information, which represents the message header information encapsulated based on the transport layer protocol; Network layer header information, which represents the message header information encapsulated based on the network layer protocol; Link layer header information, which represents the message header information encapsulated based on the link layer protocol.

4. The method according to claim 1, characterized in that, The first data buffer includes the application layer header information corresponding to the first message; the application layer header information represents the message header information encapsulated based on the application layer protocol. The application layer header information includes one or more of the following: The first field is used to indicate the protocol type and / or protocol version corresponding to the first message; The second field is used to indicate the service type corresponding to the first message; The third field is used to identify the first data buffer; The fourth field is used to indicate the memory state of the first data buffer.

5. The method according to claim 4, characterized in that, After the application process writes the first data into the first data buffer, and before the first device responds to the application process's first system call, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or, After the first device responds to the first system call of the application process and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or, After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state; The second message represents the message returned by the second device based on the first message.

6. The method according to claim 5, characterized in that, The second message contains the same fields as the third field.

7. The method according to claim 5, characterized in that, If the fourth field indicates that the memory state of the first data buffer is the set second memory state, the application process is allowed to perform modification and / or release operations on the first data buffer; and / or, If the fourth field indicates that the memory state of the first data buffer is the set third memory state, the application process is not allowed to perform modification and / or release operations on the first data buffer.

8. The method according to claim 1, characterized in that, Prior to responding to the first system call from the application process, the method further includes: The first system call was hijacked.

9. A data transmission device, characterized in that, Applied to the first device, including: A segmentation unit is used to divide the first data buffer into multiple segments in response to a first system call from the application process; The first data buffer is used to buffer the first data written by the application process; the first system call is used to instruct the sending of the first data; An allocation unit is configured to allocate a first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is used to store the header information of a first message, the first message representing a message used to transmit the first data; each of the plurality of second mbufs is used to indicate one of the segments. A linking unit is used to link the first mbuf with the plurality of second mbufs into a first linked list, and map the first linked list to the network interface card (NIC) so that the NIC can send the first packet to the second device; the first linked list is used by the NIC to read the first data from the first data buffer and encapsulate the first data into the first packet; The linking unit maps the first linked list to the network interface card (NIC), which includes: adding the first linked list to a first queue, and then adding the first queue to the NIC. The network interface card (NIC) sends the first message to the second device, including: the NIC reading the first data in the first data buffer into the NIC's internal cache via DMA based on the instruction of the second mbuf in the first linked list; the NIC reading the message header information corresponding to the first message based on the first mbuf in the first linked list; and the NIC encapsulating the first data into a first message based on the read first data and the message header information, and then sending the first message to the second device.

10. An electronic device, characterized in that, include: A processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images