Data transmission method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a data transmission method and device, equipment, a storage medium and a computer program product, and the method comprises the steps that first equipment responds to first system call of an application process, and divides a first data buffer area into a plurality of sections; the first data buffer area is used for buffering first data written by an application process; allocating a first mbuf and a plurality of second mbufs from the first memory pool based on the plurality of sections; the first mbuf is used for storing message header information corresponding to a first message, and the first message represents a message used for transmitting first data; each second mbuf in the plurality of second mbufs is used for indicating one section; linking one first mbuf and a plurality of second mbufs into a first linked list, and mapping the first linked list into a network card, so that the network card sends a first message to the second equipment; the first linked list is used for the network card to read the first data from the first data buffer area and encapsulate the first data into a first message.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a data transmission method and device, equipment, storage medium and computer program product. BACKGROUND

[0002] In the related art, the device needs to perform multiple copy operations and context switching between a user space and a kernel space of an operating system in the process of transmitting data generated by an application process, resulting in high delay of data transmission. SUMMARY

[0003] To solve the problems in the related art, the embodiments of the present application provide a data transmission method and device, equipment, storage medium and computer program product.

[0004] The technical scheme of the embodiments of the present application is implemented as follows: The embodiments of the present application provide a data transmission method, applied to a first device, and the method comprises the following steps: In response to a first system call of an application process, a first data buffer is divided into multiple segments; the first data buffer is used to buffer first data written by the application process; the first system call is used to instruct to send the first data; Based on the multiple segments, a first mbuf and multiple second mbufs are allocated from a first memory pool; the first mbuf is used to store message header information corresponding to a first message, and the first message represents a message used to transmit the first data; each second mbuf in the multiple second mbufs is used to indicate a segment; The first mbuf and the multiple second mbufs are linked into a first linked list, and the first linked list is mapped to a network card, so that the network card sends the first message to a second device; the first linked list is used for the network card to read the first data from the first data buffer and encapsulate the first data into the first message.

[0005] In the above scheme, the second mbuf includes one or more of the following: a virtual address of a corresponding segment; a data length of a corresponding segment.

[0006] In the above scheme, the method further comprises the following steps: The following information corresponding to the first message is sequentially added to the first mbuf: transport layer header information, which represents message header information encapsulated based on a transport layer protocol; network layer header information, which represents message header information encapsulated based on a network layer protocol; link layer header information, the link layer header information representing message header information encapsulated based on a link layer protocol.

[0007] In the above solution, the first data buffer includes application layer header information corresponding to the first message; the application layer header information represents message header information encapsulated based on an application layer protocol. The application layer header information includes one or more of the following: a first field, the first field being used to indicate a protocol type and / or a protocol version corresponding to the first message; a second field, the second field being used to indicate a service type corresponding to the first message; a third field, the third field being used to identify the first data buffer; a fourth field, the fourth field being used to indicate a memory state of the first data buffer.

[0008] In the above solution, before the application process writes the first data into the first data buffer and before the first device responds to the first system call of the application process, the fourth field is used to indicate that the memory state of the first data buffer is a first memory state set; and / or, after the first device responds to the first system call of the application process and before the first device receives a second message, the fourth field is used to indicate that the memory state of the first data buffer is a second memory state set; and / or, after the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a third memory state set. The second message represents a message returned by the second device based on the first message.

[0009] In the above solution, the second message encapsulates the same field as the third field.

[0010] In the above solution, in a case where the fourth field indicates that the memory state of the first data buffer is the second memory state set, a modification operation and / or a release operation of the first data buffer by the application process are allowed; and / or, In a case where the fourth field indicates that the memory state of the first data buffer is the third memory state set, the modification operation and / or the release operation of the first data buffer by the application process are not allowed.

[0011] In the above solution, before responding to the first system call of the application process, the method further includes: hijacking the first system call.

[0012] The embodiment of the present application further provides a data transmission device, which is applied to a first device and comprises: a splitting unit, configured to split a first data buffer into a plurality of segments in response to a first system call of an application process; the first data buffer is used for buffering first data written by the application process; the first system call is used for indicating to send the first data; an allocating unit, configured to allocate one first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is used for storing message header information corresponding to a first message; the first message represents a message used for transmitting the first data; each of the plurality of second mbufs is used for indicating one of the segments; a linking unit, configured to link the one first mbuf and the plurality of second mbufs into a first linked list, and map the first linked list to a network card, so that the network card sends the first message to a second device; the first linked list is used for the network card to read the first data from the first data buffer and encapsulate the first data into the first message.

[0013] The embodiment of the present application further provides a first device, comprising a processor and a memory for storing a computer program capable of running on the processor, wherein the processor is used to run the computer program, and execute the steps of any one of the foregoing methods.

[0014] The embodiment of the present application further provides a storage medium, which stores a computer program, and the computer program is characterized in that, when the computer program is executed by a processor, the steps of any one of the foregoing methods are implemented.

[0015] The embodiment of the present application further provides a computer program product, comprising a computer program, and the computer program is characterized in that, when the computer program is executed by a processor, the steps of any one of the foregoing methods are implemented.

[0016] In the embodiment of the present application, the first device divides the first data buffer into multiple segments in response to a first system call of an application process, wherein the first data buffer is used to buffer first data written by the application process, and the first system call is used to indicate sending the first data; then, based on the multiple segments, a first mbuf and multiple second mbufs are allocated from a first memory pool, wherein the first mbuf is used to store packet header information corresponding to a first packet, the first packet represents a packet used to transmit the first data, and each of the multiple second mbufs is used to indicate a segment; then, the first mbuf and the multiple second mbufs are linked into a first linked list, and the first linked list is mapped to a network card, so that the network card sends the first packet to a second device, and the first linked list is used for the network card to read the first data from the first data buffer and encapsulate the first data into the first packet. In the above scheme, the first device links the first mbuf and the multiple second mbufs allocated from the first memory pool into the first linked list in the case that the application process calls the first system call, so that the network card can directly read the first data in the first data buffer based on the first linked list, and then encapsulate and send the first packet in combination with the packet header information stored in the second mbuf. Compared with the related art, the first device avoids multiple copying and context switching between the user space and the kernel space in the process of sending data, thereby reducing the data transmission delay. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 An implementation flow diagram of a data transmission method provided by an embodiment of the present application; Figure 2 A schematic diagram of a first data buffer provided by an embodiment of the present application; Figure 3 A processing flow diagram of a data transmission method provided by an application embodiment of the present application; Figure 4 A memory state change schematic diagram provided by an application embodiment of the present application; Figure 5 A structural schematic diagram of a data transmission device provided by an embodiment of the present application; Figure 6 A hardware composition structural schematic diagram of a first device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In the related art, the device needs to perform multiple copy operations and context switching between the user space and the kernel space of the operating system in the process of transmitting data generated by an application process. For example, in the process of sending data generated by an application process, the device writes the data into a buffer in the user space, then switches the context of the operating system from the user space to the kernel space through a system call of the application process, copies the data from the user space to the kernel space, then sends the data from the kernel space to a device for receiving the data through a network card driver, and switches the context of the operating system from the kernel space to the user space. In this way, the load of the central processing unit (CPU) of the device is increased, resulting in high delay of data transmission.

[0019] Based on this, in the embodiments of the present application, the first device, in response to a first system call of an application process, divides a first data buffer into multiple segments, wherein the first data buffer is used to buffer first data written by the application process, and the first system call is used to instruct to send the first data; then, based on the multiple segments, allocates one first mbuf and multiple second mbufs from a first memory pool, wherein the first mbuf is used to store packet header information corresponding to a first packet, the first packet represents a packet used to transmit the first data, and each second mbuf of the multiple second mbufs is used to indicate a segment; then, links one first mbuf and multiple second mbufs into a first linked list, and maps the first linked list to a network card, so that the network card sends the first packet to a second device, and the first linked list is used for the network card to read the first data from the first data buffer and encapsulate the first data into the first packet. In the above scheme, the first device links one first mbuf and multiple second mbufs allocated from the first memory pool into a first linked list in the case that the application process calls the first system call, so that the network card can directly read the first data in the first data buffer based on the first linked list, and then encapsulate and send the first packet in combination with the packet header information stored in the second mbuf. Compared with the related art, the first device avoids multiple copy operations and context switching between the user space and the kernel space in the process of sending data, thereby reducing the delay of data transmission.

[0020] The present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0021] The embodiments of the present application provide a data transmission method, applied to a first device.

[0022] In practical applications, the first device can be understood as a device for transmitting data, for example, the first device can include a client device or a server device, etc.

[0023] The first device can be provided with an operating system, which can provide the first device with functions such as resource management, task scheduling, user interface provision, and system service support. The operating system can include a user-mode protocol stack, which runs in the user space of the operating system. In actual application, the data transmission method provided in the embodiments of the present application can be applied to the user-mode protocol stack of the first device.

[0024] The first device can also include a network card, which can be understood as a hardware interface between the first device and a network. The network card can be used for data transmission between the first device and other devices outside the first device.

[0025] Here, referring to Figure 1 The data transmission method provided in the embodiments of the present application includes: Step 101: In response to a first system call of an application process, a first data buffer is divided into multiple segments.

[0026] The first data buffer is used to buffer first data written by the application process. The first system call is used to instruct to send the first data.

[0027] In actual application, the application process can be understood as an instance of an application program in the first device when the application program is running, that is, a process corresponding to the application program.

[0028] The application process can generate first data during running, and write the first data into the first data buffer, and then call the first system call to instruct the first device to send the first data to other devices outside the first device.

[0029] Exemplarily, the first system call can include a write system call and / or a send system call.

[0030] In actual application, the first data buffer can represent a data buffer used to buffer the first data. The data buffer can be understood as a memory pool or a memory buffer, and the data in the data buffer can be stored continuously. Before the application process writes the first data into the first data buffer, the first device can allocate the first data buffer from a pre-applied data buffer based on the application of the application process.

[0031] In actual application, the processing of the first device in response to the first system call of the application process can also be understood as the processing of the first device in the case where the application process calls the first system call.

[0032] In actual application, when the first device processes the first system call of the application process, the operating system of the first device can only process the first system call in the user space, without processing in the kernel space, that is, without entering the kernel mode, thereby avoiding the interaction and context switching between the user space and the kernel space, and improving the processing efficiency.

[0033] In an embodiment, before the first system call of the application process, the data transmission method provided by the embodiment of the application further includes: intercepting the first system call.

[0034] In actual application, the processing function corresponding to the first system call can be rewritten from the built-in processing function of the operating system to a customized processing function. Then, when the application program calls the first system call, the first device can no longer execute the built-in function of the operating system, but execute the customized processing function. The customized processing function can make the first device avoid entering the kernel mode in the processing process, thereby avoiding the interaction and context switching between the user space and the kernel space in the processing process of the first device, and improving the processing efficiency.

[0035] The processing logic of the customized processing function can be understood by referring to the processing of the first device in response to the first system call in the embodiment of the application.

[0036] Here, the first device divides the first data buffer into multiple segments in response to the first system call of the application process. In actual application, the first device can divide the first data buffer into multiple segments based on the first parameter, so that the data length of each segment does not exceed the data length indicated by the first parameter in the process of dividing the first data buffer into multiple segments. The first parameter can also be referred to as the maximum segment size or the maximum segment length.

[0037] The first parameter can be determined based on the value of the maximum transmission unit (MTU) of the network card.

[0038] In actual application, after the first data buffer is divided into multiple segments, the first data included in the first data buffer can be distributed in each segment, that is, one segment in the first data buffer can store part of the first data.

[0039] In actual application, in the process of sending the first data, the first device needs to encapsulate the first data into a first packet, and send the first packet by the network card in the first device. The data length corresponding to the first packet can exceed the length of the data frame that can be transmitted by the network card at a time. In this case, the network card can divide the first packet into multiple packet segments for sending.

[0040] In actual application, the data in each segment of the first data buffer can be used to compose a message segment corresponding to the first message, and therefore, the segmentation of the first data buffer into multiple segments can also be understood as the segmentation of the first data buffer into multiple message segments.

[0041] In the embodiment, the first data buffer is segmented into multiple segments before the network card transmits the first message, and therefore, the network card can more efficiently transmit the first message by dividing the first message into multiple message segments when transmitting the first message, thereby improving the data transmission efficiency and reducing the data transmission delay.

[0042] Step 102: allocating a first mbuf and multiple second mbufs from the first memory pool based on the multiple segments.

[0043] The first mbuf is used to store message header information corresponding to the first message, and the first message represents a message used to transmit the first data; and each second mbuf of the multiple second mbufs is used to indicate a segment.

[0044] In actual application, the first memory pool and the data buffer can be two discontinuous memory regions, and the first memory pool and the data buffer can be respectively applied by different memory applications, thereby increasing the flexibility of memory management.

[0045] In actual application, the data in the data buffer can be continuously stored, and therefore, if the message header information is directly written in the first data buffer or the data buffer adjacent to the first data buffer after the first data is written into the first data buffer, the originally stored data in the written data buffer can be damaged. In the embodiment, the first mbuf in the first memory pool is used to store the message header information, thereby avoiding the damage to the data in the data buffer, and on this basis, the accuracy and reliability of data transmission are improved.

[0046] In actual application, the message header information stored in the first mbuf can be used to encapsulate the message header information of the first message and / or the message header information of the message segment corresponding to the first message. The first mbuf can also be referred to as a head-mbuf.

[0047] In an embodiment, the data transmission method provided by the embodiment further includes: adding the following information corresponding to the first message into the first mbuf in sequence: transport layer header information, the transport layer header information representing message header information encapsulated based on a transport layer protocol; network layer header information, the network layer header information representing message header information encapsulated based on a network layer protocol; The link layer header information represents the message header information encapsulated based on a link layer protocol.

[0048] In actual applications, the user mode protocol stack in the first device can include a network protocol stack. Exemplarily, the network protocol stack can represent an Open System Interconnect Reference Model (OSI) network protocol stack. The network protocol stack can encapsulate transport layer header information, network layer header information, and link layer header information, and then add these message header information to the first mbuf in sequence, which is equivalent to realizing the downward transmission of the first message in the network protocol stack.

[0049] It should be noted that in the process of adding the message header information corresponding to the first message to the first mbuf, the first message can not have been encapsulated, but the header information corresponding to each layer of the network protocol stack has been encapsulated and added to the first mbuf in sequence, which is equivalent to simulating the downward transmission process of the first message in the network protocol stack. Therefore, it can also be regarded as realizing the downward transmission of the first message in the network protocol stack.

[0050] For the plurality of message segments corresponding to the first message, the field types and field positions of the fields in the message header information of these message segments can be the same, so the message header information of each message segment can be encapsulated by the message header information stored in one first mbuf. In the process of encapsulating the message header information of each message segment, the message header information of the message segment can be encapsulated by modifying the field values of part of the fields in the message header information stored in the first mbuf. In this way, the message header information of a plurality of message segments can be encapsulated by the message header information stored in one first mbuf, thereby saving memory space.

[0051] It should be noted that in the encapsulation process, the modification of the message header information stored in the first mbuf can be a modification after reading or copying the message header information from the first mbuf. The message header information stored in the first mbuf can not change before and after the modification.

[0052] Here, each of the plurality of second mbufs is used to indicate a section. In actual applications, the second mbuf can also be referred to as a data-mbuf.

[0053] In an embodiment, the second mbuf includes one or more of the following: a virtual address of the corresponding section; a data length of the corresponding section.

[0054] In actual application, the second mbuf can be used to determine the storage position of the data in the corresponding section in the first data buffer and / or determine the length of the data in the corresponding section.

[0055] Step 103: Linking a first mbuf and a plurality of second mbufs into a first linked list and mapping the first linked list to a network card, so that the network card sends a first packet to a second device.

[0056] The first linked list is used for the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

[0057] In actual application, each node in the first linked list can be regarded as a mbuf. The head node of the first linked list can be represented as the first mbuf, and the order of each second mbuf in the plurality of second mbufs in the first linked list can be determined based on the order of the data corresponding to the second mbuf in the data buffer.

[0058] In the process of mapping the first linked list to the network card, the first linked list can be added to a first queue, and then the first queue can be added to the network card. After that, the network card can read the first linked list in the first queue and perform corresponding processing based on the first linked list.

[0059] In actual application, the nodes in the first linked list can be connected through pointers. On this basis, the head node of the first linked list can read the nodes in the first linked list. Therefore, in the process of adding the first linked list to the first queue, only the head node of the first linked list, i.e., only the first mbuf, can be added to the first queue. The first queue can be referred to as a descriptor queue or a sending queue, and the first mbuf added to the first queue can be referred to as a mbuf descriptor.

[0060] In actual application, the network card can read the first data in the first data buffer into the internal cache of the network card through direct memory access (DMA) based on the indication of the second mbuf in the first linked list. The network card can read the packet header information corresponding to the first packet based on the first mbuf in the first linked list. Then, the network card can encapsulate the first data into the first packet based on the read first data and packet header information, and then send the first packet to the second device. The second device can be understood as other devices other than the first device.

[0061] In the process of encapsulating and sending the first packet by the network card based on the first linked list, the first device can not need to perform copy operation and context switching between the user space and the kernel space, and data transmission can be realized only in the user space, thereby improving the data transmission efficiency and reducing the data transmission delay.

[0062] In practical applications, the first device can send data without copying between user space and kernel space, which can be understood as achieving zero-copy transmission.

[0063] In this embodiment, when the application process calls the first system call, the first device links a first mbuf allocated from the first memory pool with multiple second mbufs to form a first linked list. This allows the network card to directly read the first data in the first data buffer based on the first linked list, and then encapsulate the first packet with the packet header information stored in the second mbufs and send it. Compared with related technologies, the first device avoids multiple copies and context switches between user space and kernel space during data transmission, thus enabling data transmission only in user space and reducing data transmission latency.

[0064] In practical applications, the application process in the first device can control the memory ownership of the first data, that is, the application process can modify and / or release the first data buffer where the first data is located.

[0065] The following section further explains how application processes control memory ownership.

[0066] In one embodiment, the first data buffer includes application layer header information corresponding to the first message; the application layer header information represents the message header information encapsulated based on the application layer protocol. Application layer header information includes one or more of the following: The first field is used to indicate the protocol type and / or protocol version corresponding to the first message; The second field is used to indicate the service type corresponding to the first message; The third field is used to identify the first data buffer. The fourth field is used to indicate the memory status of the first data buffer.

[0067] In practical applications, when an application process requests a first data buffer, it can determine the required buffer length based on the amount of data in the application layer header information corresponding to the first data and the first message, and then request the first data buffer based on that buffer length. The buffer length can also be understood as the memory size of the corresponding data buffer.

[0068] The first data buffer can be divided into two regions: a first region and a second region. The first region can be used to store the application layer header information corresponding to the first message, and the second region can be used to store the first data. During the process of dividing the first data buffer into multiple segments, the second region can also be divided into multiple segments.

[0069] In actual application, the gateway can determine the application layer header information to be carried in the first message based on the application layer header information in the data buffer, and encapsulate the determined application layer header information into the first message. For example, the gateway can encapsulate the application layer header information in the data buffer into the first message.

[0070] Here, the first field is used to indicate the protocol type and / or protocol version corresponding to the first message. In actual application, the protocol type and / or protocol version corresponding to the first message can be used to ensure the reliability and consistency of the first message in network transmission process. For example, the protocol type indicated by the first field can include at least one or more of the following: Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Internet Protocol (IP).

[0071] Here, the second field is used to indicate the service type corresponding to the first message. In actual application, the service type can be understood as the service function of the first message. For example, the service type indicated by the second field can include at least one or more of the following: write request, data message, and acknowledgement message.

[0072] Here, the third field is used to identify the first data buffer. In actual application, the first data buffer can be understood as a memory block, and the field value of the third field can be equivalent to the serial number of the memory block.

[0073] Here, the fourth field is used to indicate the memory state of the first data buffer. In actual application, the memory state can be used for the application process to track the life cycle and accessibility of the memory block represented by the first data buffer. The field value of the fourth field can be understood as the state bit of the memory state of the first data buffer.

[0074] For example, the memory state can include: ready, inflight, and acked, and the corresponding field values of the fourth field can be: ready, inflight, and acked, respectively.

[0075] For example, Figure 2 A schematic diagram of the first data buffer is given, in which protocol version corresponds to the first field, type corresponds to the second field, sequence number corresponds to the third field, state corresponds to the fourth field, and data corresponds to the first data.

[0076] In the embodiment of the present application, the first data buffer includes the application layer header information corresponding to the first message. On this basis, the application process can track the life cycle and accessibility of the memory block represented by the first data buffer based on the application layer header information stored in the first data buffer, and then control the memory ownership of the first data.

[0077] In an embodiment, after the application process writes the first data into the first data buffer, and before the first device responds to the first system call of the application process, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or, After the first device responds to the first system call of the application process, and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or, After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set third memory state. The second message represents a message returned by the second device based on the first message.

[0078] In actual application, after the application process applies for the first data buffer and writes the first data into the first data buffer, the first device can set the field value of the fourth field in the application layer header information in the first data buffer, so that the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state. Exemplarily, the set first memory state can be represented as ready, and the first device can set the field value of the fourth field as ready.

[0079] After the application process calls the first system call, when the first device processes in response to the first system call, the first device can set the field value of the fourth field in the application layer header information in the first data buffer, so that the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state. Exemplarily, the set second memory state can be represented as inflight, and the first device can set the field value of the fourth field as inflight.

[0080] After the first device receives the second message, the first device can set the field value of the fourth field in the application layer header information in the first data buffer, so that the fourth field is used to indicate that the memory state of the first data buffer is a set third memory state. Exemplarily, the set third memory state can be represented as acked, and the first device can set the field value of the fourth field as acked.

[0081] In actual application, the first device can be set by the user mode protocol stack of the first device in setting the field value of the fourth field.

[0082] In actual application, the second message returned by the first device based on the first message can be used to indicate that the second device has confirmed to receive the first message, and the second message can also be understood as an acknowledgement message.

[0083] In an embodiment, the second message encapsulates the same field as the third field.

[0084] In actual application, after receiving the first message, the second device can verify the data integrity of the first message, and return the second message to the first device if the verification is passed. The second message encapsulates the same field as the third field, that is, the second message carries the sequence number of the first data buffer.

[0085] After receiving the second message, the first device can parse the second message to determine the sequence number of the data buffer carried by the second message, and then set the field value of the fourth field in the data buffer corresponding to the sequence number, that is, set the field value of the fourth field in the first data buffer, so that the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state.

[0086] In an embodiment, in the case where the fourth field indicates that the memory state of the first data buffer is the set second memory state, the modification operation and / or release operation of the application process on the first data buffer is allowed; and / or, In the case where the fourth field indicates that the memory state of the first data buffer is the set third memory state, the modification operation and / or release operation of the application process on the first data buffer is not allowed.

[0087] In actual application, in the case where the fourth field indicates that the memory state of the first data buffer is the set second memory state, it can be considered that the first device has performed the sending process of the first data, but has not received the acknowledgement message of the second device. In this case, the modification operation and / or release operation of the application process on the first data buffer is not allowed, which can avoid the modification of the data in the first data buffer, thereby avoiding the data inconsistency caused by retransmission in the sending process, and improving the reliability of data transmission.

[0088] In actual application, in the case that the fourth field indicates that the memory state of the first data buffer is the third memory state set, it can be considered that the first device has received the acknowledgement message of the second device, that is, the sending of the first message has been completed, in this case, the modification operation and / or the release operation of the first data buffer by the protocol stack can be not allowed, and the modification operation and / or the release operation of the first data buffer by the application process is allowed, so as to provide the memory ownership of the first data to the application process, and the application process can flexibly allocate and release the memory according to the business logic and memory use demand of the application process.

[0089] In the embodiment of the application, by setting the application layer header information, the memory life cycle of the first data buffer is managed, so that the application process can control the memory ownership of the first data, and the problems such as memory leakage and repeated release are avoided, and on this basis, the stability, security and scalability of memory management are improved.

[0090] The application will be further described in detail in combination with application embodiments.

[0091] The application embodiment of the application provides a data transmission method applied to a first device, referring to Figure 3 The overall processing flow of the data transmission method can mainly include the following steps: Step 1: An application process writes first data into a first data buffer, and calls a write system call.

[0092] Here, the write system call is equivalent to the first system call in the embodiment of the application, and the write system call can be hijacked by a user-mode protocol stack.

[0093] In actual application, before the application process writes the first data into the first data buffer, the first device can allocate the first data buffer from an application program memory pool (app memory pool) based on the application of the application process. The first data buffer can be used to buffer the first data written by the application process, that is, to perform data caching. The application program memory pool can be equivalent to the data buffer in the embodiment of the application.

[0094] Step 2: The user-mode protocol stack splits the first data buffer into multiple segments in response to the write system call of the application process.

[0095] Step 3: The user-mode protocol stack allocates a head-mbuf and multiple data-mbufs from an mbuf pool based on the multiple segments.

[0096] Here, the mbuf pool is equivalent to the first memory pool in the embodiments of the present application, the head-mbuf is equivalent to the first mbuf in the embodiments of the present application, and the data-mbuf is equivalent to the second mbuf in the embodiments of the present application.

[0097] The head-mbuf can be used to store packet header information corresponding to the data packet, and the packet header information can be obtained based on encapsulation of a network protocol stack. The data packet is equivalent to the first packet in the embodiments of the present application. Each data-mbuf in the plurality of data-mbufs can be used to indicate a section, and on this basis, the plurality of data-mbufs can be regarded as sharing the first data in the first data buffer.

[0098] Step 4: The user-mode protocol stack links the head-mbuf and the plurality of data-mbufs into a first linked list, and adds the first linked list into a descriptor queue.

[0099] In actual application, the head node of the first linked list can be represented as the head-mbuf, and the user-mode protocol stack can add the head-mbuf into the descriptor queue. The user-mode protocol stack adding the head-mbuf into the descriptor queue can also be understood as adding the mbuf descriptor into the descriptor queue by the user-mode driver.

[0100] Step 5: The user-mode protocol stack adds the descriptor queue into a network card.

[0101] In actual application, the user-mode protocol stack can add the descriptor queue into the network card by means of DMA mapping (map).

[0102] Step 6: The network card sends the data packet.

[0103] In actual application, the network card can obtain the first linked list based on reading of the descriptor queue, and then based on the indication of the data-mbuf in the first linked list, read the first data in the first data buffer into the internal cache of the network card by means of DMA. The network card can read the packet header information corresponding to the data packet based on the head-mbuf in the first linked list, and then encapsulate the first data as the data packet based on the read first data and the packet header information, and then send the data packet to a second device.

[0104] In actual application, referring to Figure 2 , the first data buffer can include application layer header information corresponding to the data packet, and the application layer header information can include the following fields: protocol version, type, sequence number, and state.

[0105] In actual application, the change of the memory state of the first data buffer during the process that the first device transmits the first data can refer to the change of the field value of the state field shown in Figure 4 The change of the memory state of the first data buffer can also be understood as the change of the field value of the state field; wherein, after the application process writes the first data into the first data buffer, and before the user mode protocol stack responds to the write system call of the application process, the field value of the state field can be ready; after the user mode protocol stack responds to the write system call of the application process, and before the first device receives the acknowledgement message, the field value of the state field can be inflight; after the first device receives the acknowledgement message, the field value of the state field can be acked.

[0106] In the change process of the field value of the state field shown in Figure 4 The field value of the sequence number field in the first data buffer can be 1, that is, the sequence number of the first data buffer is 1, based on which, the data message and the acknowledgement message can carry the sequence number field, and the field value of the sequence number field can be 1, on this basis, the user mode protocol stack can set the field value of the state field in the data buffer with the sequence number 1 as acked after the first device receives the acknowledgement message, that is, set the field value of the state field of the first data buffer as acked.

[0107] In the case that the field value of the state field in the first data buffer is inflight, the application process cannot modify and / or release the first data buffer, in the case that the field value of the state field in the first data buffer is acked, the application process can modify and / or release the first data buffer.

[0108] In the application embodiment, the first device links one head-mbuf and a plurality of data-mbufs allocated from the mbuf pool into a first linked list in the case that the application process has performed a write system call, so that the network card can directly read out the first data in the first data buffer based on the first linked list, and then encapsulate and send a data packet in combination with the packet header information stored in the head-mbuf. Compared with the related art, the first device avoids multiple copying and context switching between the user space and the kernel space in the process of sending data, so that data transmission can be realized only in the user space, the data transmission efficiency is improved, and the data transmission delay is reduced. Moreover, in the application embodiment, the memory life cycle of the first data buffer is managed through the setting of the application layer header information, so that the application process can control the memory ownership of the first data, and problems such as memory leakage and repeated release are avoided, and on this basis, the stability, security and scalability of memory management are improved.

[0109] Based on the above embodiment, the application further provides a data transmission device applied to a first device, referring to Figure 5 The data transmission device comprises: A segmentation unit 51 configured to segment a first data buffer into a plurality of segments in response to a first system call of an application process; the first data buffer is used to buffer first data written by the application process; the first system call is used to instruct to send the first data; An allocation unit 52 configured to allocate one first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is used to store packet header information corresponding to a first packet, and the first packet represents a packet used to transmit the first data; each second mbuf in the plurality of second mbufs is used to indicate a segment; A linking unit 53 configured to link the one first mbuf and the plurality of second mbufs into a first linked list, and map the first linked list to a network card, so that the network card sends the first packet to a second device; the first linked list is used for the network card to read out the first data from the first data buffer and encapsulate the first data into the first packet.

[0110] In an embodiment, the second mbuf includes one or more of the following: A virtual address of the corresponding segment; A data length of the corresponding segment.

[0111] In an embodiment, the data transmission device further comprises an encapsulation unit, and the encapsulation unit is configured to: Add the following information corresponding to the first packet into the first mbuf in sequence: transport layer header information, the transport layer header information representing message header information encapsulated based on a transport layer protocol; network layer header information, the network layer header information representing message header information encapsulated based on a network layer protocol; link layer header information, the link layer header information representing message header information encapsulated based on a link layer protocol.

[0112] In an embodiment, the first data buffer includes application layer header information corresponding to the first message; the application layer header information represents message header information encapsulated based on an application layer protocol. The application layer header information includes one or more of: a first field, the first field being used to indicate a protocol type and / or a protocol version corresponding to the first message; a second field, the second field being used to indicate a service type corresponding to the first message; a third field, the third field being used to identify the first data buffer; a fourth field, the fourth field being used to indicate a memory state of the first data buffer.

[0113] In an embodiment, before the application process writes the first data into the first data buffer and before the first device responds to the first system call of the application process, the fourth field is used to indicate that the memory state of the first data buffer is a first memory state set by the application process; and / or, after the first device responds to the first system call of the application process and before the first device receives a second message, the fourth field is used to indicate that the memory state of the first data buffer is a second memory state set by the first device; and / or, after the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a third memory state set by the first device; wherein the second message represents a message returned by the second device based on the first message.

[0114] In an embodiment, the second message encapsulates a field identical to the third field.

[0115] In an embodiment, the data transmission apparatus further includes a control unit, the control unit being used to: in a case where the fourth field indicates that the memory state of the first data buffer is the second memory state set by the first device, allow a modification operation and / or a release operation of the first data buffer by the application process; and / or, In a case where the fourth field indicates that the memory state of the first data buffer is the third memory state of the setting, the modification operation and / or the release operation of the first data buffer by the application process is not allowed.

[0116] In an embodiment, the control unit is further configured to: hijack the first system call before the first system call is responded to by the segmentation unit 51.

[0117] In practice, the segmentation unit 51, the allocation unit 52, the linking unit 53, the encapsulation unit, and the control unit can be implemented by a processor in the data transmission device.

[0118] It should be noted that the data transmission device provided in the above embodiments is only exemplified by the division of the above program modules when performing data transmission. In actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the data transmission device and the data transmission method provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0119] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of the present application, the present application further provides a first device, referring to Figure 6 The first device comprises: a communication interface 1 capable of information interaction with other devices; a processor 2 connected with the communication interface 1 to realize information interaction with other devices, for running a computer program, executing the method provided by one or more technical solutions in the above embodiments. And the computer program is stored on the memory 3.

[0120] Specifically, the processor 2 is configured to segment a first data buffer into a plurality of segments in response to a first system call of an application process; the first data buffer is configured to buffer first data written by the application process; and the first system call is configured to indicate sending the first data. allocate one first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is configured to store packet header information corresponding to a first packet; the first packet represents a packet for transmitting the first data; each of the plurality of second mbufs is configured to indicate one of the segments; and link the one first mbuf and the plurality of second mbufs as a first linked list, and map the first linked list to a network card, so that the network card sends the first packet to a second device; the first linked list is used for the network card to read the first data from the first data buffer and encapsulate the first data as the first packet.

[0121] In an embodiment, the second mbuf includes one or more of the following: a virtual address of a corresponding segment; a data length of a corresponding segment.

[0122] In an embodiment, the processor 2 is further configured to: add the following information corresponding to the first packet to the first mbuf in sequence: transport layer header information, the transport layer header information representing packet header information encapsulated based on a transport layer protocol; network layer header information, the network layer header information representing packet header information encapsulated based on a network layer protocol; link layer header information, the link layer header information representing packet header information encapsulated based on a link layer protocol.

[0123] In an embodiment, the first data buffer includes application layer header information corresponding to the first packet; the application layer header information represents packet header information encapsulated based on an application layer protocol. The application layer header information includes one or more of the following: a first field, the first field being used to indicate a protocol type and / or a protocol version corresponding to the first packet; a second field, the second field being used to indicate a service type corresponding to the first packet; a third field, the third field being used to identify the first data buffer; a fourth field, the fourth field being used to indicate a memory state of the first data buffer.

[0124] In an embodiment, the first processor is further configured to: after the application process writes the first data into the first data buffer, and before the first device responds to the first system call of the application process, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or, after the first device responds to the first system call of the application process, and before the first device receives a second packet, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or, After the first device receives the second packet, the fourth field is used to indicate that the memory state of the first data buffer is a third memory state; The second packet is returned by the second device based on the first packet.

[0125] In an embodiment, the processor 2 is further configured to: Hijack the first system call before responding to the first system call of the application process.

[0126] It should be noted that the specific processing process of the communication interface 1 can be understood with reference to the above method.

[0127] Of course, in actual applications, various components in the first device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus, and a state signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the bus system 4 in the Figure 6

[0128] The memory 3 in the embodiment of the application is used to store various types of data to support the operation in the first device. Examples of these data include: any computer program used to operate on the first device.

[0129] The method disclosed in the above embodiment of the application can be applied to the processor 2 or implemented by the processor 2. The processor 2 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 2 or the instruction in the form of software. The processor 2 mentioned above can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the application, the hardware decoding processor can be directly embodied to execute the completion, or the combination of hardware and software modules in the decoding processor can be executed to complete. The software module can be located in the storage medium, which is located in the memory 3, and the processor 2 reads the information in the memory 3 and combines the hardware to complete the steps of the foregoing method.

[0130] In the exemplary embodiment, the first device can be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, Microprocessors, or other electronic elements, for executing the foregoing method.

[0131] ​It can be understood that the memory 3 of the embodiments of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0132] In the example embodiments, the embodiments of the present application also provide a storage medium, i.e. a computer storage medium, specifically a computer readable storage medium, such as the memory 3 storing the computer program executable by the processor 2 of the first device to complete the steps of the aforementioned data transmission method.

[0133] The computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0134] In the example embodiments, the embodiments of the present application also provide a computer program product comprising the computer program executable by the processor 2 of the first device to complete the steps of the aforementioned data transmission method.

[0135] It should be noted that "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0136] The term "and / or" herein is merely a description of the association relationship between the associated objects, and can represent three relationships, for example, A and / or B can represent three cases of existence of A alone, existence of A and B simultaneously, and existence of B alone. In addition, the term "one or more" herein represents any combination of at least two of any one or more of a plurality, for example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C. In addition, the term "one or more" herein is an exemplary expression, which can be replaced by any possible expression, such as one or more, at least one, or at least one, etc.

[0137] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0138] The above is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application.

Claims

1. A data transmission method, characterized in that, Applied to a first device, the method includes: In response to a first system call from the application process, a first data buffer is divided into multiple segments; the first data buffer is used to buffer first data written by the application process; the first system call is used to instruct the sending of the first data. Based on the multiple segments, a first mbuf and multiple second mbufs are allocated from the first memory pool; the first mbuf is used to store the header information of the first message, the first message representing a message used to transmit the first data; each of the multiple second mbufs is used to indicate one of the segments. The first mbuf is linked with the plurality of second mbufs to form a first linked list, and the first linked list is mapped to the network card so that the network card can send the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

2. The method according to claim 1, characterized in that, The second mbuf includes one or more of the following: The virtual address of the corresponding segment; The data length of the corresponding segment.

3. The method according to claim 1, characterized in that, The method further includes: Add the following information corresponding to the first message to the first mbuf in sequence: Transport layer header information, which represents the message header information encapsulated based on the transport layer protocol; Network layer header information, which represents the message header information encapsulated based on the network layer protocol; Link layer header information, which represents the message header information encapsulated based on the link layer protocol.

4. The method according to claim 1, characterized in that, The first data buffer includes the application layer header information corresponding to the first message; the application layer header information represents the message header information encapsulated based on the application layer protocol. The application layer header information includes one or more of the following: The first field is used to indicate the protocol type and / or protocol version corresponding to the first message; The second field is used to indicate the service type corresponding to the first message; The third field is used to identify the first data buffer; The fourth field is used to indicate the memory state of the first data buffer.

5. The method according to claim 4, characterized in that, After the application process writes the first data into the first data buffer, and before the first device responds to the application process's first system call, the fourth field is used to indicate that the memory state of the first data buffer is a set first memory state; and / or, After the first device responds to the first system call of the application process and before the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is a set second memory state; and / or, After the first device receives the second message, the fourth field is used to indicate that the memory state of the first data buffer is the set third memory state; The second message represents the message returned by the second device based on the first message.

6. The method according to claim 5, characterized in that, The second message contains the same fields as the third field.

7. The method according to claim 5, characterized in that, If the fourth field indicates that the memory state of the first data buffer is the set second memory state, the application process is allowed to perform modification and / or release operations on the first data buffer; and / or, If the fourth field indicates that the memory state of the first data buffer is the set third memory state, the application process is not allowed to perform modification and / or release operations on the first data buffer.

8. The method according to claim 1, characterized in that, Prior to responding to the first system call from the application process, the method further includes: The first system call was hijacked.

9. A data transmission device, characterized in that, Applied to the first device, including: A segmentation unit is configured to, in response to a first system call from an application process, divide a first data buffer into multiple segments; the first data buffer is configured to buffer first data written by the application process; the first system call is configured to instruct the sending of the first data. An allocation unit is configured to allocate a first mbuf and a plurality of second mbufs from a first memory pool based on the plurality of segments; the first mbuf is used to store the header information of a first message, the first message representing a message used to transmit the first data; each of the plurality of second mbufs is used to indicate one of the segments. The linking unit is used to link the first mbuf with the plurality of second mbufs into a first linked list, and map the first linked list to the network card, so that the network card can send the first packet to the second device; the first linked list is used by the network card to read the first data from the first data buffer and encapsulate the first data into the first packet.

10. A first device, characterized in that, include: A processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images