Data processing

Abstract

Provided in the embodiments of the present disclosure are a data processing method, apparatus and system. The method is applied to a data transmission device, and comprises: in response to a data sending notification of a data processing device, acquiring a target data descriptor of target message data; when it is determined, on the basis of the target data descriptor, that the target message data requires order preservation, placing the target data descriptor in a target buffer area for ordering, so as to obtain an ordering result; and on the basis of the ordering result, acquiring the target message data from a target memory address corresponding to the target data descriptor. A message order-preserving mode is changed to reordering based on a target data descriptor, and the size of the target data descriptor is much smaller than the size of target message data, so that the usage requirements of hardware resources are effectively reduced while an order-preserving capability that accommodates a higher CPU processing latency is achieved, thereby ensuring message order-preserving and performance requirements under high-latency CPU processing.

Description

Data processing Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to data processing. Background Technology

[0002] For some traffic in network transmission, especially elephant traffic (used to describe traffic that occupies a large amount of bandwidth in the network, characterized by high volume and long duration), there is often a requirement to maintain order, that is, to ensure that the order in which the same type of network packet enters the network interface card and the order in which the packets are sent after processing are consistent.

[0003] When packets can be distributed across different CPU (Central Processing Unit) cores for processing, the different loads of different CPU cores and the different logical processing required for the packets may lead to out-of-order packets after processing. The network card must reorder the packets before sending them out.

[0004] Limited by the hardware resources of the network card, its order preservation capability is limited, and it can accommodate out-of-order packet reordering at the microsecond level. In some complex logical processing flows, it is difficult to withstand the high latency introduced by CPU processing, thus causing problems such as packet loss and performance failure. Summary of the Invention

[0005] In view of this, the present disclosure provides a data processing method. One or more embodiments of the present disclosure simultaneously involve two data processing devices, a data processing system, a computing device, a computer-readable storage medium, and a computer program product, in order to solve the technical defects in the related art, such as limited order preservation capability due to hardware resource constraints of network cards, difficulty in withstanding the high latency introduced by CPU processing, resulting in packet loss and unsatisfactory performance.

[0006] According to a first aspect of the present disclosure, a data processing method is provided, applied to a data transmission device, comprising: in response to a data transmission notification from the data processing device, obtaining a target data descriptor for target message data; if it is determined from the target data descriptor that the target message data needs to be sorted, placing the target data descriptor into a target buffer for sorting to obtain a sorting result; and obtaining the target message data from a target memory address corresponding to the target data descriptor according to the sorting result.

[0007] According to a second aspect of the present disclosure, a data processing method is provided, applied to a data transmission device, comprising: responding to target message data sent by a data transceiver device; determining an initial data descriptor corresponding to the target message data from a target queue corresponding to the target message data; if it is determined that the target message data needs to be ordered, updating the initial data descriptor according to the target message data to obtain a target data descriptor; sending the target message data to a target memory address according to the target data descriptor; updating the initial data descriptor in the target queue to the target data descriptor, and sending data transmission information for the target message data to a data processing device corresponding to the target queue.

[0008] According to a third aspect of the present disclosure, a data processing apparatus is provided, applied to a data transmission device, comprising: a descriptor acquisition module configured to acquire a target data descriptor of target message data in response to a data transmission notification from the data processing device; a descriptor sorting module configured to, when it is determined based on the target data descriptor that the target message data needs to be sorted, place the target data descriptor into a target buffer for sorting to obtain a sorting result; and a message acquisition module configured to acquire the target message data from a target memory address corresponding to the target data descriptor based on the sorting result.

[0009] According to a fourth aspect of the present disclosure, another data processing apparatus is provided, applied to a data transmission device, comprising: a descriptor acquisition module configured to, in response to target message data sent by a data transceiver device, determine an initial data descriptor corresponding to the target message data from a target queue corresponding to the target message data; a descriptor update module configured to, when it is determined that the target message data needs to be ordered, update the initial data descriptor according to the target message data to obtain a target data descriptor; a message sending module configured to send the target message data to a target memory address according to the target data descriptor; and an event notification module configured to update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

[0010] According to a fifth aspect of the present disclosure, a data processing system is provided, including a data transceiver device, a data transmission device, and a data processing device. The data transceiver device is configured to receive target message data, allocate a target queue for the target message data, and select an order-preserving mode for the target message data. The data transmission device is configured to, in response to the target message data sent by the data transceiver device, determine an initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data; if it is determined that the target message data requires order preservation, update the initial data descriptor according to the target message data to obtain a target data descriptor; send the target message data to a target memory address according to the target data descriptor; update the initial data descriptor in the target queue to the target data descriptor; and... The system sends data transmission information for the target message data to the data processing device corresponding to the target queue. The data processing device is configured to, based on the data transmission information, obtain the target data descriptor of the target message data from the target queue, obtain the target message data from the target memory address corresponding to the target data descriptor, process the target message data, and send a data transmission notification to the data transmission device after processing. The data transmission device is further configured to, in response to the data transmission notification from the data processing device, obtain the target data descriptor of the target message data, and, if it is determined based on the target data descriptor that the target message data needs to be sorted, place the target data descriptor into a target buffer for sorting to obtain a sorting result, and obtain the target message data from the target memory address corresponding to the target data descriptor based on the sorting result.

[0011] According to a sixth aspect of the present disclosure, a computing device is provided, comprising: a memory and a processor; the memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, wherein the computer programs / instructions, when executed by the processor, implement the steps of the above-described data processing method.

[0012] According to a seventh aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0013] According to an eighth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0014] This disclosure provides a data processing method applied to a data transmission device. Upon responding to a data transmission notification from a data processing device, the data transmission device obtains a target data descriptor for a target message. Based on the target data descriptor, it can determine whether the target message needs to be ordered. If order preservation is required, the target data descriptor is placed in a target buffer for sorting. Using the sorted result, the target message is retrieved from the target memory address corresponding to the target data descriptor. This changes the message order preservation method from data-level reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the size of the target message, the target buffer can hold more target data descriptors. This allows for order preservation with greater CPU processing latency while effectively reducing hardware resource usage, ensuring message order preservation and performance requirements under high CPU latency processing conditions. Attached Figure Description

[0015] Figure 1 is a schematic diagram of a data processing method provided in an embodiment of this disclosure;

[0016] Figure 2 is a flowchart of a data processing method applied to a data transmission device according to an embodiment of the present disclosure;

[0017] Figure 3 is a flowchart of another data processing method applied to a data transmission device according to an embodiment of the present disclosure;

[0018] Figure 4a is a schematic diagram of the processing flow of a data processing method provided in an embodiment of this disclosure;

[0019] Figure 4b is a schematic diagram of the processing flow of a packet receiving direction and data processing method provided in an embodiment of this disclosure;

[0020] Figure 4c is a schematic diagram of a Ring structure provided in an embodiment of this disclosure;

[0021] Figure 4d is a schematic diagram of the message processing flow in a data processing method provided in an embodiment of this disclosure;

[0022] Figure 4e is a schematic diagram of the processing flow of a packet sending direction and data processing method provided in an embodiment of this disclosure;

[0023] Figure 5 is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure;

[0024] Figure 6 is a schematic diagram of the structure of another data processing apparatus provided in an embodiment of the present disclosure;

[0025] Figure 7 is a schematic diagram of the structure of a data processing system provided in an embodiment of this disclosure;

[0026] Figure 8 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0027] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0028] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0030] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0031] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0032] DMA (Direct Memory Access): A function provided by the computer bus architecture that enables data to be read directly from or written to the computer's host memory from an additional device (such as a network card); it allows for large-volume data transfers between host-side memory and other hardware devices without the need for CPU intervention.

[0033] Descriptor: Used to describe the basic information of a DMA data transfer, including the memory address and length of the data.

[0034] Elephant Flow: Used to describe a flow (traffic) that consumes a large amount of bandwidth in a network, characterized by high traffic volume and long duration.

[0035] Preservation of order: The order in which traffic packets of the same network packet type enter the network interface card (NIC) is consistent with the order in which the packets are sent after processing.

[0036] This disclosure provides two data processing methods, and also relates to two data processing apparatuses, a data processing system, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0037] Referring to Figure 1, Figure 1 shows a scenario diagram of a data processing method provided according to an embodiment of the present disclosure.

[0038] Specifically, the network card receives multiple data packets from the network and can distribute these packets to different CPU cores. After processing by the different CPU cores, the packets are returned to the network card. In cases where order preservation is required, the processed packets are sorted according to the order in which they were received, thus achieving order preservation.

[0039] When multiple packet data are reordered in the buffer, the high latency introduced by CPU processing due to the limited buffer capacity of the network card and the hardware resources may lead to packet loss. Therefore, in this embodiment, the order preservation method of multiple packet data is changed from sorting the packet data in the buffer to sorting the descriptors of the packet data in the buffer. Based on the order descriptors, the packet data at the corresponding memory address is read to ensure the order of packet data acquisition and achieve packet data order preservation.

[0040] Specifically, the data processing method provided in this embodiment is applied to a data transmission device, which can be understood as a DMA engine responsible for the interaction between the network card and the CPU. In response to a data transmission notification from the data processing device (i.e., the CPU), the data transmission device obtains a target data descriptor for the target packet data. If, based on the target data descriptor, it is determined that the target packet data needs to be ordered, the target data descriptor is placed in a target buffer for sorting to obtain a sorting result. Based on the sorting result, the target packet data is obtained from the target memory address corresponding to the target data descriptor.

[0041] This disclosure provides a data processing method applied to a data transmission device. Upon responding to a data transmission notification from a data processing device, the data transmission device obtains a target data descriptor for a target message. Based on the target data descriptor, it can determine whether the target message needs to be ordered. If order preservation is required, the target data descriptor is placed in a target buffer for sorting. Using the sorted order, the target message is retrieved from the target memory address corresponding to the target data descriptor. This changes the message order preservation method from data-level reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the size of the target message, the target buffer can hold more target data descriptors. This effectively reduces hardware resource usage while accommodating greater CPU processing latency, ensuring message order preservation and performance requirements under high CPU latency processing conditions.

[0042] Referring to Figure 2, Figure 2 shows a flowchart of a data processing method applied to a data transmission device according to an embodiment of the present disclosure, which specifically includes the following steps.

[0043] Step 202: In response to the data transmission notification from the data processing device, obtain the target data descriptor of the target message data.

[0044] In this context, data transmission equipment can be understood as the data transmission channel between data processing equipment and data transceiver equipment; for example, a data transmission equipment might be a DMA engine. Data processing equipment is used to process target message data, such as a CPU. Data transceiver equipment is used to receive and send target message data, such as a network interface card (NIC).

[0045] The target data descriptor is metadata used to represent or point to the target message data, including but not limited to the memory address of the target message data, message length, and other information.

[0046] Specifically, since the target message data needs to undergo processing by the protocol stack, data decoding and decryption, and context switching, it needs to be sent to the data processing device for processing. After the data processing device finishes processing the target message data, it sends a data transmission notification to the data transmission device so that the data transceiver device can obtain the processed target message data through the data transmission device.

[0047] In response to a data transmission notification sent by a data processing device, the data transmission device needs to determine the target data descriptor of the target message data through the data transmission notification, thereby obtaining the target message data based on the memory address in the target data descriptor, and further sending the target message data to the data transceiver device.

[0048] Specifically, the data transmission notification carries data transmission information, which includes the queue position identifier of the target data descriptor in the target queue. The target queue is the queue corresponding to the data processing device where the target data descriptor is located. The data transmission information is used to determine the position of the target data descriptor corresponding to the target message data in the target queue, thereby quickly obtaining the target data descriptor corresponding to the target message data. The specific implementation is as follows: Obtaining the target data descriptor of the target message data in response to the data transmission notification from the data processing device includes: determining the data transmission information carried in the data transmission notification, wherein the data transmission information includes the queue position identifier of the target data descriptor in the target queue, and the target queue is the queue corresponding to the data processing device where the target data descriptor is located; and obtaining the target data descriptor of the target message data from the target position of the target queue corresponding to the queue position identifier of the target data descriptor in the target queue.

[0049] Specifically, the target queue includes target data descriptors for multiple target message data. Therefore, the data transmission notification needs to carry the queue position identifier of the target data descriptor in the target queue. Based on the queue position identifier of the target data descriptor in the target queue, the corresponding target data descriptor can be obtained from the target position corresponding to the queue position identifier. This target data descriptor is the descriptor of the target message data.

[0050] If the target data descriptor contains information including but not limited to whether the target packet data needs to be ordered, the packet length of the target packet data, and the memory address of the target packet data, after obtaining the target data descriptor of the target packet data, the target packet data can be read correctly. When it is determined that the target packet data needs to be ordered, the target data descriptor is cached in the target buffer, and the multiple data descriptors in the target buffer are sorted according to the ordering information in the target data descriptor.

[0051] The data processing method provided in this disclosure can accurately obtain the target data descriptor of the target message data from the target position of the target queue corresponding to the queue position identifier in the data transmission notification, thereby providing data support for the subsequent sorting of the target data descriptor.

[0052] Step 204: If it is determined that the target message data needs to be ordered based on the target data descriptor, the target data descriptor is placed into the target buffer for sorting to obtain the sorting result.

[0053] Since the target data descriptor includes an order preservation flag to determine whether the target message data needs to be ordered, the order preservation flag in the target data descriptor can be used to determine whether the target message data needs to be ordered.

[0054] The target buffer, also known as the buffer in the above embodiments, is used to temporarily store the acquired target data descriptors. When it is determined that the target message data needs to be ordered based on the target data descriptors, the corresponding target data descriptors are cached in the target buffer. The target data descriptors are then sorted in the target buffer to obtain the sorting result of the target data descriptors and determine the reading order of the target data descriptors.

[0055] In one or more embodiments of this disclosure, the target data descriptor contains an order preservation flag, which indicates whether the target message data needs to be ordered. A specific implementation is described below: determining that the target message data needs to be ordered based on the target data descriptor includes: checking the order preservation flag in the target data descriptor; if the order preservation flag is a target identifier, determining that the target message data needs to be ordered.

[0056] Specifically, when a data transceiver receives target packet data, it can select a sequence-keeping mode for it. The sequence-keeping mode includes two modes: one requiring sequence-keeping and one not requiring sequence-keeping. When the data transceiver sends the target packet data to the data transmission device, it carries the sequence-keeping mode information of the target packet data. When the data transmission device determines that the target packet data needs to be sequence-keeping based on the sequence-keeping mode information of the target packet data, it sets the sequence-keeping flag in the initial data descriptor to 1 when updating the initial data descriptor corresponding to the target packet data. It also uses the sequence-keeping engine to allocate a sequence-keeping information for the target packet data and updates the initial data descriptor with this sequence-keeping information to obtain the target data descriptor of the target packet data.

[0057] The target identifier can be understood as 1. Therefore, after obtaining the target data descriptor corresponding to the target message data, the order preservation flag bit in the target data descriptor can be checked. If the order preservation flag bit is the target identifier 1, it can be determined that the target message data needs to be ordered.

[0058] The data processing method provided in this embodiment sets an order preservation flag in the target data descriptor and sets different flags according to the order preservation mode information of the target message data. Therefore, when determining whether the target message data needs to be ordered, the order preservation flag of the target data descriptor corresponding to the target message data can be checked, and the flag in the order preservation flag can be used to efficiently and quickly determine whether the target message data needs to be ordered.

[0059] In one or more embodiments of this disclosure, the target data descriptor contains order preservation information for the target message data. This order preservation information is used to characterize the receiving order of the target message data. Therefore, the target message data can be sorted according to the order preservation information to ensure that the sorted target message data remains consistent with the receiving order. Specific implementation methods are as follows: When there are multiple data transmission notifications and multiple target data descriptors, and when it is determined that the corresponding target message data needs to be ordered based on the multiple target data descriptors, the multiple target data descriptors are placed into a target buffer; according to the order preservation information in the multiple target data descriptors, the multiple target data descriptors in the target buffer are sorted to obtain the sorting result.

[0060] Among them, the order information can be understood as information that can identify or represent the receiving order of the target message data.

[0061] Specifically, the target data descriptors corresponding to multiple target message data are checked to determine whether the target message data corresponding to them needs to maintain the receiving order based on the order preservation flag in the target data descriptor; for example, when the order preservation flag in the target data descriptor is 1, it is determined that the target message data corresponding to the target data descriptor needs to maintain the order, and when the order preservation flag in the target data descriptor is 0, it is determined that the target message data corresponding to the target data descriptor does not need to maintain the order.

[0062] Once it is determined that the target message data needs to be ordered (the ordering flag in the target data descriptor is 1), the target data descriptors corresponding to these target message data are placed into the data structure of the target buffer. The target buffer can be understood as a temporary storage area used to save these target data descriptors before sorting.

[0063] These target data descriptors are sorted using the order-preserving information in the target data descriptors. The order-preserving information can be a sequence number, timestamp, or other marker that can uniquely identify the receiving order. The sorting algorithm compares and sorts multiple target data descriptors based on their order-preserving information to ensure that the sorted target data descriptors are in the same order as the original receiving order.

[0064] In practical applications, the sequence information can be understood as a sequence number. The sequence number is incremented by one each time the data transmission device receives target message data sent by a data transceiver device.

[0065] The data processing method provided in this disclosure can quickly sort multiple target data descriptors by using the order-preserving information in the target data descriptors, and ensure that the sorting order of the multiple target data descriptors in the sorting result is consistent with the order at the time of reception.

[0066] Step 206: Based on the sorting result, obtain the target packet data from the target memory address corresponding to the target data descriptor.

[0067] Specifically, based on the sorting result of the target data descriptors in the target buffer, the reading order of the target data descriptors can be determined. Thus, by reading the target data descriptors in the order they are read, the corresponding target message data can be obtained from the target memory address corresponding to the target data descriptor.

[0068] In other words, by sorting the target data descriptors in the target buffer, the order in which the target packet data is obtained from the target memory address is guaranteed based on the reading order of the target data descriptors. The order preservation method is changed from reordering the packet data to reordering based on the target data descriptors. This not only accommodates greater CPU processing latency but also effectively reduces the demand on hardware resources.

[0069] In one or more embodiments of this disclosure, the sorting result is the order of the target data descriptor and other target data descriptors in the target buffer, determined according to the order-preserving information in the target data descriptor. Specifically, when other previously stored target data descriptors exist in the target buffer and the target data descriptor is placed in the target buffer, the order of the target data descriptor among other target data descriptors is determined. A specific implementation is as follows: obtaining the target packet data from the target memory address corresponding to the target data descriptor according to the sorting result includes: obtaining the target packet data from the target memory address corresponding to the target data descriptor according to the order of the target data descriptor and other target data descriptors in the target buffer.

[0070] Specifically, before the target data descriptor is placed into the target buffer, the target buffer may contain other target data descriptors of other target message data. These other target data descriptors have an order-keeping flag of 1 and include corresponding order-keeping information. Therefore, when the target data descriptor is placed into the target buffer, the target data descriptor and other target data descriptors in the target buffer are sorted according to the order-keeping information to determine the arrangement order of the target data descriptor and other target data descriptors in the target buffer. This allows the data transmission device to sequentially read the descriptors (including the target data descriptor and other target data descriptors) in the target buffer based on this arrangement order and obtain the message data corresponding to the memory address of the descriptor.

[0071] The data processing method provided in this disclosure sorts the descriptors in the target buffer to obtain the arrangement order between the descriptors, and then obtains the message data corresponding to the descriptors by reading the order of the descriptors based on the arrangement order, thereby achieving the preservation of message data order.

[0072] In one or more embodiments of this disclosure, the data transmission notification includes multiple data packets, and the target message data includes multiple data packets, each corresponding to a multiple target data descriptor. Therefore, when it is determined that the target message data corresponding to multiple target data descriptors needs to maintain order, the multiple target data descriptors are cached in a target buffer and sorted to obtain the arrangement order of the multiple target data descriptors. A specific implementation is as follows: obtaining the target data descriptor of the target message data based on the data transmission information carried in the data transmission notification includes: obtaining the target data descriptor of the multiple target message data based on the data transmission information carried in the multiple data transmission notifications.

[0073] The step of placing the target data descriptors into a target buffer for sorting to obtain a sorting result when it is determined that the target message data needs to be ordered based on the target data descriptors includes: placing the multiple target data descriptors into a target buffer for sorting to obtain a sorting result when it is determined that the corresponding target message data needs to be ordered based on multiple target data descriptors, wherein the sorting result is the arrangement order of the multiple target data descriptors determined based on the order preservation information in the multiple target data descriptors.

[0074] Multiple target data descriptors can come from the same queue or from different queues; there is no restriction on this.

[0075] Specifically, in scenarios where the CPU processes large data streams, the network interface card (NIC) can distribute the packet data in the large data stream across different CPU cores for processing. Each CPU core is assigned a certain queue, so it can receive multiple data transmission notifications sent by the CPU. These multiple data transmission notifications can be sent from different queues of a single CPU core, or from different queues of different CPU cores, without any limitation.

[0076] Based on the data transmission information carried in multiple data transmission notifications, the queue position identifier of the target data descriptor corresponding to each target message data can be determined. Thus, based on the queue position identifier of each target data descriptor, the target data descriptor of multiple target message data can be obtained from the corresponding position.

[0077] If the order preservation flags in multiple target data descriptors indicate that the target message data needs to be ordered, then the multiple target data descriptors need to be sorted.

[0078] Specifically, target data descriptors that need to be ordered are cached, for example, in the target buffer of the data transmission device. The target data descriptors in the target buffer are sorted to obtain the arrangement order of multiple target data descriptors.

[0079] In practical implementation, the Tx descriptor engine in the data transmission device (DMA engine) is used to obtain the target data descriptor of the target packet data; the Tx descriptor sorting engine in the DMA engine can sort the target data descriptors in the target buffer, and when a preset time interval is reached, send multiple target data descriptors in sequence to the Tx DMA engine for packet data reading according to the sorting result. The Tx DMA engine reads the target packet data at the corresponding memory address according to the memory address information in the target data descriptor.

[0080] Of course, in practical applications, multiple target data descriptors in the target buffer can be sorted and sent sequentially to the Tx DMA engine responsible for data transfer when a preset time interval is reached; alternatively, multiple target data descriptors can be sorted again when a preset time interval is reached and sent out sequentially. There are no restrictions on this.

[0081] The data processing method provided in this disclosure can greatly reduce resource consumption when the cache is limited by hardware resources by caching multiple target data descriptors that need to be ordered, thereby accommodating more target data descriptors that need to be ordered and achieving order preservation of target message data.

[0082] In one or more embodiments of this disclosure, by reading the target data descriptors according to their arrangement order, the target packet data at the target memory address corresponding to the target data descriptor can be obtained, thus ensuring the order of the target packet data. The specific implementation is as follows: obtaining the target packet data from the target memory address corresponding to the target data descriptor according to the sorting result includes: obtaining the multiple target packet data from the target memory address corresponding to the multiple target data descriptors according to their arrangement order.

[0083] Specifically, by obtaining the arrangement order of multiple target data descriptors, the order-keeping order of the corresponding target packet data can be determined based on the arrangement order of the multiple target data descriptors. That is, by reading the target data descriptors sequentially, the corresponding target packet data can be obtained sequentially from the target memory address corresponding to the target data descriptor.

[0084] For example, target message data A corresponds to target data descriptor a, target message data B corresponds to target data descriptor b, and target message data C corresponds to target data descriptor c. If the receiving order of target message data A, B, and C is 1, 2, 3, then the ordering information in target data descriptor a can be 1, in target data descriptor b can be 2, and in target data descriptor c can be 3. After CPU processing, the order in which the target data descriptors are obtained can be target data descriptor b - target data descriptor c - target data descriptor a. Using the ordering information in the target data descriptors, target data descriptors b, c, and a are sorted. Based on the ordering information 1 corresponding to target data descriptor a, the ordering information 2 corresponding to target data descriptor b, and the ordering information 3 corresponding to target data descriptor c, the arrangement order of target data descriptors b, c, and a is determined to be target data descriptor a - target data descriptor b - target data descriptor c.

[0085] Based on the order of target data descriptors a-b-c, target data descriptors can be read sequentially: first, target data descriptor a is read, and target packet data A is obtained from the memory address corresponding to target data descriptor a; then, target data descriptor b is read, and target packet data B is obtained from the memory address corresponding to target data descriptor b; finally, target data descriptor c is read, and target packet data C is obtained from the memory address corresponding to target data descriptor c. In other words, by reading multiple target data descriptors in a sequential manner, the corresponding target packet data can be obtained from the target memory address corresponding to the target data descriptor, thus achieving order preservation of the target packet data.

[0086] In one or more embodiments of this disclosure, if it is determined from the target data descriptor that the target packet data does not need to be ordered, then there is no need to cache or sort the target data descriptor; that is, the target packet data at the corresponding memory location can be read directly from the target data descriptor.

[0087] In response to a data transmission notification from a data processing device, after obtaining the target data descriptor of the target message data based on the data transmission information carried in the data transmission notification, the method further includes: if it is determined from the target data descriptor that the target message data does not need to be ordered, obtaining the target message data from the target memory address corresponding to the target data descriptor.

[0088] This disclosure provides a data processing method applied to a data transmission device. In response to a data transmission notification from a data processing device, the data transmission device obtains a target data descriptor for a target message based on the data transmission information carried in the notification. The target data descriptor determines whether the target message needs to be ordered. If order preservation is required, the target data descriptor is placed in a target buffer for sorting. The sorted result is then used to retrieve the target message from the target memory address corresponding to the target data descriptor. This changes the message order preservation method from data-level reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the target message size, the target buffer can hold more target data descriptors. This effectively reduces hardware resource usage while accommodating greater CPU processing latency, ensuring message order preservation and performance requirements under high CPU latency processing conditions.

[0089] Referring to Figure 3, which is a flowchart of another data processing method applied to a data transmission device according to an embodiment of the present disclosure, specifically including the following steps.

[0090] Step 302: In response to the target message data sent by the data transceiver device, determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data.

[0091] The data transceiver device assigns a queue ID (identifier) ​​to the target message data. Therefore, in response to the target message data sent by the data transceiver device, the data transmission device can determine the corresponding target queue based on the queue ID and allocate an initial data descriptor for the target message data from the target queue. This initial data descriptor includes a memory address that the CPU has pre-specified for the target queue.

[0092] Step 304: If it is determined that the target message data needs to be ordered, the initial data descriptor is updated according to the target message data to obtain the target data descriptor.

[0093] As mentioned in the above embodiments, the data transceiver device will also select an order-keeping mode for the target packet data. When it is determined that the target packet data needs to be ordered, the order-keeping flag in the initial data descriptor is set to 1, and the order-keeping information is requested from the order-keeping engine. Then, the order-keeping information is added to the initial data descriptor, the initial data descriptor is updated, and the target data descriptor is obtained.

[0094] Of course, in practical applications, information such as the message length of the target message data will also be added to the initial data descriptor; therefore, if the target message data does not need to maintain order, the initial data descriptor will also be updated, but the update content will not include the order information.

[0095] Step 306: Send the target message data to the target memory address according to the target data descriptor.

[0096] The interaction between the network card and the CPU is achieved through memory. The target packet data will be sent to memory according to the target memory address corresponding to the target data descriptor.

[0097] Step 308: Update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

[0098] The initial data descriptor in the target queue is updated to the target data descriptor, and the CPU is made aware of the packet being sent through interrupt upload / CPU polling, etc., and the CPU begins to process the packet.

[0099] This disclosure provides a data processing method applied to a data transmission device. When receiving target message data sent by a data transceiver device, the method updates the initial data descriptor of the target message data to obtain a target data descriptor containing information such as a sequence identifier, sequence information, and message length. The updated target data descriptor facilitates subsequent transmission of target message data by ensuring the order of the target message data based on the sequence information within the target data descriptor.

[0100] Referring to Figure 4a, Figure 4a shows a schematic diagram of the processing flow of a data processing method provided in an embodiment of the present disclosure; wherein, message reception and mode selection is responsible for the selection of the DMA queue for receiving messages and the selection of the message order-keeping mode (such as whether order-keeping is required).

[0101] The order-keeping engine is responsible for allocating order-keeping information in the uplink (Rx) direction and retrieving order-keeping information in the downlink (Tx) direction. The Rx descriptor acquisition and Rx DMA engines are responsible for the execution of the uplink direction; the Tx descriptor engine, Tx descriptor sorting, and Tx DMA engine are responsible for the hardware and software interaction in the downlink direction.

[0102] The network card and the CPU communicate via the PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) interface.

[0103] The Rx descriptor, or Receive Descriptor, is used to instruct the network card to send the received data packets to a specified location in the host-side memory, specifying the size and other information. The Tx descriptor, or Transmit Descriptor, is used to instruct the network card to send the data packets to a specified location in the host-side memory, specifying the address, length, and other information.

[0104] Figure 4a shows the complete processing flow of message reception (black line) and message transmission (blue line); the specific processing flow of this data processing method is described in detail through the explanation of Figures 4b-4e.

[0105] Referring to Figure 4b, Figure 4b is a schematic diagram of the processing flow of a packet receiving direction and data processing method provided in an embodiment of this disclosure.

[0106] The network interface card (NIC) (i.e., data transceiver) receives data packets from the network and performs project logic processing, which includes, but is not limited to, rate limiting, statistics, mirroring, and partial offloading processing, etc., without restriction here; it allocates queue IDs to data packets and selects the order-keeping mode (whether order-keeping is required); it sends the data packets to the DMA engine (i.e., data transmission device). When the DMA engine receives the data packet, it allocates an Rx descriptor for the queue corresponding to the queue ID (the Rx descriptor at this time can be understood as the initial data descriptor in the above embodiment). The Rx descriptor includes information such as the memory address allocated by the CPU to its queue in advance.

[0107] Of course, in this embodiment, the Rx descriptor also includes an order preservation identifier and order preservation information. Specifically, the order preservation identifier in the Rx descriptor is different depending on the order preservation mode selected for the data packet. For example, for data packets that need to be ordered, the order preservation identifier in the Rx descriptor is 1, and the corresponding order preservation information is added to the Rx descriptor. This order preservation information is obtained from the order preservation engine (the order preservation information is generally an ID number, which is incremented by 1 for each packet sent on the hardware side); for data packets that do not need to be ordered, the order preservation identifier in the Rx descriptor is 0.

[0108] The Rx DMA engine in the DMA engine sends the data packet to the memory address corresponding to its Rx descriptor and updates the Rx descriptor information back to the queue. The descriptor update mainly informs the CPU of information such as the length of the current packet, so that the CPU can know the length of the packet. The CPU is made aware of the packet sending through interrupt sending / CPU polling and the CPU (i.e., the data processing device) begins to process the data packet.

[0109] In practical applications, queue-based DMA hardware and software (CPU and network card) interact with tasks based on a Ring mechanism; see Figure 4c, which shows a schematic diagram of a Ring structure provided in an embodiment of this disclosure.

[0110] In practical applications, the Ring represents a contiguous block of memory allocated by the CPU. The hardware and software act as the producer and consumer, respectively. For example, when the CPU needs to send data, it updates the relevant information about the data to be sent in the descriptor at the Producer pointer in the Ring. It then notifies the hardware that the Producer pointer has changed and a new task needs to be processed. Upon receiving the software notification, the hardware retrieves the descriptor from the corresponding location and reads or writes the data at that location based on the information filled in the descriptor.

[0111] Referring to Figure 4d, Figure 4d shows a schematic diagram of the message processing flow in a data processing method provided in an embodiment of the present disclosure.

[0112] When the CPU core receives a notification from the hardware that a new message has been sent, it reads the Rx descriptor information at the corresponding position in its Ring, and then obtains the address, length, whether order preservation is required, and order preservation information of the data message based on the Rx descriptor information.

[0113] After the CPU processes the network packet, it sends the data packet. At this time, the CPU calls the DMA Tx direction driver to fill the content of the data packet into the Tx descriptor of the downlink direction. (Note that the memory address of the data packet may change after CPU processing, so the content in the Tx descriptor and Rx descriptor may be different. However, whether the data packet needs to be ordered and the ordering information remain unchanged. Therefore, the ordering flag and ordering information in the Tx descriptor and Rx descriptor are the same.) In other words, the packet information, whether ordering is required, and the ordering information are filled into the Tx descriptor of the corresponding Ring and the hardware network card is notified that there is a new data transmission task to be performed.

[0114] Referring to Figure 4e, Figure 4e is a schematic diagram of the processing flow of a packet sending direction and data processing method provided in an embodiment of this disclosure.

[0115] When the hardware network card receives a packet transmission notification request from the CPU, it obtains the corresponding data packet through the DMA engine. Specifically, the Tx descriptor engine in the DMA engine obtains the Tx descriptor from the corresponding position in the Ring and checks the ordering flag in the Tx descriptor. When the ordering flag in the Tx descriptor is 1 (meaning that the data packet corresponding to the descriptor needs to be ordered), the Tx descriptor is cached. When a preset time interval is reached, the cached Tx descriptors from different queues are sorted according to the ordering information to be reclaimed provided by the ordering engine (the ordering information to be reclaimed is the ordering information that needs to be sorted and sent out for reclaiming, and the corresponding Tx descriptor can be determined through the ordering information to be reclaimed). The Tx descriptors sorted according to the ordering information are sent to the Tx DMA engine in order. When the ordering flag in the Tx descriptor is 0 (meaning that the data packet corresponding to the descriptor does not need to be ordered), the Tx descriptor is sent directly to the Tx DMA engine.

[0116] The Tx DMA engine reads data packets from the memory addresses in the Tx descriptors according to the order of the received Tx descriptors, performs DMA data transfer, and ensures the order of the data packets. The read data packets are processed by the project processing logic, and after processing, the data packets are sent out through the network interface.

[0117] This disclosure provides a data processing method. Since the existing order-keeping function is based on the caching and sorting of the packet data itself, and due to limited hardware resources and the fact that network packets are often long, the number of packets that can be cached is small. Therefore, it is impossible to achieve the order-keeping capability while ensuring line speed in high CPU latency scenarios.

[0118] This disclosure provides a data processing method that uses DMA multi-queue descriptors for packet caching and sorting. The storage space occupied by the descriptor is fixed, only a dozen bytes. Regardless of the packet length, the storage space consumed is fixed. Compared with packet-based caching and sorting, it can achieve tens to hundreds of times higher CPU latency sorting capability, thereby tolerating line-speed millisecond-level CPU latency. The longer the packet, the greater the benefit.

[0119] Corresponding to the above method embodiments, this disclosure also provides a data processing apparatus embodiment. Figure 5 shows a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this disclosure. As shown in Figure 5, the apparatus includes: a descriptor acquisition module 502, configured to acquire a target data descriptor of target message data in response to a data transmission notification from a data processing device; a descriptor sorting module 504, configured to sort the target data descriptor in a target buffer to obtain a sorting result when it is determined that the target message data needs to be sorted according to the target data descriptor; and a message acquisition module 506, configured to acquire the target message data from the target memory address corresponding to the target data descriptor according to the sorting result.

[0120] Optionally, the descriptor acquisition module 502 is further configured to: in response to a data transmission notification from the data processing device, determine the data transmission information carried in the data transmission notification, wherein the data transmission information includes a queue position identifier of the target data descriptor in the target queue, the target queue being the queue corresponding to the data processing device where the target data descriptor is located; and, based on the queue position identifier of the target data descriptor in the target queue, acquire the target data descriptor of the target message data from the target position of the target queue corresponding to the queue position identifier.

[0121] Optionally, the descriptor sorting module 504 is further configured to: check the order preservation flag in the target data descriptor, and if the order preservation flag is the target identifier, determine that the target message data needs to be ordered.

[0122] Optionally, the message acquisition module 506 is further configured to: acquire the target message data from the target memory address corresponding to the target data descriptor according to the arrangement order of the target data descriptor and other target data descriptors in the target buffer.

[0123] Optionally, the descriptor acquisition module 502 is further configured to: acquire target data descriptors of multiple target message data based on the data transmission information carried in multiple data transmission notifications.

[0124] Optionally, the descriptor sorting module 504 is further configured to: when it is determined that the corresponding target message data needs to be sorted based on multiple target data descriptors, put the multiple target data descriptors into the target buffer for sorting to obtain a sorting result, wherein the sorting result is the arrangement order of the multiple target data descriptors determined based on the order preservation information in the multiple target data descriptors.

[0125] Optionally, the message acquisition module 506 is further configured to: acquire the multiple target message data from the target memory addresses corresponding to the multiple target data descriptors according to the arrangement order of the multiple target data descriptors.

[0126] The apparatus further includes a data acquisition module configured to acquire the target packet data from the target memory address corresponding to the target data descriptor when it is determined from the target data descriptor that the target packet data does not need to be ordered.

[0127] This disclosure provides a data processing apparatus applied to a data transmission device. Upon responding to a data transmission notification from the data processing device, the data transmission device obtains a target data descriptor for a target message based on data transmission information carried in the notification. The target data descriptor determines whether the target message needs to be ordered. If order preservation is required, the target data descriptor is placed in a target buffer for sorting. The sorted result is then used to retrieve the target message from the target memory address corresponding to the target data descriptor. This changes the message order preservation method from data-level reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the target message size, the target buffer can hold more target data descriptors. This effectively reduces hardware resource usage while accommodating greater CPU processing latency, ensuring message order preservation and performance requirements under high CPU latency processing conditions.

[0128] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0129] Corresponding to the above method embodiments, this disclosure also provides a data processing apparatus embodiment. Figure 6 shows a schematic diagram of another data processing apparatus provided in one embodiment of this disclosure. As shown in Figure 6, the apparatus includes: a descriptor acquisition module 602, configured to, in response to target message data sent by a data transceiver device, determine an initial data descriptor corresponding to the target message data from the target queue corresponding to the target message data; a descriptor update module 604, configured to, when it is determined that the target message data needs to be ordered, update the initial data descriptor according to the target message data to obtain a target data descriptor; a message sending module 606, configured to send the target message data to a target memory address according to the target data descriptor; and an event notification module 608, configured to update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

[0130] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0131] Corresponding to the above method embodiments, this disclosure also provides a data processing system embodiment. Figure 7 shows a schematic diagram of the structure of a data processing system provided in one embodiment of this disclosure. As shown in Figure 7, the system includes a data transceiver device 702, a data transmission device 704, and a data processing device 706, wherein: the data transceiver device 702 is used to receive target message data, allocate a target queue for the target message data, and select an order-preserving mode for the target message data; the data transmission device 704 is used to respond to the target message data sent by the data transceiver device 702, determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data, update the initial data descriptor according to the target message data to obtain a target data descriptor when it is determined that the target message data needs to be ordered, send the target message data to the target memory address according to the target data descriptor, update the initial data descriptor in the target queue to the target data descriptor, and send the target message data to the target memory address according to the target data descriptor. The data processing device 706 corresponding to the queue sends data transmission information for the target message data. The data processing device 706 is configured to, based on the data transmission information, obtain the target data descriptor of the target message data from the target queue, obtain the target message data from the target memory address corresponding to the target data descriptor, process the target message data, and send a data transmission notification to the data transmission device after processing. The data transmission device 704 is further configured to, in response to the data transmission notification from the data processing device 706, obtain the target data descriptor of the target message data, and, if it is determined based on the target data descriptor that the target message data needs to be sorted, place the target data descriptor into a target buffer for sorting to obtain a sorting result, and, based on the sorting result, obtain the target message data from the target memory address corresponding to the target data descriptor.

[0132] The above is an illustrative scheme of a data processing system according to this embodiment. It should be noted that the technical solution of this data processing system and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing system, please refer to the description of the technical solution of the data processing method described above.

[0133] Figure 8 shows a structural block diagram of a computing device 800 according to an embodiment of the present disclosure. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0134] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0135] In one embodiment of this disclosure, the aforementioned components of the computing device 800, as well as other components not shown in FIG. 8, may be interconnected, for example, via a bus. It should be understood that the computing device block diagram shown in FIG. 8 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0136] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0137] The processor 820 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0138] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the data processing method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the data processing method embodiments.

[0139] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0140] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the data processing method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the data processing method embodiments.

[0141] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0142] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0143] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0145] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0146] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0147] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied to a data transmission device, comprising: In response to a data transmission notification from the data processing device, obtain the target data descriptor of the target message data; If it is determined that the target message data needs to be ordered based on the target data descriptor, the target data descriptor is placed into the target buffer for sorting to obtain the sorting result; Based on the sorting result, the target message data is obtained from the target memory address corresponding to the target data descriptor.

2. The data processing method according to claim 1, wherein obtaining the target data descriptor of the target message data in response to the data transmission notification from the data processing device includes: In response to a data transmission notification from a data processing device, data transmission information carried in the data transmission notification is determined, wherein the data transmission information includes a queue position identifier of the target data descriptor in a target queue, and the target queue is the queue corresponding to the data processing device where the target data descriptor is located; Based on the queue position identifier of the target data descriptor in the target queue, the target data descriptor of the target message data is obtained from the target position of the target queue corresponding to the queue position identifier.

3. The data processing method according to claim 1, wherein determining that the target message data needs to be ordered based on the target data descriptor includes: Check the order preservation flag in the target data descriptor. If the order preservation flag is the target identifier, determine that the target message data needs to be ordered.

4. The data processing method according to any one of claims 1-3, wherein the sorting result is determined based on the order-preserving information in the target data descriptor, and the sorting result characterizes the arrangement order of the target data descriptor and other target data descriptors in the target buffer; The step of obtaining the target packet data from the target memory address corresponding to the target data descriptor according to the sorting result includes: Based on the order of the target data descriptor and other target data descriptors in the target buffer, the target message data is obtained from the target memory address corresponding to the target data descriptor.

5. The data processing method according to any one of claims 1-3, wherein the data transmission notification includes multiple data sources, the target message data includes multiple data sources, and the multiple target message data correspond to multiple target data descriptors; The step of obtaining the target data descriptor of the target message data based on the data transmission information carried in the data transmission notification includes: Based on the data transmission information carried in multiple data transmission notifications, obtain the target data descriptors for multiple target message data.

6. The data processing method according to any one of claims 1-3, wherein the data transmission notification includes multiple data sources, the target message data includes multiple data sources, and the multiple target message data correspond to multiple target data descriptors; When it is determined that the target message data needs to be ordered based on the target data descriptor, the target data descriptor is placed into the target buffer for sorting to obtain the sorting result, including: When it is determined that the corresponding target message data needs to be ordered based on multiple target data descriptors, the multiple target data descriptors are placed into the target buffer and sorted to obtain a sorting result, wherein the sorting result is the arrangement order of the multiple target data descriptors determined based on the order preservation information in the multiple target data descriptors.

7. The data processing method according to claim 6, wherein obtaining the target packet data from the target memory address corresponding to the target data descriptor according to the sorting result includes: Based on the order of the multiple target data descriptors, the multiple target message data are obtained from the target memory addresses corresponding to the multiple target data descriptors.

8. The data processing method according to any one of claims 1-3 or 7, further comprising, after obtaining the target data descriptor of the target message data based on the data transmission information carried in the data transmission notification in response to the data transmission notification from the data processing device, the method further comprises: If it is determined from the target data descriptor that the target packet data does not need to be ordered, the target packet data is obtained from the target memory address corresponding to the target data descriptor.

9. A data processing method applied to a data transmission device, comprising: In response to target message data sent by the data transceiver device, an initial data descriptor corresponding to the target message data is determined from the target queue according to the target queue corresponding to the target message data; If it is determined that the target message data needs to be ordered, the initial data descriptor is updated according to the target message data to obtain the target data descriptor; According to the target data descriptor, the target message data is sent to the target memory address; The initial data descriptor in the target queue is updated to the target data descriptor, and data transmission information for the target message data is sent to the data processing device corresponding to the target queue.

10. A data transmission system, comprising data transceiver equipment, data transmission equipment, and data processing equipment, wherein, The data transceiver device is used to receive target message data, allocate a target queue for the target message data, and select an order-preserving mode for the target message data. The data transmission device is configured to respond to target message data sent by the data transceiver device, determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data, update the initial data descriptor according to the target message data if it is determined that the target message data needs to be in order, obtain the target data descriptor, send the target message data to the target memory address according to the target data descriptor, update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue. The data processing device is configured to obtain the target data descriptor of the target message data from the target queue according to the data transmission information, obtain the target message data from the target memory address corresponding to the target data descriptor, process the target message data, and send a data transmission notification to the data transmission device after processing is completed. The data transmission device is further configured to respond to a data transmission notification from the data processing device, obtain a target data descriptor for the target message data, and, if it is determined from the target data descriptor that the target message data needs to be ordered, place the target data descriptor into a target buffer for sorting to obtain a sorting result, and, based on the sorting result, obtain the target message data from the target memory address corresponding to the target data descriptor.

11. A data processing apparatus, applied to a data transmission device, comprising: The descriptor acquisition module is configured to acquire the target data descriptor of the target message data in response to a data transmission notification from the data processing device. The descriptor sorting module is configured to, when it is determined from the target data descriptor that the target message data needs to be sorted, put the target data descriptor into the target buffer for sorting to obtain the sorting result; The message acquisition module is configured to acquire the target message data from the target memory address corresponding to the target data descriptor according to the sorting result.

12. A data processing apparatus, applied to a data transmission device, comprising: The descriptor acquisition module is configured to, in response to target packet data sent by the data transceiver device, determine the initial data descriptor corresponding to the target packet data from the target queue corresponding to the target packet data. The descriptor update module is configured to update the initial data descriptor according to the target data when it is determined that the target packet data needs to be ordered, so as to obtain the target data descriptor; The message sending module is configured to send the target message data to the target memory address according to the target data descriptor; The event notification module is configured to update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

13. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the following: In response to a data transmission notification from the data processing device, obtain the target data descriptor of the target message data; If it is determined that the target message data needs to be ordered based on the target data descriptor, the target data descriptor is placed into the target buffer for sorting to obtain the sorting result; Based on the sorting result, the target message data is obtained from the target memory address corresponding to the target data descriptor.

14. The computing device according to claim 13, wherein obtaining the target data descriptor of the target message data in response to a data transmission notification from the data processing device includes: In response to a data transmission notification from a data processing device, data transmission information carried in the data transmission notification is determined, wherein the data transmission information includes a queue position identifier of the target data descriptor in a target queue, and the target queue is the queue corresponding to the data processing device where the target data descriptor is located; Based on the queue position identifier of the target data descriptor in the target queue, the target data descriptor of the target message data is obtained from the target position of the target queue corresponding to the queue position identifier.

15. The computing device according to claim 13, wherein determining that the target message data needs to be ordered based on the target data descriptor includes: Check the order preservation flag in the target data descriptor. If the order preservation flag is the target identifier, determine that the target message data needs to be ordered.

16. The computing device according to any one of claims 13-15, wherein the sorting result is determined based on the order-preserving information in the target data descriptor, and the sorting result characterizes the arrangement order of the target data descriptor and other target data descriptors in the target buffer; The step of obtaining the target packet data from the target memory address corresponding to the target data descriptor according to the sorting result includes: Based on the order of the target data descriptor and other target data descriptors in the target buffer, the target message data is obtained from the target memory address corresponding to the target data descriptor.

17. The computing device according to any one of claims 13-15, wherein the data transmission notification includes a plurality, the target message data includes a plurality, and the plurality of target message data corresponds to a plurality of target data descriptors; The step of obtaining the target data descriptor of the target message data based on the data transmission information carried in the data transmission notification includes: Based on the data transmission information carried in multiple data transmission notifications, obtain the target data descriptors for multiple target message data.

18. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the following: In response to target message data sent by the data transceiver device, an initial data descriptor corresponding to the target message data is determined from the target queue according to the target queue corresponding to the target message data; If it is determined that the target message data needs to be ordered, the initial data descriptor is updated according to the target message data to obtain the target data descriptor; According to the target data descriptor, the target message data is sent to the target memory address; The initial data descriptor in the target queue is updated to the target data descriptor, and data transmission information for the target message data is sent to the data processing device corresponding to the target queue.

19. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 9.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 9.

Images