Data processing method, device and system

Abstract

The embodiments of the present specification provide a data processing method, apparatus and system, wherein the method is applied to a data transmission device, comprising: in response to a data sending notification of a data processing device, obtaining a target data descriptor of target message data; when it is determined according to the target data descriptor that the target message data needs to be sequenced, placing the target data descriptor into a target buffer for sorting to obtain a sorting result; according to the sorting result, obtaining the target message data from a target memory address corresponding to the target data descriptor; changing the message sequence preservation method to a reordering based on the target data descriptor, wherein the size of the target data descriptor is much smaller than the target message data size, thereby effectively reducing the use requirements of hardware resources while accommodating the sequence preservation capability of a larger CPU processing delay, thereby ensuring the message sequence preservation and performance requirements under high CPU latency processing conditions.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular, to data processing methods, devices, and systems. Background Art

[0002] For some traffic in network transmission, especially elephant traffic (used to describe traffic that occupies a large amount of bandwidth in the network, with characteristics such as high traffic and long duration), there is often a need to maintain order, that is, the order in which the same network message type traffic and the messages enter the network card must be consistent with the order in which the messages are sent after processing.

[0003] When packets can be distributed to different CPU (Central Processing Unit) cores for processing, due to factors such as different loads on different CPU cores and different logical processing required for packets, the packets may be out of order after processing. The network card needs to reorder the packets before sending them out.

[0004] Limited by the hardware resources of the network card, its order-preserving capability is limited and can accommodate the reordering of out-of-order messages at the microsecond level. In some complex logical processing flows, it is difficult to withstand the high latency introduced by CPU processing, which leads to packet loss and performance that does not meet expectations. Summary of the invention

[0005] In view of this, the embodiments of this specification provide a data processing method. One or more embodiments of this specification simultaneously involve two data processing devices, a data processing system, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art, which are limited by the hardware resource limitations of the network card, have limited order-keeping capabilities, and are unable to withstand the high delay introduced by CPU processing, thereby causing packet loss, performance not meeting expectations, and the like.

[0006] According to a first aspect of an embodiment of this specification, a data processing method is provided, which is applied to a data transmission device, including:

[0007] In response to a data transmission notification of the data processing device, obtaining a target data descriptor of the target message data;

[0008] When it is determined according to the target data descriptor that the target message data needs to be sequenced, the target data descriptor is placed in a target buffer for sequencing to obtain a sequencing result;

[0009] According to the sorting result, the target message data is obtained from the target memory address corresponding to the target data descriptor.

[0010] According to a second aspect of an embodiment of this specification, a data processing method is provided, which is applied to a data transmission device, including:

[0011] In response to target message data sent by a data transceiver device, according to a target queue corresponding to the target message data, determining an initial data descriptor corresponding to the target message data from the target queue;

[0012] When it is determined that the target message data needs to be sequenced, the initial data descriptor is updated according to the target message data to obtain a target data descriptor;

[0013] According to the target data descriptor, the target message data is sent to the target memory address;

[0014] The initial data descriptor in the target queue is updated to be the target data descriptor, and data transmission information for the target message data is sent to a data processing device corresponding to the target queue.

[0015] According to a third aspect of an embodiment of this specification, a data processing apparatus is provided, which is applied to a data transmission device, including:

[0016] a descriptor acquisition module, configured to acquire a target data descriptor of target message data in response to a data transmission notification of a data processing device;

[0017] A descriptor sorting module is configured to put the target data descriptor into a target buffer for sorting to obtain a sorting result when it is determined according to the target data descriptor that the target message data needs to be kept in order;

[0018] The message acquisition module is configured to acquire the target message data from the target memory address corresponding to the target data descriptor according to the sorting result.

[0019] According to a fourth aspect of an embodiment of this specification, another data processing apparatus is provided, which is applied to a data transmission device, including:

[0020] a descriptor acquisition module configured to respond to target message data sent by a data transceiver device and determine an initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data;

[0021] A descriptor updating module is configured to update the initial data descriptor according to the target message data to obtain a target data descriptor when it is determined that the target message data needs to be sequenced;

[0022] a message sending module, configured to send the target message data to a target memory address according to the target data descriptor;

[0023] The event notification module is configured to update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

[0024] According to a fifth aspect of the embodiments of this specification, a data processing system is provided, including a data transceiver device, a data transmission device, and a data processing device, wherein:

[0025] The data transceiver device is used to receive target message data, allocate a target queue for the target message data, and select an order-preserving mode for the target message data;

[0026] The data transmission device is used to respond to the target message data sent by the data transceiver device, determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data, and when it is determined that the target message data needs to be sequenced, update the initial data descriptor according to the target message data to obtain the target data descriptor, send the target message data to the target memory address according to the target data descriptor, update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue;

[0027] The data processing device is used to obtain the target data descriptor of the target message data from the target queue according to the data transmission information, obtain the target message data from the target memory address corresponding to the target data descriptor, process the target message data, and send a data sending notification to the data transmission device after the processing is completed;

[0028] The data transmission device is also used to respond to the data sending notification of the data processing device, obtain the target data descriptor of the target message data, and when it is determined according to the target data descriptor that the target message data needs to be kept in order, put the target data descriptor into the target buffer for sorting to obtain the sorting result, and according to the sorting result, obtain the target message data from the target memory address corresponding to the target data descriptor.

[0029] According to a sixth aspect of an embodiment of this specification, a computing device is provided, including:

[0030] Memory and processor;

[0031] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0032] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and the computer program / instruction implements the steps of the above-mentioned data processing method when executed by a processor.

[0033] According to an eighth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0034] An embodiment of the present specification provides a data processing method, which is applied to a data transmission device. In response to a data sending notification from a data processing device, the data transmission device obtains a target data descriptor of target message data. Based on the target data descriptor, it can be determined whether the target message data needs to be sequenced. If sequence preservation is required, the target data descriptor is placed in a target buffer for sorting, and then the target message data is obtained from a target memory address corresponding to the target data descriptor using the sequential sorting result, that is, the message sequence preservation method is changed from data granularity reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the target message data size, more target data descriptors can be accommodated in the target buffer. While accommodating the sequence preservation capability of a larger CPU processing delay, the demand for the use of hardware resources is effectively reduced, thereby ensuring the message sequence preservation and performance requirements under CPU high-latency processing conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a scenario diagram of a data processing method provided by an embodiment of this specification;

[0036] Figure 2 is a flow chart of a data processing method applied to a data transmission device provided by an embodiment of this specification;

[0037] Figure 3 is a flow chart of another data processing method applied to a data transmission device provided by an embodiment of this specification;

[0038] Figure 4a It is a processing flow diagram of a data processing method provided by an embodiment of this specification;

[0039] Figure 4b It is a processing flow diagram of a packet receiving direction and a data processing method provided by an embodiment of this specification;

[0040] Figure 4c It is a schematic diagram of the structure of a Ring provided by an embodiment of this specification;

[0041] Figure 4d It is a schematic diagram of a processing flow of message processing in a data processing method provided by an embodiment of this specification;

[0042] Figure 4e It is a processing flow diagram of a contract sending direction and a data processing method provided by an embodiment of this specification;

[0043] Figure 5 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0044] Figure 6 is a schematic diagram of the structure of another data processing device provided by an embodiment of this specification;

[0045] Figure 7 is a structural diagram of a data processing system provided by an embodiment of this specification;

[0046] Figure 8 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0047] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0048] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0049] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0050] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0051] First, the terms involved in one or more embodiments of this specification are explained.

[0052] DMA (Direct Memory Access): Direct memory access is a function provided by the computer bus architecture that enables data to be read / written directly from an attached device (such as a network card) to the computer's host memory; it allows large-scale data movement between the host side memory and other hardware devices without the involvement of the CPU.

[0053] Descriptor: It is used to describe the basic information of a DMA data transfer, including the memory address and length of the data.

[0054] Elephant Flow: It is used to describe the flow (traffic) that occupies a large amount of bandwidth in the network, and has the characteristics of high flow and long duration.

[0055] Sequence preservation: The order in which traffic messages of the same network message type enter the network card is consistent with the order in which they are sent out after being processed.

[0056] In this specification, two data processing methods are provided. This specification also involves two data processing devices, a data processing system, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0057] See also Figure 1 , Figure 1A scenario diagram of a data processing method provided according to an embodiment of the present specification is shown.

[0058] Specifically, the network card receives multiple message data from the network, and can distribute the multiple message data, send the multiple message data to different CPU cores, and after processing the multiple message data using different CPU cores, return the multiple message data to the network card. In the case where the order of the multiple message data needs to be preserved, the processed multiple message data needs to be sorted in the order of receipt, thereby achieving the order preservation of the multiple message data.

[0059] In the case where multiple message data are reordered in the cache area, since the cache capacity of the network card is limited by hardware resources, the high latency introduced by CPU processing is likely to cause packet loss. Therefore, in the embodiments of this specification, the order preservation method of multiple message data is changed from sorting the message data in the cache area to sorting the descriptors of the message data in the cache area, and based on the sequential descriptors, the message data of the corresponding memory address is read, so as to ensure the order of obtaining the message data and realize the order preservation of the message data.

[0060] Specifically, the data processing method provided in the embodiments of this specification is applied to a data transmission device, which can be understood as a DMA engine responsible for the interaction between a network card and a CPU; the data transmission device responds to a data transmission notification from a data processing device (i.e., the CPU) to obtain a target data descriptor of target message data; when it is determined according to the target data descriptor that the target message data needs to be kept in order, the target data descriptor is placed in a target buffer for sorting to obtain a sorting result; based on the sorting result, the target message data is obtained from a target memory address corresponding to the target data descriptor.

[0061] An embodiment of the present specification provides a data processing method, which is applied to a data transmission device. In response to a data sending notification from a data processing device, the data transmission device obtains a target data descriptor of target message data. Based on the target data descriptor, it can be determined whether the target message data needs to be sequenced. If sequence preservation is required, the target data descriptor is placed in a target buffer for sorting, and then the target message data is obtained from a target memory address corresponding to the target data descriptor using the sequential sorting result, that is, the message sequence preservation method is changed from data granularity reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the target message data size, more target data descriptors can be accommodated in the target buffer, which effectively reduces the demand for hardware resource usage while accommodating the sequence preservation capability of larger CPU processing delays, thereby ensuring message sequence preservation and performance requirements under CPU high-latency processing conditions.

[0062] See also Figure 2 , Figure 2 A flow chart of a data processing method applied to a data transmission device provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0063] Step 202: In response to the data sending notification of the data processing device, obtain the target data descriptor of the target message data.

[0064] The data transmission device can be understood as a data transmission channel between a data processing device and a data transceiver device, such as a data transmission device being a DMA engine. The data processing device is a device for processing target message data, such as a CPU. The data transceiver device is a device for receiving and sending target message data, such as a network card.

[0065] The target data descriptor is metadata used to represent or point to the target message data, including but not limited to the memory address of the target message data, message length and other information.

[0066] Specifically, since the target message data needs to be processed by the protocol stack, data decoding and decryption, and context switching, the target message data needs to be sent to a data processing device for processing. After the data processing device completes processing the target message data, the data processing device sends a data sending notification to the data transmission device, so that the data transceiver device obtains the processed target message data through the data transmission device.

[0067] The data transmission device responds to the data sending notification sent by the data processing device, and needs to determine the target data descriptor of the target message data through the data sending notification, thereby obtaining the target message data based on the memory address in the target data descriptor, and further sending the target message data to the data transceiver device.

[0068] Specifically, the data transmission notification carries data transmission information, and the data transmission information includes the queue position identifier of the target data descriptor in the target queue, and the target queue is the queue corresponding to the data processing device and where the target data descriptor is located; the target data descriptor corresponding to the target message data and the position in the target queue are determined by the data transmission information, so as to quickly obtain the target data descriptor corresponding to the target message data. The specific implementation method is as follows:

[0069] The step of obtaining the target data descriptor of the target message data in response to the data sending notification of the data processing device comprises:

[0070] In response to a data transmission notification of a data processing device, determining data transmission information carried in the data transmission notification, wherein the data transmission information includes a queue position identifier of a target data descriptor in a target queue, and the target queue is a queue corresponding to the data processing device and where the target data descriptor is located;

[0071] According to the queue position identifier of the target data descriptor in the target queue, the target data descriptor of the target message data is obtained from the target position of the target queue corresponding to the queue position identifier.

[0072] Specifically, the target queue includes target data descriptors of multiple target message data. Therefore, the queue position identifier of the target data descriptor in the target queue needs to be carried in the data sending notification. Thus, according to the queue position identifier of the target data descriptor in the target queue, the corresponding target data descriptor is obtained from the target position corresponding to the queue position identifier. The target data descriptor is the descriptor of the target message data.

[0073] In the case where the target data descriptor includes but is not limited to information such as whether the target message data needs to be kept in order, the message length of the target message data, the memory address of the target message data, etc., after obtaining the target data descriptor of the target message data, the target message data can be correctly read and when it is determined that the target message data needs to be kept in order, the target data descriptor is cached to the target buffer, and multiple data descriptors of the target buffer are sorted according to the order preservation information in the target data descriptor.

[0074] The data processing method provided in the embodiment of this specification can accurately obtain the target data descriptor of the target message data from the target position of the target queue corresponding to the queue position identifier in the data sending notification, thereby providing data support for the subsequent sorting of the target data descriptors.

[0075] Step 204: When it is determined according to the target data descriptor that the target message data needs to be sequenced, the target data descriptor is placed in a target buffer for sequencing to obtain a sequencing result.

[0076] Among them, since the target data descriptor includes an order preservation flag indicating whether the target message data needs to be sequence preserved, whether the target message data needs to be sequence preserved can be determined by the order preservation flag in the target data descriptor.

[0077] The target buffer is the cache area in the above embodiment, which is used to temporarily store the acquired target data descriptors. When it is determined according to the target data descriptors that the target message data needs to be kept in order, the corresponding target data descriptors are cached in the target buffer, thereby sorting the target data descriptors in the target buffer, obtaining the sorting results of the target data descriptors, and determining the reading order of the target data descriptors.

[0078] In one or more embodiments of the present specification, there is an order preservation flag in the target data descriptor, and the flag in the order preservation flag is used to indicate whether the target message data needs to be ordered. The specific implementation is as follows:

[0079] The step of determining, according to the target data descriptor, that the target message data needs to be sequenced includes:

[0080] An order preservation flag in the target data descriptor is checked, and when the order preservation flag is a target flag, it is determined that the target message data needs to be ordered.

[0081] Specifically, when the data transceiver device receives the target message data, it can select an order preservation mode for it, and the order preservation mode includes two modes: one that requires order preservation and one that does not require order preservation. When the data transceiver device sends the target message data to the data transmission device, it will carry the order preservation mode information of the target message data. When the data transmission device determines that the target message data needs to be preserved based on the order preservation mode information of the target message data, it will update the initial data descriptor corresponding to the target message data, set the order preservation flag in the initial data descriptor to 1, and use the order preservation engine to assign an order preservation information to the target message data, and also update the initial data descriptor with the order preservation information to obtain the target data descriptor of the target message data.

[0082] Among them, the target identifier can be understood as 1. Therefore, after obtaining the target data descriptor corresponding to the target message data, the order preservation identifier in the target data descriptor can be checked. When the order preservation identifier is the target identifier 1, it is determined that the target message data needs to be ordered.

[0083] The data processing method provided in the embodiments of the present specification sets an order preservation flag in the target data descriptor and sets the order preservation flag to different flags according to the order preservation mode information of the target message data. Therefore, when determining whether the target message data needs to be ordered, by checking the order preservation flag of the target data descriptor corresponding to the target message data, it can be determined efficiently and quickly whether the target message data needs to be ordered according to the flag in the order preservation flag.

[0084] In one or more embodiments of the present specification, the target data descriptor contains sequence preservation information of the target message data, and the sequence preservation information is used to characterize the receiving order of the target message data. Therefore, the target message data can be sorted according to the sequence preservation information, thereby ensuring that the sorted target message data can be consistent with the order when it is received. The specific implementation is as follows:

[0085] In the case where the data transmission notification includes multiple data transmission notifications and the target data descriptors include multiple data descriptors, in the case where it is determined according to the multiple target data descriptors that the corresponding target message data needs to be sequenced, the multiple target data descriptors are placed in the target buffer;

[0086] The multiple target data descriptors of the target buffer are sorted according to the order preservation information in the multiple target data descriptors to obtain the sorting result.

[0087] The sequence preservation information may be understood as information that can identify or indicate the order in which target message data is received.

[0088] Specifically, the target data descriptors corresponding to multiple target message data are checked to determine whether the corresponding target message data need to maintain the receiving order according to the order preservation identifier in the target data descriptor; for example, when the order preservation identifier in the target data descriptor is 1, it is determined that the target message data corresponding to the target data descriptor needs to maintain the order, and when the order preservation identifier in the target data descriptor is 0, it is determined that the target message data corresponding to the target data descriptor does not need to maintain the order.

[0089] Once it is determined that the target message data needs to be kept in order (the order-keeping flag in the target data descriptor is 1), the target data descriptors corresponding to these target message data are placed in the data structure of the target buffer. The target buffer can be understood as a temporary storage area for storing these target data descriptors before sorting.

[0090] These target data descriptors are sorted using the order preservation information in the target data descriptors, where the order preservation information can be a serial number, timestamp or other mark that can uniquely identify the receiving order. The sorting algorithm will compare and sort multiple target data descriptors based on the order preservation information to ensure that the order of the sorted target data descriptors is consistent with the original receiving order.

[0091] In practical applications, the sequence preservation information can be understood as a sequence number. Every time the data transmission device receives a target message data sent by a data transceiver device, the sequence number increases by one.

[0092] The data processing method provided in the embodiments of the present specification can quickly sort multiple target data descriptors through the order preservation information in the target data descriptors, and ensure that the arrangement order of the multiple target data descriptors in the sorting result is consistent with the order when they are received.

[0093] Step 206: According to the sorting result, the target message data is obtained from the target memory address corresponding to the target data descriptor.

[0094] Specifically, according to the sorting result of the target data descriptors in the target buffer, the reading order of the target data descriptors can be determined, so that the corresponding target message data can be obtained from the target memory address corresponding to the target data descriptors through the reading order of the target data descriptors.

[0095] That is, by sorting the target data descriptors in the target buffer, the order of obtaining the target message data from the target memory address is guaranteed according to the reading order of the target data descriptors, and the order preservation method is changed from reordering the message data to reordering based on the target data descriptors. While accommodating the order preservation capability of larger CPU processing delays, the demand for hardware resources is effectively reduced.

[0096] In one or more embodiments of the present specification, the sorting result is the order of the target data descriptor and other target data descriptors in the target buffer determined according to the order preservation information in the target data descriptor; specifically, when there are other target data descriptors previously stored in the target buffer and the target data descriptor is placed in the target buffer, the order of the target data descriptor and other target data descriptors is determined. The specific implementation is as follows:

[0097] The step of acquiring the target message data from a target memory address corresponding to the target data descriptor according to the sorting result includes:

[0098] The target message data is obtained from the target memory address corresponding to the target data descriptor according to the arrangement order of the target data descriptor and other target data descriptors in the target buffer.

[0099] Specifically, before the target data descriptor is placed in the target buffer, there may be other target data descriptors of other target message data in the target buffer, and the order preservation identifier of these other target data descriptors is 1, and includes corresponding order preservation information. Therefore, when the target data descriptor is placed in the target buffer, the target data descriptor and other target data descriptors in the target buffer are sorted according to the order preservation information, and the arrangement order of the target data descriptor and other target data descriptors in the target buffer is determined, so that the data transmission device sequentially reads the descriptors (including the target data descriptor and other target data descriptors) in the target buffer based on the arrangement order, and obtains the message data at the memory address corresponding to the descriptor.

[0100] The data processing method provided in the embodiment of this specification obtains the arrangement order between the descriptors by sorting the descriptors in the target buffer, and then realizes the order preservation of the message data by reading the sequential descriptors and obtaining the message data corresponding to the descriptors based on the arrangement order.

[0101] In one or more embodiments of the present specification, the data sending notification includes multiple, the target message data includes multiple, and the multiple target message data correspond to multiple target data descriptors; therefore, when it is determined that the target message data corresponding to the multiple target data descriptors need to be sequenced, the multiple target data descriptors are cached in the target buffer for sorting, thereby obtaining the arrangement order of the multiple target data descriptors. The specific implementation is as follows:

[0102] The acquiring the target data descriptor of the target message data according to the data transmission information carried in the data sending notification includes:

[0103] According to the data transmission information carried in the multiple data sending notifications, target data descriptors of multiple target message data are obtained.

[0104] When it is determined according to the target data descriptor that the target message data needs to be sequenced, the target data descriptor is placed in a target buffer for sequencing to obtain a sequencing result, including:

[0105] When it is determined based on multiple target data descriptors that the corresponding target message data needs to be kept in order, the multiple target data descriptors are placed in a target buffer for sorting to obtain a sorting result, wherein the sorting result is an arrangement order of the multiple target data descriptors determined based on the order preservation information in the multiple target data descriptors.

[0106] Among them, multiple target data descriptors can come from the same queue or from different queues, and there is no restriction here.

[0107] Specifically, in the scenario where the CPU processes the elephant flow, the network card can distribute the message data in the elephant flow to different CPU cores for processing. For each CPU core, a certain queue will be allocated to it, so it can receive multiple data sending notifications sent by the CPU. Multiple data sending notifications can be issued by different queues of one CPU core, or by different queues of different CPU cores, which is not limited here.

[0108] According to the data transmission information carried in the multiple data sending notifications, the queue position identifier of the target data descriptor corresponding to each target message data can be determined, so as to obtain the target data descriptors of multiple target message data from the corresponding positions according to the queue position identifier of each target data descriptor.

[0109] When it is determined that the target message data needs to be sequenced according to the sequence-preserving identifiers in the multiple target data descriptors, the multiple target data descriptors need to be sorted.

[0110] Specifically, the target data descriptors that need to be kept in order are cached, for example, cached to a target buffer of a data transmission device, and the target data descriptors in the target buffer are sorted to obtain an arrangement order of the multiple target data descriptors.

[0111] During specific implementation, the Tx descriptor engine in the data transmission device (DMA engine) is used to obtain the target data descriptor of the target message data; the Tx descriptor sorting engine in the DMA engine can sort the target data descriptors in the target buffer, and when a preset time interval is reached, according to the sorting result, send multiple sequential target data descriptors to the Tx DMA engine that reads the message data, and the Tx DMA engine reads the target message data of the corresponding memory address according to the memory address information in the target data descriptor.

[0112] Of course, in actual applications, multiple target data descriptors in the target buffer can be sorted, and when a preset time interval is reached, the sequential multiple target data descriptors can be sent to the Tx DMA engine responsible for data transfer; or when a preset time interval is reached, multiple target data descriptors can be sorted again and the sequential multiple target data descriptors can be sent out, and there is no limitation here.

[0113] The data processing method provided in the embodiments of this specification can greatly reduce the resource occupation when the cache is limited by hardware resources by caching multiple target data descriptors that need to be ordered, thereby accommodating more target data descriptors that need to be ordered and achieving order preservation of target message data.

[0114] In one or more embodiments of the present specification, by arranging the order of multiple target data descriptors, the target data descriptors are read, and the target data descriptors correspond to the target message data of the target memory address, thereby ensuring the order of the target message data. The specific implementation is as follows:

[0115] The step of acquiring the target message data from a target memory address corresponding to the target data descriptor according to the sorting result includes:

[0116] According to the arrangement order of the multiple target data descriptors, the multiple target message data are acquired from the target memory addresses corresponding to the multiple target data descriptors.

[0117] Specifically, by obtaining the arrangement order of multiple target data descriptors, the order of preserving the corresponding target message data can be determined according to the arrangement order of the multiple target data descriptors, that is, by sequentially reading the target data descriptors, the corresponding target message data can be sequentially obtained from the target memory address corresponding to the target data descriptors.

[0118] For example, target message data A corresponds to target data descriptor a, target message data B corresponds to target data descriptor b, and target message data C corresponds to target data descriptor c. When the receiving order of target message data A, B, and C is 1, 2, and 3, the order preservation information in target data descriptor a can be 1, the order preservation information in target data descriptor b can be 2, and the order preservation information in target data descriptor c can be 3. At this time, after CPU processing, the order of obtaining the target data descriptors after processing can be target data descriptor b-target data descriptor c-target data descriptor a. According to the order preservation information in the target data descriptors, target data descriptors b, target data descriptor c, and target data descriptor a are sorted. According to the order preservation information 1 corresponding to target data descriptor a, the order preservation information 2 corresponding to target data descriptor b, and the order preservation information 3 corresponding to target data descriptor c, the arrangement order of target data descriptors b, c, and a is determined to be target data descriptor a-target data descriptor b-target data descriptor c.

[0119] According to the arrangement order of target data descriptor a-target data descriptor b-target data descriptor c, the target data descriptor a can be read first, and the target message data A can be obtained from the memory address corresponding to the target data descriptor a, and then the target data descriptor b can be read, and the target message data B can be obtained from the memory address corresponding to the target data descriptor b, and finally the target data descriptor c can be read, and the target message data C can be obtained from the memory address corresponding to the target data descriptor c; that is, by sequentially reading the arranged multiple target data descriptors, the corresponding target message data can be obtained from the target memory address corresponding to the target data descriptor, thereby realizing the order preservation of the target message data.

[0120] In one or more embodiments of the present specification, when it is determined based on the target data descriptor that the target message data does not need to be sequenced, there is no need to cache or sort the target data descriptors, that is, the target message data at the corresponding memory location can be directly read based on the target data descriptors.

[0121] After the step of responding to the data transmission notification of the data processing device and acquiring the target data descriptor of the target message data according to the data transmission information carried in the data transmission notification, the step further includes:

[0122] When it is determined according to the target data descriptor that the target message data does not need to be sequence-preserved, the target message data is acquired from a target memory address corresponding to the target data descriptor.

[0123] An embodiment of the present specification provides a data processing method, which is applied to a data transmission device. In response to a data sending notification from a data processing device, the data transmission device obtains a target data descriptor of target message data according to the data transmission information carried in the data sending notification. According to the target data descriptor, it can be determined whether the target message data needs to be sequenced. If sequence preservation is required, the target data descriptor is placed in a target buffer for sorting, and then the target message data is obtained from a target memory address corresponding to the target data descriptor using the sequential sorting result, that is, the message sequence preservation method is changed from data granularity reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the target message data size, more target data descriptors can be accommodated in the target buffer, which effectively reduces the demand for hardware resource usage while accommodating the sequence preservation capability of larger CPU processing delays, thereby ensuring message sequence preservation and performance requirements under CPU high-latency processing conditions.

[0124] See also Figure 3 , Figure 3 It is a flowchart of another data processing method applied to a data transmission device provided by an embodiment of the present specification; it specifically includes the following steps.

[0125] Step 302: In response to target message data sent by a data transceiver device, according to a target queue corresponding to the target message data, an initial data descriptor corresponding to the target message data is determined from the target queue.

[0126] The data transceiver device assigns a queue ID (identifier) ​​to the target message data. Therefore, the data transmission device can determine the corresponding target queue according to the queue ID in response to the target message data sent by the data transceiver device, and assign an initial data descriptor to the target message data from the target queue. The initial data descriptor includes a memory address specified in advance by the CPU for the target queue.

[0127] Step 304: When it is determined that the target message data needs to maintain order, the initial data descriptor is updated according to the target message data to obtain a target data descriptor.

[0128] As mentioned in the above embodiment, the data transceiver device will also select an order preservation mode for the target message data. When it is determined that the target message data needs to be ordered, the order preservation flag in the initial data descriptor is set to 1, and the order preservation information is requested from the order preservation engine, thereby adding the order preservation information to the initial data descriptor, updating the initial data descriptor, and obtaining the target data descriptor.

[0129] Of course, in actual applications, information such as the message length of the target message data will be added to the initial data descriptor; therefore, when the target message data does not need to be sequenced, the initial data descriptor will also be updated, but the update content does not include the sequence information.

[0130] Step 306: According to the target data descriptor, the target message data is sent to the target memory address.

[0131] The interaction between the network card and the CPU is achieved through memory. The target message data will be sent to the memory according to the target memory address corresponding to the target data descriptor.

[0132] Step 308: Update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

[0133] The initial data descriptor in the target queue is updated to the target data descriptor, and the CPU is made aware of the message being sent through interrupt sending / CPU polling, and the CPU starts to process the message.

[0134] An embodiment of the present specification provides a data processing method, which is applied to a data transmission device, and is used to update an initial data descriptor for the target message data when receiving target message data sent by a data transceiver device, so as to obtain a target data descriptor including information such as an order preservation identifier, order preservation information, and message length; when the target data descriptor obtained by updating is convenient for subsequent sending of the target message data, the order of the target message data is ensured according to the order preservation information in the target data descriptor.

[0135] See also Figure 4a , Figure 4a A processing flow diagram of a data processing method provided by an embodiment of this specification is shown;

[0136] Among them, message reception and mode selection is responsible for selecting the DMA queue for receiving messages and selecting the message order preservation mode (such as whether order preservation is required).

[0137] The order-keeping engine is responsible for allocating order-keeping information in the message uplink (Rx) direction and retrieving order-keeping information in the message downlink (Tx) direction. The Rx descriptor acquisition and Rx DMA engine are responsible for the execution of the message uplink direction; the Tx descriptor engine, Tx descriptor sorting and Tx DMA engine are responsible for the interaction between the software and hardware of the message downlink.

[0138] The network card and the CPU communicate data via the PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) interface.

[0139] Among them, the Rx descriptor is the Receive Descriptor, which is used to instruct the network card to send the received data message to the specified location, size and other information in the host side memory; the Tx descriptor is the Transmit Descriptor, which is used to indicate the address, length and other information of the hardware network card data message in the host side memory.

[0140] Figure 4a The whole process of receiving the message (black line) and sending the message (blue line) is included in the figure. Figure 4b-4e The description is described in detail.

[0141] See also Figure 4b , Figure 4b It is a processing flow diagram of a packet receiving direction and a data processing method provided by an embodiment of this specification.

[0142] The network card (i.e., the data transceiver device) receives the data message from the network and performs project logic processing, where the project logic processing includes but is not limited to speed limit, statistics, mirroring, and partial unloading processing, which are not limited here; a queue ID is assigned to the data message and an order preservation mode is selected (whether order preservation is required); the data message is sent to the DMA engine (i.e., the data transmission device). When the DMA engine receives the data message, an Rx descriptor of the queue corresponding to the queue ID is assigned to the data message (the Rx descriptor at this time can be understood as the initial data descriptor in the above embodiment), and the Rx descriptor includes information such as the memory address allocated by the CPU to its queue in advance.

[0143] Of course, in the embodiment of the present specification, the Rx descriptor also includes an order preservation identifier and order preservation information; specifically, the order preservation identifier in the Rx descriptor is different depending on the order preservation mode selected by the data message. For example, for a data message that requires order preservation, the order preservation identifier in its Rx descriptor is 1, and the corresponding order preservation information is added in the Rx descriptor. The order preservation information is obtained from the order preservation engine (the order preservation information is generally an ID number, which is +1 for each message sent on the hardware side); for a data message that does not require order preservation, the order preservation identifier in its Rx descriptor is 0.

[0144] The Rx DMA engine in the DMA engine sends the data message to the corresponding memory address in its Rx descriptor, and updates the Rx descriptor information and fills it back into the queue. The descriptor update mainly informs the CPU of the length of the current message and other information, so that the CPU can know the length of the message. The CPU is aware of the message sending through interrupt sending / CPU polling, and the CPU (i.e., data processing device) starts to process the data message.

[0145] In actual applications, queue-based DMA hardware and software (CPU and network card) interact with each other based on the Ring mechanism; see Figure 4c , Figure 4c A schematic diagram of the structure of a Ring provided in one embodiment of this specification is shown.

[0146] In actual applications, the Ring is a continuous memory address requested by the CPU side. The software and hardware are producers and consumers respectively. For example, when the CPU needs to send data, it updates the relevant information of the data to be sent to the descriptor at the Producer pointer in the Ring, and then notifies the hardware that the Producer pointer has changed and there is a new task to be processed. After receiving the software notification, the hardware side goes to the corresponding location to obtain the descriptor, and reads and writes the data at the corresponding location according to the information filled in the descriptor.

[0147] See also Figure 4d, Figure 4d A schematic diagram of a processing flow of message processing in a data processing method provided by an embodiment of this specification is shown.

[0148] When the CPU core receives a notification from the hardware that a new message has been sent, it reads the Rx descriptor information at the corresponding position from its corresponding Ring, and obtains the address, length, whether order preservation is required, order preservation information, and other contents of the data message based on the Rx descriptor information.

[0149] After the CPU processes the network message, it sends the data message. At this time, the CPU calls the DMA Tx direction driver to fill the content of the data message into the Tx descriptor in the downstream direction (wherein, because the memory address of the data message may change after CPU processing, the contents in the Tx descriptor and the Rx descriptor may be different, but whether the data message needs to be kept in order and the order preservation information remain unchanged, so the order preservation identifier and order preservation information in the Tx descriptor and the Rx descriptor are the same), that is, fill the message information, whether the order preservation is needed, the order preservation information and other contents into the Tx descriptor of the corresponding Ring ring, and notify the hardware network card that there is a new data sending task that needs to be transmitted.

[0150] See also Figure 4e , Figure 4e It is a processing flow diagram of a packet sending direction and a data processing method provided by an embodiment of this specification.

[0151] When the hardware network card receives the packet notification request sent by the CPU, the corresponding data message is obtained through the DMA engine; specifically, the Tx descriptor engine in the DMA engine obtains the Tx descriptor from the corresponding position of the Ring ring, and checks the order preservation flag in the Tx descriptor. When the order preservation flag in the Tx descriptor is 1 (that is, the data message corresponding to the descriptor needs to be ordered), the Tx descriptor is cached; and when the preset time interval is reached, the cached Tx descriptors from different queues are sorted according to the order preservation information to be recycled provided by the order preservation engine (the order preservation information to be recycled is the order preservation information that needs to be sorted and sent out for recycling, and the corresponding Tx descriptor can be determined by the order preservation information to be recycled), and the Tx descriptors sorted according to the order preservation information are sent to the Tx DMA engine in sequence; and when the order preservation flag in the Tx descriptor is 0 (that is, the data message corresponding to the descriptor does not need to be ordered), the Tx descriptor is directly sent to the Tx DMA engine.

[0152] The Tx DMA engine reads data packets from the memory address in the Tx descriptor according to the order of the received Tx descriptors, performs DMA data transfer, and ensures the order of the data packets; the read data packets will be processed using the project processing logic, and after processing, the data packets will be sent out through the network port.

[0153] The embodiments of this specification provide a data processing method. Since the existing order preservation function is implemented based on the caching and sorting of the message data itself, due to limited hardware resources and the fact that network messages are often long, the number of message caches that can be implemented is small, and thus it is impossible to achieve order preservation while ensuring line speed in high CPU latency scenarios.

[0154] The embodiments of this specification provide a data processing method for caching and sorting messages based on DMA multi-queue descriptors. The storage space occupied by the descriptors is fixed, only a dozen bytes. Regardless of the length of the message, the storage space consumed is fixed. Compared with message-based caching and sorting, it can achieve tens to hundreds of times higher CPU high-latency sorting capabilities, thereby withstanding line-speed millisecond-level CPU delays. The longer the message, the greater the benefit.

[0155] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 5 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0156] A descriptor acquisition module 502 is configured to acquire a target data descriptor of target message data in response to a data transmission notification of a data processing device;

[0157] The descriptor sorting module 504 is configured to put the target data descriptor into the target buffer for sorting to obtain a sorting result when it is determined according to the target data descriptor that the target message data needs to be kept in order;

[0158] The message acquisition module 506 is configured to acquire the target message data from the target memory address corresponding to the target data descriptor according to the sorting result.

[0159] Optionally, the descriptor acquisition module 502 is further configured to:

[0160] In response to a data transmission notification of a data processing device, determining data transmission information carried in the data transmission notification, wherein the data transmission information includes a queue position identifier of a target data descriptor in a target queue, and the target queue is a queue corresponding to the data processing device and where the target data descriptor is located;

[0161] According to the queue position identifier of the target data descriptor in the target queue, the target data descriptor of the target message data is obtained from the target position of the target queue corresponding to the queue position identifier.

[0162] Optionally, the descriptor sorting module 504 is further configured to:

[0163] An order preservation flag in the target data descriptor is checked, and when the order preservation flag is a target flag, it is determined that the target message data needs to be ordered.

[0164] Optionally, the message acquisition module 506 is further configured to:

[0165] The target message data is obtained from the target memory address corresponding to the target data descriptor according to the arrangement order of the target data descriptor and other target data descriptors in the target buffer.

[0166] Optionally, the descriptor acquisition module 502 is further configured to:

[0167] According to the data transmission information carried in the multiple data sending notifications, target data descriptors of multiple target message data are obtained.

[0168] Optionally, the descriptor sorting module 504 is further configured to:

[0169] When it is determined based on multiple target data descriptors that the corresponding target message data needs to be kept in order, the multiple target data descriptors are placed in a target buffer for sorting to obtain a sorting result, wherein the sorting result is an arrangement order of the multiple target data descriptors determined based on the order preservation information in the multiple target data descriptors.

[0170] Optionally, the message acquisition module 506 is further configured to:

[0171] According to the arrangement order of the multiple target data descriptors, the multiple target message data are acquired from the target memory addresses corresponding to the multiple target data descriptors.

[0172] The device further comprises:

[0173] The data acquisition module is configured to acquire the target message data from the target memory address corresponding to the target data descriptor when it is determined according to the target data descriptor that the target message data does not need to be sequenced.

[0174] An embodiment of the present specification provides a data processing device, which is applied to a data transmission device. In response to a data sending notification from the data processing device, the data transmission device obtains a target data descriptor of the target message data according to the data transmission information carried in the data sending notification. According to the target data descriptor, it can be determined whether the target message data needs to be sequenced. If sequence preservation is required, the target data descriptor is placed in a target buffer for sorting, and then the target message data is obtained from a target memory address corresponding to the target data descriptor using the sequential sorting result, that is, the message sequence preservation method is changed from data granularity reordering to target data descriptor-based reordering. Since the size of the target data descriptor is much smaller than the target message data size, more target data descriptors can be accommodated in the target buffer, which effectively reduces the demand for hardware resources while accommodating the sequence preservation capability of larger CPU processing delays, thereby ensuring message sequence preservation and performance requirements under CPU high-latency processing conditions.

[0175] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0176] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 6 FIG. 2 shows a schematic diagram of the structure of another data processing device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0177] The descriptor acquisition module 602 is configured to respond to the target message data sent by the data transceiver device and determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data;

[0178] The descriptor updating module 604 is configured to update the initial data descriptor according to the target message data to obtain a target data descriptor when it is determined that the target message data needs to be sequenced;

[0179] The message sending module 606 is configured to send the target message data to the target memory address according to the target data descriptor;

[0180] The event notification module 608 is configured to update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue.

[0181] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0182] Corresponding to the above method embodiment, this specification also provides a data processing system embodiment. Figure 7 FIG. 1 shows a schematic diagram of a data processing system provided by an embodiment of the present specification. Figure 7 As shown, the system includes a data transceiver device 702, a data transmission device 704, and a data processing device 706, wherein:

[0183] The data transceiver device 702 is used to receive target message data, allocate a target queue for the target message data, and select an order-preserving mode for the target message data;

[0184] The data transmission device 704 is used to respond to the target message data sent by the data transceiver device 702, determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data, and when it is determined that the target message data needs to be sequenced, update the initial data descriptor according to the target message data to obtain the target data descriptor, send the target message data to the target memory address according to the target data descriptor, update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device 706 corresponding to the target queue;

[0185] The data processing device 706 is used to obtain the target data descriptor of the target message data from the target queue according to the data transmission information, obtain the target message data from the target memory address corresponding to the target data descriptor, process the target message data, and send a data sending notification to the data transmission device after the processing is completed;

[0186] The data transmission device 704 is also used to respond to the data sending notification of the data processing device 706, obtain the target data descriptor of the target message data, and when it is determined according to the target data descriptor that the target message data needs to be kept in order, put the target data descriptor into the target buffer for sorting to obtain the sorting result, and according to the sorting result, obtain the target message data from the target memory address corresponding to the target data descriptor.

[0187] The above is a schematic scheme of a data processing system of this embodiment. It should be noted that the technical scheme of the data processing system and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the data processing system can be referred to the description of the technical scheme of the above data processing method.

[0188] Figure 8 The block diagram of a computing device 800 according to an embodiment of the present specification is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0189] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0190] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0191] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 may also be a mobile or stationary server.

[0192] The processor 820 is used to execute the following computer program / instructions, which implement the steps of the above-mentioned data processing method when executed by the processor.

[0193] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the data processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the data processing method embodiment.

[0194] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned data processing method when executed by a processor.

[0195] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the data processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the data processing method embodiment.

[0196] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above data processing method when executed by a processor.

[0197] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above data processing method.

[0198] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0199] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0200] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0201] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0202] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied to a data transmission device, comprising: In response to a data transmission notification of the data processing device, obtaining a target data descriptor of the target message data; When it is determined according to the target data descriptor that the target message data needs to be sequenced, the target data descriptor is placed in a target buffer for sequencing to obtain a sequencing result; According to the sorting result, the target message data is obtained from the target memory address corresponding to the target data descriptor, Among them, the data sending notification includes multiple, the target message data includes multiple, the multiple target message data correspond to multiple target data descriptors, the target data descriptors are obtained from the target queue corresponding to the data processing device, and when the multiple data sending notifications come from different target queues, the multiple target data descriptors are determined from different target queues.

2. The data processing method according to claim 1, wherein the step of obtaining the target data descriptor of the target message data in response to the data transmission notification of the data processing device comprises: In response to a data transmission notification of a data processing device, determining data transmission information carried in the data transmission notification, wherein the data transmission information includes a queue position identifier of a target data descriptor in a target queue, and the target queue is a queue corresponding to the data processing device and where the target data descriptor is located; According to the queue position identifier of the target data descriptor in the target queue, the target data descriptor of the target message data is obtained from the target position of the target queue corresponding to the queue position identifier.

3. The data processing method according to claim 1, wherein determining that the target message data needs to be sequenced according to the target data descriptor comprises: An order preservation flag in the target data descriptor is checked, and when the order preservation flag is a target flag, it is determined that the target message data needs to be ordered.

4. The data processing method according to any one of claims 1 to 3, wherein the sorting result is determined according to the order preservation information in the target data descriptor, and the sorting result represents the arrangement order of the target data descriptor and other target data descriptors in the target buffer; The step of acquiring the target message data from a target memory address corresponding to the target data descriptor according to the sorting result includes: The target message data is obtained from the target memory address corresponding to the target data descriptor according to the arrangement order of the target data descriptor and other target data descriptors in the target buffer.

5. The data processing method according to any one of claims 1 to 3, wherein the data sending notification comprises a plurality of data, the target message data comprises a plurality of data, and the plurality of target message data corresponds to a plurality of target data descriptors; When it is determined according to the target data descriptor that the target message data needs to be sequenced, the target data descriptor is placed in a target buffer for sequencing to obtain a sequencing result, including: When it is determined based on multiple target data descriptors that the corresponding target message data needs to be kept in order, the multiple target data descriptors are placed in a target buffer for sorting to obtain a sorting result, wherein the sorting result is an arrangement order of the multiple target data descriptors determined based on the order preservation information in the multiple target data descriptors.

6. The data processing method according to claim 5, wherein obtaining the target message data from the target memory address corresponding to the target data descriptor according to the sorting result comprises: According to the arrangement order of the multiple target data descriptors, the multiple target message data are obtained from the target memory addresses corresponding to the multiple target data descriptors.

7. The data processing method according to any one of claims 1 to 3 or 6, wherein after obtaining the target data descriptor of the target message data according to the data transmission information carried in the data transmission notification in response to the data transmission notification of the data processing device, the method further comprises: When it is determined according to the target data descriptor that the target message data does not need to be sequence-preserved, the target message data is acquired from a target memory address corresponding to the target data descriptor.

8. A data processing method, applied to a data transmission device, comprising: In response to target message data sent by a data transceiver device, according to a target queue corresponding to the target message data, determining an initial data descriptor corresponding to the target message data from the target queue; When it is determined that the target message data needs to be sequenced, the initial data descriptor is updated according to the target message data to obtain a target data descriptor; According to the target data descriptor, the target message data is sent to the target memory address; Update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue, so that the data processing device obtains the target data descriptor of the target message data from the target queue according to the data transmission information, obtains the target message data from the target memory address corresponding to the target data descriptor, processes the target message data, and sends a data sending notification to the data transmission device after the processing is completed, and uses any one of the data processing methods of claims 1-7 to obtain the target message data processed by the data processing device.

9. A data transmission system, comprising a data transceiver device, a data transmission device, and a data processing device, wherein: The data transceiver device is used to receive target message data, allocate a target queue for the target message data, and select an order-preserving mode for the target message data; The data transmission device is used to respond to the target message data sent by the data transceiver device, determine the initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data, and when it is determined that the target message data needs to be sequenced, update the initial data descriptor according to the target message data to obtain the target data descriptor, send the target message data to the target memory address according to the target data descriptor, update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue; The data processing device is used to obtain the target data descriptor of the target message data from the target queue according to the data transmission information, obtain the target message data from the target memory address corresponding to the target data descriptor, process the target message data, and send a data sending notification to the data transmission device after the processing is completed; The data transmission device is further used to respond to the data transmission notification of the data processing device, obtain the target data descriptor of the target message data, and when it is determined according to the target data descriptor that the target message data needs to be kept in order, put the target data descriptor into the target buffer for sorting to obtain the sorting result, and obtain the target message data from the target memory address corresponding to the target data descriptor according to the sorting result. Among them, the data sending notification includes multiple, the target message data includes multiple, the multiple target message data correspond to multiple target data descriptors, the target data descriptors are obtained from the target queue corresponding to the data processing device, and when the multiple data sending notifications come from different target queues, the multiple target data descriptors are determined from different target queues.

10. A data processing device, applied to a data transmission device, comprising: a descriptor acquisition module, configured to acquire a target data descriptor of target message data in response to a data transmission notification of a data processing device; A descriptor sorting module is configured to put the target data descriptor into a target buffer for sorting to obtain a sorting result when it is determined according to the target data descriptor that the target message data needs to be kept in order; a message acquisition module, configured to acquire the target message data from the target memory address corresponding to the target data descriptor according to the sorting result, Among them, the data sending notification includes multiple, the target message data includes multiple, the multiple target message data correspond to multiple target data descriptors, the target data descriptors are obtained from the target queue corresponding to the data processing device, and when the multiple data sending notifications come from different target queues, the multiple target data descriptors are determined from different target queues.

11. A data processing device, applied to a data transmission device, comprising: a descriptor acquisition module configured to respond to target message data sent by a data transceiver device and determine an initial data descriptor corresponding to the target message data from the target queue according to the target queue corresponding to the target message data; A descriptor updating module is configured to update the initial data descriptor according to the target message data to obtain a target data descriptor when it is determined that the target message data needs to be sequenced; a message sending module, configured to send the target message data to a target memory address according to the target data descriptor; An event notification module is configured to update the initial data descriptor in the target queue to the target data descriptor, and send data transmission information for the target message data to the data processing device corresponding to the target queue, so that the data processing device obtains the target data descriptor of the target message data from the target queue according to the data transmission information, obtains the target message data from the target memory address corresponding to the target data descriptor, processes the target message data, and sends a data sending notification to the data transmission device after the processing is completed, and uses any one of the data processing methods of claims 1-7 to obtain the target message data processed by the data processing device.

12. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the data processing method according to any one of claims 1 to 8 are implemented.

13. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 8.

14. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 8.

Images