Data processing methods and electronic devices

Abstract

This application discloses a data processing method and electronic device, relating to the field of computer technology. The method includes: determining whether the data to be processed is urgent data, the urgent data including data whose processing latency is less than a preset latency requirement threshold; when the data to be processed is urgent data, writing the data to be processed into the large page memory corresponding to a pre-configured first queue; writing the address offset and data length of the data to be processed in the large page memory into the network interface card (NIC), and triggering the NIC to read the data to be processed from the large page memory and process it according to the address offset and data length. This application can solve the technical problems of low data processing efficiency of NICs and the inability to prioritize the processing of urgent data in related technologies.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to a data processing method and an electronic device. Background Technology

[0002] In related technologies, the data processing flow of a network interface card (NIC) relies on the collaborative work of multiple queues, including a send queue, a send completion queue, a receive queue, a receive completion queue, and an event queue.

[0003] Taking the data transmission process as an example, the transmission of each data packet requires multiple read and write operations to the host memory. These operations consume the host memory access bandwidth and reduce the overall data processing efficiency. In addition, all data packets are queued equally in the queue, making it impossible to prioritize urgent data, which results in the inability to meet the needs of certain special services. Summary of the Invention

[0004] This application provides a data processing method and an electronic device that can solve the technical problems of low data processing efficiency of network cards and inability to prioritize urgent data in related technologies.

[0005] This application provides a data processing method, the method comprising:

[0006] Determine whether the data to be processed is urgent data, which includes data whose processing latency is less than a preset latency requirement threshold; if the data to be processed is urgent data, write the data to be processed into the large page memory corresponding to the pre-configured first queue; write the address offset and data length of the data to be processed in the large page memory into the network card, and trigger the network card to read the data to be processed from the large page memory and process it according to the address offset and data length.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the above-described data processing method.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described data processing method.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described data processing method.

[0010] The data processing method and electronic device provided in this application divide the network card queue resources into an urgent queue and a normal queue. By combining large page memory and dedicated registers, the network card is exempted from reading and writing the queue descriptor multiple times when sending a single data packet. This not only prioritizes the processing of urgent data but also reduces the number of accesses to the host memory, thereby improving the data processing efficiency of the network card while meeting the needs of special services. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of an architecture provided in this application embodiment, in which a processor and a network card are connected and transmit data via a high-speed serial computer expansion (Peripheral Component Interconnect-express, PCIe) bus;

[0013] Figure 2 This is a schematic diagram of a queue and operation process used in a data transmission process provided in an embodiment of this application;

[0014] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of another architecture provided in this application embodiment for connecting and transmitting data between a processor and a network card via a PCIe bus;

[0016] Figure 5 This is a schematic diagram of the queue and operation process used in another data transmission process provided in this application embodiment;

[0017] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] In this embodiment of the application, "multiple" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0021] Network interface cards (NICs), as devices requiring high-speed data processing, are typically connected to the processor via the PCIe bus, such as... Figure 1 As shown, Figure 1 This is a schematic diagram of an architecture in which a processor and a network card are connected and transmit data via a PCIe bus, as provided in an embodiment of this application.

[0022] Optionally, the processor 101 internally includes:

[0023] Processor core: The core computing unit of the processor, responsible for executing various instructions and operations.

[0024] Local bus: Used for communication between the processor core and other components. Software access interacts with Double Data Rate Synchronous Dynamic Random Access Memory (DDR) controllers, etc., through the local bus.

[0025] PCIe Root Complex (RC): As the root complex of the PCIe bus, it is the hub connecting the processor and PCIe devices (which can be network cards). It is responsible for managing PCIe bus transactions, including initiating and responding to hardware accesses.

[0026] DDR controller: Responsible for managing the processor's access to host memory 103 (such as DDR) and coordinating data transfer between the processor core and memory.

[0027] Optionally, network card 102 internally includes:

[0028] PCIe IP: This is the PCIe interface module on the network card, used to communicate with the PCIe RC of the processor 101 to realize high-speed data transmission between the network card 102 and the processor 101.

[0029] Network data processing logic: Responsible for processing received and sent network data, including protocol parsing, data encapsulation / decapsulation, and other operations.

[0030] DDR: Dynamic Random Access Memory inside the network card, used to temporarily store network data, possibly for buffering received or pending data packets.

[0031] Direct Memory Access (DMA): Allows the network card to directly access host memory without relying on the processor, improving data transfer efficiency and reducing the processor's workload.

[0032] Media Access Control (MAC): Responsible for controlling access to network media, handling data frame structure, address identification, and other functions, and interacting with the descriptor queue through the descriptor queue address register.

[0033] Descriptor queue address register: Stores the address information of the descriptor queue, indicating the location of the descriptor queue in memory.

[0034] The processor 101 communicates with the network card 102 via the PCIe bus. The PCIe RC and PCIe IP serve as interfaces on the processor side and the network card side, respectively, to enable hardware access and data transmission.

[0035] After processing network data through network data processing logic, network card 102 can directly access host memory via DMA to store data in a specified data cache area, and use descriptor queues to manage the data transmission and processing status.

[0036] The host memory 103 (DDR) contains a descriptor queue and a data cache. The descriptor queue stores a descriptor for data transmission, such as the data's address, length, and status, helping the network card and processor coordinate data transmission and processing.

[0037] The data buffer area is used to store the actual network data. The network card can directly write the received data into this area or read the data to be sent from this area via DMA.

[0038] In some implementations, when sending data via the network interface card (NIC), the program running on the processor first writes the data to the host memory and then notifies the NIC (via the PCIe bus and internal bus) to read it. Receiving data is the reverse: the NIC first stores the data in the host memory and then notifies the processor to read it. This method of direct access to host memory by the NIC device (without going through the processor) is called DMA (Direct Memory Access). DMA operations can be further divided into two types based on the data flow: one where data is copied from host memory to the device; and the other where data is copied from the device to system memory.

[0039] A queue is a form of data organization for transferring data between software and hardware. It is a special type of linear list, unique in that it only allows insertion operations at one end and deletion operations at the other. The end where insertions are performed is called the head, and the end where deletions are performed is called the tail. A queue is called an empty queue when it contains no elements.

[0040] The data elements in a queue are also called queue elements. Inserting a queue element into a queue is called enqueueing, and deleting a queue element from a queue is called dequeueing. Because a queue only allows insertion at one end and deletion at the other end, only the element that entered the queue earliest can be deleted from the queue first. Therefore, a queue is also called a first-in-first-out (FIFO) linear list.

[0041] In the field of data transmission and reception, circular queues are generally used, employing a producer-consumer pattern. Taking a network interface card (NIC) as an example, the sending queue used for data transmission is as follows: Software (driver) is the producer of data in the queue, and hardware is the consumer. Objects (or queue elements) in the queue are called descriptors. Each descriptor stores information such as the physical address and length of a data buffer (located in host memory, corresponding to a data packet). Descriptors are primarily filled by the producer, but sometimes the consumer modifies the descriptor content as feedback. Both the producer and consumer have their own head and tail pointers for accessing descriptors in the queue.

[0042] Taking the data transmission process as an example (data reception is similar), before sending data through the network card, the software (running on the processor core in the processor) first allocates a segment of physical memory with contiguous physical addresses in the host memory to store the descriptor queue, and writes the address of this memory segment into the descriptor queue address register in the network card (see...). Figure 1 ).

[0043] The descriptor queue contains many descriptors. For the same type of network interface card (NIC), each descriptor has the same data format. For different NICs, the data format of the descriptors may be different, but they will all contain at least two pieces of information: the address of the data buffer where the data to be sent is located and the length of the data.

[0044] When sending data through the network card, the software (which could be the network card driver) first requests a contiguous data buffer with a single physical address, and then writes the data to be sent into the buffer. Next, the software selects a descriptor (it can start from the first descriptor and select sequentially; let's assume it's the Nth descriptor in the descriptor queue), and writes the address of the data buffer and the data length into the descriptor. Then, the software writes the number N to the doorbell register (also known as the head register of the send queue) provided by the network card, notifying the network card to start processing the Nth descriptor.

[0045] The network card first calculates the address of the Nth descriptor by adding "N times the length of each descriptor" to the base address of the descriptor queue. Then, it reads the contents of the descriptor from the host memory via the PCIe bus to obtain the address of the data cache and the data length. After that, the network card reads the data from the data cache again via the PCIe bus into its own internal cache or the DDR on the hardware board (when the data volume is large and the internal cache is insufficient), and then sends the data to the network.

[0046] The following embodiments illustrate the implementation scheme of network card using queues for data transmission. It is understood that the data reception process can adopt a similar principle, which will not be elaborated in the embodiments of this application.

[0047] Before sending and receiving data, queues must be created, including a send queue, a send-complete queue, a receive queue, a receive-complete queue, and an event queue (from the application and operating system's perspective, these different types of queues actually belong to the same internal implementation of a single queue. The expedited queue mentioned later is also explained from the application and operating system's perspective). The number of each type of queue, as well as the number of elements (descriptors) within each queue, varies, depending on the hardware capabilities and software configuration. However, the send queue and the send-complete queue must be in a one-to-one correspondence, meaning that the number of these two queues and the number of descriptors within each queue are the same. Similarly, the receive queue and the receive-complete queue are also in a one-to-one correspondence.

[0048] The queues used in data sending and receiving are different; refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram illustrating the queue and operation flow used in a data transmission process provided in an embodiment of this application. Figure 2 The middle arrow indicates the direction in which primary data or commands are transmitted.

[0049] In some implementations, the data transmission process described above is as follows.

[0050] S1. The kernel network protocol stack calls the data sending function registered during driver initialization.

[0051] S2, the data sending function populates the "send queue" (fills its descriptor with the address, length, etc. of the data packet) and updates the head of the "send queue" in the software count.

[0052] S3. The data sending function writes the new head of the "send queue" to the head register of the "send queue" provided by the network card. The network card reads the new descriptor in the "send queue" to obtain the address and length of the data to be sent.

[0053] S4. After the network card finishes sending the data packet, it fills the "send completed queue" (mainly by filling its descriptor with the length of the data packet sent and the descriptor index in the "send queue" for processing), and updates the head of the "send completed queue" counted by the network card.

[0054] S5. The network card continues to fill the "event queue" (mainly by filling its descriptor with the completed event types (send completed) and the index of the filled "send completed queue"), updates the head of the network card's counted "event queue", and then initiates an interrupt.

[0055] S6. The processor receives an interrupt and calls the interrupt handler function registered by the driver. The interrupt handler function reads the head register of the "event queue" to obtain the head of the "event queue" from the network card. If it finds that the head of the "event queue" is not equal to the tail of the "event queue" recorded by the software, it indicates that a new event has occurred. Then, it reads the descriptor from the "event queue" to obtain the new event. After judgment, it can be determined that this new event is a transmit completion event. The "event queue" descriptor also contains the index, i.e., the number, of the "transmit completion queue" corresponding to this event.

[0056] S7. The interrupt handler calls the "data transmission complete" handler.

[0057] S8. The data transmission completion processing function reads the head register of the "transmission completion queue". If the new head is found to be different from the tail of the "transmission completion queue" it has recorded, it indicates that there is a new descriptor in the "transmission completion queue" (meaning that data has been sent by the network card). Then, it reads the descriptors in the "transmission completion queue" to obtain the index of the descriptor in the "transmission queue" processed by the network card, and releases the data buffer it points to. Finally, it updates the tail count of the "transmission completion queue" in the software and writes to the tail register to inform the network card that processing is complete.

[0058] Currently, data center networks are constantly evolving towards higher bandwidths, such as 100Gbps, 200Gbps, and even higher. As the amount of data transmitted per unit time increases, higher demands are placed on the efficiency of network interface cards (NICs) in retrieving data from host memory.

[0059] In related technologies, each time a network interface card (NIC) sends a data packet, it involves multiple operations: retrieving descriptors from host memory, filling descriptors, and retrieving data packets. These operations consume host memory access bandwidth, i.e., occupy the PCIe bus and the processor's internal bus, reducing overall data transmission efficiency. Furthermore, host memory is shared by multiple processor cores; excessive memory access bandwidth usage by one processor core or device can negatively impact the performance of other cores (the programs running on them).

[0060] Furthermore, for applications, different services have varying degrees of urgency; some service data packets need to be sent urgently, while others can wait a little longer. However, in related technologies, all data packets need to be queued equally, which cannot meet the needs of certain special business operations.

[0061] To address the aforementioned technical issues, this application provides a data processing method that divides network interface card (NIC) queue resources into expedited queues and regular queues. By combining large page memory, dedicated registers, and a new tagging mechanism, the method reduces the frequency of host memory access, prioritizes expedited data processing, thereby improving data transmission efficiency and meeting differentiated service requirements.

[0062] To better understand the present application, the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0063] This application's embodiments are applicable to high-bandwidth, low-latency scenarios such as data centers, cloud computing, and real-time video transmission. In a typical network architecture, the host connects to the network interface card (NIC) via a PCIe bus, and the NIC is responsible for data transmission and reception. This application prioritizes high-priority data packets (such as real-time communication and emergency alarms) through an expedited queuing mechanism, and reduces memory access overhead by using large page memory and dedicated registers to reduce descriptor operations. This makes it better suited for business scenarios that are sensitive to latency and have high throughput requirements.

[0064] See Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. In some embodiments, the data processing method includes:

[0065] S301. Determine if the data to be processed is urgent; if so, proceed to S302.

[0066] Optionally, the data to be processed can be data to be sent or data to be received; this application embodiment does not impose any restrictions.

[0067] Optionally, the aforementioned expedited data includes data whose processing latency is less than a preset latency requirement threshold.

[0068] Processing latency refers to the time it takes for data to travel from entering the system or network node to completing the corresponding processing operation. When this time is less than a pre-set threshold, the data can be identified as expedited data.

[0069] For example, in video conferencing, participants' voice and video data need to be transmitted and processed in a timely manner to ensure the continuity of the conversation and the synchronization of the visuals. Therefore, voice and video data in video conferencing can be treated as urgent data.

[0070] Optionally, the latency requirement threshold can be determined comprehensively based on various factors such as the specific application scenario, business needs, and system performance. For example, in a real-time voice interaction system, to ensure the fluency and real-time nature of the voice, the latency requirement threshold can be set to 50 milliseconds, and data with a latency lower than this threshold should be processed as expedited data.

[0071] In some implementations, it can be determined whether the data to be processed carries an expedited flag; when the data to be processed carries an expedited flag, the data to be processed is determined to be expedited data; when the data to be processed does not carry an expedited flag, the data to be processed is determined to be non-expedited data or ordinary data.

[0072] S302. Write the data to be processed into the large page memory corresponding to the pre-configured first queue.

[0073] The first queue is a pre-configured expedited queue used to process urgent data.

[0074] For example, network interface card (NIC) queue resources can be divided into expedited queues and regular queues. For instance, the first 10 queues (0-9) of the NIC can be designated as expedited queues, and the rest as regular queues.

[0075] The expedited queue is used to prioritize the processing of high-priority data packets and is isolated from the resources of the regular queue.

[0076] In some implementations, a large page of memory can be allocated in system memory for each first queue to store data that needs to be processed urgently.

[0077] In some implementations, the address of the aforementioned large page memory can be written to the network card.

[0078] In some implementations, when sending data that requires expedited processing, the data can be written to the aforementioned large page memory first.

[0079] In some implementations, if the data to be processed is not urgent, it can be written to a regular queue, for example, using... Figure 2 The processing flow shown in this embodiment will not be repeated here.

[0080] S303. Write the address offset and data length of the data to be processed in the big page memory into the network card, and trigger the network card to read the data to be processed from the big page memory and process it according to the address offset and data length.

[0081] In some implementations, upon receiving a trigger signal, the network interface card (NIC) can prioritize allocating DMA engine and MAC module resources, directly reading data packets from large page memory and continuing processing. During this process, urgent data packets do not require maintaining a descriptor queue, thereby reducing the number of accesses to host memory.

[0082] The data processing method provided in this application divides the network card queue resources into an urgent queue and a normal queue. By combining large page memory and dedicated registers, it eliminates the need for the network card to read and write the queue descriptor multiple times when sending a single data packet. This not only prioritizes the processing of urgent data but also reduces the number of accesses to host memory, thereby improving the data processing efficiency of the network card while meeting the needs of special services.

[0083] In some embodiments, the above method further includes:

[0084] Configure a first queue (or expedited queue) and a second queue (or normal queue) for the network card. The first queue is used to process expedited data, and the second queue is used to process non-expedited data. Select a portion of the storage resources in the host memory to be designated as the large page memory corresponding to the first queue.

[0085] In some implementations, the network interface card's (NIC) processing queue resources can be divided into two parts. For example, the first 10 queues (0 to 9) are dedicated to processing urgent data, while the remaining queues are used to process regular data. This division allows urgent data to bypass the queuing process for regular data and directly access processing resources.

[0086] In some implementations, a large page can be allocated in system memory for each expedited queue to store data that needs to be processed urgently.

[0087] In some implementations, an expedited data base address register can be configured in the network interface card (NIC) for the first queue. After selecting a portion of the storage resources in the host memory to determine the large page memory corresponding to the first queue, the address of the large page memory is written into the expedited data base address register.

[0088] In the above implementation, by writing the allocated big page address into the corresponding expedited data base address register, it can be ensured that the network card can correctly access expedited data.

[0089] In some implementations, an expedited processing trigger register can be configured in the network interface card for the first queue.

[0090] In some implementations, the address offset and data length of the data to be processed in the large page memory can be written into the expedited processing trigger register, and the network card can be triggered to read the address of the large page memory stored in the expedited data base address register, as well as the address offset and data length stored in the expedited processing trigger register, to read the data to be processed from the large page memory and process it.

[0091] For example, the software can write urgent data packets into large page memory and send a trigger signal (containing address offset and length) to the network card via the urgent processing trigger register. When the network card receives the trigger signal, it can prioritize processing the urgent data, read the data from the large page memory via DMA, and send it.

[0092] In the above implementation, by writing the address offset and data length of the data to be processed in the big page memory to the expedited processing trigger register, the network card can be triggered to read the data to be processed from the big page memory and process it according to the address offset and data length of the data to be processed in the big page memory.

[0093] In some implementations, the network interface card (NIC) is provided with an expedited processing completion count register; the expedited processing completion count register is used to count the pending data that the NIC has completed processing.

[0094] In some embodiments, the above method further includes:

[0095] Based on the count value of the expedited processing completion counter register and the amount of pending data written to the large page memory, determine whether the large page memory storage resources are sufficient; when the large page memory storage resources are insufficient, update the large page memory according to the pending data processed by the network card.

[0096] In some implementations, if the amount of a single urgent data request is relatively small, the software can continuously write multiple urgent data requests to the network card, i.e., write the urgent processing trigger register multiple times consecutively. In this case, the network card needs an internal buffer to temporarily store this content, and delete it from the buffer after processing is complete. The number of these temporarily unprocessed urgent data requests is reflected in the urgent processing completion count register. The number of unprocessed requests in the buffer is equal to the number of times the software has written to the urgent processing trigger register minus the value of the urgent processing completion count register.

[0097] In some embodiments, the above method further includes:

[0098] Send the identification information of the first queue to the network card. This identification information is used to instruct the network card to use the first queue to process the data to be processed.

[0099] For example, when an application calls a data sending function, it specifies the expedited queue number by marking it with identification information and writes the data to the large page memory. When urgent data arrives, the expedited queue has priority over other queues in occupying shared resources (such as the DMA engine, MAC module, etc.).

[0100] The data processing method provided in this application not only prioritizes urgent data processing and supports certain special services (eliminating the cost of building a dedicated network for these services), but also avoids multiple read / write operations of the network card to the queue descriptor, thus accelerating data processing speed.

[0101] For example, refer to Figure 4 , Figure 4 This is a schematic diagram of another architecture provided in this application embodiment for connecting and transmitting data between a processor and a network card via a PCIe bus.

[0102] exist Figure 4 In the host memory 103, there is a large page memory, and the address of the large page memory is written into the expedited data base address register.

[0103] Urgent data 0 to urgent data N can be written into the aforementioned large page memory. The address offset and data length of urgent data 0 to urgent data N in the large page memory are written into the urgent processing trigger register.

[0104] The priority judgment and preemption module can be used to determine whether the data to be processed is urgent data. If so, the urgent queue can preempt public resources (such as DMA, MAC, etc.).

[0105] The expedited processing completion count register is used to count the pending data that has been processed by the network card.

[0106] For example, in some embodiments, the big page function can be enabled in the system startup configuration file; the directory corresponding to the big page can be created by running a script.

[0107] In some implementations, large page memory can be divided into multiple parts (e.g., 100 parts, the specific number of which can be the same as the number of caches for the expedited processing trigger register in the network card). The application can record the number of parts used (by continuously accumulating). When the number of data parts that have not yet been processed (which can be calculated by subtracting the number of times the expedited processing trigger register has been written from the number of times it has been processed) exceeds 100, the memory is updated based on the number of data packets that have been processed by the network card to determine which memory can be reused.

[0108] In some implementations, when sending urgent data by calling the data sending function (send function), the urgent data can be marked with an urgent flag, the function of which is explained later.

[0109] Additionally, the application needs to specify the expedited queue number (0 to 9) of the network card it wants to use. Since the maximum length of expedited data is 1GB (31 bits), and the third parameter of the data transmission function is a 64-bit variable in a 64-bit system, the queue number can be placed in the high 32 bits of the third parameter and passed to the operating system.

[0110] In some implementations, a new flag is added to the fourth parameter of the data sending function to mark the data to be sent as urgent data.

[0111] Reference Figure 5 , Figure 5 This is a schematic diagram of the queue and operation process used in another data transmission process provided in this application embodiment.

[0112] In some implementations, the urgency flag can be used to determine whether the data packet to be sent is a regular data packet or an urgency data packet. If it is a regular data packet, steps S1 to S8 can be executed; see the details below. Figure 1 The embodiment shown; if it is an urgent data packet, the urgent queue specified by the urgent queue number is used to send the data, that is, the address of the urgent processing trigger register corresponding to this queue is obtained according to the urgent queue number.

[0113] The system can count and update the number of data packets being processed in real time. When the number of data packets being processed is less than or equal to the maximum number of expedited data packets that can be processed, S9 is executed. When the number of data packets being processed is greater than the maximum number of expedited data packets that can be processed, the number of data packets being processed is updated based on the data to be processed that has been processed by the network card.

[0114] S9 includes writing the address offset and data length of the expedited data packet in the large page memory into the expedited processing trigger register.

[0115] For example, refer to Table 1, which is an example table of definitions for the expedited processing trigger register.

[0116] Table 1: Example of Definitions for the Urgent Processing Trigger Register

[0117]

[0118] In some implementations, a network card cache can be added to the expedited processing trigger register to temporarily store relevant information before data transmission is complete.

[0119] In some implementations, urgent data can be processed immediately using the corresponding expedited queue resources when software writes to the expedited processing trigger register are detected.

[0120] In some implementations, data can be moved from system memory to the network card via DMA based on the address offset in the expedited processing trigger register (plus the address of the large page memory written in the expedited data base address register) and the data length, and then processed for subsequent transmission.

[0121] In some implementations, the expedited processing completion counter register is incremented by 1 whenever an expedited data transmission is completed.

[0122] The data processing method provided in this application has the following beneficial effects:

[0123] (1) It can not only prioritize the processing of urgent data, but also avoid the network card from repeatedly reading and writing the queue descriptor, thus speeding up the data processing speed.

[0124] (2) By prioritizing the processing of urgent data, certain special services can be supported, eliminating the cost of building a dedicated network for these services.

[0125] (3) Special optimizations were made on the basis of the existing queue processing mechanism to avoid the network card from repeatedly reading and writing the queue descriptor, thus speeding up the data processing speed.

[0126] (4) The architecture is simple and highly feasible.

[0127] This application embodiment also provides a data processing apparatus, the apparatus comprising:

[0128] The judgment module is used to determine whether the data to be processed is urgent data, which includes data whose processing latency is less than a preset latency requirement threshold.

[0129] The write module is used to write the data to be processed into the large page memory corresponding to the pre-configured first queue when the data to be processed is urgent data.

[0130] The trigger module is used to write the address offset and data length of the data to be processed in the big page memory into the network card, and trigger the network card to read the data to be processed from the big page memory and process it according to the address offset and data length.

[0131] In some embodiments, the above-described apparatus further includes a configuration module for:

[0132] Configure a first queue and a second queue for the network card. The first queue is used to process urgent data, and the second queue is used to process non-urgent data. The first queue has a higher priority than the second queue.

[0133] In some embodiments, the configuration module is further configured to:

[0134] Select a portion of the storage resources in the host memory and determine it as the large page memory corresponding to the first queue.

[0135] In some embodiments, the network interface card is provided with an expedited data base address register; the writing module is used for:

[0136] After selecting a portion of the storage resources in the host memory and determining the large page memory corresponding to the first queue, the address of the large page memory is written into the expedited data base address register.

[0137] In some embodiments, the network interface card is provided with an expedited processing trigger register; the writing module is used for:

[0138] Write the address offset and data length of the data to be processed in the large page memory into the expedited processing trigger register, and trigger the network card to read the address of the large page memory stored in the expedited data base address register, as well as the address offset and data length stored in the expedited processing trigger register, and read the data to be processed from the large page memory and process it.

[0139] In some embodiments, the network interface card (NIC) is provided with an expedited processing completion count register; the expedited processing completion count register is used to count the pending data that the NIC has completed processing.

[0140] In some embodiments, the above-mentioned determination module is further configured to:

[0141] Based on the count value of the expedited processing completion counter register and the amount of pending data written to the large page memory, determine whether the large page memory storage resources are sufficient; when the large page memory storage resources are insufficient, update the large page memory according to the amount of pending data processed by the network card.

[0142] In some embodiments, the triggering module is further configured to:

[0143] Send the identification information of the first queue to the network card. This identification information is used to instruct the network card to use the first queue to process the data to be processed.

[0144] In some embodiments, the above-mentioned determination module is used for:

[0145] Determine whether the data to be processed contains an urgent flag; if the data to be processed contains an urgent flag, determine that the data to be processed is urgent data.

[0146] This application also provides an electronic device, exemplarily, in this embodiment. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the electronic device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus.

[0147] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to execute the embodiment corresponding to the above data processing method.

[0148] The specific implementation process of processor 601 can be found in the embodiments corresponding to the above data processing method. The implementation principle and technical effect are similar, and will not be repeated here.

[0149] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the data processing method disclosed in the application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0150] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0151] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0152] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above data processing method embodiments when it is run.

[0153] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), RAM, portable hard drives, magnetic disks, or optical disks.

[0154] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above data processing method embodiments.

[0155] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0156] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the technical solutions and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data processing method, characterized in that, The method includes: Determine whether the data to be processed is urgent data, wherein urgent data includes data whose processing latency is less than a preset latency requirement threshold; When the data to be processed is urgent data, the data to be processed is written into the large page memory corresponding to the pre-configured first queue, which is used to process urgent data. The address offset and data length of the data to be processed in the large page memory are written into the expedited processing trigger register of the network card, and the network card is triggered to use the DMA engine and MAC module resources to read the address of the large page memory stored in the expedited data base address register of the network card, and to read the address offset and data length stored in the expedited processing trigger register, and to read the data to be processed from the large page memory and process it; wherein, the first queue occupies the DMA engine and MAC module resources first compared with other queues; The method further includes: When the data to be processed consists of multiple urgent data items, the address offset and data length of the multiple urgent data items in the large page memory are written into the urgent processing trigger register, and the network card is triggered to read the address offset and data length written in the urgent processing trigger register. According to the address of the large page memory and the address offset and data length, the multiple urgent data items are read from the large page memory and stored in a preset cache. The number of urgent data items that the network card has not yet processed in the cache is equal to the number of times the urgent processing trigger register is written minus the count value of the network card's urgent processing completion counter register.

2. The method according to claim 1, characterized in that, The method further includes: Configure the network card with a first queue and a second queue, the second queue being used to process non-urgent data; the first queue has a higher priority than the second queue.

3. The method according to claim 2, characterized in that, The method further includes: A portion of the storage resources in the host memory is selected and determined as the large page memory corresponding to the first queue.

4. The method according to claim 3, characterized in that, The network interface card is equipped with an expedited data base address register; the method further includes: After selecting a portion of the storage resources in the host memory and determining it as the large page memory corresponding to the first queue, the address of the large page memory is written into the expedited data base address register.

5. The method according to any one of claims 1 to 4, characterized in that, The network card is equipped with an expedited processing completion count register; the expedited processing completion count register is used to count the pending data that has been processed by the network card.

6. The method according to claim 5, characterized in that, The method further includes: Based on the count value of the expedited processing completion counter register and the amount of data to be processed written to the large page memory, determine whether the storage resources of the large page memory are sufficient. When the storage resources of the large page memory are insufficient, the large page memory is updated according to the amount of pending data processed by the network card.

7. The method according to claim 1, characterized in that, The method further includes: The identification information of the first queue is sent to the network card, and the identification information is used to instruct the network card to process the data to be processed using the first queue.

8. The method according to claim 1, characterized in that, Determining whether the data to be processed is urgent data includes: Determine whether the data to be processed contains an expedited flag; When the data to be processed carries an urgent flag, the data to be processed is determined to be urgent data.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data processing method as described in any one of claims 1 to 8.

Images