Data processing

Abstract

Provided is a data processing method, which is applied to a server in which an application process and a network service process are deployed, wherein the server comprises data processing hardware which comprises a queue management module and an input / output (IO) acceleration module. The method comprises: by means of a first process, storing, in the queue management module, description information of data to be transmitted, so as to send the description information to a second process by means of the queue management module, wherein the first process is the application process and the second process is the network service process, or the first process is the network service process and the second process is the application process; and sending the description information to the IO acceleration module by means of the second process, such that the IO acceleration module copies said data from a virtual address space of the first process to a virtual address space of the second process on the basis of the description information.

Description

Data processing technology field

[0001] The present application relates to the field of data storage, and more specifically, to data processing.

[0002] With the surge in network traffic and the rapid development of network cards, the network in the data center has undergone multiple upgrades and iterations, from 1Gb, 10Gb, 25Gb to the current 100Gb. The traditional kernel-mode network can no longer meet the requirements of high performance, and the network in the data center has begun to evolve to the user-mode network. User-mode network protocol stacks such as Seastar, TLDK, and VPP, as well as low-latency, high-throughput RDMA networks, have begun to be widely deployed in data centers.

[0003] There are usually two implementation modes for user-mode network systems. One is the run-to-completion (RTC) mode, that is, the user-mode network system directly accesses the upper-layer application in the libOS mode. In this mode, the network needs to be strongly coupled with the application, and the logic of the network and the application will affect each other, which has poor flexibility. The other is the mode in which the user-mode network system is an independent service. In this mode, the network and the application are decoupled and flexible enough, but the interaction cost between the network service and the upper-layer application is increased, and the performance overhead is increased. Therefore, there is an urgent need for a mode that takes into account both flexibility and high performance. Contents of the invention

[0004] The present application provides a data processing method, an apparatus, an electronic device, and a storage medium to implement a user-mode network system that combines flexibility and high performance.

[0005] In a first aspect, the present application provides a data processing method, which is applied to a server, wherein an application process and a network service process are deployed in the server, and the server includes data processing hardware, and the data processing hardware includes a queue management module and an input and output 10 acceleration module. The method includes: storing description information of data to be transmitted in the queue management module through a first process, so as to send the description information to a second process through the queue management module; wherein the first process is the application process, and the second process is the network service process; or, the first process is the network service process, and the second process is the application process; sending the description information to the 10 acceleration module through the second process, so that the 10 acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information.

[0006] In one implementation, the method further includes: the first process generates the to-be-transmitted The method further comprises: generating, by the acceleration module 10, first verification information of the data to be transmitted in the virtual address space of the second process; and feeding back to the second process a verification pass if the first verification information is consistent with the second verification information.

[0007] In one implementation, the server includes one or more application processes, and for each application process, the queue management module includes a first queue and a second queue corresponding to the application process, wherein the first queue is used to store description information of data to be transmitted from the application process to be transmitted to the network service process, and the second queue is used to store description information of data to be transmitted from the network service process to be transmitted to the application process.

[0008] In one implementation, the first queue and the second queue are both multi-producer, multi-consumer queues, the first queue is used to store description information of data to be transmitted from multiple threads of the application process to be transmitted to multiple threads of the network service process, and the second queue is used to store description information of data to be transmitted from multiple threads of the network service process to be transmitted to multiple threads of the application process.

[0009] In one implementation, the description information includes a virtual address and a length of the data to be transmitted.

[0010] In one implementation, the second process sends the description information to the 10 acceleration module, including: the second process determines the size of the data to be transmitted according to the description information; if the size of the data to be transmitted exceeds a preset threshold, sending the description information to the 10 acceleration module.

[0011] In one implementation, the method further includes: if the size of the data to be transmitted does not exceed a preset threshold, the second process copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process through the CPU of the server.

[0012] In one implementation, the description information includes the priority of the data to be transmitted. When the description information of multiple data to be transmitted is included in the queue management module, the queue management module preferentially sends the description information of the data to be transmitted with a high priority to the second process.

[0013] In a second aspect, an embodiment of the present application provides a data processing device, which is applied to a server. An application process and a network service process are deployed in the server. The server includes a queue management module and an input / output (I / O) acceleration module. The device includes: a first processing module, configured to store the description information of the data to be transmitted into the queue management module through a first process, so as to send the description information to a second process through the queue management module; where the first process is the application process, and the second process is the network service process; or the first process is the network service process, and the second process is the application process; a second processing module, configured to send the description information to the I / O acceleration module through the second process, so that the I / O acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information. In one implementation, it further includes: a third processing module, configured to: the first process generates first check information of the data to be transmitted through the I / O acceleration module, and adds the first check information to the description information; the I / O acceleration module generates second check information of the data to be transmitted in the virtual address space of the second process; if the first check information is consistent with the second check information, feedback that the check is passed to the second process.

[0014]

[0015] In one implementation, the server includes one or more application processes. For each application process, the queue management module includes a first queue and a second queue corresponding to the application process. The first queue is used to store the description information of the data to be transmitted from the application process to the network service process, and the second queue is used to store the description information of the data to be transmitted from the network service process to the application process.

[0016] ​In one implementation, both the first queue and the second queue are multi-producer and multi-consumer queues. The first queue is used to store the description information of the data to be transmitted from multiple threads of the application process to multiple threads of the network service process, and the second queue is used to store the description information of the data to be transmitted from multiple threads of the network service process to multiple threads of the application process.

[0017] In one implementation, the description information includes the virtual address and length of the data to be transmitted.

[0018] In one implementation, the second processing module is configured to: the second process determines the size of the data to be transmitted according to the description information; if the size of the data to be transmitted exceeds a preset threshold, the description information is sent to the 10 acceleration module.

[0019] In one implementation, the second processing module is further configured to: if the size of the data to be transmitted does not exceed the preset threshold, the second process copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process through the CPU of the server.

[0020] In one implementation, the description information includes the priority of the data to be transmitted. When there are description information of multiple data to be transmitted in the queue management module, the queue management module preferentially sends the description information of the data to be transmitted with a high priority to the second process.

[0021] In a third aspect, the present application provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, and the computer program when running causes the processor to execute the method described in the first aspect.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor is caused to execute the method described in the first aspect.

[0023] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0024] In the data processing method, apparatus, electronic device, and storage medium provided in this application, based on regarding the network system as an independent service, in combination with the queue management module and the I / O acceleration module of the hardware, through the queue management capability of the queue management module, using a low-latency hardware queue to replace the software queue, reducing the complexity of inter-process queue management, and reducing the enqueue and dequeue latency, offloading the CPU-intensive tasks in the user-space network service and upper-layer application data interaction to the I / O acceleration module for fast processing, saving the CPU resources of the upper-layer application and the network service while reducing the latency of data interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0026] FIG. 1 is a schematic diagram of a mode of a user-space network system I;

[0027] FIG. 2 is a schematic diagram of a mode of a user-space network system II;

[0028] FIG. 3 is a schematic diagram of the implementation of a user-space network system provided by an embodiment of this application;

[0029] FIG. 4 is a schematic flowchart of a data processing method provided by an embodiment of this application;

[0030] FIG. 5 is a schematic diagram of a queue management module implementing queue management provided by an embodiment of this application;

[0031] FIG. 6 is a schematic diagram of implementing data copy based on a queue management module and an I / O acceleration module provided by an embodiment of this application;

[0032] FIG. 7 is a schematic structural diagram of a data processing apparatus provided by an embodiment of this application;

[0033] FIG. 8 is a schematic block diagram of an electronic device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Apparently, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0035] First, the terms related to the embodiments of this application will be introduced.

[0036] l.DSA: Data Streaming Accelerator, a data stream accelerator, which is a type of I / O acceleration (I / O Acceleration Technology, I0AT) device and has the ability to accelerate CPU-intensive tasks such as memory copy and CRC operation.

[0037] 2.DLB: Dynamic Load Balancer, a hardware for managing queues, which has the ability of high-priority scheduling and traffic load balancing and realizes low-latency queue interaction.

[0038] 3.CRC: Cyclic Redundancy Check, a data error detection technology that can detect errors in the data transmission process and take corresponding measures to correct the errors. It achieves this purpose by calculating a value called the cyclic redundancy check code (CRC code).

[0039] Figure 1 shows the RTC mode, that is, a schematic diagram of the user-mode network system directly accessing the upper-layer application in the mode of libOS. The advantage of this method is that it can achieve low request latency and high throughput. The disadvantage is that the network needs to be strongly coupled with the application, and the logic of the network and the application will affect each other, and it is difficult to implement centralized traffic scheduling. Figure 2 shows the mode of the user-mode network system as an independent service. The txq and rxq between the application process and the user-mode network system are the send queue and the receive queue respectively. The advantage of this mode is that the independent service mode is decoupled from the application and is flexible enough to facilitate the implementation of functions such as elastic resource expansion and centralized traffic scheduling. The disadvantage is that it increases the interaction cost between the network service and the upper-layer application, resulting in an increase in latency and a decrease in the ultimate throughput capacity.

[0040] In view of this, the embodiments of the present application propose a new mode of realizing flexibility and high performance in a user-mode network system in a data center. On the basis of taking the network system as an independent service, the hardware queue management module and the 10 acceleration module are combined, and the queue management capability of the queue management module is used to replace the software queue with a low-latency hardware queue, thereby reducing the complexity of queue management between processes and reducing the delay of queue entry and exit. The CPU-intensive tasks in the data interaction between the user-mode network service and the upper-layer application are offloaded to the 10 acceleration module for rapid processing, thereby saving the CPU resources of the upper-layer application and the network service, and reducing the delay of data interaction.

[0041] 3, the equipment in the data center is deployed with a network service process and different application processes. For example, the application process may be a storage process. Each application process and network service process may include multiple threads. The network system provides network services to upper layer applications as an independent network service process. The application process exchanges data description information with the network service process through the queue management module DLB. The queue management module may include an entry logic module and an exit logic module. The queue management module shields the upper layer application and the network service process from queue implementation details, reduces the complexity of the software implementation of the application and the network service process, and transmits data description information through the hardware queue to achieve lower interaction delay.

[0042] On the basis of the description information of the data transmitted through the queue management module, the specific data is carried by the 10 acceleration hardware DSA. After the upper-layer application or network service process obtains the description information of the data transmitted by the other party through the queue management module, the memory copy acceleration capability provided by the 10 acceleration hardware is used to copy the other party's data to its own memory space, saving the CPU resources of the software copy. In addition, the 10 acceleration hardware provides the CRC generation and verification capability, which can be used to regularly verify the correctness of the data to ensure that the data is not lost or not correct. Through the mutual cooperation of the 10 acceleration hardware and the queue management module, low latency and high throughput can be achieved on the basis of the network system as an independent service, while taking into account flexibility and high performance.

[0043] Figure 4 is a schematic flowchart of a data processing method provided by an embodiment of the present application. This method is applied to a server, where an application process and a network service process are deployed in the server, and data processing hardware is in the server. The data processing hardware includes a queue management module and an input / output (I / O) acceleration module. Among them, the queue management module and the I / O acceleration module can be two functional modules in a data processing hardware. For example, the data processing hardware is a Cloud Infrastructure Processing Unit (CIPU) or a Data Processing Unit (DPU), etc. Or, the queue management module and the I / O acceleration module can be two independent hardwares.

[0044] As shown in Figure 4, this method includes steps S401 to S402.

[0045] S401. Store the description information of the data to be transmitted into the queue management module through the first process, so as to send the description information to the second process through the queue management module.

[0046] Among them, the first process is an application process, and the second process is a network service process; or, the first process is a network service process, and the second process is an application process.

[0047] S402. Send the description information to the I / O acceleration module through the second process, so that the I / O acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information.

[0048] The data to be transmitted is the data that the first process needs to transmit to the second process. The description information of the data to be transmitted may include information such as the virtual address (VA) and size of the data to be transmitted. The data to be transmitted that the first process needs to transmit to the second process is stored in the virtual address space of the first process, and the virtual address space can also be called the private memory space.

[0049] In the case where the first process is an application process and the second process is a network service process, that is, the application process sends data to the network service process. In the case where the first process is a network service process and the second process is an application process, that is, the network service process sends data to the application process. That is, the method of the embodiment of the present application can be applied to the mutual data transmission process between the application process and the network service process.

[0050] The first process sends the description information of the data to be transmitted to the second process through the queue management module, and the second process sends the description information to the 10 acceleration module. The 10 acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process. In this way, it realizes that the first process sends the data to be transmitted to the second process. The queue management module transfers the description information of the data to be transmitted between the two processes, and then the 10 acceleration module performs memory copying based on the description information, without relying on software queues and CPU memory copying, improving efficiency, reducing latency, and reducing CPU occupancy.

[0051] In the embodiments of the present application, the server includes one or more application processes. For each application process, the queue management module includes a first queue and a second queue corresponding to the application process. Among them, the first queue is used to store the description information of the data to be transmitted from the application process to the network service process, and the second queue is used to store the description information of the data to be transmitted from the network service process to the application process. That is to say, when the first process is an application process and the second process is a network service process, the first process stores the description information of the data to be transmitted in the first queue. When the first process is a network service process and the second process is an application process, the first process stores the description information of the data to be transmitted in the second queue.

[0052] Optionally, since the application process or the network service process may include multiple threads, both the first queue and the second queue are multi-producer and multi-consumer queues. The first queue is used to store the description information of the data to be transmitted from multiple threads of the application process to multiple threads of the network service process, and the second queue is used to store the description information of the data to be transmitted from multiple threads of the network service process to multiple threads of the application process.

[0053] Figure 5 is a schematic diagram of a queue management module implementing queue management provided by an embodiment of the present application. As shown in Figure 5, between each application process and network service process, two multi-producer (MP), multi-consumer (MC) queues are maintained through the queue management module. The first queue is used for the application process to send data to the network service process, and the second queue is used for the application process to receive data from the network service process. The application process and the network service process exchange description information (meta information) of the data to be transmitted, that is, the description information of the data to be transmitted. For example, the description information may include virtual address, length, CRC of the data, priority, and other information describing the data. Through a unified enqueue and dequeue module, the low latency of the description information transmission is ensured.

[0054] Optionally, the queue management module can perform scheduling according to the 10 priority in the description information to ensure that the 10 with high priority can dequeue first and avoid the head-of-line blocking problem. That is, the description information includes the priority of the data to be transmitted. When the queue management module includes description information of multiple data to be transmitted, the queue management module preferentially sends the description information of the data to be transmitted with high priority to the second process. For example, thread 1 of the application process first stores the description information of the data to be transmitted with low priority in the first queue of the queue management module, and thread 2 of the application process then stores the description information of the data to be transmitted with high priority in the first queue of the queue management module. In this way, the queue management module will preferentially send the description information of the data to be transmitted with high priority of thread 2 to the second process.

[0055] In addition, the queue management module has the scheduling ability of load balancing. For an application process that can synchronize the states between threads, it can break up the large elephant flow that cannot be efficiently processed by a single core from the network service process and distribute it to each thread of the application process, and use multi-core parallel acceleration processing to reduce the processing latency of the large elephant flow.

[0056] Optionally, to ensure data accuracy, in the embodiments of this application, a 10 acceleration module can be used to verify the data. Before the first process sends the data to be transmitted to the second process, the first process generates the first verification information of the data to be transmitted through the 10 acceleration module and adds the first verification information to the description information of the data to be transmitted. Then, after the 10 acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information, the 10 acceleration module generates the second verification information of the data to be transmitted in the virtual address space of the second process. If the first verification information and the second verification information are consistent, the 10 acceleration module feeds back that the verification is passed to the second process. In the case where the verification is passed, the second process can then perform subsequent processing on the data to be transmitted.

[0057] Optionally, in the method of the embodiments of this application, it is possible to select whether to use direct CPU copying or copying through the 10 acceleration module based on the size of the data to be transmitted. Optionally, the description information of the data to be transmitted includes the virtual address and length of the data to be transmitted. The second process determines the size of the data to be transmitted according to the description information. If the size of the data to be transmitted exceeds a preset threshold, the description information is sent to the 10 acceleration module so that the 10 acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information. If the size of the data to be transmitted does not exceed the preset threshold, the second process copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process through the CPU of the server.

[0058] That is to say, when the size of the data to be transmitted does not exceed the preset threshold, that is, is less than 10, it is still possible to choose to use the direct CPU copying method. At this time, the CPU occupancy is not high and the processing speed is also relatively fast. When the size of the data to be transmitted exceeds the preset threshold, that is, is greater than 10, the 10 acceleration module is selected for copying, thus avoiding the high CPU occupancy of large 10 for a long time and improving the data copying speed.

[0059] Figure 6 is a schematic diagram of data copying implemented based on a queue management module and a 10 acceleration module provided by the embodiments of this application, taking the queue management module as DLB, the 10 acceleration module as DSA, and an application process sending data to a network service process as an example.

[0060] The application process first generates the CRC code of the data to be transmitted through DSA, which is the first check information. The CRC code is filled into the description information of 10 and placed in the TX queue of DLB, that is, the first queue, and sent to the network service process. After the network service process obtains the description information, it determines the size of the data to be copied according to the 10 length in the description information through the 10 selector, so as to select whether to directly copy by the CPU or asynchronously copy through DSA. For small 10, direct CPU copying occupies less CPU resources and does not need to go through the DSA interaction process, which can reduce latency. For larger 10, the network service process will hand it over to DSA for processing. DSA copies this large 10 from the virtual address space of the application process to the virtual address space of the network service process through its own function, and regenerates the CRC code of the copied data, that is, the second check information, and compares it with the CRC in the description information. After confirming that the two generated CRCs are consistent, it is handed over to the network service process for further processing at the network layer, so as to ensure the correctness of the transmitted data. Since the speed of DSA copying large 10 is faster than that of the CPU, in this way, the CPU occupancy is reduced and the data copying duration is shortened.

[0061] The method of the embodiment of the present application realizes data interaction between processes through hardware, reduces interaction overhead, and realizes the performance of low latency and high throughput. In this way, it can completely decouple the network service process and the application process in software. While maintaining the flexibility of the network service process itself, it achieves performance similar to the libOS mode. It is a user-state network system implementation method that takes into account both performance and flexibility. Through this mode, while meeting the requirements of key performance indicators such as latency and throughput, it can conveniently realize functions such as flexible scheduling of resources within the network service process and traffic priority control.

[0062] The method of the embodiment of the present application can be applied to the scenario of high-performance storage networks, providing a new application scenario of a new type of acceleration hardware in storage networks. It reduces the software complexity of inter-process queue management through the hardware queue management of DLB, and reduces the enqueue and dequeue latency. It offloads operations that occupy CPU resources such as memory copying and CRC verification of inter-process interaction to hardware acceleration processing through DSA.

[0063] Figure 7 is a schematic structural diagram of a data processing device provided by an embodiment of the present application. The device is applied to a server, where an application process and a network service process are deployed. The server includes data processing hardware, and the data processing hardware includes a queue management module and an input / output (I / O) acceleration module. As shown in Figure 7, the data processing device 700 includes a first processing module 701 and a second processing module 702.

[0064] The first processing module 701 is configured to store the description information of the data to be transmitted into the queue management module through a first process, so as to send the description information to a second process through the queue management module; wherein, the first process is an application process, and the second process is a network service process; alternatively, the first process is a network service process, and the second process is an application process.

[0065] The second processing module 702 is configured to send the description information to the I / O acceleration module through the second process, so that the I / O acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information.

[0066] In one implementation, it further includes: a third processing module, configured to: the first process generates first check information of the data to be transmitted through the I / O acceleration module, and adds the first check information to the description information; the I / O acceleration module generates second check information of the data to be transmitted in the virtual address space of the second process; if the first check information and the second check information are consistent, a check pass is fed back to the second process.

[0067] In one implementation, the server includes one or more application processes. For each application process, the queue management module includes a first queue and a second queue corresponding to the application process. The first queue is used to store the description information of the data to be transmitted from the application process to the network service process, and the second queue is used to store the description information of the data to be transmitted from the network service process to the application process.

[0068] In one implementation, both the first queue and the second queue are multi-producer, multi-consumer queues. The first queue is used to store the description information of the data to be transmitted from multiple threads of the application process to multiple threads of the network service process, and the second queue is used to store the description information of the data to be transmitted from multiple threads of the network service process to multiple threads of the application process.

[0069] In one implementation, the description information includes the virtual address and length of the data to be transmitted.

[0070] In one implementation, the second processing module 702 is configured to: determine the size of the data to be transmitted by the second process according to the description information; if the size of the data to be transmitted exceeds a preset threshold, send the description information to the acceleration module 10.

[0071] In one implementation, the second processing module 702 is further configured to: if the size of the data to be transmitted does not exceed the preset threshold, the second process copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process through the CPU of the server.

[0072] In one implementation, the description information includes the priority of the data to be transmitted. When the queue management module includes the description information of multiple data to be transmitted, the queue management module preferentially sends the description information of the data to be transmitted with a high priority to the second process.

[0073] The device according to the embodiment of the present application can be used to execute the data processing method in the foregoing embodiment, and its implementation principle and technical effect are similar, which will not be described in detail here.

[0074] FIG. 8 is a schematic block diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 8, the electronic device 800 may include at least one processor 801 for implementing the data processing method provided by the embodiment of the present application. The electronic device 800 may be the server in the foregoing embodiment.

[0075] Optionally, the electronic device 800 further includes at least one memory 802 for storing program instructions and / or data. The memory 802 is coupled to the processor 801. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms for information interaction between devices, units or modules. The processor 801 may cooperate with the memory 802. The processor 801 may execute the program instructions stored in the memory 802. At least one of the at least one memory may be included in the processor.

[0076] Optionally, the electronic device 800 further includes a communication interface 803 for communicating with other devices through a transmission medium, so that the electronic device 800 can communicate with other devices. The communication interface 803 may be, for example, a transceiver, an interface, a bus, a circuit or a device capable of implementing a transceiver function. The processor 801 may use the communication interface 803 to transmit and receive data and / or information and is used to implement the method provided by the embodiment of the present application. For specific details, refer to the detailed description in the foregoing embodiment, which will not be elaborated here.

[0077] In the embodiments of the present application, the specific connection medium between the above-mentioned processor 801, memory 802, and communication interface 803 is not limited. In the embodiments of the present application, in FIG. 8, the processor 801, memory 802, and communication interface 803 are connected through a bus 804. The bus 804 is represented by a thick line in FIG. 8. The connection manners between other components are only for illustrative purposes and are not restrictive. This bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, only one thick line is used in FIG. 8, but it does not mean that there is only one bus or one type of bus.

[0078] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or the instructions in software form. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0079] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include but not be limited to these and any other suitable types of memory.

[0080] The present application also provides a computer-readable storage medium, which stores a computer program (which can also be referred to as code or instructions). When the computer program is run, it causes the computer to execute the method in any of the foregoing embodiments.

[0081] The terms "unit", "module", etc. used in this specification can be used to represent computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution.

[0082] Those of ordinary skill in the art will realize that the various illustrative logical blocks and steps described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application. In several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed between each other can be through some interfaces, indirect couplings or communication connections of devices or units, and can be electrical, mechanical, or other forms.

[0083] The unit described as a separate component may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0085] In the above embodiments, the functions of each functional unit can be fully or partially implemented through software, hardware, firmware, or It can be implemented by any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center integrating one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0086] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the related technology, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods according to the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0087] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to choose to authorize or refuse.

[0088] As described above, this is only a specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims described above.

Claims

Claims 1. A data processing method, wherein, Applied to a server, where an application process and a network service process are deployed in the server, and the server includes data processing hardware, the data processing hardware includes a queue management module and an input / output (I / O) acceleration module. The method includes: storing, by a first process, description information of data to be transmitted into the queue management module, so as to send the description information to a second process through the queue management module; where the description information is used to indicate the storage location of the data to be transmitted; the first process is the application process, and the second process is the network service process; or the first process is the network service process, and the second process is the application process; sending, by the second process, the description information to the I / O acceleration module, so that the I / O acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information.

2. The method according to claim 1, further comprising: The first process generates first check information of the data to be transmitted through the I / O acceleration module, and adds the first check information to the description information. After the I / O acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information, the method further includes: the I / O acceleration module generates second check information of the data to be transmitted in the virtual address space of the second process. If the first check information and the second check information are consistent, a check passed is fed back to the second process.

3. The method according to claim 1, wherein The server includes one or more application processes. For each application process, the queue management module includes a first queue and a second queue corresponding to the application process, where the first queue is used to store description information of data to be transmitted that is from the application process and to be transmitted to the network service process, and the second queue is used to store description information of data to be transmitted that is from the network service process and to be transmitted to the application process.

4. The method according to claim 3, wherein Both the first queue and the second queue are multi-producer, multi-consumer queues. The first queue is used to store description information of data to be transmitted that is from multiple threads of the application process and to be transmitted to multiple threads of the network service process, and the second queue is used to store description information of data to be transmitted that is from multiple threads of the network service process and to be transmitted to multiple threads of the application process.

5. The method according to claim 1, wherein The description information includes the virtual address and length of the data to be transmitted.

6. The method according to claim 5, wherein The second process sends the description information to the I / O acceleration module, including: the second process determines the size of the data to be transmitted according to the description information. If the size of the data to be transmitted exceeds a preset threshold, the description information is sent to the I / O acceleration module.

7. The method according to claim 6, further comprising: If the size of the data to be transmitted does not exceed a preset threshold, the second process copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process through the CPU of the server.

8. The method according to claim 5, wherein, The description information includes the priority of the data to be transmitted. When the description information of multiple data to be transmitted is included in the queue management module, the queue management module preferentially sends the description information of the data to be transmitted with a high priority to the second process.

9. A data processing device, wherein, Applied to a server, where an application process and a network service process are deployed in the server, and the server includes a queue management module and an input / output (I / O) acceleration module. The device includes: a first processing module, configured to store the description information of the data to be transmitted into the queue management module through a first process, so as to send the description information to a second process through the queue management module; where the first process is the application process and the second process is the network service process; or the first process is the network service process and the second process is the application process; a second processing module, configured to send the description information to the I / O acceleration module through the second process, so that the I / O acceleration module copies the data to be transmitted from the virtual address space of the first process to the virtual address space of the second process based on the description information.

10. An electronic device, wherein, Comprising: A memory and a processor; The memory is used to store a computer program; The processor is configured to execute the computer program stored in the memory, and when the computer program runs, the processor executes the method according to any one of claims 1-8.

11. A computer-readable storage medium, wherein, A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the processor executes the method according to any one of claims 1-8.

12. A computer program product, wherein, Comprising a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-8.

Images