Data processing method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: receiving a data processing request, and allocating a target cache queue to the data processing request; the data processing request is analyzed through a target processing thread corresponding to the target cache queue to obtain a data packet, and different cache queues correspond to different processing threads; and processing the data packet according to the target cache queue through the target processing thread so as to respond to the data processing request. By adopting the method, the performance of the target device in a high-performance data processing scene can be improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] With the explosion of data volume and the complexity of network topologies, the implementation of the traditional Internet Small Computer Systems Interface (iSCSI) transport protocol often faces problems such as performance bottlenecks, increased latency, and insufficient reliability when dealing with high-concurrency and large-scale data transmissions. Therefore, the SCSI Target Subsystem for Linux (SCST) framework that can implement the iSCSI transport protocol in the Linux kernel was proposed. The SCST framework allows users to configure a Linux system as an iSCSI target, so that other clients can access the data stored on this Linux system through the iSCSI protocol.

[0003] However, due to the fact that SCST depends on the Linux network protocol stack and the thread model of SCST is complex, there is a problem of poor target performance in high-performance data processing scenarios. Summary of the Invention

[0004] Based on this, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the performance of the target in high-performance data processing scenarios for the above technical problems.

[0005] In a first aspect, this application provides a data processing method. The method includes:

[0006] Receiving a data processing request and determining an allocation target cache queue corresponding to the data processing request;

[0007] Parsing the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet, where different cache queues correspond to different processing threads;

[0008] Processing the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0009] In one of the embodiments, the processing the data packet according to the target processing thread and the target cache queue includes:

[0010] Copy the data packet to the physical memory space corresponding to the target cache queue, and determine a second logical address according to the first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0011] Process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0012] In one embodiment, the determining the second logical address according to the first logical address corresponding to the target cache queue includes:

[0013] Determine an offset address through the target processing thread according to the first logical address and the base address corresponding to the target cache queue, and determine the second logical address according to the offset address and the pre-allocated base address.

[0014] In one embodiment, the processing the data packet in the physical memory space corresponding to the second logical address through the target processing thread includes:

[0015] Parse the data packet through the target processing thread to obtain a parsing result, where the parsing result at least includes a data processing command;

[0016] Process the data packet in the physical memory space according to the parsing result.

[0017] In one embodiment, the processing the data packet in the physical memory space according to the parsing result includes:

[0018] If the parsing result includes a first data processing command and data to be processed, then store the data to be processed in a first physical storage device through the target processing thread in response to the first data processing command.

[0019] In one embodiment, the processing the data packet in the physical memory space according to the parsing result includes:

[0020] If the parsing result includes a second data processing command, then read target data from a second physical storage device through the target processing thread in response to the second data processing command, and process the target data in the physical memory space.

[0021] In one embodiment, before processing the data packet in the physical memory space corresponding to the second logical address through the target processing thread, the method further includes:

[0022] Determine the processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests;

[0023] The processing of the data packet in the physical memory space corresponding to the second logical address by the target processing thread includes:

[0024] Process the data packet in the physical memory space by the target processing thread according to the processing frequency.

[0025] In a second aspect, the present application also provides a data processing device. The device includes:

[0026] A receiving module, configured to receive a data processing request and allocate a target cache queue for the data processing request;

[0027] A parsing module, configured to parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet, and different cache queues correspond to different processing threads;

[0028] A processing module, configured to process the data packet according to the target cache queue through the target processing thread to respond to the data processing request.

[0029] In a third aspect, the present application also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect above are implemented.

[0030] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0031] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0032] For the above data processing method, apparatus, computer device, computer-readable storage medium, and computer program product, the server first receives a data processing request and determines a target cache queue corresponding to the data processing request. Then, the data processing request is parsed through a target processing thread corresponding to the target cache queue to obtain a data packet. Subsequently, the data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request. Since different cache queues correspond to different processing threads, when the data packet included in the data processing request is processed through the target processing thread, no interrupt needs to be initiated, and no context switching is required, reducing the memory redundantly occupied by context switching and avoiding the impact of a large number of interrupts on the server stability. Moreover, since the target processing thread adopts a serial processing method, no processing resources need to be locked, avoiding lock competition and reducing the overhead of locking and unlocking. Therefore, while improving the system stability, the memory occupancy rate is reduced, the cache utilization rate and the response efficiency to the data processing request are improved, and thus the performance of the server is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the related art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is an application environment diagram of the data processing method in an embodiment;

[0035] Figure 2 It is a flowchart of the data processing method in an embodiment;

[0036] Figure 3 It is a schematic diagram of the corresponding relationship between cache queues and processing threads in an embodiment;

[0037] Figure 4 It is a flowchart of step 203 in another embodiment;

[0038] Figure 5 It is a schematic diagram of multiple logical addresses and the same physical memory space in an embodiment;

[0039] Figure 6 It is a flowchart of step 402 in another embodiment;

[0040] Figure 7 It is a flowchart of the data processing method in another embodiment;

[0041] Figure 8It is a structural block diagram of a data processing device in an embodiment;

[0042] Figure 9 It is an internal structure diagram of a computer device in an embodiment. Specific implementation manners

[0043] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In the field of data transmission, a computer device can send commands from a host to a storage device through a Small Computer System Interface (SCSI) protocol, and can transmit SCSI commands that can only be used for the local computer of the computer to a device in the network through a standard Ethernet through an Internet Small Computer Systems Interface (iSCSI) protocol. However, with the explosion of data volume and the complexity of network topologies, the implementation manner of iSCSI will face problems such as increased latency and insufficient reliability when processing high-concurrency and large-scale data transmission. Therefore, an SCST (SCSI Target Subsystem for Linux) framework that can implement the iSCSI transmission protocol in the Linux kernel is proposed. Through SCST, a user can configure the Linux system as an iSCSI target, enabling other computer devices to access resources stored on the Linux system through the iSCSI protocol.

[0045] SCST mainly relies on the TCP / IP network protocol stack of Linux for data transmission. SCST can be divided into three layers, namely the SCST TARGET driver layer, the SCST layer, and the device interface layer. Among them, the SCST TARGET driver layer is mainly responsible for receiving and replying to iSCSI messages. This layer contains two types of IO threads, the iscsird thread and the iscsiwd thread. The iscsird thread is responsible for receiving network messages and preprocessing iSCSI commands, and the iscsiwd thread is responsible for replying to the command execution results. The SCST layer is mainly responsible for executing SCSI commands and creates multiple groups of processes according to the types of SCSI commands. For example, the SCSI IO thread is responsible for processing read / write SCSI commands, and the SCSI TM thread is responsible for processing management SCSI commands. SCST MGMT is responsible for creating and destroying sessions. In the above process of SCST's data transmission processing, when a data frame arrives at the computer device, the computer device will write the data frame into the allocated circular buffer queue in the way of direct memory access. Then, it notifies the CPU that there is data arrival in the way of hard interrupt. Next, the CPU responds to the hard interrupt, sets the register after that, initiates a soft interrupt request, and performs context switching processing. Due to frequent context switching, it leads to insufficient resources such as IO and memory, causing the problem of lock competition. Moreover, in this processing process, multiple memory copies of data are involved. Therefore, in high-performance application scenarios, SCST will face a large number of problems. For example, interrupt processing will affect the response ability and performance stability of the computer device; context switching requires saving information such as the register state, program counter, and stack pointer of the current task, resulting in CPU cache invalidation; memory copy will increase the occupancy of memory bandwidth and the burden on the CPU, restricting the data transmission efficiency; lock competition will reduce the throughput of the computer system, extend the response time, and even cause the system to enter a deadlock state. In summary, in high-performance application scenarios, SCST will affect the performance of the computer device, that is, it will affect the performance of the iSCSI target.

[0046] In view of this, the present application proposes a data processing method. First, the server receives a data processing request and allocates a target cache queue for the data processing request. Then, the data processing request is parsed through the target processing thread corresponding to the target cache queue to obtain a data packet. Subsequently, the data packet is processed by the target processing thread according to the target cache queue to respond to the data processing request. Since the processing threads corresponding to different cache queues are different, when the data packet included in the data processing request is processed by the target processing thread, no interrupt needs to be initiated, no context switching is required, the memory redundantly occupied by context switching is reduced, and the impact of a large number of interrupts on the server stability is avoided. Moreover, since the target processing thread adopts a serial processing method, no processing resources need to be locked, lock competition is avoided, and the overhead of locking and unlocking is reduced. Therefore, while improving the system stability, the memory occupancy rate is reduced, the cache utilization rate and the response efficiency of the data processing request are improved, and thus the performance of the server is improved.

[0047] The data processing method provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the server 102 communicates with the terminal 104 through a network. The data storage system can store the data that the server 102 needs to process. The data storage system can be integrated on the server 102, or can be placed in the cloud or on other network servers. The server 102 can receive a data processing request and allocate a target cache queue for the data processing request. Then, the data processing request is parsed through the target processing thread corresponding to the target cache queue to obtain a data packet. The processing threads corresponding to different cache queues are different. Subsequently, the packet is processed by the target processing thread according to the target cache queue to respond to the data processing request. Among them, the terminal 104 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 102 can be implemented by an independent server or a server cluster composed of multiple servers.

[0048] In an exemplary embodiment, as Figure 2 shown, a data processing method is provided. Taking the server in Figure 1 as an example, the method includes the following steps 201-step 203:

[0049] Step 201, receive a data processing request and determine the target cache queue corresponding to the data processing request.

[0050] Among them, the data processing request is a data frame sent by the client to the server through a network request. The server processes the data frame by responding to the data processing request and returns the processing result to the client to respond to the client's data processing requirements. For example, the data processing request can be a data query request, a data read request, a data storage request, etc.

[0051] Among them, the target cache queue is a storage space in the memory for temporarily storing data processing requests, so as to quickly access and reduce the number of accesses to the main storage system. Through the target cache queue, the received data processing requests can be stored sequentially, and each data processing request can be read and processed in the stored order. It can be understood that the server can pre-set multiple cache queues in the memory to store multiple data processing requests to improve the response speed. Exemplarily, the cache queue can be a circular cache queue.

[0052] It can be understood that the receive queue corresponding to the network card in the server can be used to store the received data processing requests, and then determine the cache queue corresponding to the data processing request according to the correspondence between the data packet in the data processing request and the preset cache queue. In this embodiment, this cache queue is determined as the target cache queue.

[0053] In this embodiment, the server can receive the data processing requests sent by other devices through the receive queue of the network card, and determine the target cache queue corresponding to the currently received data processing request according to the number of processing requests already stored in each cache queue, or according to the correspondence between the cache queue and the network card queue, and bind the target cache queue to the data processing request to establish the correspondence between the data processing request and the target cache queue. Exemplarily, the server can determine the cache queue with the least number of currently stored processing requests as the target cache queue.

[0054] Step 202, parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet, and different cache queues correspond to different processing threads.

[0055] Among them, the target processing thread is a processing thread for processing data processing requests. It should be noted that in this embodiment, in order to avoid problems such as interruption and context switching during the data processing request process, multiple processing threads can be set, and each processing thread is allocated to each cache queue, so that multiple processing threads can concurrently execute the processing threads in the corresponding cache queue. Among them, the processing thread can be a polling thread for periodically determining whether there are new data processing requests allocated to the cache queue. Among them, each processing thread can share system resources and corresponds to its own register environment, call stack, etc.

[0056] It should be noted that each stage in the process of responding to a data processing request can be processed by a target processing thread in a serial execution manner, thereby avoiding lock contention and the resource overhead of locking and unlocking. Moreover, in a scenario with a high concurrency of data processing requests, a large number of CPU interrupts can be avoided, and thus the stability and response speed of the system can be improved. For multiple processing threads included in the server, a simple lock can be used to lock resources.

[0057] Among them, the data packet refers to the data carried in the data processing request. The data processing request may include a request line, an Ethernet frame header, an IP header, a TCP header, a data packet, etc. Among them, the request line includes the request type, the request resource path, the version and type of the transport protocol, the message header contains the sender and receiver of the data processing request, and the data packet includes the actual data to be processed.

[0058] Exemplarily, as Figure 3 shown, the cache queue and the processing thread are in one-to-one correspondence. For example, Figure 3 cache queue 1 corresponds to processing thread 1, and cache queue N corresponds to processing thread N. Processing thread 1 is used to process the data processing requests stored in cache queue 1, and the data processing requests in cache queue 1 cannot be processed by other processing threads. For example, cache queue N cannot access and process the data processing requests in cache queue 1.

[0059] In this embodiment, the send queue corresponding to the network card in the server can send the data processing requests stored in the receive queue to the target cache queue. After the data processing request is allocated to the target cache queue, the server can use the target processing thread corresponding to the target cache queue to parse the data processing request, strip the Ethernet frame header, IP header, and TCP header in the data processing request to obtain a parsing result, and obtain the data packet from the parsing result.

[0060] As a possible implementation manner, the network card may correspond to multiple network card queues, each network card queue corresponds to at least one processing thread, and the processing thread can identify the card slot corresponding to the network card to determine the correspondence between the processing thread and the network card.

[0061] Step 203, process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0062] It can be understood that during the process of the target processing thread processing the data packet, the types of data processing requests are different, and the corresponding data packets are also different. The target processing thread can process the data packet accordingly according to the type of the data processing request.

[0063] In this embodiment, when the server obtains a data packet, it can use the target processing thread to process the data packet in the target cache queue, and then, it can return the processing result to the target cache queue.

[0064] In the above data processing method, the server first receives a data processing request and determines the target cache queue corresponding to the data processing request. Then, it parses the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet. After that, it processes the data packet according to the target processing thread and the target cache queue to respond to the data processing request. Since different cache queues correspond to different processing threads, when processing the data packet included in the data processing request through the target processing thread, no interrupt needs to be initiated, and no context switch is required, reducing the memory redundantly occupied by the context switch and avoiding the impact of a large number of interrupts on the server stability. Moreover, since the target processing thread adopts a serial processing method, no processing resources need to be locked, avoiding lock competition and reducing the overhead of locking and unlocking. Therefore, while improving the system stability, it can reduce the memory occupancy rate, improve the cache utilization rate and the response efficiency of the data processing request, and further improve the performance of the server.

[0065] In one embodiment, based on Figure 2 the embodiment shown, refer to Figure 4 , this embodiment relates to the process of the server processing a data packet according to the target cache queue through the target processing thread. As Figure 4 shown, step 203 includes Figure 4 the steps 401 and 402 shown in

[0066] Step 401, copy the data packet to the physical memory space corresponding to the target cache queue, and determine a second logical address according to the first logical address corresponding to the target cache queue. The physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address.

[0067] Among them, the physical memory space refers to the actual memory space corresponding to the virtual memory space where the target cache queue is located. It can be understood that the physical memory space can be the actual storage space on the storage device, and the data in the physical memory space can be stored for a long time. Therefore, the data in the data processing request can be stored in the physical memory space.

[0068] Among them, the first logical address is the storage address of the target cache queue in the virtual memory space, and the second logical address is the logical address of the virtual memory space that needs to be accessed during the data request processing. To avoid the physical memory space corresponding to the target cache queue being different from the physical memory space corresponding to the data request processing, it is necessary to copy the data packet from one memory space to another. In this embodiment, the physical memory space corresponding to the first logical address and the physical memory space corresponding to the second logical address are set to the same physical memory space.

[0069] It should be noted that in the embodiments of the present application, a physical memory can be pre-allocated as the memory space corresponding to the data processing request, so that the input / output operations during the processing can be completed in this physical memory without additional data copying, so as to achieve zero-copy of memory between multiple processes. In practical applications, the server can map this physical memory to different virtual memory spaces. For example, this physical memory can be mapped to the virtual memory space corresponding to the user-mode process, or this physical memory can be mapped to the virtual memory space corresponding to the kernel-mode process. This virtual memory space can include the virtual memory space corresponding to the first logical address, or can include the virtual memory space corresponding to the second logical address.

[0070] Exemplarily, a schematic diagram of different logical addresses corresponding to the same physical memory space is as Figure 5 shown, Figure 5 In, the same physical memory space can be mapped to the virtual memory space 1 corresponding to process 1, or can be mapped to the virtual memory space 2 corresponding to process 2, and the offset addresses of the virtual memory space 1 and the virtual memory space 2 are the same. Therefore, the physical memory spaces accessed by process 1 and process 2 are the same physical memory.

[0071] In this embodiment, before processing the data packet, the server can copy the data packet in the target cache queue to the physical memory space corresponding to the target cache queue through the network card. Then, based on the fact that the physical memory space corresponding to the first logical address of the target cache queue is the same as the physical memory space corresponding to the second logical address, the first logical address is converted to obtain the physical address of the physical memory space, and then the second logical address is determined based on this physical address.

[0072] Step 402, process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0073] In this embodiment, after determining the second logical address, the server can access the physical memory space storing the data packet, that is, the physical memory space corresponding to the second logical address, through the target processing thread, and perform corresponding data processing on the data packet.

[0074] In this embodiment, the server first copies the data packet to the physical memory space corresponding to the target cache queue, determines the second logical address according to the first logical address corresponding to the target cache queue, and then processes the data packet in the physical memory space corresponding to the second logical address through the target processing thread. Since the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address, the physical memory space accessed by the server when processing the data packet is the same as the physical memory space where the data packet is stored, thus avoiding the problem of increasing memory occupancy by copying the data packet from one physical memory space to another physical memory space, reducing the memory occupancy, and improving the data transmission efficiency.

[0075] In one embodiment, based on Figure 4 the embodiment shown, this embodiment relates to the process in which the server determines the second logical address according to the first logical address corresponding to the target cache queue. The server implements this process in the following manner: the target processing thread determines the offset address according to the first logical address and the base address corresponding to the target cache queue, and determines the second logical address according to the offset address and the pre-allocated base address.

[0076] Among them, the base address refers to the address of the starting position of the memory segment used to store data in the physical memory space, and the offset address refers to the offset of the end position of this memory segment relative to the base address. Among them, the pre-allocated base address refers to the base address of the memory space allocated for the target processing thread to process data according to the memory mapping relationship when creating the target processing thread.

[0077] It can be understood that since the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address, the offset address of the physical memory space corresponding to the first logical address is also the same as the offset address of the physical memory space corresponding to the second logical address.

[0078] Exemplarily, if vAddr1 is the first logical address mapped by process 1, gVMapStart1 is the starting address of process 1, vAddr2 is the second logical address mapped by process 2, and gVMapStart2 is the starting address of process 2, then the corresponding offset address Offset1 is: Offset1 = vAddr1 - gVMapStart1; the second logical address can be determined according to offset address 1: vAddr2 = gVMapStart0 + offset1.

[0079] In this embodiment, the server can obtain the base address corresponding to the target cache queue through the target processing thread, and obtain the pre-allocated base address. Then, according to the address calculation method, the offset address is determined based on the first logical address and the base address. Subsequently, based on the offset address and the pre-allocated base address, the second logical address is determined.

[0080] In this embodiment, the server determines the offset address through the target processing thread based on the first logical address and the base address corresponding to the target cache queue. Then, based on the offset address and the pre-allocated base address, the second logical address is determined. Since the first logical address and the second logical address correspond to the same physical memory space, the second logical address can be quickly determined based on the offset address and the pre-allocated base address, thereby improving the efficiency of determining the second logical address.

[0081] In one embodiment, based on Figure 2 the embodiment shown, refer to Figure 6 , this embodiment relates to the process of the server processing the data packet in the physical memory space corresponding to the second logical address. As Figure 6 shown, step 402 includes Figure 6 steps 601 and 602 shown in

[0082] Step 601, parse the data packet through the target processing thread to obtain a parsing result, and the parsing result includes at least a data processing command.

[0083] Among them, the data processing command refers to the instructions and operations used in the data processing process. For example, the data processing command can be: a data storage command, a data update command, a data read command, a data query command, a data delete command, etc. It can be understood that some data processing commands need to process the data included in the data processing request, and some data processing commands need to process the data stored in the server. Therefore, the parsing result can include a data processing command, or the parsing result can also include a data processing command and the data to be processed.

[0084] In this embodiment, the server can parse the data packet according to the encapsulation format of the data packet through the target processing thread to obtain a parsing result.

[0085] Step 602, process the data packet in the physical memory space according to the parsing result.

[0086] In this embodiment, the server can determine the processing type corresponding to the data processing request according to the content of the parsing result, and thus process the data packet in the physical memory space according to the processing type.

[0087] In this embodiment, the server can obtain a parsing result by parsing the data packet through a target processing thread. Then, based on the parsing result, the server processes the data packet. Since the parsing result at least includes a data processing command, the server can determine the request type corresponding to the data processing request according to the data processing command, so as to respond to the data processing request according to the request type, avoiding the problem of operation errors caused by inconsistent data processing operations and data processing requests, and further improving the processing performance of the server in the scenario of high-concurrency data processing requests.

[0088] Based on the above embodiment, in the parsing result obtained by the server parsing the data packet, it may include a data processing command, or may include a data processing command and data to be processed. The following describes the processing processes of the server for different parsing results respectively.

[0089] In one embodiment, based on Figure 6 the embodiment shown, this embodiment relates to the first process of the server processing the data packet according to the parsing result: if the parsing result includes a first data processing command and data to be processed, then through the target processing thread in response to the first data processing command, the data to be processed is stored in a first physical storage device.

[0090] Among them, the first data processing command refers to a data processing command that needs to store data. It can be understood that if the parsing result includes a first data processing command and data to be processed, the server can determine that this data processing command is a type that needs to process the data included in the data processing request. For example, a data write command. Among them, the physical storage device refers to a device for storing the data to be processed. For example, the physical storage device can be a random access memory, a cache memory, a disk, a solid-state drive, an optical disc, etc. In this embodiment, the physical storage device storing the data to be processed corresponding to the first data processing command is used as the first data processing command.

[0091] In this embodiment, when the parsing result includes a first data processing command and data to be processed, the server can determine that the data processing request needs to perform corresponding operations on the data to be processed, and through the target processing thread in response to the first data processing command, the data to be processed is stored in a first physical storage device.

[0092] In this embodiment, when the parsing result includes a first data processing command and data to be processed, the server can quickly determine the request type corresponding to the data processing request. Then, through the target processing thread in response to the first data processing command, the data to be processed can be stored in a first physical storage device, thereby quickly responding to the data processing request and improving the response speed of the server to the data processing request.

[0093] In one embodiment, based on Figure 6 the embodiment shown, this embodiment relates to the process of the second server processing the data packet according to the parsing result: If the parsing result includes a second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0094] Among them, the second data processing command refers to a data processing command that does not need to store data and processes the already stored data. It can be understood that if the parsing result includes a second data processing command, the server can determine that the data processing command is of the type that needs to process the data stored in the server. Among them, the second physical storage device refers to the storage device that stores the data corresponding to the second data processing command. In this embodiment, the data corresponding to the second data processing command stored in the second physical storage device is used as the target data.

[0095] In this embodiment, when the parsing result includes a second data processing command, the server can determine that the data processing request needs to perform corresponding operations on the data to be processed, and the target processing thread responds to the second data processing command. Then, the target data is read from the second physical storage device and stored in the physical memory space, and then the target data is processed.

[0096] In this embodiment, when the parsing result includes a second data processing command, the server can quickly determine the request type corresponding to the data processing request. Then, the target processing thread responds to the second data processing command, and can process the target data read from the second physical storage device in the physical memory space, so as to quickly respond to the data processing request and improve the response speed of the server to the data processing request.

[0097] In one embodiment, based on Figure 3 the embodiment shown, this embodiment relates to the process of determining the working frequency corresponding to the target processing thread before the server processes the data packet through the target processing thread: Determine the processing frequency corresponding to the target processing thread, and the processing frequency is related to the number of currently received data processing requests.

[0098] It is understandable that the number of data processing requests received by the server varies at different time periods, and the corresponding business processing status is also different. For example, if the number of services shows a linear growth trend, it can be determined that the server is in a busy business state; if the number of services shows an exponential decreasing trend, it can be determined that the server is in an idle business state. Correspondingly, when the server is in a busy business state, the processing frequency of the processing thread can be reduced. For example, the poll mechanism can be used to reduce the response speed of the processing thread by extending the timeout period or increasing the polling interval, thereby reducing the CPU usage rate; when the server is in an idle business state, the processing frequency of the processing thread can be increased. For example, the full-load mode can be adopted to improve the response speed of the processing thread. Therefore, the load balancing of the server can be achieved by dynamically adjusting the working frequency of the processing thread.

[0099] Among them, the processing frequency refers to the frequency at which the server responds to data processing commands through the processing thread per unit time. In this embodiment, the server can determine the processing frequency of the current processing thread according to the number of data processing requests received in the current time period, and then determine the processing frequency corresponding to the target processing thread as this processing frequency. Then, the server can process the data packet in the physical memory space through the target processing thread according to the processing frequency.

[0100] In this embodiment, by determining the processing frequency corresponding to the target processing thread, the server can process the data packet in the physical memory space through the target processing thread according to the processing frequency. Since the processing frequency is related to the number of currently received data processing requests, the processing frequency can be adjusted in a timely manner according to the number of currently received data processing requests, so that the processing frequency can match the current processing resources of the server, thereby realizing the dynamic load balancing of the server, and further achieving the goal of low-power operation of the server in the idle scenario while ensuring high performance.

[0101] In one embodiment, a data processing method for a server is provided, as Figure 7 shown. The method includes the following steps:

[0102] Step 701, receive a data processing request and determine the target cache queue corresponding to the data processing request.

[0103] Step 702, parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet.

[0104] Step 703, copy the data packet to the physical memory space corresponding to the target cache queue through the network card, and determine the offset address through the target processing thread according to the first logical address and the base address corresponding to the target cache queue.

[0105] Step 704: Determine a second logical address according to the offset address and the pre-allocated base address.

[0106] Step 705: Determine the processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests.

[0107] Step 706: Parse the data packet through the target processing thread according to the processing frequency to obtain a parsing result, where the parsing result includes at least a data processing command.

[0108] Step 707: If the parsing result includes a first data processing command and data to be processed, store the data to be processed in the first physical storage device through the target processing thread in response to the first data processing command.

[0109] Step 708: If the parsing result includes a second data processing command, read the target data from the second physical storage device through the target processing thread in response to the second data processing command, and process the target data in the physical memory space.

[0110] Step 709: Return the processing result to the target cache queue through the target processing thread.

[0111] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least some of the steps or stages in other steps or other steps.

[0112] Based on the same inventive concept, an embodiment of the present application further provides a data processing device for implementing the data processing method described above. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the data processing device provided below can refer to the limitations on the data processing method in the above text, and will not be repeated here.

[0113] In an exemplary embodiment, as Figure 8 shown, a data processing device is provided, including:

[0114] A receiving module 801, configured to receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0115] A parsing module 802, configured to parse the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet, where the processing threads corresponding to different cache queues are different;

[0116] A processing module 803, configured to process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0117] In one embodiment, the above-mentioned processing module 803 includes:

[0118] A determining unit, configured to copy the data packet to a physical memory space corresponding to the target cache queue through the target processing thread, and determine a second logical address according to a first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0119] A processing unit, configured to process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0120] In one embodiment, the above-mentioned determining unit is specifically configured to:

[0121] Determine an offset address through the target processing thread according to the first logical address and a base address corresponding to the target cache queue, and determine the second logical address according to the offset address and the pre-allocated base address.

[0122] In one embodiment, the above-mentioned processing unit is specifically configured to:

[0123] Parse the data packet through the target processing thread to obtain a parsing result, where the parsing result includes at least a data processing command;

[0124] Process the data packet in the physical memory space according to the parsing result.

[0125] In one embodiment, the above-mentioned processing unit is specifically configured to:

[0126] If the parsing result includes a first data processing command and data to be processed, store the data to be processed in a first physical storage device through the target processing thread in response to the first data processing command.

[0127] In one embodiment, the above-mentioned processing unit is specifically configured to:

[0128] If the parsing result includes a second data processing command, the target processing thread responds to the second data processing command, reads target data from the second physical storage device, and processes the target data in the physical memory space.

[0129] In one embodiment, the above device further includes:

[0130] A determination module, configured to determine the processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests.

[0131] The above processing unit is specifically configured to: process the data packet in the physical memory space through the target processing thread according to the processing frequency.

[0132] Each module in the above data processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the server in hardware form or independent of the processor, or stored in the memory in the server in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0133] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 9 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data to be processed. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. The computer program, when executed by the processor, implements a data processing method.

[0134] Those skilled in the art can understand that Figure 9 the structure shown in

[0135] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0136] Receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0137] Parse the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet. Different cache queues correspond to different processing threads;

[0138] Process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0139] In one of the embodiments, when the processor executes the computer program, the following steps are specifically implemented:

[0140] Copy the data packet to a physical memory space corresponding to the target cache queue, and determine a second logical address according to a first logical address corresponding to the target cache queue. The physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0141] Process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0142] In one of the embodiments, when the processor executes the computer program, the following steps are specifically implemented:

[0143] The target processing thread determines an offset address according to the first logical address and a base address corresponding to the target cache queue, and determines a second logical address according to the offset address and a pre-allocated base address.

[0144] In one of the embodiments, when the processor executes the computer program, the following steps are specifically implemented:

[0145] Parse the data packet through the target processing thread to obtain a parsing result, and the parsing result at least includes a data processing command;

[0146] Process the data packet in the physical memory space according to the parsing result.

[0147] In one of the embodiments, when the processor executes the computer program, the following steps are specifically implemented:

[0148] If the parsing result includes a first data processing command and data to be processed, the target processing thread stores the data to be processed in a first physical storage device in response to the first data processing command.

[0149] In one embodiment, when the processor executes the computer program, the following steps are specifically implemented:

[0150] If the parsing result includes a second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0151] In one embodiment, when the processor executes the computer program, the following steps are specifically implemented:

[0152] Determine the processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests;

[0153] Process the data packet in the physical memory space through the target processing thread according to the processing frequency.

[0154] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0155] Receive a data processing request and determine the target cache queue corresponding to the data processing request;

[0156] Parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet. Different cache queues correspond to different processing threads;

[0157] Process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0158] In one embodiment, when the computer program is executed by the processor, the following steps are specifically implemented:

[0159] Copy the data packet to the physical memory space corresponding to the target cache queue, and determine a second logical address according to the first logical address corresponding to the target cache queue. The physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0160] Process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0161] In one embodiment, when the computer program is executed by the processor, the following steps are specifically implemented:

[0162] The target processing thread determines an offset address according to the first logical address and the base address corresponding to the target cache queue, and determines a second logical address according to the offset address and the pre-allocated base address.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0164] Parse the data packet through the target processing thread to obtain a parsing result, where the parsing result includes at least a data processing command;

[0165] Process the data packet in the physical memory space according to the parsing result.

[0166] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0167] If the parsing result includes a first data processing command and data to be processed, then in response to the first data processing command through the target processing thread, store the data to be processed in the first physical storage device.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0169] If the parsing result includes a second data processing command, then in response to the second data processing command through the target processing thread, read the target data from the second physical storage device and process the target data in the physical memory space.

[0170] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0171] Determine the processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests;

[0172] Process the data packet in the physical memory space through the target processing thread according to the processing frequency.

[0173] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0174] Receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0175] Parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet, where different cache queues correspond to different processing threads;

[0176] Process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0178] Copy the data packet to the physical memory space corresponding to the target cache queue, and determine the second logical address according to the first logical address corresponding to the target cache queue. The physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0179] Process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0181] The target processing thread determines the offset address according to the first logical address and the base address corresponding to the target cache queue, and determines the second logical address according to the offset address and the pre-allocated base address.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0183] The target processing thread parses the data packet to obtain a parsing result, and the parsing result includes at least a data processing command;

[0184] Process the data packet in the physical memory space according to the parsing result.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0186] If the parsing result includes a first data processing command and data to be processed, the target processing thread stores the data to be processed in the first physical storage device in response to the first data processing command.

[0187] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0188] If the parsing result includes a second data processing command, the target processing thread reads the target data from the second physical storage device in response to the second data processing command, and processes the target data in the physical memory space.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0190] Determine the processing frequency corresponding to the target processing thread, and the processing frequency is related to the number of currently received data processing requests;

[0191] Process the data packet in the physical memory space by the target processing thread according to the processing frequency.

[0192] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0193] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0194] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: Receiving a data processing request and determining a target cache queue corresponding to the data processing request; Parsing the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet, where the processing threads corresponding to different cache queues are different; Processing the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

2. The method according to claim 1, characterized in that The processing the data packet according to the target processing thread and the target cache queue includes: Copying the data packet to a physical memory space corresponding to the target cache queue, and determining a second logical address according to a first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address; Processing the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

3. The method according to claim 2, wherein The determining the second logical address according to the first logical address corresponding to the target cache queue includes: Determining an offset address through the target processing thread according to the first logical address and a base address corresponding to the target cache queue, and determining the second logical address according to the offset address and a pre-allocated base address.

4. The method according to claim 2, wherein The processing the data packet in the physical memory space corresponding to the second logical address through the target processing thread includes: Parsing the data packet through the target processing thread to obtain a parsing result, where the parsing result at least includes a data processing command; Processing the data packet in the physical memory space according to the parsing result.

5. The method according to claim 4, wherein The processing the data packet in the physical memory space according to the parsing result includes: If the parsing result includes a first data processing command and data to be processed, storing the data to be processed in a first physical storage device through the target processing thread in response to the first data processing command; If the parsing result includes a second data processing command, reading target data from a second physical storage device through the target processing thread in response to the second data processing command, and processing the target data in the physical memory space.

6. The method according to claim 2, characterized in that Before the processing the data packet in the physical memory space corresponding to the second logical address through the target processing thread, the method further includes: Determining a processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests; The processing the data packet in the physical memory space corresponding to the second logical address through the target processing thread includes: Processing the data packet in the physical memory space through the target processing thread according to the processing frequency.

7. A data processing device, characterized in that, The apparatus includes: A receiving module, configured to receive a data processing request and allocate a target cache queue for the data processing request; A parsing module, configured to parse the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet, and different cache queues correspond to different processing threads; A processing module, configured to process the data packet according to the target cache queue through the target processing thread to respond to the data processing request.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images