Data processing method, device, computer equipment, storage medium and program product

Abstract

The present application relates to a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product. The method comprises: receiving a data processing request and assigning a target cache queue to the data processing request; parsing the data processing request via a target processing thread corresponding to the target cache queue to obtain a data packet, where different cache queues correspond to different processing threads; and processing the data packet via the target processing thread according to the target cache queue to respond to the data processing request. This method can improve the performance of a target device in high-performance data processing scenarios.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the surge in data volumes and increasing complexity of network topologies, traditional implementations of the Internet Small Computer Systems Interface (iSCSI) transport protocol often face performance bottlenecks, increased latency, and insufficient reliability when handling highly concurrent and large-scale data transfers. Therefore, the SCSI Target Subsystem for Linux (SCST) framework was proposed to implement the iSCSI transport protocol within the Linux kernel. The SCST framework allows users to configure a Linux system as an iSCSI target, allowing other clients to access data stored on the Linux system via the iSCSI protocol.

[0003] However, since SCST relies on the Linux network protocol stack and its thread model is complex, it suffers from poor target performance in high-performance data processing scenarios. Summary of the Invention

[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the performance of the target device in high-performance data processing scenarios in response to the above technical problems.

[0005] In a first aspect, the present application provides a data processing method. The method comprises:

[0006] receiving a data processing request and determining an allocation target cache queue corresponding to the data processing request;

[0007] Parsing the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet, where different cache queues correspond to different processing threads;

[0008] The data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request.

[0009] In one embodiment, processing the data packet according to the target processing thread and the target cache queue includes:

[0010] Copying the data packet to the physical memory space corresponding to the target cache queue, and determining a second logical address according to the first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0011] The data packet is processed in the physical memory space corresponding to the second logical address by the target processing thread.

[0012] In one embodiment, determining the second logical address according to the first logical address corresponding to the target cache queue includes:

[0013] The target processing thread determines an offset address according to the first logical address and a base address corresponding to the target cache queue, and determines the second logical address according to the offset address and a pre-assigned base address.

[0014] In one embodiment, processing the data packet in the physical memory space corresponding to the second logical address by the target processing thread includes:

[0015] Parsing the data packet through the target processing thread to obtain a parsing result, wherein the parsing result at least includes a data processing command;

[0016] According to the analysis result, the data packet is processed in the physical memory space.

[0017] In one embodiment, processing the data packet in the physical memory space according to the parsing result includes:

[0018] If the parsing result includes a first data processing command and data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in a first physical storage device.

[0019] In one embodiment, processing the data packet in the physical memory space according to the parsing result includes:

[0020] If the parsing result includes a second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0021] In one embodiment, before processing the data packet in the physical memory space corresponding to the second logical address by the target processing thread, the method further includes:

[0022] Determining a processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests;

[0023] The processing of the data packet in the physical memory space corresponding to the second logical address by the target processing thread includes:

[0024] The data packet is processed in the physical memory space according to the processing frequency by the target processing thread.

[0025] In a second aspect, the present application further provides a data processing device. The device comprises:

[0026] A receiving module, configured to receive a data processing request and allocate a target cache queue to the data processing request;

[0027] A parsing module, configured to parse the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet, wherein different cache queues correspond to different processing threads;

[0028] A processing module is configured to process the data packet according to the target cache queue through the target processing thread to respond to the data processing request.

[0029] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect above when executing the computer program.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0031] In a fifth aspect, the present application further provides a computer program product, comprising a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0032] In the above-mentioned data processing method, apparatus, computer equipment, computer-readable storage medium and computer program product, the server first receives a data processing request and determines the target cache queue corresponding to the data processing request. Then, the data processing request is parsed by the target processing thread corresponding to the target cache queue to obtain a data packet. Then, the data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request. Since different cache queues correspond to different processing threads, when the data packet included in the data processing request is processed by the target processing thread, there is no need to initiate an interrupt or perform context switching, which reduces the unnecessary memory occupied by the context switching and avoids the impact of a large number of interrupts on the stability of the server. Moreover, since the target processing thread adopts a serial processing method, there is no need to lock the processing resources, which avoids lock contention and reduces the overhead of switching locks. Therefore, while improving system stability, it can reduce memory occupancy, improve cache utilization and the response efficiency to data processing requests, thereby improving server performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 A diagram of an application environment of a data processing method in one embodiment;

[0035] Figure 2 1 is a flow chart of a data processing method in one embodiment;

[0036] Figure 3 A schematic diagram of the correspondence between cache queues and processing threads in one embodiment;

[0037] Figure 4 is a flow chart of step 203 in another embodiment;

[0038] Figure 5 A schematic diagram of multiple logical addresses and the same physical memory space in one embodiment;

[0039] Figure 6 is a flow chart of step 402 in another embodiment;

[0040] Figure 7 is a flow chart of a data processing method in another embodiment;

[0041] Figure 8is a structural block diagram of a data processing device in one embodiment;

[0042] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0044] In the data transmission field, computer devices can send commands from the host to storage devices using the Small Computer System Interface (SCSI) protocol. The Internet Small Computer Systems Interface (iSCSI) protocol then transmits these SCSI commands, which are limited to the computer itself, to networked devices via standard Ethernet. However, with the surge in data volumes and the increasing complexity of network topologies, iSCSI implementations face challenges with increased latency and reliability when handling high-concurrency and large-scale data transmission. Therefore, the SCST (SCSI Target Subsystem for Linux) framework was proposed to implement the iSCSI transport protocol within the Linux kernel. Using SCST, users can configure a Linux system as an iSCSI target, allowing other computer devices to access storage resources on the Linux system using the iSCSI protocol.

[0045] SCST primarily relies on the Linux TCP / IP network protocol stack for data transmission. SCST can be divided into three layers: the SCST target driver layer, the SCST layer, and the device interface layer. The SCST target driver layer is primarily responsible for receiving and responding to iSCSI messages. This layer contains two types of I / O threads: the iscsird thread and the iscsiwd thread. The iscsird thread is responsible for receiving network messages and pre-processing iSCSI commands, while the iscsiwd thread is responsible for responding to command execution results. The SCST layer is primarily responsible for executing SCSI commands and creates multiple processes based on the type of SCSI command. For example, the SCSI I / O thread handles read and write SCSI commands, while the SCSI TM thread handles management SCSI commands. The SCST MGMT is responsible for creating and destroying sessions. During the SCST data transmission process, when a data frame arrives at a computer device, it writes it to an allocated ring buffer queue using direct memory access. The device then notifies the CPU of the data arrival via a hard interrupt. The CPU then responds to the hard interrupt, sets a register, initiates a soft interrupt request, and performs a context switch. Frequent context switching leads to insufficient resources like I / O and memory, resulting in lock contention. Furthermore, this processing requires multiple memory copies of data. Therefore, in high-performance applications, SCST faces numerous challenges. For example, interrupt handling can affect the responsiveness and performance stability of computer devices. Context switching requires saving information such as the current task's register state, program counter, and stack pointer, causing CPU cache invalidation. Memory copies increase memory bandwidth usage and CPU burden, limiting data transfer efficiency. Lock contention can reduce computer system throughput, increase response time, and even cause system deadlock. In summary, SCST in high-performance applications can impact computer device performance, specifically the performance of the iSCSI target.

[0046] In view of this, the present application proposes a data processing method, in which the server first receives a data processing request and allocates a target cache queue to the data processing request. Then, the data processing request is parsed by the target processing thread corresponding to the target cache queue to obtain a data packet. Then, the data packet is processed by the target processing thread according to the target cache queue to respond to the data processing request. Since different cache queues correspond to different processing threads, when the target processing thread processes the data packet included in the data processing request, there is no need to initiate an interrupt or perform context switching, which reduces the unnecessary memory occupied by the context switching and avoids the impact of a large number of interrupts on the stability of the server. Moreover, since the target processing thread adopts a serial processing method, there is no need to lock the processing resources, which avoids lock contention and reduces the overhead of switching locks. This can improve system stability while reducing memory occupancy, improving cache utilization and response efficiency to data processing requests, thereby improving server performance.

[0047] The data processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the server 102 communicates with the terminal 104 through the network. The data storage system can store the data that the server 102 needs to process. The data storage system can be integrated on the server 102, or it can be placed on the cloud or other network servers. The server 102 can receive a data processing request and assign a target cache queue to the data processing request. Then, the target processing thread corresponding to the target cache queue parses the data processing request to obtain a data packet. Different cache queues correspond to different processing threads. Then, the target processing thread processes the packet according to the target cache queue to respond to the data processing request. Among them, the terminal 104 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 102 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0048] In an exemplary embodiment, Figure 2 As shown, a data processing method is provided, which is applied to Figure 1 Taking the server in FIG. 1 as an example, the process includes the following steps 201 to 203:

[0049] Step 201: Receive a data processing request and determine a target cache queue corresponding to the data processing request.

[0050] A data processing request is a data frame sent by a client to a server via a network request. The server responds to the data processing request by processing the data frame and returning the processing result to the client. For example, a data processing request can be a data query request, a data read request, or a data storage request.

[0051] The target cache queue is a storage space in memory used to temporarily store data processing requests, enabling faster access and reducing the number of accesses to the primary storage system. The target cache queue allows for sequential storage of received data processing requests and for each request to be read and processed in the order in which they were stored. It is understood that the server may pre-set multiple cache queues in memory to store multiple data processing requests to improve response speed. For example, the cache queue may be a ring cache queue.

[0052] It is understood that the receiving queue corresponding to the network card in the server can be used to store the received data processing request, and then the cache queue corresponding to the data processing request is determined based on the correspondence between the data packet in the data processing request and the preset cache queue. In this embodiment, the cache queue is determined as the target cache queue.

[0053] In this embodiment, the server can receive data processing requests sent by other devices through the network card's receive queue and determine the target cache queue corresponding to the currently received data processing request based on the number of processing requests already stored in each cache queue, or based on the correspondence between cache queues and network card queues. The server then binds the target cache queue to the data processing request, establishing a correspondence between the data processing request and the target cache queue. For example, the server can determine the cache queue with the least number of currently stored processing requests as the target cache queue.

[0054] Step 202: The data processing request is parsed by the target processing thread corresponding to the target cache queue to obtain a data packet. Different cache queues correspond to different processing threads.

[0055] The target processing thread is a processing thread for processing data processing requests. It should be noted that, in this embodiment, in order to avoid problems such as interruptions and context switching during the data processing request process, multiple processing threads can be set, and each processing thread can be assigned to each cache queue, so that multiple processing threads can concurrently execute the processing threads in the corresponding cache queue, wherein the processing thread can be a polling thread for periodically determining whether there is a new data processing request assigned to the cache queue. wherein each processing thread can share system resources and correspond to its own register environment, call stack, etc.

[0056] It should be noted that each stage of responding to a data processing request can be handled serially by the target processing thread, thereby avoiding lock contention and the resource overhead of opening and closing locks. Furthermore, in scenarios with high concurrency of data processing requests, this can avoid a large number of CPU interrupts, thereby improving system stability and responsiveness. For multiple processing threads included in the server, simple locks can be used to lock resources.

[0057] A data packet refers to the data carried in a data processing request. A data processing request may include a request line, Ethernet frame header, IP header, TCP header, and data packet. The request line includes the request type, requested resource path, and transport protocol version and type. The message header contains the sender and receiver of the data processing request. The data packet contains the actual data to be processed.

[0058] For example, Figure 3 As shown, the cache queues correspond one to one with the processing threads, for example, Figure 3 Cache queue 1 corresponds to processing thread 1, and cache queue N corresponds to processing thread N. Processing thread 1 is used to process data processing requests stored in cache queue 1, and data processing requests in cache queue 1 cannot be processed by other processing threads. For example, cache queue N cannot access and process data processing requests in cache queue 1.

[0059] In this embodiment, the sending queue corresponding to the network card in the server can send the data processing request stored in the receiving queue to the target cache queue. After the data processing request is assigned to the target cache queue, the server can use the target processing thread corresponding to the target cache queue to parse the data processing request, strip the Ethernet frame header, IP header, and TCP header in the data processing request, obtain the parsing result, and obtain the data packet from the parsing result.

[0060] As a possible implementation, the network card may correspond to multiple network card queues, each network card queue corresponds to at least one processing thread, and the processing thread can identify the card slot corresponding to the network card to determine the correspondence between the processing thread and the network card.

[0061] Step 203: Process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0062] It is understandable that, when the target processing thread processes the data packet, different types of data processing requests correspond to different data packets, and the target processing thread can process the data packet accordingly according to the type of data processing request.

[0063] In this embodiment, when the server obtains a data packet, it can use the target processing thread to process the data packet in the target cache queue, and then return the processing result to the target cache queue.

[0064] In the above data processing method, the server first receives a data processing request and determines the target cache queue corresponding to the data processing request. Then, the data processing request is parsed by the target processing thread corresponding to the target cache queue to obtain a data packet. Then, the data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request. Since different cache queues correspond to different processing threads, when the data packet included in the data processing request is processed by the target processing thread, there is no need to initiate an interrupt or perform context switching, which reduces the unnecessary memory occupied by the context switching and avoids the impact of a large number of interrupts on the stability of the server. Moreover, since the target processing thread adopts a serial processing method, there is no need to lock the processing resources, which avoids lock contention and reduces the overhead of switching locks. This can improve system stability while reducing memory usage, improving cache utilization and response efficiency to data processing requests, thereby improving server performance.

[0065] In one embodiment, based on Figure 2 The embodiment shown, see Figure 4 , this embodiment relates to the process of the server processing the data packet according to the target cache queue through the target processing thread. Figure 4 As shown, step 203 includes Figure 4 Steps 401 and 402 are shown.

[0066] Step 401, copy the data packet to the physical memory space corresponding to the target cache queue, and determine the second logical address according to the first logical address corresponding to the target cache queue, the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address.

[0067] Among them, the physical memory space refers to the actual memory space corresponding to the virtual memory space where the target cache queue is located. It can be understood that the physical memory space can be the actual storage space on the storage device, and the data in the physical memory space can be stored for a long time. Therefore, the data in the data processing request can be stored in the physical memory space.

[0068] The first logical address is the storage address of the target cache queue in the virtual memory space, and the second logical address is the logical address of the virtual memory space to be accessed during data request processing. To avoid the situation where the physical memory space corresponding to the target cache queue differs from the physical memory space corresponding to the data request processing, which requires copying the data packet from one memory space to another, in this embodiment, the physical memory space corresponding to the first logical address and the physical memory space corresponding to the second logical address are set to the same physical memory space.

[0069] It should be noted that, in an embodiment of the present application, a block of physical memory can be pre-allocated as the memory space corresponding to the data processing request, so that the input / output operations of the processing process can be completed in the physical memory without the need for additional data copying, thereby achieving zero memory copy between multiple processes. In actual applications, the server can map the physical memory to different virtual memory spaces. For example, the physical memory can be mapped to the virtual memory space corresponding to the user-mode process, or the physical memory can be mapped to the virtual memory space corresponding to the kernel-mode process. The virtual memory space can include the virtual memory space corresponding to the first logical address, or the virtual memory space corresponding to the second logical address.

[0070] For example, different logical addresses correspond to the same physical memory space as shown in the following diagram: Figure 5 As shown, Figure 5 In the figure, the same physical memory space can be mapped to the virtual memory space 1 corresponding to process 1, and can also be mapped to the virtual memory space 2 corresponding to process 2, and the offset address of virtual memory space 1 is the same as the offset address of virtual memory space 2. Therefore, the physical memory space accessed by process 1 and process 2 is the same physical memory.

[0071] In this embodiment, before processing the data packet, the server can copy the data packet in the target cache queue to the physical memory space corresponding to the target cache queue through the network card, and then, based on the fact that the first logical address corresponding to the target cache queue is the same as the physical memory space corresponding to the second logical address, convert the first logical address to obtain the physical address of the physical memory space, and then determine the second logical address based on the physical address.

[0072] Step 402: Process the data packet in the physical memory space corresponding to the second logical address through the target processing thread.

[0073] In this embodiment, after determining the second logical address, the server can access the physical memory space storing the data packet, that is, the physical memory space corresponding to the second logical address, through the target processing thread to perform corresponding data processing on the data packet.

[0074] In this embodiment, the server first copies the data packet to the physical memory space corresponding to the target cache queue, and determines the second logical address based on the first logical address corresponding to the target cache queue. Then, the data packet is processed in the physical memory space corresponding to the second logical address through the target processing thread. Since the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address, the physical memory space accessed by the server when processing the data packet is the same as the physical memory space where the data packet is stored, thereby avoiding the need to copy the data packet from one physical memory space to another physical memory space, which increases memory usage, thereby reducing memory usage and improving data transmission efficiency.

[0075] In one embodiment, based on Figure 4 The embodiment shown in FIG. 1 is directed to a process in which a server determines a second logical address based on a first logical address corresponding to a target cache queue. The server implements this process by having a target processing thread determine an offset address based on the first logical address and a base address corresponding to the target cache queue, and then determine the second logical address based on the offset address and a pre-assigned base address.

[0076] The base address refers to the address of the starting location of the memory segment used to store data in the physical memory space, and the offset address refers to the offset of the ending location of the memory segment relative to the base address. The pre-allocated base address refers to the base address of the memory space for data processing allocated to the target processing thread based on the memory mapping relationship when the target processing thread is created.

[0077] It can be understood that since the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address, the offset address of the physical memory space corresponding to the first logical address and the offset address of the physical memory space corresponding to the second logical address are also the same.

[0078] For example, if vAddr1 is the first logical address mapped by process 1, gVMapStart1 is the starting address of process 1, vAddr2 is the second logical address mapped by process 2, and gVMapStart2 is the starting address of process 2, then the corresponding offset address Offset1 is: Offset1=vAddr1-gVMapStart1; the second logical address can be determined according to offset address 1: vAddr2=gVMapStart2+offset1.

[0079] In this embodiment, the server can obtain the base address corresponding to the target cache queue and the pre-allocated base address through the target processing thread, and then determine the offset address based on the first logical address and the base address according to the address calculation method, and then determine the second logical address based on the offset address and the pre-allocated base address.

[0080] In this embodiment, the server determines the offset address through the target processing thread based on the first logical address and the base address corresponding to the target cache queue, and then determines the second logical address based on the offset address and the pre-allocated base address. Since the first logical address and the second logical address correspond to the same physical memory space, the second logical address can be quickly determined based on the offset address and the pre-allocated base address, thereby improving the efficiency of determining the second logical address.

[0081] In one embodiment, based on Figure 2 The embodiment shown, see Figure 6 This embodiment relates to a process in which the server processes a data packet in the physical memory space corresponding to the second logical address through a target processing thread. Figure 6 As shown, step 402 includes Figure 6 Steps 601 and 602 are shown.

[0082] Step 601: parse the data packet through the target processing thread to obtain a parsing result, which at least includes a data processing command.

[0083] Data processing commands refer to instructions and operations used during data processing. For example, data processing commands may include data storage commands, data update commands, data read commands, data query commands, and data delete commands. It is understood that some data processing commands require processing the data included in the data processing request, while others require processing data stored on the server. Therefore, the parsing results may include data processing commands, or they may also include data processing commands and the data to be processed.

[0084] In this embodiment, the server may parse the data packet according to the encapsulation format of the data packet through the target processing thread to obtain a parsing result.

[0085] Step 602: Process the data packet in the physical memory space according to the analysis result.

[0086] In this embodiment, the server may determine the processing type corresponding to the data processing request according to the content of the parsing result, and thus process the data packet according to the processing type in the physical memory space.

[0087] In this embodiment, the server parses the data packet through the target processing thread and can obtain the parsing result. Then, the data packet is processed according to the parsing result. Since the parsing result includes at least the data processing command, the server can determine the request type corresponding to the data processing request according to the data processing command, and thus can respond to the data processing request according to the request type, avoiding the problem that the data processing operation does not match the data processing request, resulting in operational errors, and thus can improve the processing performance of the server in high-concurrency data processing request scenarios.

[0088] Based on the above embodiment, the parsing result obtained by the server when parsing the data packet may include a data processing command or a data processing command and data to be processed. The following describes the server's processing process for different parsing results respectively.

[0089] In one embodiment, based on Figure 6 The embodiment shown relates to a process in which a first server processes a data packet based on a parsing result: if the parsing result includes a first data processing command and data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in a first physical storage device.

[0090] The first data processing command refers to a data processing command that requires data storage. It is understandable that if the parsing result includes the first data processing command and the data to be processed, the server can determine that the data processing command is of a type that requires processing the data included in the data processing request, for example, a data write command. The physical storage device refers to a device that stores the data to be processed, for example, a random access memory, a cache memory, a magnetic disk, a solid-state drive, an optical disk, etc. In this embodiment, the physical storage device that stores the data to be processed corresponding to the first data processing command is used as the first data processing command.

[0091] In this embodiment, when the parsing result includes a first data processing command and data to be processed, the server can determine that the data processing request requires corresponding operations to be performed on the data to be processed, respond to the first data processing command through the target processing thread, and store the data to be processed in the first physical storage device.

[0092] In this embodiment, when the parsing result includes a first data processing command and data to be processed, the server can quickly determine the request type corresponding to the data processing request, and then respond to the first data processing command through the target processing thread, and can store the data to be processed in the first physical storage device, thereby quickly responding to the data processing request and improving the server's response speed to the data processing request.

[0093] In one embodiment, based on Figure 6 The embodiment shown relates to a process in which the second server processes a data packet based on the parsing result: if the parsing result includes a second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0094] The second data processing command refers to a data processing command that does not require data storage and processes already stored data. It is understood that if the parsing result includes the second data processing command, the server can determine that the data processing command is of a type that requires processing data stored in the server. The second physical storage device refers to a storage device that stores data corresponding to the second data processing command. In this embodiment, the data corresponding to the second data processing command stored in the second physical storage device is used as the target data.

[0095] In this embodiment, when the parsing result includes a second data processing command, the server can determine that the data processing request requires corresponding operations on the data to be processed, and respond to the second data processing command through the target processing thread, then read the target data from the second physical storage device, store the target data in the physical memory space, and then process the target data.

[0096] In this embodiment, when the parsing result includes the second data processing command, the server can quickly determine the request type corresponding to the data processing request, and then respond to the second data processing command through the target processing thread, and can process the target data read from the second physical storage device in the physical memory space, thereby being able to quickly respond to the data processing request, thereby improving the server's response speed to the data processing request.

[0097] In one embodiment, based on Figure 3 The embodiment shown in this embodiment relates to a process in which the server determines the working frequency corresponding to the target processing thread before processing the data packet through the target processing thread: determining the processing frequency corresponding to the target processing thread, and the processing frequency is related to the number of data processing requests currently received.

[0098] It is understandable that the number of data processing requests received by the server in different time periods is different, and the corresponding business processing status is also different. For example, if the number of businesses shows a linear growth trend, it can be determined that it is in a busy business state; if the number of businesses shows an exponential decrease trend, it can be determined that it is in an idle business state. Correspondingly, when the server is in a busy business state, the processing frequency of the processing thread can be reduced. For example, the poll mechanism can be used to extend the timeout period or increase the polling interval to reduce the response speed of the processing thread, thereby reducing the CPU usage; when the server is in an idle business state, the processing frequency of the processing thread can be increased. For example, the full load mode can be used to increase the response speed of the processing thread. Therefore, the load balancing of the server can be achieved by dynamically adjusting the working frequency of the processing thread.

[0099] The processing frequency refers to the frequency at which the server responds to data processing commands via a processing thread per unit time. In this embodiment, the server can determine the processing frequency of the current processing thread based on the number of data processing requests received during the current time period, and then determine this processing frequency as the processing frequency corresponding to the target processing thread. The server can then process the data packets in the physical memory space via the target processing thread according to the processing frequency.

[0100] In this embodiment, the server determines the processing frequency corresponding to the target processing thread, and can process the data packet in the physical memory space according to the processing frequency through the target processing thread. Since the processing frequency is related to the number of data processing requests currently received, the processing frequency can be adjusted in time according to the number of data processing requests currently received, so that the processing frequency can match the current processing resources of the server, thereby realizing dynamic load balancing of the server, and then achieving the goal of low power consumption operation of the server in idle scenarios while ensuring high performance.

[0101] In one embodiment, a data processing method is provided for use in a server, such as Figure 7 As shown, the method includes the following steps:

[0102] Step 701: Receive a data processing request and determine a target cache queue corresponding to the data processing request.

[0103] Step 702: parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet.

[0104] Step 703: The data packet is copied to the physical memory space corresponding to the target cache queue through the network card, and the target processing thread determines the offset address according to the first logical address and the base address corresponding to the target cache queue.

[0105] Step 704: Determine a second logical address according to the offset address and the pre-assigned base address.

[0106] Step 705: Determine the processing frequency corresponding to the target processing thread. The processing frequency is related to the number of currently received data processing requests.

[0107] Step 706 , parsing the data packet through the target processing thread according to the processing frequency to obtain a parsing result, which at least includes a data processing command.

[0108] Step 707: If the parsing result includes the first data processing command and the data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in the first physical storage device.

[0109] Step 708: If the parsing result includes the second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0110] Step 709: Return the processing result to the target cache queue through the target processing thread.

[0111] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0112] Based on the same inventive concept, the present application also provides a data processing device for implementing the aforementioned data processing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations in one or more data processing device embodiments provided below can be found in the above-mentioned limitations on the data processing method and will not be repeated here.

[0113] In an exemplary embodiment, Figure 8 As shown, a data processing device is provided, comprising:

[0114] The receiving module 801 is configured to receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0115] Parsing module 802, used to parse the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet. Different cache queues correspond to different processing threads;

[0116] The processing module 803 is configured to process the data packet according to the target processing thread and the target cache queue to respond to the data processing request.

[0117] In one embodiment, the processing module 803 includes:

[0118] a determining unit, configured to copy the data packet to a physical memory space corresponding to a target cache queue through a target processing thread, and determine a second logical address based on a first logical address corresponding to the target cache queue, wherein the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0119] The processing unit is configured to process the data packet in the physical memory space corresponding to the second logical address through a target processing thread.

[0120] In one embodiment, the determining unit is specifically configured to:

[0121] The target processing thread determines an offset address according to the first logical address and a base address corresponding to the target cache queue, and determines a second logical address according to the offset address and a pre-assigned base address.

[0122] In one embodiment, the processing unit is specifically configured to:

[0123] Parsing the data packet through the target processing thread to obtain a parsing result, the parsing result at least including a data processing command;

[0124] Based on the parsing results, the data packet is processed in the physical memory space.

[0125] In one embodiment, the processing unit is specifically configured to:

[0126] If the parsing result includes the first data processing command and the data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in the first physical storage device.

[0127] In one embodiment, the processing unit is specifically configured to:

[0128] If the parsing result includes the second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0129] In one embodiment, the apparatus further comprises:

[0130] The determination module is used to determine the processing frequency corresponding to the target processing thread, and the processing frequency is related to the number of data processing requests currently received.

[0131] The processing unit is specifically configured to process the data packet in the physical memory space through the target processing thread according to the processing frequency.

[0132] Each module in the above-mentioned data processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the server in hardware form, or can be stored in the memory of the server in software form, so that the processor can call and execute the corresponding operations of each module.

[0133] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data to be processed. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0134] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0135] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0136] Receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0137] The target processing thread corresponding to the target cache queue parses the data processing request to obtain the data packet. Different cache queues correspond to different processing threads.

[0138] The data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request.

[0139] In one embodiment, when the processor executes the computer program, the processor specifically implements the following steps:

[0140] Copy the data packet to the physical memory space corresponding to the target cache queue, and determine the second logical address according to the first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0141] The data packet is processed in the physical memory space corresponding to the second logical address by the target processing thread.

[0142] In one embodiment, when the processor executes the computer program, the processor specifically implements the following steps:

[0143] The target processing thread determines an offset address according to the first logical address and a base address corresponding to the target cache queue, and determines a second logical address according to the offset address and a pre-assigned base address.

[0144] In one embodiment, when the processor executes the computer program, the processor specifically implements the following steps:

[0145] Parsing the data packet through the target processing thread to obtain a parsing result, the parsing result at least including a data processing command;

[0146] Based on the parsing results, the data packet is processed in the physical memory space.

[0147] In one embodiment, when the processor executes the computer program, the processor specifically implements the following steps:

[0148] If the parsing result includes the first data processing command and the data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in the first physical storage device.

[0149] In one embodiment, when the processor executes the computer program, the processor specifically implements the following steps:

[0150] If the parsing result includes the second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0151] In one embodiment, when the processor executes the computer program, the processor specifically implements the following steps:

[0152] Determine the processing frequency corresponding to the target processing thread, which is related to the number of data processing requests currently received;

[0153] According to the processing frequency, the data packets are processed in the physical memory space through the target processing thread.

[0154] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0155] Receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0156] The target processing thread corresponding to the target cache queue parses the data processing request to obtain the data packet. Different cache queues correspond to different processing threads.

[0157] The data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request.

[0158] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0159] Copy the data packet to the physical memory space corresponding to the target cache queue, and determine the second logical address according to the first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0160] The data packet is processed in the physical memory space corresponding to the second logical address by the target processing thread.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0162] The target processing thread determines an offset address according to the first logical address and a base address corresponding to the target cache queue, and determines a second logical address according to the offset address and a pre-assigned base address.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0164] Parsing the data packet through the target processing thread to obtain a parsing result, the parsing result at least including a data processing command;

[0165] Based on the parsing results, the data packet is processed in the physical memory space.

[0166] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0167] If the parsing result includes the first data processing command and the data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in the first physical storage device.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0169] If the parsing result includes the second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0170] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0171] Determine the processing frequency corresponding to the target processing thread, which is related to the number of data processing requests currently received;

[0172] According to the processing frequency, the data packets are processed in the physical memory space through the target processing thread.

[0173] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0174] Receive a data processing request and determine a target cache queue corresponding to the data processing request;

[0175] The target processing thread corresponding to the target cache queue parses the data processing request to obtain the data packet. Different cache queues correspond to different processing threads.

[0176] The data packet is processed according to the target processing thread and the target cache queue to respond to the data processing request.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0178] Copy the data packet to the physical memory space corresponding to the target cache queue, and determine the second logical address according to the first logical address corresponding to the target cache queue, where the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address;

[0179] The data packet is processed in the physical memory space corresponding to the second logical address by the target processing thread.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0181] The target processing thread determines an offset address according to the first logical address and a base address corresponding to the target cache queue, and determines a second logical address according to the offset address and a pre-assigned base address.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0183] Parsing the data packet through the target processing thread to obtain a parsing result, the parsing result at least including a data processing command;

[0184] Based on the parsing results, the data packet is processed in the physical memory space.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0186] If the parsing result includes the first data processing command and the data to be processed, the target processing thread responds to the first data processing command and stores the data to be processed in the first physical storage device.

[0187] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0188] If the parsing result includes the second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0190] Determine the processing frequency corresponding to the target processing thread, which is related to the number of data processing requests currently received;

[0191] According to the processing frequency, the data packets are processed in the physical memory space through the target processing thread.

[0192] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0193] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A data processing method, characterized in that: The method comprises: receiving a data processing request and determining a target cache queue corresponding to the data processing request; Parsing the data processing request through the target processing thread corresponding to the target cache queue to obtain a data packet, where different cache queues correspond to different processing threads; The data packet is copied to the physical memory space corresponding to the target cache queue, the target processing thread determines the offset address according to the first logical address corresponding to the target cache queue and the base address corresponding to the target cache queue, and the second logical address is determined according to the offset address and the pre-allocated base address, the target processing thread processes the data packet in the physical memory space corresponding to the second logical address to respond to the data processing request, and the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address.

2. The method according to claim 1, characterized in that The processing of the data packet in the physical memory space corresponding to the second logical address by the target processing thread includes: Parsing the data packet through the target processing thread to obtain a parsing result, wherein the parsing result at least includes a data processing command; According to the analysis result, the data packet is processed in the physical memory space.

3. The method according to claim 2, characterized in that The processing of the data packet in the physical memory space according to the parsing result includes: If the parsing result includes a first data processing command and data to be processed, storing the data to be processed in a first physical storage device in response to the first data processing command by the target processing thread; If the parsing result includes a second data processing command, the target processing thread responds to the second data processing command, reads the target data from the second physical storage device, and processes the target data in the physical memory space.

4. The method according to claim 1, wherein Before processing the data packet in the physical memory space corresponding to the second logical address by the target processing thread, the method further includes: Determining a processing frequency corresponding to the target processing thread, where the processing frequency is related to the number of currently received data processing requests; The processing of the data packet in the physical memory space corresponding to the second logical address by the target processing thread includes: The data packet is processed in the physical memory space by the target processing thread according to the processing frequency.

5. The method according to claim 1, wherein The base address is a base address allocated to the target processing thread according to a memory mapping relationship when the target processing thread is created.

6. The method according to claim 1, characterized in that The offset address of the physical memory space corresponding to the first logical address is the same as the offset address of the physical memory space corresponding to the second logical address.

7. A data processing device, characterized in that: The device comprises: A receiving module, configured to receive a data processing request and allocate a target cache queue to the data processing request; A parsing module, configured to parse the data processing request through a target processing thread corresponding to the target cache queue to obtain a data packet, wherein different cache queues correspond to different processing threads; A processing module is used to copy the data packet to the physical memory space corresponding to the target cache queue, determine the offset address through the target processing thread according to the first logical address corresponding to the target cache queue and the base address corresponding to the target cache queue, and determine the second logical address according to the offset address and the pre-assigned base address, and process the data packet in the physical memory space corresponding to the second logical address through the target processing thread to respond to the data processing request, and the physical memory space corresponding to the first logical address is the same as the physical memory space corresponding to the second logical address.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images