Data processing method and device and computer equipment

Abstract

The invention relates to a data processing method and device and computer equipment, and relates to the technical field of databases. The method comprises the steps that under the condition that a data processing request sent by a client is received, a target main thread determines to-be-processed data based on the data processing request, current data blocks matched with a first preset number are determined in the to-be-processed data, and sub-threads are created; executing a data processing action matched with the type of the data processing request on the current data block through the sub-thread; and after the sub-threads complete processing of the current data blocks, the target main thread takes the remaining to-be-processed data as new to-be-processed data, and the steps of determining the current data blocks matched with the first preset number in the to-be-processed data and creating the sub-threads are continuously executed until the corresponding sub-threads complete processing of the current data blocks. And if no data to be processed is left, returning the processing result to the client. By adopting the method, the problem of performance bottleneck in large-scale data operation can be solved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data processing method, device and computer equipment. Background Art

[0002] In the field of modern database technology, performance and response speed are key factors affecting system availability and user experience. Redis (Remote Dictionary Server) is a high-performance in-memory database that is widely used in scenarios such as caching, message queues, and real-time data processing. With the explosive growth of data volume and the complexity of application scenarios, traditional database operation methods face huge challenges in processing large-scale data sets. Large-scale data operations may cause long-term system congestion, which significantly affects the overall performance and response speed of the system, especially in applications with high real-time and concurrency requirements. Summary of the invention

[0003] Based on this, it is necessary to provide a data processing method, device and computer equipment to solve the performance bottleneck problem in large-scale data operations in response to the above technical problems.

[0004] In a first aspect, the present application provides a data processing method, which is applied to a Redis server, and the method includes:

[0005] In the case of receiving a data processing request sent by a client, the target main thread determines the data to be processed based on the data processing request, determines the current data blocks matching the first preset number in the data to be processed, and creates a child thread;

[0006] Execute a data processing action matching the type of the data processing request on the current data block through a child thread;

[0007] After the child thread completes processing the current data block, the target main thread takes the remaining data to be processed as new data to be processed, continues to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

[0008] In one embodiment, multiple sub-threads are created; and a data processing action matching the type of the data processing request is executed on the current data block by the sub-threads, including:

[0009] Split the current data block into multiple sub-data blocks;

[0010] Data processing actions matching the types of data processing requests are respectively performed on the multiple sub-data blocks through the multiple sub-threads.

[0011] In one embodiment, after the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, including:

[0012] Get the processing progress of multiple sub-threads;

[0013] After the processing progress of any sub-thread indicates that the processing of the corresponding sub-data block is completed, the thread resources of the corresponding sub-thread are released;

[0014] After the sub-threads matching the second preset number indicate that the corresponding sub-data blocks have been processed, the target main thread uses the remaining data to be processed as new data to be processed.

[0015] In one embodiment, performing a data processing action matching the type of the data processing request on the current data block through a child thread includes:

[0016] When the data processing request is of insert type, each data is inserted into the Redis server according to the position of each data in the current data block through the child thread;

[0017] When the data processing request is of the modification type, the child thread replaces the data in the current data block with the data at the corresponding position in the Redis server according to the position of each data in the current data block;

[0018] When the data processing request is of the delete type, the child thread deletes the data from the Redis server according to the position of each data in the current data block;

[0019] When the data processing request is of query type, the Redis server is queried through the child thread based on the position of each data in the current data block.

[0020] In one embodiment, the data processing method further includes:

[0021] When receiving a target data processing request sent by a target client, adding the target data processing request to a request queue;

[0022] When the data processing request is processed, the target main thread takes out the target data processing request from the request queue, determines the target data to be processed based on the target data processing request, determines the target data blocks matching the first preset number in the target data to be processed, and creates a target sub-thread;

[0023] Performing a data processing action matching the type of the target data processing request on the target data block through the target subthread;

[0024] After the target sub-thread completes processing the target data block, the target main thread takes the remaining target data to be processed as new target data to be processed, continues to execute the steps of determining the target data blocks that match the first preset number in the target data to be processed, and creates a target sub-thread, until the corresponding target sub-thread completes processing the target data block and there is no remaining target data to be processed, and returns the target processing result to the target client.

[0025] In one embodiment, the data processing method further includes:

[0026] The target main thread persists the processing status information of each child thread.

[0027] In a second aspect, the present application further provides a data processing device, applied to a Redis server, the device comprising:

[0028] A processing module, configured to, upon receiving a data processing request sent by a client, determine, by a target main thread, data to be processed based on the data processing request, determine a current data block matching a first preset number in the data to be processed, and create a child thread;

[0029] An execution module, used for executing a data processing action matching the type of the data processing request on the current data block through a sub-thread;

[0030] A loop processing module is used to, after the child thread completes processing the current data block, have the target main thread use the remaining data to be processed as new data to be processed, continue to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] In the case of receiving a data processing request sent by a client, the target main thread determines the data to be processed based on the data processing request, determines the current data blocks matching the first preset number in the data to be processed, and creates a child thread;

[0033] Execute a data processing action matching the type of the data processing request on the current data block through a child thread;

[0034] After the child thread completes processing the current data block, the target main thread takes the remaining data to be processed as new data to be processed, continues to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

[0035] In the above-mentioned data processing method, device and computer equipment, the Redis server includes a target main thread for data processing. When the Redis server receives a data processing request sent by a client, the target main thread determines the data to be processed based on the data processing request, determines the current data block matching the first preset number in the data to be processed, and creates a child thread; the child thread performs a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, continues to perform the steps of determining the current data block matching the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client. This method of decomposing large-scale data operations into multiple small, continuous data processing steps and gradually executing each small step reduces the load of single data processing, avoids long-term system blocking, and improves data processing efficiency; through the main and child thread asynchronous processing mechanism, the data processing load of the target main thread is dispersed, so that the system can maintain efficient operation in a high-concurrency environment, thereby improving the overall performance of the system.

[0036] In a fourth aspect, the present application further provides a data processing method, which is applied to a client, and the method includes:

[0037] A data processing request is sent to a Redis server; the data processing request is used to instruct the Redis server that upon receiving the data processing request, the target main thread of the Redis server determines the data to be processed based on the data processing request, determines the current data block that matches the first preset number in the data to be processed, and creates a child thread; the child thread is used to perform a data processing action that matches the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block that matches the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client;

[0038] Receive the target processing result sent by the Redis server.

[0039] In a fifth aspect, the present application further provides a data processing device, applied to a client, the device comprising:

[0040] A sending module, used for sending a data processing request to a Redis server; the data processing request is used to instruct the Redis server that, upon receiving the data processing request, the target main thread of the Redis server determines the data to be processed based on the data processing request, determines the current data block that matches the first preset number in the data to be processed, and creates a child thread; the child thread is used to perform a data processing action that matches the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block that matches the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client;

[0041] The receiving module is used to receive the target processing result sent by the Redis server.

[0042] In a sixth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0043] A data processing request is sent to a Redis server; the data processing request is used to instruct the Redis server that upon receiving the data processing request, the target main thread of the Redis server determines the data to be processed based on the data processing request, determines the current data block that matches the first preset number in the data to be processed, and creates a child thread; the child thread is used to perform a data processing action that matches the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block that matches the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client;

[0044] Receive the target processing result sent by the Redis server.

[0045] In the above-mentioned data processing method, apparatus and computer equipment, when data processing is required, the client sends a data processing request to the Redis server, thereby instructing the Redis server to determine the data to be processed based on the data processing request by the target main thread of the Redis server, determine the current data block matching the first preset number in the data to be processed, and create a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, and continue to execute the current data block matching the first preset number in the data to be processed, And create a child thread step until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client; the client receives the target processing result sent by the Redis server. After the client's data processing request is sent to the Redis server, the Redis server decomposes the large-scale data operation into multiple small, continuous data processing steps, and gradually executes each small step. The method reduces the load of a single data processing, avoids long-term system blocking, and improves data processing efficiency; through the main and child thread asynchronous processing mechanism, the data processing load of the target main thread is dispersed, so that the system can maintain efficient operation in a high concurrency environment, thereby improving the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] FIG1 is an application environment diagram of a data processing method in one embodiment;

[0048] FIG2 is a schematic flow chart of a data processing method in one embodiment;

[0049] FIG3 is a schematic flow chart of a data processing method in another embodiment;

[0050] FIG4 is a schematic flow chart of a data processing method in yet another embodiment;

[0051] FIG5 is a block diagram of a data processing device in one embodiment;

[0052] FIG6 is a block diagram of a data processing device in another embodiment;

[0053] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The data processing method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1 . Among them, the client 102 interacts with the Redis server (remote dictionary server) 104 for data. The Redis server 104 includes a target main thread for data processing. When the Redis server 104 receives a data processing request sent by the client 102, the target main thread determines the data to be processed based on the data processing request, determines the current data block that matches the first preset number in the data to be processed, and creates a child thread; the child thread performs a data processing action that matches the type of the data processing request on the current data block; after the child thread completes the processing of the current data block, the target main thread uses the remaining data to be processed as the new data to be processed, and continues to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and creating a child thread, until the corresponding child thread completes the processing of the current data block, and there is no remaining data to be processed, and returns the processing result to the client 102. The client 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0056] In an exemplary embodiment, as shown in FIG2 , a data processing method is provided, which is described by taking the method applied to the Redis server 104 in FIG1 as an example, and includes the following steps 202 to 206. Among them:

[0057] Step 202, when receiving a data processing request sent by the client, the target main thread determines the data to be processed based on the data processing request, determines the current data blocks matching the first preset number in the data to be processed, and creates a child thread.

[0058] The client refers to an application or terminal that issues a data processing request. The client sends a data processing request to the Redis server, and the data processing request is used to request to insert, delete, modify or query multiple elements in the Redis server.

[0059] The target main thread refers to the thread used for data processing in the Redis server. When the Redis server receives a data processing request, it assigns the request to the target main thread. In the traditional method, the target main thread processes the data processing request. However, this method has a performance bottleneck problem when processing large-scale data.

[0060] The data to be processed refers to the data that needs to be processed as indicated by the data processing request. For example, if the data processing request is used to request to insert data, the data to be processed is the data that needs to be inserted into the Redis server; if the data processing request is used to request to delete data, the data to be processed is the data that needs to be deleted in the Redis server; if the data processing request is used to request to modify data, the data to be processed is the data used to modify the data in the Redis server; if the data processing request is used to request to query data, the data to be processed is the data used to query data in the Redis server.

[0061] The current data block refers to a plurality of data selected from the data to be processed, and the amount of data included in the current data block may be a first preset amount, which may be flexibly adjusted according to actual conditions.

[0062] The child thread is a thread created by the target main thread and is used to process the current data block. By creating a child thread to process data, the burden of the target main thread is effectively reduced, which is conducive to improving the overall performance of the system.

[0063] Step 204: execute a data processing action matching the type of the data processing request on the current data block through the child thread.

[0064] The types of data processing requests include insert type, delete type, modify type, and query type. The child thread processes the current data block, and the adopted data processing action can be determined according to the type of data processing request. For example, if the data processing request is of insert type, the data processing action can be to insert the current data block into the Redis server.

[0065] Step 206, after the child thread completes processing the current data block, the target main thread takes the remaining data to be processed as new data to be processed, continues to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

[0066] Among them, the child thread can record the processing progress of the current data block, and determine whether the current data block has been processed according to the processing progress. When the current data block is processed, the current data block is used as the processed data, and the remaining data to be processed refers to the data remaining after the processed data is removed from the data to be processed. The target main thread uses the remaining data to be processed as new data to be processed and continues to execute the data processing process in step 202 and step 204. In this way, large-scale data can be decomposed into multiple data blocks and executed in sequence by multiple child threads, which reduces the load of single data processing, avoids long-term system blocking, and improves data processing efficiency.

[0067] The processing result includes the processing result of each sub-thread. In some embodiments, when multiple sub-threads are all processed, the target main thread can return the processing result to the client so that the client can timely understand the response of the data processing request.

[0068] In the above data processing method, the Redis server includes a target main thread for data processing. When the Redis server receives a data processing request sent by a client, the target main thread determines the data to be processed based on the data processing request, determines the current data block matching the first preset number in the data to be processed, and creates a child thread; the child thread performs a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, continues to perform the steps of determining the current data block matching the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client. This method of decomposing large-scale data operations into multiple small, continuous data processing steps and gradually executing each small step reduces the load of a single data processing, avoids long-term system blocking, and improves data processing efficiency; through the main and child thread asynchronous processing mechanism, the data processing load of the target main thread is dispersed, so that the system can maintain efficient operation in a high-concurrency environment, thereby improving the overall performance of the system.

[0069] In an exemplary embodiment, multiple sub-threads are created; a data processing action matching the type of data processing request is performed on the current data block through the sub-threads, including: splitting the current data block into multiple sub-data blocks; and performing data processing actions matching the type of data processing request on the multiple sub-data blocks respectively through multiple sub-threads.

[0070] Among them, the target main thread can create multiple sub-threads at the same time, and multiple sub-threads can be processed in parallel, which is conducive to improving data processing efficiency. The number of sub-threads that the target main thread can create can be determined according to system resource conditions.

[0071] The target main thread splits the current data block into multiple sub-data blocks. The data splitting method can be an average split or a split according to the resource situation of each sub-thread. Each sub-thread can perform data processing actions on the corresponding sub-data block.

[0072] In this embodiment, multiple sub-threads are created through the target main thread, and the current data block is split into multiple sub-data blocks. In this way, each sub-thread can perform data processing actions on the corresponding sub-data block independently, further dispersing the data processing pressure of each thread, which is beneficial to improving system performance. In addition, multiple sub-threads can be executed in parallel, which is beneficial to improving data processing efficiency.

[0073] In an exemplary embodiment, after a sub-thread completes processing of a current data block, the target main thread uses the remaining data to be processed as new data to be processed, including: obtaining the processing progress of each of the multiple sub-threads; after the processing progress of any sub-thread indicates that the corresponding sub-data block has been processed, releasing the thread resources of the corresponding sub-thread; after a sub-thread matching a second preset number indicates that the corresponding sub-data block has been processed, the target main thread uses the remaining data to be processed as new data to be processed.

[0074] Among them, the processing progress is used to indicate the degree of completion of the data processing action in the corresponding sub-thread. Exemplarily, the sub-thread inserts the corresponding sub-data block into the List of the Redis server, and the sub-thread traverses each data in the corresponding sub-data block and inserts each data into the List in turn. The processing progress can indicate the amount of data processed, or it can also be the ratio of the amount of data processed to the total amount of data in the corresponding sub-data block. Among them, the List in the Redis server is a bidirectional linked list data structure that can be used to store ordered string collections. Users can push or pop elements at both ends of the List, and the elements in the List are sorted in the order of insertion.

[0075] Since each sub-thread may have different data processing efficiencies, if the processing progress of any sub-thread indicates that the corresponding sub-data block has been processed, the thread resources of the sub-thread may be released so that the released thread resources can be reused to create a new sub-thread.

[0076] The target main thread can continue to allocate data and create new sub-threads after all or part of the sub-threads have been processed. The second preset number can be flexibly selected according to actual conditions. Before the second preset number of sub-threads are processed, the target main thread can be temporarily dormant to asynchronously process other client requests during the dormant period to avoid blocking requests, thereby improving the overall performance of the system.

[0077] In this embodiment, after the processing of the second preset number of sub-threads is completed, the thread resources of the corresponding sub-threads have been released, and the target main thread can continue to process the remaining data to be processed after a short sleep. On the one hand, it can make full use of thread resources and improve the data processing efficiency of the system. On the other hand, the target main thread can asynchronously process other client requests during the short sleep process to avoid request blocking, which is conducive to solving the system performance bottleneck problem.

[0078] In an exemplary embodiment, a data processing action matching the type of data processing request is performed on the current data block by a child thread, including: when the data processing request is of an insert type, each data is inserted into the Redis server according to the position of each data in the current data block by the child thread; when the data processing request is of a modify type, the data in the current data block is replaced with the data at the corresponding position in the Redis server by the child thread according to the position of each data in the current data block; when the data processing request is of a delete type, the data is deleted from the Redis server according to the position of each data in the current data block by the child thread; when the data processing request is of a query type, the Redis server is queried based on the position of each data in the current data block by the child thread.

[0079] According to the type of data processing request, the child thread may adopt a data processing action matching the type of data processing request to process the current data block. The data processing to the List of the Redis server is used as an example for explanation.

[0080] For example, when the data processing request is of the insert type, the data processing request may also indicate the insertion position of each data in the current data block in the List, or indicate the insertion position range of the current data block in the List, and insert each data in the current data block into each insertion position in the List, or the insertion position range in the List, in sequence through a child thread.

[0081] For example, when the data processing request is of a modification type, the data processing request may also indicate the modification position of each data in the current data block in the List, or indicate the modification position range of the current data block in the List, and replace the data at each modification position in the List in sequence through a child thread, or replace the data in the modification position range of the List in sequence.

[0082] For example, when the data processing request is of the deletion type, the data processing request can also indicate the position of each data in the current data block in the List, or indicate the position range of the current data block in the List, and delete the position of each data from the List through the child thread, or the data within the position range.

[0083] For example, when the data processing request is of a query type, the data processing request can also indicate the position of each data in the current data block in the List, or indicate the position range of the current data block in the List, and query the position of each data in the List or the position range in the List through a child thread.

[0084] In this embodiment, a sub-thread is used to perform data processing actions on the current data block, including insertion, deletion, modification, query, etc., which can disperse the load of the target main thread and improve the performance and data efficiency of the system.

[0085] In an exemplary embodiment, the data processing method also includes: upon receiving a target data processing request sent by a target client, adding the target data processing request to a request queue; upon completing the processing of the data processing request, taking out the target data processing request from the request queue by the target main thread, and determining the target data to be processed based on the target data processing request, determining a target data block matching a first preset number in the target data to be processed, and creating a target sub-thread; executing a data processing action matching the type of the target data processing request on the target data block by the target sub-thread; after the target sub-thread completes processing of the target data block, the target main thread uses the remaining target data to be processed as the new target data to be processed, continues to execute the steps of determining a target data block matching the first preset number in the target data to be processed, and creating a target sub-thread, until the corresponding target sub-thread completes processing of the target data block and there is no remaining target data to be processed, and returns the target processing result to the target client.

[0086] The target client is a client other than the aforementioned client. When the Redis server receives the target data processing request sent by the target client, it adds the target data processing request to the request queue. When the data processing request of the aforementioned client is completed, the target main thread takes out the target data processing request from the request queue and processes the target data processing request with reference to the aforementioned data processing method. After the Redis server completes the processing of the target data processing request, it returns the target processing result to the target client.

[0087] In this embodiment, for high-concurrency data processing scenarios, multiple clients may initiate data processing requests at the same time, or while processing a data processing request from one client, data processing requests from other clients may be received. In this case, in order to avoid system blocking, the data processing request initiated later can be added to the request queue for queuing, thereby processing the data processing requests of each client asynchronously. In this way, when processing large-scale data objects, other client requests will not be blocked. This asynchronous processing mechanism significantly improves the response speed and stability of the system, especially in a high-concurrency environment.

[0088] In an exemplary embodiment, the data processing method further includes: the target main thread persists the processing status information of each sub-thread.

[0089] The processing status information refers to the status information of each sub-thread during the data processing process, which may include the amount of processed data, the total amount of data, the amount of remaining data, etc. The target main thread persists the processing status information of each sub-thread, and the persisted processing status information can be used for subsequent data operations or data recovery, etc.

[0090] In this embodiment, the target main thread persists the processing status information of each child thread. In this way, even if the system crashes or anomalies occur in the middle of data processing, the unfinished task can be continued after recovery, ensuring the integrity and consistency of data operations.

[0091] In an exemplary embodiment, as shown in FIG3 , a data processing method is provided, which is described by taking the method applied to the terminal 102 in FIG1 as an example, and includes the following steps 302 to 304. Among them:

[0092] Step 302, sending a data processing request to the Redis server; the data processing request is used to instruct the Redis server that when receiving the data processing request, the target main thread of the Redis server determines the data to be processed based on the data processing request, determines the current data block matching the first preset number in the data to be processed, and creates a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block matching the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client.

[0093] Step 304: Receive the target processing result sent by the Redis server.

[0094] The client refers to an application or terminal that issues a data processing request. The client sends a data processing request to the Redis server, and the data processing request is used to request to insert, delete, modify or query multiple elements in the Redis server.

[0095] The target main thread refers to the thread used for data processing in the Redis server. When the Redis server receives a data processing request, it assigns the request to the target main thread. In the traditional method, the target main thread processes the data processing request. However, this method has a performance bottleneck problem when processing large-scale data.

[0096] The data to be processed refers to the data that needs to be processed as indicated by the data processing request. For example, if the data processing request is used to request to insert data, the data to be processed is the data that needs to be inserted into the Redis server; if the data processing request is used to request to delete data, the data to be processed is the data that needs to be deleted in the Redis server; if the data processing request is used to request to modify data, the data to be processed is the data used to modify the data in the Redis server; if the data processing request is used to request to query data, the data to be processed is the data used to query data in the Redis server.

[0097] The current data block refers to a plurality of data selected from the data to be processed, and the amount of data contained in the current data block is a first preset amount, which can be flexibly adjusted according to actual conditions.

[0098] The child thread is a thread created by the target main thread and is used to process the current data block. By creating a child thread to process data, the burden of the target main thread is effectively reduced, which is conducive to improving the overall performance of the system.

[0099] The types of data processing requests include insert type, delete type, modify type, and query type. The child thread processes the current data block, and the data processing action adopted can be determined according to the type of data processing request. For example, if the data processing request is of insert type, the data processing action can be to insert the current data block into the Redis server.

[0100] Among them, the child thread can record the processing progress of the current data block, and determine whether the current data block has been processed according to the processing progress. When the current data block is processed, the current data block is used as the processed data, and the remaining data to be processed refers to the data remaining after the processed data is removed from the data to be processed. The target main thread uses the remaining data to be processed as new data to be processed and continues to execute the aforementioned data processing process. In this way, large-scale data can be decomposed into multiple data blocks and executed in sequence by multiple child threads, reducing the load of single data processing, avoiding long-term system blocking, and improving data processing efficiency.

[0101] The processing result includes the processing result of each sub-thread. When multiple sub-threads have completed processing, the target main thread returns the processing result to the client so that the client can promptly understand the response status of the data processing request.

[0102] In the above data processing method, when data processing is required, the client sends a data processing request to the Redis server, thereby instructing the Redis server to determine the data to be processed based on the data processing request by the target main thread of the Redis server, determine the current data block matching the first preset number in the data to be processed, and create a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to determine the current data block matching the first preset number in the data to be processed, and create a child thread. The steps of the thread are executed until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and the target processing result is returned to the client; the client receives the target processing result sent by the Redis server. After the client's data processing request is sent to the Redis server, the Redis server decomposes the large-scale data operation into multiple small and continuous data processing steps, and gradually executes each small step, which reduces the load of single data processing, avoids long-term system blocking, and improves data processing efficiency; through the main and child thread asynchronous processing mechanism, the data processing load of the target main thread is dispersed, so that the system can maintain efficient operation in a high concurrency environment, thereby improving the overall performance of the system.

[0103] In a specific embodiment, as shown in FIG4 , a data processing method is provided, comprising the following steps:

[0104] 1. The client sends a request:

[0105] The client sends a data processing request to insert, delete or modify multiple elements in the list of the Redis server (remote dictionary server). For example, a data processing request is an insert operation request containing multiple data items, which is intended to add data to a specified list in batches. The target client sends another target data processing request to the Redis server, which can be an insert, query or other operation.

[0106] 2. Redis server receives the request:

[0107] The Redis server receives the client's data processing request and assigns it to the target main thread for processing. The Redis server's responsibility is to ensure that the request is received and processed correctly, and to coordinate the work of each thread to ensure the task is completed smoothly.

[0108] 3. Initialization operation:

[0109] The target main thread starts processing the data processing request and first obtains or creates the target list object. If the list does not exist, the target main thread will create a new list object. Then, the target main thread verifies the input parameters and initializes the processing status and variables, such as the amount of processed data (processed) and the total amount of data (totalCount).

[0110] 4. Determine the data block size:

[0111] The target main thread calculates the current data block size based on the remaining data amount and the preset block size (CHUNK_SIZE). This step ensures that the amount of data for each operation is moderate, avoiding excessive data blocks that lead to excessive consumption of system resources.

[0112] 5. Create a child thread to process the data block:

[0113] The target main thread creates one or more child threads to handle the element insertion operation in the current data block. The task of the child thread is to reduce the burden of the target main thread. The overall efficiency of the system can be improved by parallel processing of multiple child threads.

[0114] 6. Process a block of data:

[0115] The child thread uses the iterator to traverse and process each element in the current data block and insert it into the list. The child thread performs specific insertion operations to ensure that each element is correctly added to the list.

[0116] 7. Update progress:

[0117] After the child thread completes the data block processing, it will feed back the processing progress to the target main thread. The target main thread will update the amount of processed data and record the current processing progress.

[0118] 8.Persistent task status:

[0119] The target main thread persists the processing status information, including the amount of data processed and the amount of remaining data, etc. In this way, even if the system crashes or anomalies occur during processing, the unfinished task can be continued after recovery, ensuring the integrity and consistency of data operations.

[0120] 9. Register the next incremental task:

[0121] The target main thread registers the next incremental task so that it can continue to execute after a short sleep. Through the time event mechanism, the target main thread adds the next incremental task to the event loop so that it can be automatically executed after the specified time.

[0122] 10. Short sleep:

[0123] The target main thread will sleep for a short time or give up CPU resources to process other operations. This short sleep mechanism avoids occupying system resources for a long time and ensures the responsiveness of the system in a high-concurrency environment.

[0124] 11. When the task is not completed:

[0125] The Redis server notifies the target main thread to continue processing the next data block. The target main thread repeats steps 4 to 9 above until all data blocks are processed.

[0126] 12. When the task is completed:

[0127] The target main thread marks the processing complete and performs necessary cleanup and status updates. The target main thread notifies the Redis server of the processing completion and releases resources to ensure the effective use of system resources.

[0128] The embodiments of the present application include the following technical points:

[0129] a. Incremental processing optimization mechanism: By breaking down large-scale operations into multiple small, continuous operation steps, each small step is executed gradually, thus avoiding long-term blocking and improving system performance and response speed.

[0130] The specific steps include: initializing the operation, determining the size of the data block, processing a data block, updating the progress, registering the next incremental task, short sleep, and operation completion.

[0131] b. Asynchronous processing mechanism: Introduce the collaborative working mechanism of main and sub-threads in the Redis server. The main thread is responsible for coordination and management, and the sub-thread is responsible for specific data block processing. Through short sleep and time event mechanisms, the processing load of large-scale operations is dispersed into multiple small tasks to ensure the responsiveness of the system in a high-concurrency environment.

[0132] c. Task status persistence: After each operation, the current task status is persisted, including information such as the amount of data processed and the amount of remaining data. This ensures that when the system crashes or anomalies occur, unfinished tasks can be continued after recovery, ensuring the integrity and consistency of data operations.

[0133] d. Parallel processing: Use sub-threads to process data block operations in parallel, reduce the burden on the main thread, and improve the overall efficiency of the system. Sub-threads use iterators to traverse and operate the elements in the list to ensure that each element is processed correctly.

[0134] e. Request queuing and processing: When the Redis server receives a new client request, it queues it and waits for processing after the current task is completed. This ensures that new requests are not blocked, effectively utilizes system resources, and improves the system's response speed and stability.

[0135] In this embodiment, by introducing the collaborative working mechanism of the main and child threads in the Redis server, and dispersing the processing load of large-scale operations through short sleep and time event mechanisms, it is ensured that other client requests will not be blocked even when processing large objects. Such an asynchronous processing mechanism significantly improves the response speed and stability of the system, especially in a high-concurrency environment; the incremental processing optimization technology enables the system to maintain high performance while ensuring the integrity and consistency of data operations by means of step-by-step execution and persistent state. In addition, through the incremental processing optimization mechanism, large-scale operations are decomposed into multiple small steps to avoid excessive resource consumption of a single operation, thereby improving the overall performance of the system; the parallel processing mechanism and the asynchronous processing mechanism enable the system to maintain efficient operation in a high-concurrency environment, significantly improving the response speed of the system; the incremental processing and short sleep mechanisms effectively disperse the processing load, avoid long-term system blocking, and improve the availability and stability of the system; the asynchronous processing mechanism ensures that other client requests will not be blocked when processing large-scale data objects, thereby improving the user experience; the task state persistence mechanism ensures that when the system crashes or an exception occurs, the unfinished task can be continued after recovery to ensure the integrity and consistency of data operations; the request queuing and processing mechanism effectively utilizes system resources , avoiding resource competition and performance bottlenecks, and enhancing the stability and reliability of the system; incremental processing optimization technology can be widely used in various scenarios that require processing large-scale collection objects, such as real-time data analysis, stream data processing, order management of large e-commerce systems, etc., which improves the database operation performance in these application scenarios and ensures the efficient operation of the system in complex application environments; through clear task decomposition and persistence mechanisms, it simplifies the system maintenance and fault recovery process, and reduces operation and maintenance costs. Incremental processing optimization technology provides a scalable solution that can be flexibly adjusted and applied according to actual needs. It can effectively solve the performance bottleneck problem when processing large-scale collections in Redis, significantly improve the stability and response speed of the system, and has broad application prospects and practical value.

[0136] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0137] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0138] In an exemplary embodiment, as shown in FIG5 , a data processing device 500 is provided, which is applied to a Redis server and includes: a processing module 520, an execution module 540 and a loop processing module 560, wherein:

[0139] The processing module 520 is used for, when receiving a data processing request sent by a client, determining the data to be processed based on the data processing request by the target main thread, determining the current data blocks matching the first preset number in the data to be processed, and creating a child thread;

[0140] An execution module 540 is used to execute a data processing action matching the type of the data processing request on the current data block through a sub-thread;

[0141] The loop processing module 560 is used to, after the child thread completes processing the current data block, have the target main thread use the remaining data to be processed as new data to be processed, continue to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

[0142] In one embodiment, multiple sub-threads are created; a data processing action matching the type of data processing request is executed on the current data block through the sub-threads, and the execution module 540 is also used to: split the current data block into multiple sub-data blocks; and execute data processing actions matching the type of data processing request on the multiple sub-data blocks respectively through multiple sub-threads.

[0143] In one of the embodiments, after the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, and the loop processing module 560 is also used to: obtain the processing progress of each of the multiple child threads; after the processing progress of any child thread indicates that the corresponding sub-data block has been processed, release the thread resources of the corresponding child thread; after the child thread matching the second preset number indicates that the corresponding sub-data block has been processed, the target main thread uses the remaining data to be processed as new data to be processed.

[0144] In one embodiment, a data processing action matching the type of data processing request is performed on the current data block through a child thread, and the execution module 540 is also used for: when the data processing request is of an insert type, inserting each data into the Redis server according to the position of each data in the current data block through the child thread; when the data processing request is of a modify type, replacing the data at the corresponding position in the Redis server with the data in the current data block through the child thread according to the position of each data in the current data block; when the data processing request is of a delete type, deleting the data from the Redis server according to the position of each data in the current data block through the child thread; when the data processing request is of a query type, querying the Redis server based on the position of each data in the current data block through the child thread.

[0145] In one embodiment, the data processing device 500 also includes a queue processing module, which is used to: upon receiving a target data processing request sent by a target client, add the target data processing request to a request queue; upon completing the processing of the data processing request, the target main thread takes out the target data processing request from the request queue, and determines the target data to be processed based on the target data processing request, determines a target data block matching a first preset number in the target data to be processed, and creates a target sub-thread; performs a data processing action matching the type of the target data processing request on the target data block through the target sub-thread; after the target sub-thread completes processing of the target data block, the target main thread uses the remaining target data to be processed as the new target data to be processed, continues to execute the steps of determining a target data block matching the first preset number in the target data to be processed, and creates a target sub-thread, until the corresponding target sub-thread completes processing of the target data block and there is no remaining target data to be processed, and returns the target processing result to the target client.

[0146] In one embodiment, the data processing device 500 further includes a persistence module, and the persistence module is used to: persist the processing status information of each sub-thread by the target main thread.

[0147] In the above-mentioned data processing device, the Redis server includes a target main thread for data processing. When the Redis server receives a data processing request sent by a client, the target main thread determines the data to be processed based on the data processing request, determines the current data block matching the first preset number in the data to be processed, and creates a child thread; the child thread performs a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, continues to perform the steps of determining the current data block matching the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client. This method of decomposing large-scale data operations into multiple small, continuous data processing steps and gradually executing each small step reduces the load of single data processing, avoids long-term system blocking, and improves data processing efficiency; through the main and child thread asynchronous processing mechanism, the data processing load of the target main thread is dispersed, so that the system can maintain efficient operation in a high-concurrency environment, thereby improving the overall performance of the system.

[0148] In an exemplary embodiment, as shown in FIG6 , a data processing device 600 is provided, which is applied to a client and includes: a sending module 620 and a receiving module 640, wherein:

[0149] The sending module 620 is used to send a data processing request to the Redis server; the data processing request is used to instruct the Redis server that when receiving the data processing request, the target main thread of the Redis server determines the data to be processed based on the data processing request, determines the current data block matching the first preset number in the data to be processed, and creates a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block matching the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client;

[0150] The receiving module 640 is used to receive the target processing result sent by the Redis server.

[0151] In the above-mentioned data processing device, when the client needs to process data, it sends a data processing request to the Redis server, thereby instructing the Redis server to determine the data to be processed based on the data processing request by the target main thread of the Redis server, determine the current data block matching the first preset number in the data to be processed, and create a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to determine the current data block matching the first preset number in the data to be processed, and create a child thread. The steps of the thread are executed until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and the target processing result is returned to the client; the client receives the target processing result sent by the Redis server. After the client's data processing request is sent to the Redis server, the Redis server decomposes the large-scale data operation into multiple small and continuous data processing steps, and gradually executes each small step, which reduces the load of single data processing, avoids long-term system blocking, and improves data processing efficiency; through the main and child thread asynchronous processing mechanism, the data processing load of the target main thread is dispersed, so that the system can maintain efficient operation in a high concurrency environment, thereby improving the overall performance of the system.

[0152] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0153] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data processing requests and data to be processed, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0154] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0156] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0157] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0159] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0160] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: Applied to a Redis server, the method includes: In the case of receiving a data processing request sent by a client, the target main thread determines the data to be processed based on the data processing request, determines the current data blocks matching the first preset number in the data to be processed, and creates a child thread; executing, by the child thread, a data processing action matching the type of the data processing request on the current data block; After the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, continues to execute the steps of determining the current data block that matches the first preset number in the data to be processed, and creates a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

2. The method according to claim 1, characterized in that There are multiple sub-threads created; the sub-threads executing a data processing action matching the type of the data processing request on the current data block include: Split the current data block into multiple sub-data blocks; Data processing actions matching the type of the data processing request are respectively performed on the multiple sub-data blocks through the multiple sub-threads.

3. The method according to claim 2, characterized in that After the child thread completes processing the current data block, the target main thread uses the remaining data to be processed as new data to be processed, including: Get the processing progress of multiple sub-threads; After the processing progress of any sub-thread indicates that the processing of the corresponding sub-data block is completed, the thread resources of the corresponding sub-thread are released; After the sub-threads matching the second preset number indicate that the corresponding sub-data blocks have been processed, the target main thread uses the remaining data to be processed as new data to be processed.

4. The method according to claim 1, characterized in that: The performing, by the child thread, a data processing action matching the type of the data processing request on the current data block comprises: In the case where the data processing request is of an insert type, inserting each data into the Redis server according to the position of each data in the current data block through the child thread; In the case where the data processing request is of a modification type, the child thread replaces the data in the current data block with the data at the corresponding position in the Redis server according to the position of each data in the current data block; In the case where the data processing request is of a deletion type, deleting data from the Redis server according to the position of each data in the current data block through the child thread; In the case where the data processing request is of a query type, the Redis server is queried based on the position of each data in the current data block through the child thread.

5. The method according to claim 1, characterized in that: The method further comprises: When receiving a target data processing request sent by a target client, adding the target data processing request to a request queue; When the data processing request is processed, the target main thread takes out the target data processing request from the request queue, determines target data to be processed based on the target data processing request, determines target data blocks matching a first preset number in the target data to be processed, and creates a target sub-thread; executing, by the target subthread, a data processing action matching the type of the target data processing request on the target data block; After the target sub-thread completes processing the target data block, the target main thread uses the remaining target data to be processed as new target data to be processed, continues to execute the steps of determining target data blocks matching the first preset number in the target data to be processed, and creates a target sub-thread, until the corresponding target sub-thread completes processing the target data block and there is no remaining target data to be processed, and returns the target processing result to the target client.

6. The method according to claim 1, characterized in that The method further comprises: The target main thread persists the processing status information of each child thread.

7. A data processing method, characterized in that: Applied to a client, the method comprises: Sending a data processing request to a Redis server; the data processing request is used to instruct the Redis server, upon receiving the data processing request, to determine the data to be processed based on the data processing request by the target main thread of the Redis server, determine the current data block matching the first preset number in the data to be processed, and create a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block matching the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client; Receive the target processing result sent by the Redis server.

8. A data processing device, characterized in that: Applied to a Redis server, the device comprises: a processing module, configured to, upon receiving a data processing request sent by a client, determine, by a target main thread, data to be processed based on the data processing request, determine a current data block matching a first preset number in the data to be processed, and create a child thread; An execution module, configured to execute a data processing action matching the type of the data processing request on the current data block through the sub-thread; A loop processing module is used to, after the child thread completes processing the current data block, have the target main thread use the remaining data to be processed as new data to be processed, continue to execute the steps of determining the current data block matching the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the processing result to the client.

9. A data processing device, characterized in that: Applied to a client, the device comprises: A sending module, used for sending a data processing request to a Redis server; the data processing request is used to instruct the Redis server, when receiving the data processing request, to determine the data to be processed based on the data processing request by the target main thread of the Redis server, determine the current data block matching the first preset number in the data to be processed, and create a child thread; the child thread is used to perform a data processing action matching the type of the data processing request on the current data block; after the child thread completes processing the current data block, the target main thread is also used to use the remaining data to be processed as new data to be processed, continue to perform the steps of determining the current data block matching the first preset number in the data to be processed, and create a child thread, until the corresponding child thread completes processing the current data block and there is no remaining data to be processed, and returns the target processing result to the client; A receiving module is used to receive the target processing result sent by the Redis server.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images