Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium. The method comprises: dividing a data task into a data query sub-task and a data processing sub-task to be executed respectively; caching the received data tasks by adopting a distributed task queue; acquiring a data task through a data query thread, performing data query to complete a data query sub-task, caching to-be-processed data queried by the data query thread by adopting a distributed data queue, and acquiring the to-be-processed data through a data processing thread to perform dataprocessing to complete a data processing sub-task. Through the method, the data query thread and the data processing thread only need to independently execute data query and data processing; and for each thread, the time consumption is greatly reduced, so that the digestion speed of the data tasks in the task queue can be ensured, the data tasks in the task queue do not need to wait for a long time, and the server can quickly respond to the data processing request of the client.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular to a data processing method, device, electronic equipment, and storage medium. Background technique [0002] In the information age, the number of clients that can interact with the server in the Internet is increasing. When the client initiates a data processing request to the server, its requirements for the response speed of the server are also getting higher and higher. [0003] Usually, after receiving the data processing request initiated by the client, the server adds the request message to the task queue and waits for execution; and for the data processing task in the task queue, the server calls the corresponding service to create a task thread to pass the task threads to execute the data processing tasks in the task queue; after the current processing task is completed, the next data processing task is received from the task queue for processing. [...

AI Technical Summary

Problems solved by technology

[0004] However, for each data processing task, the corresponding task thread needs to complete the data query and processing operations in sequence, so it takes a long time, resulting in a long waiting time for the processing tasks in the task queue, resulting in a slow server response effect, which can easily lead to service restart, and once the service restarts, it will cause task loss

Images