Adaptive data request processing method and apparatus, task scheduler and adaptive data request processing system

Abstract

The present invention discloses an adaptive data request processing method. The method comprises: acquiring a dependency relationship between at least two data request processing tasks in a to-be-executed data request queue, and establishing a dependency relationship table; and according to the dependency relationship table, uniformly and adaptively scheduling the at least two data request processing tasks in the data request queue, and driving and executing the at least two data request processing tasks. The present invention also discloses an adaptive data request processing apparatus, a task scheduler and an adaptive data request processing system. According to the adaptive data request processing method and apparatus, the task scheduler and the adaptive data request processing system, when the task queue with the dependency relationships is executed, the highest degree of concurrency is acquired; the task execution efficiency is improved; data request processing is efficiently completed; and meanwhile, the adaptive data request processing method and apparatus, the task scheduler and the adaptive data request processing system are beneficial to later maintenance and expansion.

Description

technical field [0001] The invention relates to the field of data processing, in particular to an adaptive data request processing method, device, task scheduler and system. Background technique [0002] A system or software (such as an application) involving data processing usually involves a lot of data requests and / or data processing tasks. Reasonable and efficient data requests and processing are very important, which can improve the experience of the entire system to the best , so that users get a better experience. [0003] The existing data request processing task (hereinafter referred to as request or task) processing process generally puts the start of the next request into the execution result of the previous request that it depends on. When the next request obtains the execution result of the previous request, it will The data is passed to the next request, and the next data request task is started when the execution of the previous request of the previous depend...

AI Technical Summary

Problems solved by technology

In this way, when a task queue with dependencies is executed, the logical relationship of its processing is that one or more tasks drive the next task and execute it sequentially. The coupling between tasks is high, and it cannot achieve high concurrent execution of tasks. , resulting in lower processing efficiency

[0007] In addition, when the dependencies between tasks change, it is difficult to clarify the dependencies between tasks, and the flexibility to adjust the dependencies between tasks is low, which further leads to low processing efficiency and inconvenient system maintenance and expansion.

Taking the above example as an example, if adding a step in the middle of the above tasks is to obtain the playlist, then the dependencies between the involved tasks will change a lot. If you want to modify it, it will be error-prone. The relationship is difficult to be clear, and there is no way to clearly see the dependencies between them through the actual implementation code, so it is difficult to flexibly adjust the execution of driving tasks

Images