Task scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a task scheduling method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a load state of each execution unit in a scheduling environment, a queue state of a to-be-processed task queue and a system resource state of a scheduling controller to generate an environment state vector; inputting the environment state vector into a reinforcement learning strategy model to obtain a scheduling strategy which is used for determining a task allocation scheme for each execution unit; according to the scheduling strategy, distributing the to-be-processed task to a corresponding execution unit for execution, and collecting feedback data in the task execution process; based on the feedback data, calculating a comprehensive reward value fusing a task execution efficiency index and a task business value index; and updating the reinforcement learning strategy model by using the comprehensive reward value. Therefore, the continuous optimal balance between the overall processing efficiency and the resource utilization efficiency of the system is realized.

Description

Technical Field

[0001] This application relates to the field of computer system resource management technology, and in particular to a task scheduling method, apparatus, electronic device and storage medium. Background Technology

[0002] As enterprises deepen their digital transformation, business auditing systems that ensure data accuracy and the correctness of business processes become crucial. Such systems need to perform large-scale Structured Query Language (SMQ) validation tasks on various data sources to detect data anomalies in real time. In this scenario, the efficiency and intelligence level of task scheduling directly affect the timeliness of anomaly detection and the overall utilization of system computing resources. Currently, a common practice is to adopt a scheduling scheme based on fixed rules. This scheme pre-sets fixed task execution parameters for different types of databases and allocates tasks to the corresponding databases for execution according to a pre-set rule sequence.

[0003] However, when dealing with diverse data source types and dynamically changing workloads, the fixed scheduling parameters and rule sequences of the aforementioned existing technical solutions are difficult to adjust adaptively, resulting in the overall task processing efficiency and resource utilization efficiency of the system failing to reach the optimal state. Summary of the Invention

[0004] The purpose of this application is to provide a task scheduling method, apparatus, electronic device, and storage medium to solve the problem that existing scheduling schemes based on fixed parameters and rules lack adaptive adjustment capabilities and cannot achieve optimal overall system efficiency under diverse data sources and dynamic load scenarios. The specific technical solution is as follows: Firstly, this application provides a task scheduling method, including: The system collects the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller to generate an environment state vector. The environmental state vector is input into the reinforcement learning policy model to obtain a scheduling policy, which is used to determine the task allocation scheme for each execution unit. According to the scheduling strategy, the tasks to be processed are distributed to the corresponding execution units for execution, and feedback data during the task execution process is collected. Based on the feedback data, a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators is calculated. The reinforcement learning policy model is updated using the comprehensive reward value.

[0005] In one possible implementation, the scheduling strategy includes a global concurrent task limit and concurrent task weights corresponding to each execution unit; The step of distributing the tasks to be processed to the corresponding execution units for execution according to the scheduling policy includes: Based on the global concurrent task limit and the concurrent task weights corresponding to each execution unit, calculate the concurrent execution quota for each execution unit; For each execution unit, determine the group of tasks to be processed corresponding to the execution unit from the queue of tasks to be processed; Based on the concurrent execution quota corresponding to the execution unit, select a corresponding number of tasks to be processed from the task group to be processed; The task to be processed is sent to the execution unit.

[0006] In one possible implementation, selecting a corresponding number of tasks to be processed from the task group based on the concurrent execution quota corresponding to the execution unit includes: For each task in the task group to be processed, a task score is calculated based on the task's priority and estimated execution time. Select a corresponding number of tasks as pending tasks according to their corresponding task scores from highest to lowest.

[0007] In one possible implementation, the scheduling strategy further includes a task execution timeout threshold for each execution unit; After the task to be processed is sent to the execution unit, the process further includes: During the execution of the task to be processed by the execution unit, the task execution time is monitored; When the execution time of the task exceeds the timeout threshold of the task execution corresponding to the execution unit, a timeout processing operation is performed.

[0008] In one possible implementation, the step of calculating a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators based on the feedback data includes: Calculate the delay reward based on the task execution time in the feedback data; Based on the resource consumption data in the feedback data, calculate the resource utilization rate reward item; Calculate the timeout penalty based on the task completion status in the feedback data; Based on the task execution results in the feedback data, calculate the reward for discovering business anomalies; Based on preset weights, the delay reward item, the resource utilization reward item, the timeout penalty item, and the business anomaly detection reward item are weighted and merged to obtain the comprehensive reward value.

[0009] In one possible implementation, updating the reinforcement learning policy model using the comprehensive reward value includes: The environment state vector and the comprehensive reward value are input into the reinforcement learning policy model. The value evaluation network in the reinforcement learning policy model calculates the value evaluation score of the current state based on the environment state vector and the comprehensive reward value. The policy generation network in the reinforcement learning policy model updates the policy parameters based on the value evaluation score and the comprehensive reward value.

[0010] In one possible implementation, the method further includes: When a new execution unit is connected to the scheduling environment, the current reinforcement learning strategy model is used as the base model. Based on the environment state vector containing the state of the new execution unit, a scheduling strategy is generated and the task distribution is performed according to the scheduling strategy. Collect feedback data on task execution and calculate the comprehensive reward value based on the feedback data; The base model is updated using the comprehensive reward value to obtain a reinforcement learning policy model adapted to the scheduling environment containing the new execution unit.

[0011] Secondly, this application provides a task scheduling apparatus, comprising: The acquisition module is used to collect the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller in order to generate an environment status vector. An input module is used to input the environment state vector into a reinforcement learning policy model to obtain a scheduling policy, which is used to determine a task allocation scheme for each execution unit. The distribution module is used to distribute the tasks to be processed to the corresponding execution units for execution according to the scheduling strategy, and to collect feedback data during the task execution process; The calculation module is used to calculate a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators based on the feedback data. An update module is used to update the reinforcement learning policy model using the comprehensive reward value.

[0012] In one possible implementation, the scheduling strategy includes a global concurrent task limit and concurrent task weights corresponding to each execution unit; The distribution module is specifically used for: Based on the global concurrent task limit and the concurrent task weights corresponding to each execution unit, calculate the concurrent execution quota for each execution unit; For each execution unit, determine the group of tasks to be processed corresponding to the execution unit from the queue of tasks to be processed; Based on the concurrent execution quota corresponding to the execution unit, select a corresponding number of tasks to be processed from the task group to be processed; The task to be processed is sent to the execution unit.

[0013] In one possible implementation, the distribution module is further configured to: For each task in the task group to be processed, a task score is calculated based on the task's priority and estimated execution time. Select a corresponding number of tasks as pending tasks according to their corresponding task scores from highest to lowest.

[0014] In one possible implementation, the scheduling strategy further includes a task execution timeout threshold for each execution unit; The device further includes a timeout processing module for: During the execution of the task to be processed by the execution unit, the task execution time is monitored; When the execution time of the task exceeds the timeout threshold of the task execution corresponding to the execution unit, a timeout processing operation is performed.

[0015] In one possible implementation, the computing module is specifically used for: Calculate the delay reward based on the task execution time in the feedback data; Based on the resource consumption data in the feedback data, calculate the resource utilization rate reward item; Calculate the timeout penalty based on the task completion status in the feedback data; Based on the task execution results in the feedback data, calculate the reward for discovering business anomalies; Based on preset weights, the delay reward item, the resource utilization reward item, the timeout penalty item, and the business anomaly detection reward item are weighted and merged to obtain the comprehensive reward value.

[0016] In one possible implementation, the update module is specifically used for: The environment state vector and the comprehensive reward value are input into the reinforcement learning policy model. The value evaluation network in the reinforcement learning policy model calculates the value evaluation score of the current state based on the environment state vector and the comprehensive reward value. The policy generation network in the reinforcement learning policy model updates the policy parameters based on the value evaluation score and the comprehensive reward value.

[0017] In one possible implementation, the device further includes an adapter module for: When a new execution unit is connected to the scheduling environment, the current reinforcement learning strategy model is used as the base model. Based on the environment state vector containing the state of the new execution unit, a scheduling strategy is generated and the task distribution is performed according to the scheduling strategy. Collect feedback data on task execution and calculate the comprehensive reward value based on the feedback data; The base model is updated using the comprehensive reward value to obtain a reinforcement learning policy model adapted to the scheduling environment containing the new execution unit.

[0018] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0019] Fourthly, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0020] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to execute any of the task scheduling methods described above.

[0021] Beneficial effects of the embodiments in this application: This application provides a task scheduling method, apparatus, electronic device, and storage medium. In this embodiment, the load status of each execution unit, the queue status of the task queue, and the system resource status of the scheduling controller are first collected in real time and fused to generate an environmental state vector, thereby achieving comprehensive perception of the multi-dimensional state of the system during operation. Next, this environmental state vector is input into a reinforcement learning strategy model, which outputs a dynamic scheduling strategy to determine the task allocation scheme for each execution unit. Then, tasks are distributed to the corresponding execution units according to the dynamic scheduling strategy, and feedback data generated during task execution is collected synchronously. Finally, a comprehensive reward value that balances task execution efficiency and business value is calculated based on the feedback data, and this comprehensive reward value is used to continuously optimize and update the reinforcement learning strategy model. Through this solution, the system can generate and execute the optimal strategy based on the real-time environmental status in each scheduling cycle, and continuously improve itself through feedback learning, thereby achieving a continuous optimal balance between overall system processing efficiency and resource utilization efficiency in complex environments with diverse data source types and dynamically fluctuating loads.

[0022] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0026] Figure 1 A flowchart illustrating a task scheduling method provided in an embodiment of this application; Figure 2 A flowchart illustrating another task scheduling method provided in this application embodiment; Figure 3 A flowchart illustrating yet another task scheduling method provided in this application embodiment; Figure 4 This is a schematic diagram of the system architecture of the intelligent scheduling engine provided in the embodiments of this application; Figure 5 A schematic diagram illustrating the workflow and learning mechanism of the task scheduling method provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of a task scheduling device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0029] Figure 1 This is a flowchart illustrating a task scheduling method provided in an embodiment of this application. This method can be applied to one or more electronic devices such as smartphones, laptops, desktop computers, portable computers, and servers. Furthermore, the execution entity of this method can be hardware or software. When the execution entity is hardware, it can be one or more of the aforementioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the execution entity is software, this method can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are imposed here.

[0030] like Figure 1 As shown, the method specifically includes: S101. Collect the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller to generate an environment state vector.

[0031] Execution unit: refers to the computing entity or service used to carry out and run specific tasks in the scheduling environment, such as heterogeneous data sources such as relational database MySQL, distributed database TiDB, batch processing engine Hive, and real-time analysis database StarRocks.

[0032] Load status: refers to the real-time performance indicators of the execution unit during runtime, including at least the number of active connections, the high percentile value of query response time such as P99, and the computing resource utilization rate of the execution unit itself.

[0033] The pending task queue refers to a collection of tasks that are organized in a certain order and are waiting to be scheduled for execution. In business auditing scenarios, each task usually contains a "validation structured query language" statement to determine whether the data is abnormal.

[0034] Queue status: refers to the characteristic information of the queue of tasks to be processed at a certain moment, including at least the total number of tasks in the queue, the preset business priority of each task, the estimated execution time of the task based on historical data statistics, and the waiting time of the task.

[0035] Scheduling controller: refers to the core service process or module that runs this scheduling method and is responsible for coordinating the execution of the entire scheduling process.

[0036] System resource status: refers to the real-time hardware resource usage of the server where the scheduling controller is located, including CPU utilization, available memory, and network input / output bandwidth pressure.

[0037] Environment state vector: refers to a multi-dimensional numerical vector formed by splicing or encoding the collected load state, queue state and system resource state after normalization and other preprocessing. It is used to comprehensively and standardizedly represent the overall status of the current scheduling environment.

[0038] In this embodiment, the system periodically (e.g., every 5 seconds) or triggeredly collects the above three types of status information through probes or agents deployed on the execution unit side and a monitoring module local to the scheduling controller. Subsequently, the collected numerical indicators (such as response time and CPU (Central Processing Unit) utilization) are normalized to eliminate dimensional differences, and the processed data is concatenated in a predetermined order to ultimately form a fixed-dimensional environment state vector. For example, for a system containing 3 execution units, each execution unit collects 3 load indicators, queue status extracts 5 features, and system resource status collects 3 indicators, thus generating a (3... The environmental state vector is calculated as 3 + 5 + 3 = 17. This step enables real-time quantitative perception of the performance of heterogeneous data sources, the pressure of pending tasks, and the health of the system itself, providing accurate data input for subsequent intelligent decision-making.

[0039] S102. Input the environment state vector into the reinforcement learning policy model to obtain a scheduling policy, which is used to determine the task allocation scheme for each execution unit.

[0040] Reinforcement learning policy model: refers to an artificial intelligence model trained based on reinforcement learning algorithms, which can output a suggested action decision to be taken in the input environment state. In this application embodiment, the model is preferably trained using a proximal policy optimization algorithm.

[0041] Scheduling strategy: refers to the decision result output by the reinforcement learning strategy model after calculating based on the current environment state vector. Its specific form is a series of executable scheduling instructions or parameter sets.

[0042] Task allocation scheme: refers to the specific plan determined according to the scheduling strategy on how to allocate tasks to be processed to each execution unit. It defines which tasks should be selected from the task queue and which specific execution unit should be assigned to them in the current scheduling cycle.

[0043] In this embodiment, the system feeds the environment state vector generated in step S101 as input to the loaded reinforcement learning policy model. This model typically contains a policy network (also known as an actor network). After receiving the environment state vector, this network performs calculations through its internal neural network layers, ultimately outputting a set of action parameters. These parameters collectively constitute the scheduling policy. For example, in a specific implementation, this policy might explicitly include: 1) a numerical value K representing the global concurrent task limit; 2) a vector W, where each component represents the concurrent task weight assigned to the corresponding execution unit; and 3) a series of scores for task ranking. The set of these parameters defines the complete scheme for selecting tasks from the queue and assigning them to execution units in the current cycle. The core of this step is to utilize a reinforcement learning model to replace manually preset static rules, enabling the dynamic generation of optimal or near-optimal scheduling policies based on real-time, ever-changing environmental states.

[0044] S103. According to the scheduling strategy, the tasks to be processed are distributed to the corresponding execution units for execution, and feedback data during the task execution process is collected.

[0045] Feedback data refers to all recorded information about the execution process and results generated after the task to be processed is sent to the corresponding execution unit and actually executed.

[0046] In this embodiment, the system executes task distribution logic defined by a scheduling strategy. Specifically, the system parses the scheduling strategy and, based on the rules determined by the strategy (e.g., the concurrency quota allocated to each execution unit or the selection of priority), selects specific tasks to be dispatched from the task queue and assigns them to the corresponding execution units for execution. During and after task execution, the system collects data generated during the execution process through a monitoring interface, forming feedback data. This feedback data covers the objective results and process indicators of task execution, providing a basis for subsequent performance evaluation. This step transforms the abstract scheduling strategy into concrete system behavior and completes the collection of key information.

[0047] S104. Based on the feedback data, calculate the comprehensive reward value that integrates the task execution efficiency index and the task business value index.

[0048] Task execution efficiency indicators refer to indicators that quantify and evaluate the task execution process from the dimensions of execution speed and resource consumption efficiency.

[0049] Task business value metrics refer to indicators that quantify and evaluate task execution results from the perspective of achieving business objectives (such as whether key anomalies were discovered).

[0050] The overall reward value is a scalar value that combines the two types of indicators mentioned above. It is used to quantitatively evaluate the overall effect of the scheduling strategy implemented in the reinforcement learning framework.

[0051] In this embodiment, the system processes and analyzes the feedback data to calculate a quantitative value, namely the comprehensive reward value, used to evaluate the effectiveness of the scheduling decision. The calculation process involves two core evaluation dimensions: first, task execution efficiency indicators, which reflect the efficiency of task execution in terms of time and resource consumption, and can consider basic indicators such as task duration and system resource utilization; second, task business value indicators, which reflect the contribution of the task execution result to the business level, and can be determined based on whether the task triggered a specific, business-significant conclusion (e.g., whether data anomalies were found in an audit scenario). The system integrates these two types of indicators into a unified scalar value, namely the comprehensive reward value, through predefined functions or rules. The calculation of this comprehensive reward value aims to guide the scheduling strategy towards optimization that simultaneously improves system efficiency and business value.

[0052] S105. Update the reinforcement learning policy model using the comprehensive reward value.

[0053] In this embodiment, the system uses the environment state vector generated during the current scheduling cycle, the scheduling policy output by the reinforcement learning policy model based on this vector, and the comprehensive reward value calculated after executing the policy to form a set of empirical data. Based on the update mechanism of the reinforcement learning algorithm (such as the proximal policy optimization algorithm), the system uses this set of empirical data to optimize and adjust the internal parameters of the reinforcement learning policy model. This optimization process aims to enable the model to output a scheduling policy that yields a higher expected comprehensive reward value when facing similar environment states in the future. This step endows the system with the ability to learn and continuously improve from historical scheduling experience, and is the core mechanism for the scheduling method to achieve dynamic adaptation and continuous optimization.

[0054] The technical solution provided in this application first collects the load status of each execution unit, the queue status of the task queue, and the system resource status of the scheduling controller in real time, and fuses them to generate an environmental state vector, thereby achieving a comprehensive perception of the multi-dimensional state of the system during operation. Next, this environmental state vector is input into a reinforcement learning strategy model, which outputs a dynamic scheduling strategy to determine the task allocation scheme for each execution unit. Then, tasks are distributed to the corresponding execution units for execution according to the dynamic scheduling strategy, and feedback data generated during task execution is collected synchronously. Finally, a comprehensive reward value that balances task execution efficiency and business value is calculated based on the feedback data, and this comprehensive reward value is used to continuously optimize and update the reinforcement learning strategy model. Through this solution, the system can generate and execute the optimal strategy based on the real-time environmental status in each scheduling cycle, and continuously improve itself through feedback learning, thereby achieving a continuous optimal balance between overall system processing efficiency and resource utilization efficiency in complex environments with diverse data source types and dynamically fluctuating loads.

[0055] See Figure 2 This is a flowchart illustrating an embodiment of another task scheduling method provided in this application. Figure 2 The process shown above Figure 1 Based on the illustrated process, this section describes how, when the scheduling strategy includes a global concurrent task limit and the concurrent task weights corresponding to each execution unit, the tasks to be processed are distributed to the corresponding execution units for execution according to the scheduling strategy. For example... Figure 2 As shown, the process may include the following steps: S201. Calculate the concurrent execution quota of each execution unit based on the global concurrent task limit and the concurrent task weight corresponding to each execution unit.

[0056] Global concurrent task limit: This refers to the maximum number of tasks that the scheduler controller allows to be issued and executed simultaneously within the current scheduling cycle. It is a core parameter output by the reinforcement learning policy model and used to control the overall concurrency pressure of the system.

[0057] Concurrent task weights: These are proportional coefficients assigned to each execution unit, representing its relative share of the global concurrent resource pool. These weights collectively form a weight vector, which is also output by the reinforcement learning policy model.

[0058] Concurrent execution quota: refers to the maximum number of tasks that can be executed concurrently within the current scheduling period, calculated based on the above parameters for an execution unit. This value is a dynamically calculated instantaneous quota.

[0059] In this embodiment, the system parses the scheduling strategy and extracts the global concurrent task limit (denoted as K) generated by the model decision and the concurrent task weights corresponding to each execution unit (denoted as weight vector W = [w1, w2, ..., wn]). Subsequently, the system calculates the concurrent execution quota (denoted as quota_i) for each execution unit i based on these parameters. A specific and feasible calculation method is to allocate according to the weight ratio, with the calculation formula being, for example: quota_i = floor(K) Alternatively, a similar rounding strategy can be used to ensure that the total limit does not exceed K. The core of this step is to accurately decode and quantify the model's decision on macro-level resource allocation (K and W) and the specific, executable concurrent task limit for each execution unit, thereby laying the mathematical foundation for subsequent precise resource allocation and task selection.

[0060] S202. For each execution unit, determine the group of tasks to be processed corresponding to the execution unit from the queue of tasks to be processed.

[0061] Pending task group: refers to a subset of pending tasks belonging to a specific execution unit, selected from the global pending task queue based on the inherent association between the task and the execution unit (e.g., the database where the data table operated by the task's SQL statement is located).

[0062] In this embodiment, the system iterates through the task queue for each execution unit (e.g., MySQL, Hive) in the scheduling environment. For each task in the queue, the system determines the target execution unit based on its configuration information or task content (e.g., in a business audit system, each audit item has its verification data source configured in the "Audit Item Configuration Center"). All tasks targeting the same execution unit are grouped together, forming a task group corresponding to that execution unit. This step is essentially a pre-classification for routing distribution based on the static attributes of the tasks, ensuring that tasks are only scheduled to units where they can be correctly executed, laying the foundation for subsequent quota selection.

[0063] S203. Select a corresponding number of tasks to be processed from the task group according to the concurrent execution quota corresponding to the execution unit.

[0064] In this embodiment, for each execution unit, the system obtains the concurrent execution quota calculated in step S201 (e.g., quota_i=5) and accesses the task group to be processed corresponding to that unit as determined in step S202. The system needs to select specific tasks from this task group, with a number not exceeding its quota (i.e., a maximum of 5), for distribution. One or more rules can be used to optimize the effect during selection.

[0065] As one possible implementation, S203 may include the following steps: for each task in the task group to be processed, calculate a task score based on the task's priority and estimated execution time; select a corresponding number of tasks as tasks to be processed according to the order of their corresponding task scores from high to low.

[0066] Task score: A numerical value used to quantify the priority of task scheduling. This value comprehensively reflects the urgency of the task (business priority) and the expected resource consumption (estimated execution time).

[0067] In this embodiment, when the system needs to select tasks from a task group corresponding to an execution unit, a quantitative evaluation mechanism is employed. Specifically, for each task in the task group, the system obtains two key characteristics: priority (determined by business configuration, indicating the importance of the task) and estimated execution time (expected time derived from historical execution data statistics). The system calculates a task score based on these two parameters according to a predefined function or rule. A typical calculation principle is that the score is positively correlated with priority and negatively correlated with estimated execution time; for example, the formula can be: Score = Priority Coefficient / (1 + Estimated Execution Time). After calculating the scores of all tasks in the group, the system sorts all tasks in descending order of score. Finally, based on the concurrent execution quota determined for the execution unit (i.e., the number of tasks to be selected), the system selects a corresponding number of tasks sequentially from the top of the sorted list as the tasks to be processed in this round.

[0068] This solution calculates a comprehensive score for each task within a task group, incorporating both priority and estimated duration. Tasks are then selected based on their scores, allowing the system to intelligently prioritize "high-value, short-duration" tasks while meeting concurrency constraints. This resolves the "head-of-the-queue blocking" problem, where critical tasks are blocked by long-tail tasks, and ensures that high-value tasks (such as core audit items) are executed promptly. Consequently, under resource-constrained conditions, this significantly optimizes the business effectiveness and overall processing efficiency of task scheduling.

[0069] S204. The task to be processed is sent to the execution unit.

[0070] In this embodiment, the system formally submits the specific tasks selected for each execution unit in step S203 to that execution unit for execution via the corresponding communication protocol or database connection driver. For example, the "verify SQL" statement in the selected audit task is sent to the corresponding MySQL database for execution via a JDBC connection. This step is the final physical implementation of the scheduling decision, enabling the task to enter the actual running state.

[0071] This solution dynamically calculates the global concurrency limit (K) and weights (W) output by the reinforcement learning policy model as the concurrency quota for each execution unit, achieving precise concurrency control over heterogeneous data sources and preventing load imbalance at the source. Furthermore, by grouping tasks by task attributes and intelligently selecting tasks within groups based on priority and time consumption, it ensures that high-value tasks are prioritized for scheduling, effectively avoiding head-of-line blocking. Finally, reliable delivery completes the scheduling loop. This mechanism enables the system to automatically adapt to heterogeneous environments and dynamic loads, significantly improving overall throughput and reducing critical task latency while ensuring the stability of each data source, achieving an optimal balance between resource utilization and processing efficiency.

[0072] In another embodiment, the scheduling strategy further includes a task execution timeout threshold corresponding to each execution unit; after the task to be processed is sent to the execution unit, the following steps may also be included: during the execution of the task to be processed by the execution unit, the task execution time is monitored; when the task execution time exceeds the task execution timeout threshold corresponding to the execution unit, a timeout processing operation is performed.

[0073] Task execution timeout threshold: This refers to the maximum time limit set for an execution unit to determine whether the tasks executed under it have timed out. This threshold is a component of the scheduling strategy and can be dynamically output by the reinforcement learning policy model based on the performance characteristics of different execution units.

[0074] Timeout handling operations: These refer to predefined response actions that the system automatically triggers when a task execution timeout is detected, such as forcibly terminating the execution of the task, marking it as a failure and logging it, or putting it back into the queue.

[0075] In this embodiment, the scheduling strategy is further expanded to include, in addition to concurrency control parameters, a task execution timeout threshold configured independently for each execution unit. After the system distributes the task to be processed to the target execution unit and starts its execution according to the scheduling strategy, it enters the monitoring phase. The system continuously monitors the task execution duration (i.e., the time elapsed from the start of task execution to the current or end time). The system compares this duration with the corresponding task execution timeout threshold used when it was distributed to the execution unit in real time or periodically. Once the system detects that the execution duration of a task exceeds the timeout threshold set for its execution unit, it determines that a timeout event has occurred. At this time, the system no longer waits passively but actively triggers predefined timeout handling operations. For example, in an audit scenario, for a Hive query that has not returned a result for a long time, the system can forcibly cancel the query, release the occupied resources, mark the audit task as "timeout failed," and trigger an alarm.

[0076] This solution sets differentiated dynamic timeout thresholds for different execution units and monitors execution duration, automatically triggering processing operations when a task times out. It intelligently prevents long-running tasks from excessively consuming resources, quickly releases resources and isolates problems when tasks malfunction, thus significantly enhancing the system's stability and self-healing capabilities in heterogeneous environments and avoiding overall scheduling chain blockage caused by individual task anomalies.

[0077] See Figure 3 This is a flowchart illustrating another embodiment of the task scheduling method provided in this application. Figure 3 The process shown above Figure 1 Based on the illustrated process, describe how to calculate a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators based on the feedback data. For example... Figure 3 As shown, the process may include the following steps: S301. Calculate the delay reward based on the task execution time in the feedback data; S302. Calculate the resource utilization rate reward based on the resource consumption data in the feedback data; S303. Calculate the timeout penalty based on the task completion status in the feedback data; S304. Calculate the business anomaly detection reward based on the task execution results in the feedback data; S305. Based on preset weights, the delay reward item, the resource utilization reward item, the timeout penalty item, and the business anomaly detection reward item are weighted and integrated to obtain the comprehensive reward value.

[0078] For ease of understanding, the following provides a unified explanation of S301-305: Delayed reward: This refers to a numerical item that is negatively correlated with the time taken to complete a task. Its purpose is to encourage the system to shorten the average processing time of a task.

[0079] Resource utilization bonus item: refers to a numerical item related to the consumption level of system resources (such as CPU and memory), and its design purpose is to guide the system to maintain resource utilization within an ideal range.

[0080] Timeout penalty: This refers to a negative value that is positively correlated with the proportion of task timeouts. Its purpose is to provide clear negative feedback for the adverse event of task execution timeout.

[0081] Business Anomaly Discovery Reward: This refers to a numerical item that is positively correlated with the preset business anomalies discovered through task execution. Its core design is to directly quantify business value goals into a driving force for scheduling optimization.

[0082] Weighted fusion: refers to the calculation process of multiplying multiple numerical items by their corresponding preset weight coefficients and then summing them to obtain a comprehensive scalar value.

[0083] In this embodiment, the system first extracts four key information categories from the feedback data: task execution time, resource consumption data, task completion status, and task execution result. In step S301, a latency reward is calculated based on the task execution time (e.g., calculating average latency), with the principle that the lower the latency, the larger the value of this item. In step S302, a resource utilization reward is calculated based on resource consumption data (e.g., CPU utilization), with this item receiving a higher value when the utilization is within the target high-efficiency range. In step S303, the task timeout ratio is statistically analyzed based on the task completion status, and a timeout penalty is calculated accordingly; the higher the timeout ratio, the larger the absolute value of the negative value of this item. In step S304, the system analyzes the task execution results to determine if a preset type of business anomaly (such as data errors detected during audits) is discovered, and a business anomaly discovery reward is calculated accordingly; if an anomaly is discovered, this item has a positive value. Finally, in step S305, the system performs a weighted fusion calculation on the above four items according to the predefined weight coefficients (w1, w2, w3, w4), that is, using the formula: Comprehensive reward value = w1 Delayed reward items + w2 Resource utilization bonus items +w3 Business Anomaly Discovery Rewards +w4 The timeout penalty is used to obtain the final overall reward value. Here, w4 is usually negative, or the timeout penalty itself is negative, to achieve the penalty effect.

[0084] Figure 3 The process illustrated uses a clearly structured and targeted mathematical function to map the four scheduling optimization objectives—"reducing latency," "optimizing resource utilization," "encouraging the detection of business anomalies," and "suppressing task timeouts"—into a quantifiable scalar signal. This signal serves as the direct optimization objective of the reinforcement learning model, simultaneously driving the model to search the policy space for scheduling strategies that improve system performance efficiency while prioritizing core business value (timely anomaly detection). This systematically solves the fundamental problem of traditional methods' inability to balance multi-dimensional objectives in dynamic heterogeneous environments, achieving synergistic optimization of the scheduling agent in terms of system performance and business effectiveness.

[0085] In one embodiment, S105 may specifically include the following steps: inputting the environment state vector and the comprehensive reward value into the reinforcement learning policy model; calculating the value evaluation score of the current state based on the environment state vector and the comprehensive reward value by the value evaluation network in the reinforcement learning policy model; and updating the policy parameters based on the value evaluation score and the comprehensive reward value by the policy generation network in the reinforcement learning policy model.

[0086] Value evaluation network: refers to the sub-network in a reinforcement learning policy model used to evaluate the value of a given environmental state (i.e., the expected long-term cumulative reward). In the field of reinforcement learning, it is often called the "Critic" network.

[0087] Policy generation network: refers to the sub-network in a reinforcement learning policy model used to generate specific scheduling policies (actions) based on the environmental state. In the field of reinforcement learning, it is often called the "Actor" network.

[0088] Value assessment score: refers to a scalar value score calculated by the value assessment network for the system state represented by the current environmental state vector. This score represents the model's expectation of obtaining cumulative rewards in this state in the future.

[0089] Policy parameters: These refer to the adjustable internal variables (such as neural network weights and biases) that constitute the policy generation network. Updating these parameters will directly change the behavior of the model's output policy.

[0090] In this embodiment, after the current scheduling cycle ends, the system inputs the environmental state vector (i.e., the state upon which the model makes its decisions) and the comprehensive reward value (i.e., the evaluation of the decision's effectiveness) generated in this cycle into the reinforcement learning policy model. First, the value evaluation network within the model is activated. Based on the input environmental state vector and the latest feedback provided by the comprehensive reward value, it calculates and outputs a value evaluation score. This score evaluates the potential value of executing subsequent policies in the current state. Subsequently, the policy generation network within the model is activated, receiving the aforementioned value evaluation score and comprehensive reward value as key inputs. Based on these inputs and following a specific reinforcement learning algorithm (such as the proximal policy optimization algorithm preferred in this embodiment), the network calculates the difference (policy gradient) between its output (i.e., the actual scheduling policy used in this round) and the ideal policy, and adjusts (updates) its internal policy parameters accordingly. This process enables the model to learn how to adjust its output scheduling policy when facing similar environmental state vectors to obtain higher value evaluation scores and comprehensive reward values ​​in the future.

[0091] This scheme uses a value assessment network to continuously evaluate states, enabling the model to develop an understanding of the long-term value of the environment. Meanwhile, the policy generation network updates parameters based on immediate rewards and long-term value assessments, ensuring that the optimization of the scheduling policy not only focuses on immediate gains but also considers long-term system stability and efficiency. This mechanism allows the scheduling system to learn and iterate efficiently and stably from the scheduling experience of each round, continuously approaching the optimal scheduling policy in complex dynamic environments. This achieves the autonomous evolution and continuous improvement of the system's scheduling capabilities, fundamentally solving the problem of static policies being unable to adapt.

[0092] In another embodiment, the method may further include the following steps: when a new execution unit is accessed in the scheduling environment, using the current reinforcement learning policy model as the base model, generating a scheduling policy based on the environment state vector containing the state of the new execution unit, and distributing tasks according to the scheduling policy; collecting feedback data of task execution, and calculating a comprehensive reward value based on the feedback data; updating the base model using the comprehensive reward value to obtain a reinforcement learning policy model adapted to the scheduling environment containing the new execution unit.

[0093] Base model: refers to the reinforcement learning strategy model that has been trained and is in a stable operating state before the scheduling environment changes (new execution units are added). It serves as the starting point for learning in the new environment and carries the system's existing scheduling experience and knowledge.

[0094] Adaptation: refers to the process of making targeted adjustments and updates to the base model so that its decision-making capabilities can be effectively extended to the changed scheduling environment that includes new components.

[0095] In this embodiment, when a new execution unit is detected joining the scheduling environment (e.g., adding a StarRocks database as an audit data source), the system initiates this adaptation process. First, the system uses the currently used, stable reinforcement learning policy model as the base model. The system begins collecting an expanded environment state vector, including the initial state information of the new execution unit (such as the initial number of connections and baseline response time). Then, the system uses this expanded state vector as input to drive the base model to perform forward inference and generate a scheduling policy. At this point, since the base model has not yet learned the characteristics of the new unit, its output policy may assign conservative concurrency weights to the new unit. The system then executes task distribution according to the scheduling policy, which may include issuing exploratory tasks to the new execution unit. After task execution, the system collects feedback data (including the task time and success rate of the new unit) and calculates a comprehensive reward value. This reward value reflects the immediate effect of the execution policy in the new environment. Finally, the system updates the base model using the comprehensive reward value, that is, it uses the "state-action-reward" experience data generated in the new environment to fine-tune the model parameters in one (or more) rounds. The model obtained through this update process is the adapted reinforcement learning policy model, which can understand the behavioral characteristics of new units to a certain extent and adjust the scheduling policy accordingly.

[0096] This solution provides an efficient method for rapid cold start and online adaptation of models. When the system architecture expands, it avoids the need to discard existing models and undergo a lengthy and unstable retraining process from scratch. Instead, based on the base model, a usable model adapted to the new environment can be quickly fine-tuned through short-cycle online interaction and learning. This reduces the operational and time costs of system expansion and ensures a smooth transition and rapid convergence of scheduling performance during the expansion process, greatly improving the scalability and practicality of the entire scheduling system.

[0097] To facilitate understanding of the technical solution of this application, the following will be combined with Figure 4 and 5 This application provides a detailed description of a heterogeneous data source task scheduling method based on reinforcement learning, according to an embodiment of the present application. Figure 4 This is a schematic diagram of the system architecture of the intelligent scheduling engine provided in the embodiments of this application. Figure 5 This is a diagram illustrating the corresponding workflow and learning mechanism.

[0098] like Figure 4 As shown, the system architecture of this embodiment includes the following core components: Heterogeneous data source layer: This layer serves as the task execution unit and includes, but is not limited to, relational databases (such as MySQL), distributed databases (such as TiDB), batch processing engines (such as Hive), and real-time analytics databases (such as StarRocks). These data sources report performance metrics in real time by deploying probes or agents.

[0099] The state awareness module is responsible for periodically (e.g., every 5 seconds) collecting three types of state information: the load status of each data source, including the number of active connections, P99 response time, and computing resource utilization; the status of the pending task queue, including the number of tasks, priority, estimated execution time, and waiting time; and the resource status of the scheduler itself, including CPU utilization, available memory, and network I / O pressure. All collected metrics are normalized, concatenated to form a fixed-dimensional environment state vector, and stored in a feature vector pool for use by subsequent modules.

[0100] The reinforcement learning agent (RL agent) employs an Actor-Critic architecture model based on the Proximal Policy Optimization (PPO) algorithm. This module receives the environment state vector and calculates and outputs a scheduling policy through its internal network. This policy consists of a set of executable action parameters, including: a global concurrent task limit K; task allocation weight vectors W for each data source; and task execution timeout thresholds T_q for each data source.

[0101] Task generation and execution module: The task generator generates a specific task distribution sequence based on the scheduling strategy and task queue status; the dynamic concurrency controller calculates the concurrent execution quota of each data source based on the weight W and the global upper limit K; the action executor is responsible for sending the selected tasks to the corresponding data source for execution in the form of SQL.

[0102] Feedback and Learning Module: The reward calculator calculates the comprehensive reward value based on the task execution results (time spent, resource consumption, whether anomalies were found, etc.) according to a preset multi-objective reward function. The feedback loop transmits empirical data consisting of environmental state, actions (scheduling strategy), and reward value back to the RL Agent for model updates.

[0103] like Figure 5 As shown, the workflow and learning mechanism of this embodiment are as follows: Step 1: State Acquisition and Vectorization The system triggers batch audit tasks periodically (e.g., every minute), while simultaneously collecting real-time metrics from data sources, system resources, and task queues through a status awareness module. The collected raw data undergoes feature preprocessing and normalization to form an environment status vector.

[0104] Step 2: Strategy Generation and Action Execution The environment state vector is input into the Actor network in the RL Agent. The Actor network calculates the concurrency weight and global concurrency limit that each data source should obtain in the current state, and outputs a scheduling policy. Based on this policy, the system adjusts the concurrency limit K of each data source through a dynamic concurrency controller, selects tasks from the task queue according to priority and estimated time, and issues SQL to the corresponding data source for execution.

[0105] Step 3: Perform monitoring and reward calculation During task execution, the system monitors the execution results and time consumption. Upon completion, the reward calculator calculates the comprehensive reward value R for the current period based on the following four indicators: latency reward (encouraging a reduction in average query latency); resource utilization reward (encouraging maintaining CPU / memory utilization within the ideal range of 70%-80%); anomaly penalty (negative incentive for task timeouts); and business anomaly detection reward (positive reward for tasks that detect data anomalies). These four indicators are weighted and combined according to preset weights to obtain R.

[0106] Step 4: Model Update and Policy Optimization The empirical data, consisting of the state vector, scheduling policy, and overall reward value generated in this cycle, is input into the RL Agent for learning and updating: the Critic network updates its evaluation of the state value based on the state and reward; the Actor network updates its policy parameters based on the Critic's output and policy gradient (PPO algorithm). This process enables the model to gradually learn how to schedule in various environments to obtain higher long-term cumulative rewards.

[0107] Step 5: Policy Transfer and Adaptation The system continuously monitors the environment for new data sources. For example... Figure 5 As shown in the right-hand branch, when a new data source is detected (such as a newly added StarRocks cluster), the system triggers a policy migration mechanism: using the currently trained PPO model as the base model; collecting environment vectors containing the state of the new data source; performing exploratory scheduling in the new environment, collecting feedback and calculating rewards; and rapidly fine-tuning the base model through meta-learning to quickly adapt it to the characteristics of the new data source. After the migration is complete, the system enters the next scheduling cycle and continues to execute the closed-loop learning process from S501 to S504.

[0108] This embodiment achieves adaptive and efficient scheduling of batch tasks in heterogeneous data source environments through the aforementioned closed-loop perception-decision-execution-learning process. The system can perceive data source load, task queues, and system resource status in real time, and dynamically generate scheduling strategies that balance concurrency control, task priority, and timeout management based on reinforcement learning. By integrating a multi-objective reward mechanism that combines task execution efficiency and business value, the system drives the agent to prioritize the execution of high-value tasks and anomaly detection while reducing average latency and optimizing resource utilization. When accessing new data sources, a policy migration mechanism quickly fine-tunes the model, achieving smooth cold starts and continuous online optimization. This method effectively solves the problems of traditional static rule scheduling in heterogeneous and dynamic environments, such as difficulty in balancing multiple objectives and the tendency for uneven load distribution and queue head blocking, significantly improving the overall system throughput, resource utilization efficiency, and business response time.

[0109] Based on the same technical concept, embodiments of this application also provide a task scheduling device, such as... Figure 6 As shown, the device includes: The acquisition module 61 is used to acquire the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller, so as to generate an environment status vector. Input module 62 is used to input the environment state vector into the reinforcement learning policy model to obtain a scheduling policy, wherein the scheduling policy is used to determine the task allocation scheme for each execution unit; The distribution module 63 is used to distribute the tasks to be processed to the corresponding execution units for execution according to the scheduling strategy, and to collect feedback data during the task execution process; Calculation module 64 is used to calculate a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators based on the feedback data. The update module 65 is used to update the reinforcement learning policy model using the comprehensive reward value.

[0110] In one possible implementation, the scheduling strategy includes a global concurrent task limit and concurrent task weights corresponding to each execution unit; The distribution module is specifically used for: Based on the global concurrent task limit and the concurrent task weights corresponding to each execution unit, calculate the concurrent execution quota for each execution unit; For each execution unit, determine the group of tasks to be processed corresponding to the execution unit from the queue of tasks to be processed; Based on the concurrent execution quota corresponding to the execution unit, select a corresponding number of tasks to be processed from the task group to be processed; The task to be processed is sent to the execution unit.

[0111] In one possible implementation, the distribution module is further configured to: For each task in the task group to be processed, a task score is calculated based on the task's priority and estimated execution time. Select a corresponding number of tasks as pending tasks according to their corresponding task scores from highest to lowest.

[0112] In one possible implementation, the scheduling strategy further includes a task execution timeout threshold for each execution unit; The device further includes a timeout processing module for: During the execution of the task to be processed by the execution unit, the task execution time is monitored; When the execution time of the task exceeds the timeout threshold of the task execution corresponding to the execution unit, a timeout processing operation is performed.

[0113] In one possible implementation, the computing module is specifically used for: Calculate the delay reward based on the task execution time in the feedback data; Based on the resource consumption data in the feedback data, calculate the resource utilization rate reward item; Calculate the timeout penalty based on the task completion status in the feedback data; Based on the task execution results in the feedback data, calculate the reward for discovering business anomalies; Based on preset weights, the delay reward item, the resource utilization reward item, the timeout penalty item, and the business anomaly detection reward item are weighted and merged to obtain the comprehensive reward value.

[0114] In one possible implementation, the update module is specifically used for: The environment state vector and the comprehensive reward value are input into the reinforcement learning policy model. The value evaluation network in the reinforcement learning policy model calculates the value evaluation score of the current state based on the environment state vector and the comprehensive reward value. The policy generation network in the reinforcement learning policy model updates the policy parameters based on the value evaluation score and the comprehensive reward value.

[0115] In one possible implementation, the device further includes an adapter module for: When a new execution unit is connected to the scheduling environment, the current reinforcement learning strategy model is used as the base model. Based on the environment state vector containing the state of the new execution unit, a scheduling strategy is generated and the task distribution is performed according to the scheduling strategy. Collect feedback data on task execution and calculate the comprehensive reward value based on the feedback data; The base model is updated using the comprehensive reward value to obtain a reinforcement learning policy model adapted to the scheduling environment containing the new execution unit.

[0116] Based on the same technical concept, embodiments of this application also provide an electronic device, such as... Figure 7 As shown, it includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; When processor 111 executes a program stored in memory 113, it performs the following steps: The system collects the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller to generate an environment state vector. The environmental state vector is input into the reinforcement learning policy model to obtain a scheduling policy, which is used to determine the task allocation scheme for each execution unit. According to the scheduling strategy, the tasks to be processed are distributed to the corresponding execution units for execution, and feedback data during the task execution process is collected. Based on the feedback data, a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators is calculated. The reinforcement learning policy model is updated using the comprehensive reward value.

[0117] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0118] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0119] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0120] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0121] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described task scheduling methods.

[0122] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the task scheduling methods described above.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0125] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0126] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A task scheduling method, characterized in that, The method includes: The system collects the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller to generate an environment state vector. The environmental state vector is input into the reinforcement learning policy model to obtain a scheduling policy, which is used to determine the task allocation scheme for each execution unit. According to the scheduling strategy, the tasks to be processed are distributed to the corresponding execution units for execution, and feedback data during the task execution process is collected. Based on the feedback data, a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators is calculated. The reinforcement learning policy model is updated using the comprehensive reward value.

2. The method according to claim 1, characterized in that, The scheduling strategy includes a global concurrent task limit and concurrent task weights corresponding to each execution unit; The step of distributing the tasks to be processed to the corresponding execution units for execution according to the scheduling policy includes: Based on the global concurrent task limit and the concurrent task weights corresponding to each execution unit, calculate the concurrent execution quota for each execution unit; For each execution unit, determine the group of tasks to be processed corresponding to the execution unit from the queue of tasks to be processed; Based on the concurrent execution quota corresponding to the execution unit, select a corresponding number of tasks to be processed from the task group to be processed; The task to be processed is sent to the execution unit.

3. The method according to claim 2, characterized in that, The step of selecting a corresponding number of tasks to be processed from the task group according to the concurrent execution quota corresponding to the execution unit includes: For each task in the task group to be processed, a task score is calculated based on the task's priority and estimated execution time. Select a corresponding number of tasks as pending tasks according to their corresponding task scores from highest to lowest.

4. The method according to claim 2, characterized in that, The scheduling strategy also includes task execution timeout thresholds for each execution unit; After the task to be processed is sent to the execution unit, the process further includes: During the execution of the task to be processed by the execution unit, the task execution time is monitored; When the execution time of the task exceeds the timeout threshold of the task execution corresponding to the execution unit, a timeout processing operation is performed.

5. The method according to claim 1, characterized in that, Based on the feedback data, a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators is calculated, including: Calculate the delay reward based on the task execution time in the feedback data; Based on the resource consumption data in the feedback data, calculate the resource utilization rate reward item; Calculate the timeout penalty based on the task completion status in the feedback data; Based on the task execution results in the feedback data, calculate the reward for discovering business anomalies; Based on preset weights, the delay reward item, the resource utilization reward item, the timeout penalty item, and the business anomaly detection reward item are weighted and merged to obtain the comprehensive reward value.

6. The method according to claim 1, characterized in that, The step of updating the reinforcement learning policy model using the comprehensive reward value includes: The environment state vector and the comprehensive reward value are input into the reinforcement learning policy model. The value evaluation network in the reinforcement learning policy model calculates the value evaluation score of the current state based on the environment state vector and the comprehensive reward value. The policy generation network in the reinforcement learning policy model updates the policy parameters based on the value evaluation score and the comprehensive reward value.

7. The method according to claim 1, characterized in that, The method further includes: When a new execution unit is connected to the scheduling environment, the current reinforcement learning strategy model is used as the base model. Based on the environment state vector containing the state of the new execution unit, a scheduling strategy is generated and the task distribution is performed according to the scheduling strategy. Collect feedback data on task execution and calculate the comprehensive reward value based on the feedback data; The base model is updated using the comprehensive reward value to obtain a reinforcement learning policy model adapted to the scheduling environment containing the new execution unit.

8. A task scheduling device, characterized in that, The device includes: The acquisition module is used to collect the load status of each execution unit in the scheduling environment, the queue status of the task queue to be processed, and the system resource status of the scheduling controller in order to generate an environment status vector. An input module is used to input the environment state vector into a reinforcement learning policy model to obtain a scheduling policy, which is used to determine a task allocation scheme for each execution unit. The distribution module is used to distribute the tasks to be processed to the corresponding execution units for execution according to the scheduling strategy, and to collect feedback data during the task execution process; The calculation module is used to calculate a comprehensive reward value that integrates task execution efficiency indicators and task business value indicators based on the feedback data. An update module is used to update the reinforcement learning policy model using the comprehensive reward value.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the task scheduling method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the task scheduling method according to any one of claims 1-7.

Images