Resource scheduling method and system based on reinforcement learning

Abstract

The invention belongs to the technical field of resource scheduling, and particularly discloses a resource scheduling method and system based on reinforcement learning, and the method comprises the steps: describing the dependency and conflict relation between tasks through constructing a task causal relation graph which can be dynamically updated; constructing a state space and an action space based on a current task state and a causal relationship, training an intelligent agent by adopting a reinforcement learning model in combination with a multi-target reward function, and generating a scheduling and resource allocation strategy; linear programming and heuristic joint resource allocation are carried out under strategy guidance, and task priorities and causal relationships are dynamically adjusted in combination with task states; by collecting execution data and analyzing strategy deviation, a model structure and parameters are further adjusted, and a complete closed-loop optimization process is formed. According to the method, task priority dynamic adjustment, resource allocation strategy self-adaptive updating and scheduling process closed-loop optimization are realized, and the method is suitable for complex project management scenes under multi-task and multi-resource constraints.

Description

Technical Field

[0001] This invention belongs to the field of resource scheduling technology, specifically relating to a resource scheduling method and system based on reinforcement learning. Background Technology

[0002] In complex project management and multi-task scheduling scenarios, effectively coordinating task execution order and resource allocation remains a core challenge in project execution systems. Existing technologies often employ fixed rules, manual experience, or static optimization methods for scheduling and resource allocation, which struggle to address uncertainties such as dynamic dependencies between tasks, changes in resource bottlenecks, and fluctuations in task progress during project execution.

[0003] Furthermore, traditional methods often fail to adequately characterize the implicit relationships or causal connections between tasks, resulting in task ranking and resource adjustment failing to reflect systematic global optimization. While some methods introduce machine learning models to assist scheduling, most are limited to static data-driven approaches, lacking dynamic adaptive mechanisms and making it difficult to form a closed-loop optimization process.

[0004] Especially in scenarios with scarce resources, compressed schedules, or fluctuations in critical tasks, existing scheduling methods are prone to accumulating resource conflicts, imbalances in task chains, and loss of overall progress control. They lack the systematic modeling and optimization capabilities to balance multiple objectives among tasks, resources, and benefits. Summary of the Invention

[0005] This invention addresses the problems in the prior art by providing a resource scheduling method and system based on reinforcement learning. This solves the problems in the prior art where existing scheduling methods easily lead to the accumulation of resource conflicts, imbalance of task chains, and loss of overall progress control, and lack the ability to systematically model and optimize the balance between multiple objectives of tasks, resources, and benefits.

[0006] The technical solution adopted in this invention is as follows: Firstly, this application provides a resource scheduling method based on reinforcement learning, which includes the following steps: Step S1: Obtain project and task information, construct a task causal relationship diagram, and dynamically update the causal relationship diagram according to the changes in task status during project execution; Step S2: Based on the causal relationship graph and the current project state, construct the state space and action space for generating reinforcement learning strategies; Step S3: Construct a multi-objective reward function, which includes resource utilization efficiency indicators, progress delay indicators, and task revenue indicators; Step S4: Based on the state space, action space, and reward function, construct a reinforcement learning model, train the intelligent agent through the reinforcement learning model, and generate the optimal strategy for task scheduling and resource allocation. Step S5: Allocate resources according to the optimal strategy, using a combination of linear programming and heuristic algorithms to generate a preliminary resource allocation scheme; Based on task priority and dynamic resource status, make partial optimizations and adjustments to the initial resource allocation plan; Step S6: Monitor the task execution status in real time, identify situations such as task progress delays, early completion, or resource bottlenecks, and update the task cause-effect graph and priority ranking according to the task execution status. Step S7: Collect execution data, including task completion status, resource utilization efficiency, and strategy benefits. Analyze the execution data and adjust the cause-effect graph update strategy, reward function parameters, and resource allocation logic.

[0007] Furthermore, in step S1, the constructed task causal relationship graph is a graph structure model. Task nodes have explicit attributes including task priority, task duration, and resource requirements. Edges between tasks represent dependency relationships or resource conflict relationships. The weight of the edges is determined based on historical task data and domain knowledge. The weight of the edges is adjusted in real time according to the task status during project execution.

[0008] Furthermore, in step S2, the state space includes the task feature vector f i The weights of the edges and the current resource allocation status; eigenvector f i Composed of the explicit attributes of the task and semantic embedding vectors, it is represented as:

[0009] Where, p i For task priority, t i For the remaining construction period, r i For resource needs, e i It is a semantic embedding vector; The action space includes tasks such as prioritizing tasks, changing resource allocation schemes, and modifying task node attributes. Each decision selects an action based on the policy function π(a|s), where π(a|s) represents the probability of choosing action a in state s.

[0010] Furthermore, in step S3, the multi-objective reward function is as follows:

[0011] Among them: U r For resource utilization efficiency; D p λ1, λ2, and λ3 represent the degree of schedule delay; G represents the task reward; and λ1, λ2, and λ3 are weight parameters.

[0012] Furthermore, step S4 includes the following steps: Step S4-1: Based on the reward function constructed in step S3, construct a reinforcement learning model for learning scheduling strategies; Step S4-2: During training, an ε-greedy strategy is used for action selection. The current reward value and the next state are obtained through interaction, and (s,a,r,s′) quadruples are constructed for experience playback. Step S4-3: Optimize the Q function using the TD error update method. The update formula is as follows:

[0013] Where: α is the learning rate; γ is the discount factor; r is the current reward; s′ is the next state after action a; Step S4-4: When the strategy converges or meets the preset performance indicators, output the final scheduling and resource allocation strategy.

[0014] Furthermore, in step S5, the constraints introduced into the linear programming model include: The total allocation of each type of resource shall not exceed the corresponding available limit; The start time of each task must not be earlier than the completion time of its predecessor task; For tasks on the critical path, their resource requirements are set as strong constraints that must be met in the model. Resource allocation for all tasks should meet the minimum demand threshold limit; After the initial resource allocation plan is generated, the following heuristic local adjustment strategy is executed: Based on the node priority in the task causal relationship graph, incremental resource iterative allocation is performed on high-priority tasks. Perform resource reduction or postponement scheduling on low-priority tasks; In the event of resource conflicts between tasks, a resource conflict matrix is ​​constructed, and a round-robin method based on a greedy strategy is used to rearrange the resource occupation order in turn; For situations where there is a mismatch between task resource requirements and available resources, a resource reallocation strategy based on neighborhood search is introduced for further optimization.

[0015] Furthermore, step S6 includes the following steps: Step S6-1: Based on real-time monitoring data during project execution, collect the current progress, actual start time, execution duration, and resource usage details of each task, and form a task execution status vector; Step S6-2: Compare the task execution status vector with the plan baseline value to identify the task status type. The status types include five types: delayed, ahead of schedule, paused, not started, and abnormal interruption. Step S6-3: When a task status of delay or resource conflict is detected, update the edge weight values ​​associated with the task in the task causal relationship graph. The edge weight values ​​are adjusted by increasing or decreasing the weight according to the task delay duration, resource occupancy ratio and critical path impact coefficient. Step S6-4: Based on the updated causal relationship graph, recalculate the priority index of each task node. The priority index is determined by a joint evaluation of the node's in-degree, out-degree, remaining task duration, and the scope of delay impact.

[0016] Furthermore, step S7 includes the following steps: Step S7-1: After each scheduling cycle ends, collect the task completion status, resource allocation results and actual revenue values ​​within the current cycle, and generate an execution dataset. The dataset includes task ID, completion status identifier, resource usage details and corresponding strategy ID. Step S7-2: Perform structured statistics on each data record in the execution dataset, extract task success rate, resource utilization rate and revenue deviation value, and construct an indicator analysis matrix. The matrix fields include: task completion rate, resource matching deviation rate and critical path execution error. Step S7-3: Based on the statistical results of the indicator analysis matrix, adjust the edge weights of the task causal relationship graph in batches. The adjustment range of the edge weights is set according to the deviation gradient threshold. Step S7-4: Based on the profit deviation distribution, the weight parameters in the reward function constructed in step S3 are numerically corrected. The correction is based on the comparison ratio between the resource deviation rate and the key task score. Step S7-5: In the policy training loop, the learning rate of the reinforcement learning model is fine-tuned based on the cumulative feedback error, and the model update frequency is dynamically adjusted using an exponential decay strategy.

[0017] Secondly, this application provides a resource scheduling system based on reinforcement learning, the system comprising: The causal reasoning and task dependency modeling module is used to receive project information and task information, construct a task causal relationship graph. The causal relationship graph is in graph structure form, where nodes represent task entities, edges represent dependencies or resource conflicts between tasks, and the weight of the edges can be dynamically adjusted based on task status, historical data, and graph structure changes. The reinforcement learning agent and optimization decision module is used to construct a state space and action space based on the causal relationship graph and project state information. The state space includes task priority, remaining duration, resource requirements and edge weights, and the action space includes task priority reordering, resource allocation adjustment and task attribute modification. The module builds a reinforcement learning model and trains the agent to generate task scheduling and resource allocation strategies. The resource allocation optimization module is used to allocate resources according to the strategy output by the optimization decision module. The resource allocation optimization module includes a linear programming unit and a heuristic adjustment unit, which are used to generate a preliminary resource allocation plan and make local adjustments based on task priority and resource status, respectively. The progress adjustment and plan update module is used to monitor the task execution status, collect task progress information, resource usage and task execution sequence, and update the edge weights and task priority ranking results in the causal relationship graph based on the current execution status. The feedback optimization and closed-loop adjustment module is used to collect execution data for each task cycle. The execution data includes task completion status, resource usage, and strategy execution deviation. The module dynamically adjusts the causal graph structure, reward function parameters, and reinforcement learning model training parameters.

[0018] Furthermore, the reinforcement learning agent and optimization decision-making module includes: The state representation unit is used to construct the state space by combining the task node attributes and edge weights in the causal relationship graph. The task node attributes include task priority, remaining duration, resource requirements, and semantic feature vectors. The action generation unit is used to set the action space according to the state space. The action space includes tasks such as adjusting the priority order, modifying resource allocation values, and updating task attributes. The strategy execution unit is used to select a specific action from the action space in each training step based on a preset probability selection mechanism, and interact with the environment to obtain reward information in the current state and the new state after execution. The experience storage unit is used to store data samples formed by states, actions, rewards and new states into the experience buffer, and supports the extraction of batch historical samples from it for model training. The policy update unit is used to adjust the internal model parameters and update the policy mapping relationship between states and actions based on the deviation between the current reward and the predicted value of the next state. The strategy output unit is used to output the task scheduling order and resource allocation results after the model training meets the termination condition, for the resource allocation optimization module to execute.

[0019] As can be seen from the above technical solutions, the advantages of the present invention are: By constructing a dynamically updatable task causal relationship graph, this method can accurately describe the explicit and implicit dependencies between tasks and reflect the impact of task state evolution on subsequent tasks. Based on reinforcement learning training, the intelligent agent can autonomously generate scheduling and resource allocation strategies under dynamic changes in task state and resource configuration, and has self-learning and adaptive capabilities.

[0020] Furthermore, the method introduces a multi-objective reward function to achieve a comprehensive trade-off between resource utilization efficiency, project schedule control, and key task benefits. It also forms an optimization closed loop through real-time status monitoring and execution data analysis, supporting dynamic iterative updates of the strategy. This method possesses advantages such as structured approach, intelligence, and dynamic evolution, making it suitable for optimizing and scheduling multi-task, multi-resource, and multi-objective tasks in highly complex project environments. Attached Figure Description

[0021] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the steps of the resource scheduling method based on reinforcement learning in this embodiment. Figure 2 This is a flowchart of a resource scheduling system based on reinforcement learning in the embodiment. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1 As shown, this invention provides a resource scheduling method based on reinforcement learning, comprising the following steps: Step S1: Obtain project and task information, construct a task causal relationship diagram, and dynamically update the causal relationship diagram according to the changes in task status during project execution; The constructed task causal relationship graph is a graph structure model. Task nodes have explicit attributes including task priority, task duration and resource requirements. Edges between tasks represent dependency relationships or resource conflict relationships. The weight of the edges is determined based on historical task data and domain knowledge. The weight of the edges is adjusted in real time according to the task status during project execution. Step S2: Based on the causal relationship graph and the current project state, construct the state space and action space for generating reinforcement learning strategies; The state space includes the task feature vector f i The weights of the edges and the current resource allocation status; eigenvector f i Composed of the explicit attributes of the task and semantic embedding vectors, it is represented as:

[0025] Where, p i For task priority, t i For the remaining construction period, r i For resource needs, e i It is a semantic embedding vector; The action space includes tasks such as prioritizing tasks, changing resource allocation schemes, and modifying task node attributes. Each decision selects an action based on the policy function π(a|s), where π(a|s) represents the probability of choosing action a in state s. Step S3: Construct a multi-objective reward function, which includes resource utilization efficiency indicators, progress delay indicators, and task revenue indicators; The multi-objective reward function is as follows:

[0026] Among them: U r For resource utilization efficiency; D p λ1 represents the degree of schedule delay; G represents the task reward; λ1, λ2, λ3 are weight parameters. Step S4: Based on the state space, action space, and reward function, construct a reinforcement learning model, train the intelligent agent through the reinforcement learning model, and generate the optimal strategy for task scheduling and resource allocation. Step S4-1: Based on the reward function constructed in step S3, construct a reinforcement learning model for learning scheduling strategies; Step S4-2: During training, an ε-greedy strategy is used for action selection. The current reward value and the next state are obtained through interaction, and (s,a,r,s′) quadruples are constructed for experience playback. Step S4-3: Optimize the Q function using the TD error update method. The update formula is as follows:

[0027] Where: α is the learning rate; γ is the discount factor; r is the current reward; s′ is the next state after action a; Step S4-4: When the strategy converges or meets the preset performance indicators, output the final scheduling and resource allocation strategy; Step S5: Allocate resources according to the optimal strategy, using a combination of linear programming and heuristic algorithms to generate a preliminary resource allocation scheme; The constraints introduced in the linear programming model include: The total allocation of each type of resource shall not exceed the corresponding available limit; The start time of each task must not be earlier than the completion time of its predecessor task; For tasks on the critical path, their resource requirements are set as strong constraints that must be met in the model. Resource allocation for all tasks should meet the minimum demand threshold limit; After the initial resource allocation plan is generated, the following heuristic local adjustment strategy is executed: Based on the node priority in the task causal relationship graph, incremental resource iterative allocation is performed on high-priority tasks. Perform resource reduction or postponement scheduling on low-priority tasks; In the event of resource conflicts between tasks, a resource conflict matrix is ​​constructed, and a round-robin method based on a greedy strategy is used to rearrange the resource occupation order in turn; For situations where there is a mismatch between task resource requirements and available resources, a resource reallocation strategy based on neighborhood search is introduced for further optimization. Based on task priority and dynamic resource status, make partial optimizations and adjustments to the initial resource allocation plan; Step S6: Monitor the task execution status in real time, identify situations such as task progress delays, early completion, or resource bottlenecks, and update the task cause-effect graph and priority ranking according to the task execution status. Step S6-1: Based on real-time monitoring data during project execution, collect the current progress, actual start time, execution duration, and resource usage details of each task, and form a task execution status vector; Step S6-2: Compare the task execution status vector with the plan baseline value to identify the task status type. The status types include five types: delayed, ahead of schedule, paused, not started, and abnormal interruption. Step S6-3: When a task status of delay or resource conflict is detected, update the edge weight values ​​associated with the task in the task causal relationship graph. The edge weight values ​​are adjusted by increasing or decreasing the weight according to the task delay duration, resource occupancy ratio and critical path impact coefficient. Step S6-4: Based on the updated causal relationship graph, recalculate the priority index of each task node. The priority index is determined by a joint evaluation of the node's in-degree, out-degree, remaining task duration, and the scope of delay impact. Step S7: Collect execution data, including task completion status, resource utilization efficiency, and strategy benefits. Analyze the execution data and adjust the cause-effect graph update strategy, reward function parameters, and resource allocation logic. Step S7-1: After each scheduling cycle ends, collect the task completion status, resource allocation results and actual revenue values ​​within the current cycle, and generate an execution dataset. The dataset includes task ID, completion status identifier, resource usage details and corresponding strategy ID. Step S7-2: Perform structured statistics on each data record in the execution dataset, extract task success rate, resource utilization rate and revenue deviation value, and construct an indicator analysis matrix. The matrix fields include: task completion rate, resource matching deviation rate and critical path execution error. Step S7-3: Based on the statistical results of the indicator analysis matrix, adjust the edge weights of the task causal relationship graph in batches. The adjustment range of the edge weights is set according to the deviation gradient threshold. Step S7-4: Based on the profit deviation distribution, the weight parameters in the reward function constructed in step S3 are numerically corrected. The correction is based on the comparison ratio between the resource deviation rate and the key task score. Step S7-5: In the policy training loop, the learning rate of the reinforcement learning model is fine-tuned based on the cumulative feedback error, and the model update frequency is dynamically adjusted using an exponential decay strategy.

[0028] In some embodiments, please refer to Figure 2 As shown, this application provides a resource scheduling system based on reinforcement learning, the system comprising: The causal reasoning and task dependency modeling module is used to receive project information and task information, construct a task causal relationship graph. The causal relationship graph is in graph structure form, where nodes represent task entities, edges represent dependencies or resource conflicts between tasks, and the weight of the edges can be dynamically adjusted based on task status, historical data, and graph structure changes. The causal reasoning and task dependency modeling module is implemented based on the user request input and processing module and is the entry module of this invention. This module is mainly used to receive user requests in various forms, including text input, voice input and direct scheduling instructions, and to convert the natural language or instructions input by the user into structured data through a semantic parser, thereby providing a clear and accurate task description for subsequent modules of this system.

[0029] User input requests can be presented in the following forms, for example: scheduling tasks (such as "schedule maintenance for equipment A"), status queries (such as "current status of equipment B"), or process adjustments (such as "optimize production line process"). The module extracts key elements from user commands using semantic parsing technology, including task type (such as maintenance, inspection, production, etc.) and parameters (such as equipment name, time requirement, priority, etc.). It constructs a causal relationship graph and determines the specific intent of the request through causal reasoning. At the same time, to ensure the integrity of complex commands, the module combines a context completion mechanism and extracts background information from previous rounds of interaction through hierarchical memory to compensate for conditions or dependencies that may be omitted in the user input.

[0030] After processing, the user request is transformed into structured instructions, such as "Task Type: Device Check; Device Name: Device A; Priority: High; Time: Immediate". These instructions are then passed to the dynamic knowledge base module for further retrieval of relevant contextual information, providing data support for scheduling generation. If the user's input is vague or lacks sufficient information, the module will guide the user to supplement the details through a prompt feedback mechanism to ensure that the generated instructions are complete and logical.

[0031] The reinforcement learning agent and optimization decision module is used to construct a state space and action space based on the causal relationship graph and project state information. The state space includes task priority, remaining duration, resource requirements and edge weights, and the action space includes task priority reordering, resource allocation adjustment and task attribute modification. The module builds a reinforcement learning model and trains the agent to generate task scheduling and resource allocation strategies. The reinforcement learning agent and optimization decision-making module generates dynamic task adjustment and resource allocation strategies based on the task causal relationship graph and the current state of the project using deep reinforcement learning algorithms. The goal of this module is to achieve multi-objective optimization through optimal strategies, including resource saving, project duration optimization, and profit maximization. The optimal strategies generated by the reinforcement learning agent include task priority adjustment and resource reallocation. Specific output actions include increasing resources for high-priority tasks and adjusting the schedule of delayed tasks. The module evaluates the effectiveness of strategy execution in real time and feeds the improved task and resource configurations back to the project management platform, forming a dynamic optimization loop. The efficient optimization capability of the reinforcement learning agent significantly improves the flexibility and adaptability of project management.

[0032] The resource allocation optimization module is used to allocate resources according to the strategy output by the optimization decision module. The resource allocation optimization module includes a linear programming unit and a heuristic adjustment unit, which are used to generate a preliminary resource allocation plan and make local adjustments based on task priority and resource status, respectively. The resource allocation optimization module is responsible for dynamically allocating resources to each task in the project to achieve multi-objective optimization of resource utilization efficiency, task execution progress, and overall benefits. This module takes the output of the reinforcement learning agent, the task causal relationship graph, and real-time resource status as input to comprehensively analyze task priorities, dependencies, and resource requirements, constructing a dynamic resource allocation mechanism. Through this module, the system can effectively allocate the resources required by tasks under limited resource conditions and adapt to changes during project execution, optimizing the overall project management effect.

[0033] The module first analyzes the key attributes of each task, including priority, duration, resource requirements, and dependencies with other tasks. Combining the weights of task nodes and the dependency strength of edges in the causal graph, the module determines the priority order of tasks. For high-priority tasks, the system allocates resources preferentially, especially when their delays might affect subsequent tasks or the overall project schedule. For low-priority tasks, the allocation is flexibly adjusted based on remaining resources to ensure maximum resource utilization efficiency.

[0034] In its implementation, the module employs a phased optimization strategy. In the initial phase, linear programming is used to quickly generate a resource allocation scheme, comprehensively considering total resources, task priorities, and time constraints. The linear programming result provides a globally optimal preliminary solution for resource allocation. However, due to the complexity and uncertainty of actual task requirements, the module introduces heuristic algorithms for local optimization. For example, when critical task resources are insufficient or priorities are adjusted due to unforeseen circumstances, the module dynamically adjusts the resource allocation ratio by reassessing task dependencies and completion times. This two-tiered optimization strategy balances global optimization with flexible local adjustments, ensuring that the resource allocation results meet the actual needs of the project.

[0035] Furthermore, the module supports dynamic updates to resource allocation schemes. As tasks progress during project execution, some tasks may complete ahead of schedule, releasing resources, or new tasks may require resource reallocation. The module triggers re-optimization of resource allocation by monitoring resource usage and task status changes in real time. Specifically, the module reassesses the priority of incomplete tasks and adjusts the allocation strategy based on the current remaining resource amount. For example, tasks on the critical path will receive priority access to newly released resources, while peripheral tasks will have their resource allocation appropriately postponed or reduced based on their impact. This dynamic adjustment mechanism significantly improves the flexibility of resource utilization and the system's adaptability to unpredictable changes.

[0036] Ultimately, the resource allocation optimization module generates a complete resource allocation plan, including the allocation amount, time window, and expected completion time for each task, and feeds it back to the project management system for execution and verification. Through the combination of multi-objective optimization and dynamic scheduling, the module ensures that resources can maximize project value under limited conditions, while improving project execution efficiency and the level of intelligent management.

[0037] The progress adjustment and plan update module is used to monitor the task execution status, collect task progress information, resource usage and task execution sequence, and update the edge weights and task priority ranking results in the causal relationship graph based on the current execution status. The schedule adjustment and plan update module is an important component of the system that dynamically responds to changes in project progress. It aims to dynamically adjust task time nodes and priorities based on task completion status, actual resource utilization, and overall project goals, thereby optimizing the continuity and efficiency of project execution.

[0038] The module monitors task execution status in real time, including start time, progress, and resource consumption, to obtain the actual execution status of tasks. Based on this, the system updates task status, such as marking delayed tasks, tasks completed ahead of schedule, or tasks paused due to insufficient resources. This status information is integrated into a task causal relationship graph to dynamically adjust task priorities and their impact on subsequent tasks. The results of task status updates are not only used to adjust current tasks but also passed to relevant nodes and edge weights in the causal relationship graph by updating task dependencies. For example, when a critical task is delayed, the module recalculates its priority based on its impact on subsequent tasks and derives the optimal adjustment strategy to minimize the impact of the delay on the overall project schedule.

[0039] When task status changes, the module reassesses task scheduling based on the reinforcement learning agent's optimization strategy and current task dependencies. For tasks on the critical path, the module prioritizes ensuring their completion time to reduce the risk to the overall project schedule. For non-critical tasks, the module makes flexible adjustments based on resource availability and project flexibility, such as appropriately delaying start times or reducing resource allocation. The decision-making process for schedule adjustments considers not only the current task status but also the project's overall objectives. For example, when maximizing overall project benefits takes precedence over time requirements, the module may choose to prioritize high-benefit tasks, even if it results in slight delays to non-critical tasks. This decision-making process is guided by a multi-objective optimization function, ensuring that the adjusted schedule strikes a balance between efficiency and benefits.

[0040] After schedule adjustments are completed, the module generates a new project plan, covering the latest timelines, priorities, and resource allocation recommendations for each task. The plan update not only covers currently affected tasks but also adjusts other tasks whose dependencies have changed, ensuring the integrity and consistency of the project plan. The module also supports real-time output and execution of the plan; the updated task plan is directly transmitted to the resource allocation module and project management platform for coordinated execution. Simultaneously, the plan adjustment results are recorded and fed back to the system's closed-loop optimization module to improve adjustment strategies for similar future scenarios.

[0041] The feedback optimization and closed-loop adjustment module is used to collect execution data for each task cycle. The execution data includes task completion status, resource usage, and strategy execution deviation. The module dynamically adjusts the causal graph structure, reward function parameters, and reinforcement learning model training parameters.

[0042] The feedback optimization and closed-loop adjustment module is the core part of the system to achieve adaptive optimization capabilities. Its main function is to collect the execution data of each module in project management, and dynamically adjust system parameters and strategies by analyzing the detection results and the effectiveness of optimization strategies in real time, so as to achieve closed-loop optimization of the entire process.

[0043] This module collects operational data from each module in real time, including: Task dependency and causal relationship data: Analyze the update frequency and accuracy of the causal reasoning module, and assess the impact of changes in task dependencies on project optimization strategies.

[0044] Reinforcement learning agent output data: Monitor the effectiveness of reinforcement learning decisions, such as the degree to which optimization strategies improve resource utilization and task completion time.

[0045] Resource allocation and schedule adjustment data: Evaluate the efficiency of the resource allocation optimization module and the responsiveness of the schedule adjustment module, including the resource conflict resolution rate and the effectiveness of schedule deviation correction.

[0046] The module quantifies and evaluates the overall system performance by setting key metrics (such as task completion rate, resource utilization, and the proportion of delayed tasks) to identify and optimize bottlenecks. For example, when task delays occur frequently or resource allocation conflicts are frequent, the module will trigger targeted optimization adjustments.

[0047] Based on the collected feedback data, the module dynamically adjusts the parameters and strategies of each submodule to improve the overall performance of the system. For example: Task priority weight adjustment: When feedback indicates that the actual priority of some tasks does not match the weight calculated by the causal inference module, the system will reallocate the weights based on the task execution results to optimize the updating of the causal relationship graph.

[0048] Reinforcement learning agent policy updates: By monitoring the success rate of agent decisions, the module can adjust the weight coefficients in the reward function. For example, when resource utilization is too low, the module increases the weights related to resource utilization efficiency in the reward function, guiding the reinforcement learning agent to pay more attention to resource allocation.

[0049] Resource allocation and plan adjustment strategy optimization: Based on the feedback from the plan execution results, the module can dynamically optimize the resource allocation strategy, such as adjusting the total resource constraints or the scheduling priority between tasks, to ensure the efficiency of resources in actual use.

[0050] The module supports lightweight fine-tuning of reinforcement learning and causal inference models to improve performance in specific domains or scenarios. For example, when the system performs poorly in handling a particular type of task (such as frequent critical path delays), the module uses feedback data to train the model specifically, generating optimized strategies that better meet actual needs. Furthermore, the module forms a complete closed-loop processing mechanism by updating the causal relationship graph and optimization parameters in real time. All adjustments are recorded and used for strategy recommendations in subsequent similar projects, enabling the system to gradually accumulate experience and improve its adaptability to complex projects.

[0051] The module outputs optimization results to other modules, coordinating updates to task dependencies, resource allocation, and plan adjustment strategies. Simultaneously, the feedback optimization module generates performance reports, including trends in key metrics and evaluations of optimization effectiveness, providing users with transparent information on system improvements. This transparency helps users understand the system's operating mechanisms and provides a reference for project management.

[0052] Through the feedback optimization and closed-loop adjustment module, the system achieves self-learning and dynamic improvement capabilities. The introduction of this module significantly improves the system's robustness and flexibility, enabling it to provide continuously optimized solutions in diverse and complex project management scenarios.

[0053] In this embodiment, the reinforcement learning agent and optimization decision module includes: The state representation unit is used to construct the state space by combining the task node attributes and edge weights in the causal relationship graph. The task node attributes include task priority, remaining duration, resource requirements, and semantic feature vectors. The action generation unit is used to set the action space according to the state space. The action space includes tasks such as adjusting the priority order, modifying resource allocation values, and updating task attributes. The strategy execution unit is used to select a specific action from the action space in each training step based on a preset probability selection mechanism, and interact with the environment to obtain reward information in the current state and the new state after execution. The experience storage unit is used to store data samples formed by states, actions, rewards and new states into the experience buffer, and supports the extraction of batch historical samples from it for model training. The policy update unit is used to adjust the internal model parameters and update the policy mapping relationship between states and actions based on the deviation between the current reward and the predicted value of the next state. The strategy output unit is used to output the task scheduling order and resource allocation results after the model training meets the termination condition, for the resource allocation optimization module to execute.

[0054] In some embodiments, this application is used in industrial production workshops and production lines to optimize resource scheduling during the production process. By collecting multi-dimensional data such as production line equipment status, process parameters, and material inventory in real time, it comprehensively coordinates aspects such as worker hours, workpiece allocation, product assembly, and spare parts management, and coordinates and guides production project optimization. An analytical model is established to accurately identify production bottlenecks and resource conflicts. Simultaneously, combined with a reinforcement learning-based scheduler, it dynamically optimizes production resource allocation, achieving the following core functions: 1) intelligent scheduling of industrial products and priority scheduling of workpieces; 2) worker skill matching and balanced allocation of working hours; 3) optimization of assembly paths for multi-process collaboration; 4) spare parts management; 5) dynamic game optimization of capacity and delivery time. This system achieves closed-loop control of virtual and physical linkage through digital twin technology. It can not only respond in real time to production disturbances such as equipment failures / urgent order insertions, but also continuously iterate and optimize scheduling strategies through a continuous learning mechanism.

[0055] Furthermore, causal reasoning techniques are used to construct a multi-level causal relationship graph between production tasks, guiding intelligent production scheduling and workpiece priority dispatch. An industrial IoT data acquisition system is built, integrating MES system work order data, PLC equipment operating parameters, RFID material tracking data, and WMS warehouse information. Through time-series data alignment and feature extraction, a feature warehouse is formed, encompassing dimensions such as equipment status, process quality, and resource availability. A sliding window statistical method is employed to extract dynamic features such as production line cycle time fluctuations and process waiting times, providing a data foundation for causal discovery.

[0056] Three causal networks are constructed based on data: 1. Equipment-level causal network: PC algorithms are used to analyze sensor data to discover equipment-level causal chains such as "spindle temperature > 65℃ → increased vibration amplitude → dimensional deviation"; 2. Process-level causal network: Granger causality tests are applied to establish the transmission relationships of process parameters such as "stamping speed → springback → assembly gap"; 3. Scheduling-level causal network: Structural causal models are used to characterize the impact paths of management decisions such as "emergency order insertion → increased mold replacement frequency → shortened preventive maintenance cycle". A domain knowledge-guided causal discovery method is adopted to ensure that the model conforms to actual production constraints. Finally, a multi-level causal relationship graph is synthesized from the three causal networks.

[0057] Preferably, a resource scheduling system is constructed using reinforcement learning technology, employing a three-layer architecture of "perception-decision-execution": The perception layer collects worker status, task characteristics, and environmental data in real time through industrial IoT devices (such as Radio Frequency Identification (RFID) and equipment sensors); the decision layer deploys a deep reinforcement learning model based on a hybrid architecture of Proximal Policy Optimization (PPO) + Deep Q-Network (DQN), whose core policy network uses 16-dimensional input states (including features such as skill level and task complexity), extracts features through a 64→32-dimensional fully connected layer, and finally outputs a 10-dimensional discrete action space (allocation combination of 5 workers × 2 task types) and continuous state value assessment; the execution layer is deeply integrated with the automatic guidance scheduling system and electronic work order dashboard to achieve closed-loop control from decision-making to implementation.

[0058] Preferably, the system takes the output of reinforcement learning agents, task causal relationship graphs, and real-time resource status as inputs to comprehensively analyze task priorities, dependencies, and resource requirements, constructing a dynamic resource allocation mechanism to guide spare parts allocation management. Through causal reasoning optimization, a project-specific dynamic point-to-point spare parts management list is provided. Equipment and material data are collected in real-time at the data perception layer, and fault propagation paths and critical spare parts dependencies are analyzed through the causal reasoning layer. Finally, a project-specific dynamic spare parts list is generated at the decision execution layer. The reinforcement learning safety stock algorithm is combined with causal reasoning fault impact analysis to support precise project-level allocation and cross-project resource coordination. Taking industrial workpiece production as an example, the system can dynamically adjust spare parts demand forecasts based on real-time operating conditions, optimizing spare parts quantities through causal reasoning, achieving a significant effect of increasing spare parts fulfillment rate by 10% and reducing obsolete inventory by 70%.

[0059] The system employs a fusion of reinforcement learning decision-making, causal reasoning, and real-time resource status to achieve dynamic resource allocation and multi-process collaborative optimization. It adopts a four-layer closed-loop architecture of "perception-decision-execution-optimization": the perception layer collects real-time data on equipment status, material flow, and process quality through the Industrial Internet of Things (RFID, sensors, vision systems); the decision-making layer uses Graph Attention Network (GAT) and Temporal Convolutional Network (TCN) to fuse multi-source features and combines Mixed Integer Programming (MIP) to solve for the optimal assembly path; the execution layer dynamically issues assembly instructions, spare parts delivery plans, and anomaly handling contingency plans; and the optimization layer uses digital twin simulation and online learning for continuous iteration strategies.

[0060] The system achieves end-to-end optimization from global production scheduling to real-time scheduling: Initial planning stage: Critical paths are identified through causal graphs, generating assembly sequences that balance efficiency and reliability, and associated with spare parts lists; Dynamic adjustment stage: When a high-precision alignment platform experiences sudden vibration anomalies, the system predicts accuracy loss based on causal inference, immediately initiates a compensation scheme, and triggers the delivery of servo encoder spare parts in advance; Closed-loop optimization stage: Actual assembly data is collected, and the edge weights of the causal graph and the reinforcement learning reward function are updated in reverse.

[0061] Employing a three-layer intelligent architecture of "perception-game-decision," the system achieves a precise balance between production capacity and delivery time through multi-source data fusion and dynamic game optimization. The perception layer integrates real-time equipment OEE monitoring, supervising the collection of parameters related to equipment availability, performance, and quality. It tracks the entire order lifecycle, including process completion and remaining buffer time, and perceives multi-dimensional resource status such as worker attendance, mold lifespan, and material availability, constructing a digital twin foundation updated in milliseconds. The game optimization layer innovatively establishes a non-cooperative game model, using Nash equilibrium as its core, designing a multi-objective reward function, and employing inverse induction to generate the optimal solution set, supporting quantitative risk-reward trade-offs. The decision-making layer, based on an improved MADDPG algorithm, enables multi-agent collaborative decision-making, dynamically outputting process priority adjustment schemes, resource reallocation instructions, and intelligent delivery commitment updates. Simultaneously, it deeply integrates with the MES / CRM system through an industrial bus, forming a closed-loop management system from production execution to customer service.

[0062] In some embodiments, this application uses an intelligent control platform for visualization, and its interactive display interface includes the following modules: 1) A work order panoramic dashboard displays the execution status of work orders for each production line in real time, showing the plan completion rate, delay warnings, quality anomaly markers, etc. The module supports multi-dimensional filtering by priority / delivery date / process type; 2) A worker time balance matrix presents the task load of workers of each skill level in the form of a heat map, determining their current working hours / standard working hours, continuous working duration and fatigue index. When an abnormal overload state occurs, the platform automatically triggers a red warning; 3) A resource dynamic scheduling diagram provides a visual representation of equipment, molds, and materials through a 3D map. The system tracks the usage of resources such as materials and marks resource conflict points identified by causal reasoning algorithms; 4) The project causal analysis view displays the causal dependencies between tasks in the form of a knowledge graph (e.g., "electrode replacement delay → welding quality degradation → assembly cycle time extension"), highlighting critical path nodes; 5) The reinforcement learning decision tracking records and compares the differences between AI-suggested solutions and actual execution results, such as the comparison between predicted mold change time and actual time consumption, thereby accumulating a decision-making experience base; 6) The predictive monitoring panel integrates forward-looking indicators such as equipment health, spare parts consumption trends, and capacity prediction, supporting sliding time windows to simulate the impact of different scheduling strategies. This control platform connects directly to workshop equipment via transmission protocols, enabling minute-level data updates and millisecond-level anomaly responses.

[0063] In some embodiments, this application provides a terminal, including: Memory, used to store resource schedulers based on reinforcement learning; A processor is used to implement the steps of the reinforcement learning-based resource scheduling method when executing the reinforcement learning-based resource scheduling system.

[0064] In some embodiments, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the reinforcement learning-based resource scheduling method.

[0065] It is understood that the systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can be a personal computer, a laptop computer, a personal digital assistant, a tablet computer, a wearable device, or any combination of these devices.

[0066] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0067] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0068] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this embodiment, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0069] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0071] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0072] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this embodiment refers to and includes any or all possible combinations of one or more associated listed items.

[0073] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0074] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A resource scheduling method based on reinforcement learning, characterized in that, Includes the following steps: Step S1: Obtain project and task information, construct a task causal relationship diagram, and dynamically update the causal relationship diagram according to the changes in task status during project execution; Step S2: Based on the causal relationship graph and the current project state, construct the state space and action space for generating reinforcement learning strategies; Step S3: Construct a multi-objective reward function, which includes resource utilization efficiency indicators, progress delay indicators, and task revenue indicators; Step S4: Based on the state space, action space, and reward function, construct a reinforcement learning model, train the intelligent agent through the reinforcement learning model, and generate the optimal strategy for task scheduling and resource allocation. Step S5: Allocate resources according to the optimal strategy, using a combination of linear programming and heuristic algorithms to generate a preliminary resource allocation scheme; Based on the task priorities determined by task scheduling, the preliminary resource allocation plan is partially optimized and adjusted; Step S6: Monitor the task execution status in real time, identify situations such as task progress delays, early completion, or resource bottlenecks, and update the task cause-effect graph and priority ranking according to the task execution status. Step S7: Collect execution data, including task completion status, resource utilization efficiency, and strategy benefits. Analyze the execution data and adjust the cause-effect graph update strategy, reward function parameters, and resource allocation logic.

2. The resource scheduling method based on reinforcement learning according to claim 1, characterized in that, In step S1, the constructed task causal relationship graph is a graph structure model. Task nodes have explicit attributes including task priority, task duration and resource requirements. Edges between tasks represent dependency relationships or resource conflict relationships. The weight of the edges is determined based on historical task data and domain knowledge. The weight of the edges is adjusted in real time according to the task status during project execution.

3. The resource scheduling method based on reinforcement learning according to claim 2, characterized in that, In step S2, the state space includes the task feature vector f i The weights of the edges and the current resource allocation status; eigenvector f i Composed of the explicit attributes of the task and semantic embedding vectors, it is represented as: Where, p i For task priority, t i For the remaining construction period, r i For resource needs, e i It is a semantic embedding vector; The action space includes tasks such as prioritizing tasks, changing resource allocation schemes, and modifying task node attributes. Each decision selects an action based on the policy function π(a|s), where π(a|s) represents the probability of choosing action a in state s.

4. The resource scheduling method based on reinforcement learning according to claim 3, characterized in that, In step S3, the multi-objective reward function is as follows: Among them: U r For resource utilization efficiency; D p λ1, λ2, and λ3 represent the degree of schedule delay; G represents the task reward; and λ1, λ2, and λ3 are weight parameters.

5. The resource scheduling method based on reinforcement learning according to claim 4, characterized in that, Step S4 includes the following steps: Step S4-1: Based on the reward function constructed in step S3, construct a reinforcement learning model for learning scheduling strategies; Step S4-2: During training, an ε-greedy strategy is used for action selection. The current reward value and the next state are obtained through interaction, and (s,a,r,s′) quadruples are constructed for experience playback. Step S4-3: Optimize the Q function using the TD error update method. The update formula is as follows: Where: α is the learning rate; γ is the discount factor; r is the current reward; s′ is the next state after action a; Step S4-4: When the strategy converges or meets the preset performance indicators, output the final scheduling and resource allocation strategy.

6. The resource scheduling method based on reinforcement learning according to claim 1, characterized in that, In step S5, the constraints introduced into the linear programming model include: The total allocation of each type of resource shall not exceed the corresponding available limit; The start time of each task must not be earlier than the completion time of its predecessor task; For tasks on the critical path, their resource requirements are set as strong constraints that must be met in the model. Resource allocation for all tasks should meet the minimum demand threshold limit; After the initial resource allocation plan is generated, the following heuristic local adjustment strategy is executed: Based on the node priority in the task causal relationship graph, incremental resource iterative allocation is performed on high-priority tasks. Perform resource reduction or postponement scheduling on low-priority tasks; In the event of resource conflicts between tasks, a resource conflict matrix is ​​constructed, and a round-robin method based on a greedy strategy is used to rearrange the resource occupation order in turn; For situations where there is a mismatch between task resource requirements and available resources, a resource reallocation strategy based on neighborhood search is introduced for further optimization.

7. The resource scheduling method based on reinforcement learning according to claim 1, characterized in that, Step S6 includes the following steps: Step S6-1: Based on real-time monitoring data during project execution, collect the current progress, actual start time, execution duration, and resource usage details of each task, and form a task execution status vector; Step S6-2: Compare the task execution status vector with the plan baseline value to identify the task status type. The status types include five types: delayed, ahead of schedule, paused, not started, and abnormal interruption. Step S6-3: When a task status of delay or resource conflict is detected, update the edge weight values ​​associated with the task in the task causal relationship graph. The edge weight values ​​are adjusted by increasing or decreasing the weight according to the task delay duration, resource occupancy ratio and critical path impact coefficient. Step S6-4: Based on the updated causal relationship graph, recalculate the priority index of each task node. The priority index is determined by a joint evaluation of the node's in-degree, out-degree, remaining task duration, and the scope of delay impact.

8. The resource scheduling method based on reinforcement learning according to claim 4 or 5, characterized in that, Step S7 includes the following steps: Step S7-1: After each scheduling cycle ends, collect the task completion status, resource allocation results and actual revenue values ​​within the current cycle, and generate an execution dataset. The dataset includes task ID, completion status identifier, resource usage details and corresponding strategy ID. Step S7-2: Perform structured statistics on each data record in the execution dataset, extract task success rate, resource utilization rate and revenue deviation value, and construct an indicator analysis matrix. The matrix fields include: task completion rate, resource matching deviation rate and critical path execution error. Step S7-3: Based on the statistical results of the indicator analysis matrix, adjust the edge weights of the task causal relationship graph in batches. The adjustment range of the edge weights is set according to the deviation gradient threshold. Step S7-4: Based on the profit deviation distribution, the weight parameters in the reward function constructed in step S3 are numerically corrected. The correction is based on the comparison ratio between the resource deviation rate and the key task score. Step S7-5: In the policy training loop, the learning rate of the reinforcement learning model is fine-tuned based on the cumulative feedback error, and the model update frequency is dynamically adjusted using an exponential decay strategy.

9. A resource scheduling system based on reinforcement learning, characterized in that, The system includes: The causal reasoning and task dependency modeling module is used to receive project information and task information, construct a task causal relationship graph. The causal relationship graph is in graph structure form, where nodes represent task entities, edges represent dependencies or resource conflicts between tasks, and the weight of the edges can be dynamically adjusted based on task status, historical data, and graph structure changes. The reinforcement learning agent and optimization decision module is used to construct a state space and action space based on the causal relationship graph and project state information. The state space includes task priority, remaining duration, resource requirements and edge weights, and the action space includes task priority reordering, resource allocation adjustment and task attribute modification. The module builds a reinforcement learning model and trains the agent to generate task scheduling and resource allocation strategies. The resource allocation optimization module is used to allocate resources according to the strategy output by the optimization decision module. The resource allocation optimization module includes a linear programming unit and a heuristic adjustment unit, which are used to generate a preliminary resource allocation plan and make local adjustments based on task priority and resource status, respectively. The progress adjustment and plan update module is used to monitor the task execution status, collect task progress information, resource usage and task execution sequence, and update the edge weights and task priority ranking results in the causal relationship graph based on the current execution status. The feedback optimization and closed-loop adjustment module is used to collect execution data for each task cycle. The execution data includes task completion status, resource usage, and strategy execution deviation. The module dynamically adjusts the causal graph structure, reward function parameters, and reinforcement learning model training parameters.

10. The resource scheduling system based on reinforcement learning according to claim 9, characterized in that, The reinforcement learning agent and optimization decision-making module includes: The state representation unit is used to construct the state space by combining the task node attributes and edge weights in the causal relationship graph. The task node attributes include task priority, remaining duration, resource requirements, and semantic feature vectors. The action generation unit is used to set the action space according to the state space. The action space includes tasks such as adjusting the priority order, modifying resource allocation values, and updating task attributes. The strategy execution unit is used to select a specific action from the action space in each training step based on a preset probability selection mechanism, and interact with the environment to obtain reward information in the current state and the new state after execution. The experience storage unit is used to store data samples formed by states, actions, rewards and new states into the experience buffer, and supports the extraction of batch historical samples from it for model training. The policy update unit is used to adjust the internal model parameters and update the policy mapping relationship between states and actions based on the deviation between the current reward and the predicted value of the next state. The strategy output unit is used to output the task scheduling order and resource allocation results after the model training meets the termination condition, for the resource allocation optimization module to execute.

Images