A scene-dependent intelligent scheduling optimization method and system

Abstract

The application relates to the technical field of computer data processing, and discloses a scene-dependent intelligent production scheduling optimization method and system. The method obtains a to-be-scheduled task set, scene hierarchical dependency relationships and production scheduling constraints, constructs a heterogeneous graph representing a containing relationship, a reference relationship and a sharing relationship, learns node embedding vectors and importance scores, establishes a mixed integer linear programming model and constructs a constraint Markov decision process, learns a production scheduling strategy through a proximal policy optimization algorithm based on a reward function and a cost function, and generates a production scheduling scheme. The application can continuously transmit scene dependency relationships, production scheduling targets and constraint violation degrees, and reduce the risk of repeated arrangement of shared tasks and unbalanced load of resource groups.

Description

Technical Field

[0001] This application relates to the field of computer data processing technology, specifically to a scenario-dependent intelligent scheduling optimization method and system. Background Technology

[0002] In computer systems, project scheduling typically involves the combined optimization of task data, resource group data, time slot data, and dependency data. This is commonly seen in business scenarios such as software development, intelligent agent scenario development, system integration, and project delivery. Existing project management systems, task scheduling systems, or production scheduling platforms usually convert task lists, task priorities, estimated durations, personnel availability time, and resource group availability time into tabular or list data, and then generate a production schedule based on fixed sorting rules or constraint solving rules. While this process can output task start times and resource group allocation results, it lacks a structured representation of multi-level scenario relationships, shared task relationships, and resource group load rate matrices.

[0003] However, in complex projects such as intelligent agent scenario development, the tasks to be scheduled are often not independent, flat tasks, but rather originate from multi-level business scenarios. A complete after-sales service scenario can be broken down into micro-scenarios such as identity verification, order inquiry, and payment confirmation. Different micro-scenarios may also share reusable meta-scenarios such as intent recognition, message generation, and exception transfer to human intervention. The same meta-scenarios may be used by multiple micro-scenarios or multiple original scenarios, and the completion status of an upstream scenario may also affect the startup order of multiple downstream tasks. If the computer system still uses a regular task list or a simple tree-like task structure for scheduling, the relationships, reference relationships, and shared relationships are difficult to convert into computable node types, edge types, initial node feature matrices, and edge type mapping information, resulting in the subsequent constraint solving process being unable to accurately read the structural data corresponding to the shared meta-scenarios.

[0004] In the above situation, the existing scheduling method is prone to computer data chain breakpoints: On the one hand, the system may only allocate resource groups according to the priority and resource availability of a single task, failing to convert the impact of a certain meta-scenario on multiple downstream micro-scenarios into node embedding vectors, importance scores, and weighted completion time calculation parameters, resulting in key tasks lacking structural weights in the constraint solving process; on the other hand, the system may mistakenly regard a meta-scenario commonly referenced by multiple micro-scenarios as multiple independent tasks, failing to eliminate duplicate actions through shared relation edges, shared constraints, and action masking mechanisms, causing the task corresponding to the same meta-scenario to be repeatedly written into the scheduling plan and repeatedly occupying resource group capacity or computing resources.

[0005] Furthermore, existing scheduling systems typically handle scheduling objectives and constraints separately: scheduling objectives focus on shortening the overall project duration, improving resource utilization, or meeting task priorities, while scheduling constraints focus on personnel availability, resource group capacity, task dependencies, and shared task restrictions. When projects simultaneously face resource constraints, complex upstream and downstream dependencies, numerous shared tasks, and frequent team switching, relying solely on static priority rules or manual adjustments makes it difficult to continuously assess the impact of current scheduling decisions on subsequent project duration, resource load balancing, and the degree of constraint violations. This can easily lead to situations where local scheduling appears reasonable, but the overall scheduling effect is poor.

[0006] Therefore, the main problem with existing technologies is that in the computer-based production scheduling data processing of complex business scenarios, scenario-level dependencies, resource constraints, timing constraints, dependency constraints, and shared constraints are not transformed into heterogeneous graphs, graph embedding matrices, resource group load matrices for future time slots, mixed-integer linear programming variables, and sets of actionable actions in the same data processing link. This results in a lack of continuous data transfer between the constraint solving process and the policy network update process. Consequently, the production scheduling scheme is prone to problems such as critical tasks being delayed, shared tasks being repeatedly scheduled, resource group load rates exceeding the upper limit of resource group load rates, or actions not satisfying timing constraints, dependency constraints, and shared constraints. Summary of the Invention

[0007] The technical problem to be solved by this application can be described as follows: how to transform the set of tasks to be scheduled, the hierarchical dependency relationship of the scenario, and the scheduling constraints into a heterogeneous graph, node initial feature matrix, graph embedding matrix, resource group load matrix for future time slots, mixed integer linear programming variables, and a set of actionable actions in computer scheduling data processing, and to continuously connect the graph embedding matrix, importance score, resource group load rate, constraint violation amount corresponding to resource constraints, reward function, cost function, and action probability distribution, so that the computer system can generate a scheduling scheme that satisfies resource constraints, time constraints, dependency constraints, and shared constraints through constraint solving and action masking.

[0008] To address the shortcomings of existing technologies, this application aims to provide a scenario-dependent intelligent scheduling optimization method and system. This addresses the difficulty in uniformly encoding scenario-level dependencies, resource constraints, temporal constraints, dependency constraints, and shared constraints into heterogeneous graph structures, matrix data, and constraint variables during computer-based scheduling data processing. This results in a lack of continuously transferable data interfaces between mixed-integer linear programming models, constrained Markov decision processes, and proximal policy optimization algorithms. Consequently, scheduling schemes are prone to issues such as critical task delays, duplicate scheduling of shared tasks, resource group load rates exceeding resource group load rate limits, or invalid scheduling actions.

[0009] To achieve the above objectives, this application adopts the following technical solution:

[0010] Firstly, this application provides a scenario-dependent intelligent scheduling optimization method, including:

[0011] Obtain the set of tasks to be scheduled, the scenario hierarchy dependencies, and the scheduling constraints;

[0012] Based on the scenario hierarchy dependency, the set of tasks to be scheduled is modeled as a heterogeneous graph. The heterogeneous graph is used to represent the inclusion relationship, reference relationship and sharing relationship between scenarios at different levels.

[0013] Based on the node type, edge type, and upstream and downstream dependencies of each node in the heterogeneous graph, the embedding vector and importance score of each node are learned.

[0014] A mixed-integer linear programming model is established based on the embedding vector, importance score, and production scheduling constraints of each node. The mixed-integer linear programming model is used to evaluate the degree of violation of production scheduling objectives and constraints.

[0015] The production scheduling decision process is modeled as a constrained Markov decision process, and a reward function to characterize the production scheduling objective and a cost function to characterize the degree of violation of the production scheduling constraints are set in the constrained Markov decision process.

[0016] A proximal policy optimization algorithm based on reward and cost functions is used to learn a scheduling strategy that satisfies scheduling constraints, and a scheduling plan is generated based on the scheduling strategy.

[0017] Preferably, the method for modeling the set of tasks to be scheduled as a heterogeneous graph based on the scene hierarchy dependency includes:

[0018] The original scene, micro scene, and meta scene are determined from the set of tasks to be scheduled and the scene hierarchy dependency relationship, and the original scene node, micro scene node, and meta scene node are generated respectively.

[0019] Different edge types are generated based on inclusion, reference, and sharing relationships to obtain a heterogeneous graph.

[0020] Preferably, after obtaining the heterogeneous graph, a meta-path is generated based on the original scene nodes, micro-scene nodes, meta-scene nodes, and edges of different edge types. The node connection range for learning the embedding vectors of each node is then determined based on the meta-path.

[0021] Preferably, the method for learning the embedding vectors of each node includes:

[0022] Two nodes connected by an edge within the node's connection range are designated as the source node and the target node, respectively. The dynamic attention coefficient is determined based on the node type of the source node, the node type of the target node, and the corresponding edge type.

[0023] The dynamic attention coefficients are normalized, and the source nodes are aggregated based on the normalized dynamic attention coefficients to obtain the embedding vector of the target node.

[0024] Preferably, the method for obtaining the embedding vector of the target node further includes:

[0025] Aggregate the source nodes according to their node types to obtain a type aggregation vector;

[0026] Determine type-level attention weights based on the type aggregation vector;

[0027] Update the embedding vector of the target node based on the type-level attention weights.

[0028] Preferably, the method for learning the importance score of each node includes:

[0029] Determine the set of downstream dependent nodes and the set of upstream dependent nodes for each node based on the upstream and downstream dependencies;

[0030] An importance score for each node is generated based on its embedding vector, the set of downstream dependent nodes, and the set of upstream dependent nodes.

[0031] Preferably, the method for establishing a mixed-integer linear programming model includes:

[0032] Determine the decision variables, which include the task start time, resource group allocation results, task completion order, task completion time, and resource group load rate.

[0033] A mixed-integer linear programming model is established based on decision variables, embedding vectors of each node, importance scores of each node, and production scheduling constraints.

[0034] Preferably, the method for generating production scheduling target evaluation values ​​using a mixed-integer linear programming model includes:

[0035] The total project duration is determined based on the task completion time.

[0036] Determine the resource load balancing evaluation value based on the resource group load rate;

[0037] The weighted sum of completion times is determined based on the importance score of each node and the task completion time.

[0038] The team switching penalty is determined based on the resource group allocation results and the preset team switching penalty rules.

[0039] The production target evaluation value is generated based on the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switchover penalty.

[0040] Preferably, the scheduling constraints include resource constraints, timing constraints, dependency constraints, and sharing constraints;

[0041] Shared constraints are used to ensure that tasks corresponding to the same meta-scenario are scheduled only once.

[0042] Preferably, the method for establishing a mixed-integer linear programming model further includes:

[0043] Determine the constraint violation amount corresponding to the resource constraint, and introduce the constraint violation amount into the objective function of the mixed integer linear programming model through Lagrange multipliers to obtain the relaxed objective function.

[0044] The production scheduling objective is evaluated based on the relaxed objective function, and the degree of constraint violation is evaluated based on the amount of constraint violation.

[0045] Preferably, the method of modeling the production scheduling decision process as a constrained Markov decision process includes:

[0046] The graph embedding matrix corresponding to the heterogeneous graph, the execution history characteristics of the set of tasks to be scheduled, the load matrix of the resource group in the future time slot, the expected completion time vector and time characteristics of each task are determined as the state in the constrained Markov decision process.

[0047] The actions of selecting tasks to be executed, allocating resource groups, and setting start times at the current moment will be determined as constraints in the Markov decision-making process.

[0048] Preferably, the production scheduling target includes a production scheduling target evaluation value, and the degree of violation of production scheduling constraints includes the amount of constraint violation corresponding to resource constraints;

[0049] The reward function is determined based on the production scheduling target evaluation value, while the cost function is determined based on the constraint violation amount corresponding to the resource constraints.

[0050] Preferably, the method of learning a scheduling strategy that satisfies scheduling constraints using a proximal policy optimization algorithm based on reward and cost functions includes:

[0051] The state inputs in the constrained Markov decision process are the policy network, value network, and cost network;

[0052] The action probability distribution is output through the policy network;

[0053] Estimate expected cumulative rewards through value networks;

[0054] Estimate the expected cumulative cost using a cost network;

[0055] The production scheduling strategy is updated based on the action probability distribution, expected cumulative reward, and expected cumulative cost.

[0056] Preferably, the method for updating the production scheduling strategy includes:

[0057] Update the Lagrange multipliers in the near-end policy optimization algorithm based on the cost function and the maximum allowed constraint violation.

[0058] Based on the reward function, cost function, and updated Lagrange multipliers, the parameters of the policy network are updated to obtain a production scheduling policy that satisfies the production scheduling constraints.

[0059] Preferably, before generating the production scheduling plan, the following steps are also included:

[0060] Obtain historical production schedule data, which includes the scenario hierarchy dependencies of completed projects and historical production schedule data.

[0061] Historical production scheduling data is used as a learning task to perform model-independent meta-learning and obtain meta-initialization parameters.

[0062] The near-end strategy optimization algorithm is initialized based on the meta-initialization parameters, and the production scheduling strategy is obtained through gradient update.

[0063] Secondly, this application provides a scenario-dependent intelligent scheduling optimization system, including:

[0064] The data acquisition module is used to acquire the set of tasks to be scheduled, the scenario hierarchy dependencies, and scheduling constraints.

[0065] The heterogeneous graph modeling module is used to model the set of tasks to be scheduled as a heterogeneous graph based on the scene hierarchy dependency relationship. The heterogeneous graph is used to represent the inclusion relationship, reference relationship and sharing relationship between different levels of scenes.

[0066] The node learning module is used to learn the embedding vector and importance score of each node based on the node type, edge type and upstream and downstream dependencies of each node in the heterogeneous graph.

[0067] The model building module is used to build a mixed-integer linear programming model based on the embedding vectors, importance scores and scheduling constraints of each node. The mixed-integer linear programming model is used to evaluate the degree of violation of scheduling objectives and scheduling constraints.

[0068] The process modeling module is used to model the production scheduling decision process as a constrained Markov decision process, and to set a reward function to characterize the production scheduling objective and a cost function to characterize the degree of violation of the production scheduling constraints in the constrained Markov decision process.

[0069] The scheme generation module is used to learn a production scheduling strategy that satisfies the production scheduling constraints using a proximal policy optimization algorithm based on reward function and cost function, and generate a production scheduling scheme based on the production scheduling strategy.

[0070] Compared with the prior art, the beneficial effects achieved by this application are as follows:

[0071] This application acquires the set of tasks to be scheduled, the hierarchical dependencies of scenarios, and scheduling constraints. Based on these dependencies, it models the set of tasks as a heterogeneous graph. This allows the conversion of the inclusion, reference, and sharing relationships between the original scenario, micro-scenarios, meta-scenarios, and scenarios at different levels into computer-processable sets of nodes, edges, initial node feature matrices, node type mapping information, and edge type mapping information. Therefore, scenario dependency information no longer exists solely as business descriptions or manual rules, but is instead input as a graph data structure into subsequent embedding vector learning, constraint solving, and policy network update processes.

[0072] This application learns the embedding vectors and importance scores of each node based on its node type, edge type, and upstream / downstream dependencies in a heterogeneous graph. This allows the node embedding vectors to participate in subsequent calculations as numerical vectors in the graph embedding matrix, and the importance scores to be used as weights in the weighted sum of completion times for production target evaluation. As a result, the computer system can transform the structural semantics formed by inclusion, reference, and sharing relationships into computational parameters for task processing time, resource requirements, weighted completion time, and production target evaluation values, reducing the inadequacy of data representation caused by allocating resource groups solely based on static priorities.

[0073] This application establishes a mixed-integer linear programming model based on the embedding vectors, importance scores, and scheduling constraints of each node. The model generates scheduling target evaluation values ​​and constraint violation quantities corresponding to resource constraints, enabling task start time, task completion time, resource group allocation results, execution allocation auxiliary variables, resource group load rate, load deviation auxiliary variables, and shared constraints to be processed within the same constraint-solving model. Therefore, the computer system can use linear constraints to calculate the load occupancy of resource groups in future time slots and quantify the extent to which the resource group load rate exceeds the upper limit of the resource group load rate through the constraint violation quantities corresponding to the resource constraints.

[0074] This application models the production scheduling decision-making process as a constrained Markov decision process, and sets up a reward function to characterize the production scheduling objective and a cost function to characterize the degree of resource constraint violation. This allows the production scheduling objective evaluation value and the constraint violation quantity corresponding to the resource constraints output by the mixed-integer linear programming model to be converted into reward and cost signals used by the policy network, value network, and cost network. Consequently, the graph embedding matrix in the state, execution history features, the load matrix of resource groups in future time slots, the expected completion time vector of each task, and time features can be continuously input into the action selection process, forming a sequential decision data link that can be executed by a computer.

[0075] This application employs a proximal policy optimization algorithm based on reward and cost functions to learn scheduling strategies that satisfy scheduling constraints, and generates scheduling plans based on these strategies. This ensures that the scheduling strategy update process is simultaneously constrained by both the scheduling target evaluation value and the constraint violation amount corresponding to resource constraints. Simultaneously, the computer system filters and masks actions that do not satisfy temporal, dependency, and shared constraints through action set filtering and action masking. This ensures that the action probability distribution output by the policy network only applies to the effective action space composed of executable tasks, available resource groups, and optional start times. Therefore, the scheduling plan generation process is no longer a manual rule-based sorting process, but a computer data processing process based on matrix state input, constraint solution results, action probability distribution, and constraint masking results.

[0076] Therefore, this application does not merely perform conventional sorting of tasks to be scheduled, nor does it simply adjust the order of tasks based on manual management rules. Instead, it constructs a continuous computer data processing chain around the set of tasks to be scheduled, scenario hierarchical dependencies, resource group load data, and scheduling constraints. Scenario dependencies are continuously passed along the heterogeneous graph modeling, node initial feature matrix generation, graph embedding matrix generation, mixed-integer linear programming model solving, constrained Markov decision process modeling, action masking, and near-end policy optimization algorithm update processes. Through the combined effect of these technical means, this application can improve the consistency of the scheduling scheme in data structure expression, matrix state construction, resource group load rate calculation, constraint violation evaluation, and action selection control, reducing the risks of critical task delays, duplicate scheduling of shared tasks, resource group load rates exceeding the upper limit, and invalid scheduling actions being output. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the implementation environment for a scenario-dependent intelligent scheduling optimization method.

[0078] Figure 2 A flowchart of a scenario-dependent intelligent scheduling optimization method;

[0079] Figure 3 A flowchart for heterogeneous graph modeling and metapath generation methods;

[0080] Figure 4 A flowchart illustrating the method for learning the embedding vectors and importance scores for each node;

[0081] Figure 5 A flowchart illustrating the method for establishing a mixed-integer linear programming model;

[0082] Figure 6 A flowchart illustrating the modeling method for constrained Markov decision processes and the methods for setting reward and cost functions;

[0083] Figure 7A flowchart illustrating a method for learning a scheduling strategy that satisfies scheduling constraints using a proximal policy optimization algorithm based on reward and cost functions;

[0084] Figure 8 A flowchart illustrating a method for performing model-independent meta-learning based on historical production scheduling data before generating a production scheduling plan;

[0085] Figure 9 This is a schematic diagram of the structure of a scenario-dependent intelligent scheduling optimization system. Detailed Implementation

[0086] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof.

[0087] It should be noted that, in this embodiment, the set of tasks to be scheduled is read by the scenario-dependent intelligent scheduling optimization device from the project management system, task management table, requirement breakdown table, or manual configuration file; each task to be scheduled in the set includes at least one or more of the following: task identifier, task processing time, resource requirements, dependent tasks, corresponding scenario level, task-owning team, executable resource group, or estimated completion time; the task identifier is used to distinguish different tasks to be scheduled; the task processing time is used to calculate the task completion time; the resource requirements are used to match with the capabilities of the resource group; the dependent tasks are used to determine upstream and downstream dependencies; the corresponding scenario level is used to establish scenario-level dependencies; the task-owning team and executable resource group are used to determine the resource group allocation results; and the estimated completion time is used to construct the state in the constrained Markov decision process.

[0088] It should be noted that the scenario hierarchy dependency relationship in this application embodiment is used to represent the hierarchical dependency between the original scenario, micro-scenario and meta-scenario; the original scenario corresponds to the complete business process or complete development scenario in the intelligent agent scenario development project; the micro-scenario corresponds to the sub-process or sub-functional scenario in the original scenario; the meta-scenario corresponds to the functional unit that can be referenced and reused by multiple micro-scenarios; the scenario dependency intelligent scheduling optimization device identifies the original scenario, micro-scenario and meta-scenario according to the scenario hierarchy dependency relationship, and establishes a heterogeneous graph according to the inclusion relationship, reference relationship and sharing relationship between the original scenario, micro-scenario and meta-scenario.

[0089] It should be noted that the scheduling constraints in this application embodiment include resource constraints, timing constraints, dependency constraints, and sharing constraints; resource constraints are used to limit the capacity of resource groups, the parallelism of resource groups, the available time of personnel, or the range of computing power occupied; timing constraints are used to limit the start time of task execution, the completion time of task, and whether it is a working day or a non-working day; dependency constraints are used to limit the downstream task to enter the executable state only after the upstream task is completed; sharing constraints are used to ensure that the task corresponding to the same meta-scenario is scheduled only once, avoiding the duplicate scheduling of development tasks corresponding to the same meta-scenario due to references by multiple micro-scenarios.

[0090] Figure 1 This is a schematic diagram illustrating the implementation environment of a scenario-dependent intelligent scheduling optimization method provided in this application embodiment; see also Figure 1 The implementation environment includes scenario-dependent intelligent scheduling optimization equipment, project management data source, resource group data source, historical scheduling project data source, and scheduling plan output terminal; the project management data source, resource group data source, and historical scheduling project data source are respectively connected to the scenario-dependent intelligent scheduling optimization equipment; the scenario-dependent intelligent scheduling optimization equipment is connected to the scheduling plan output terminal; the scenario-dependent intelligent scheduling optimization equipment reads scheduling input data from the project management data source, resource group data source, and historical scheduling project data source, and sends the generated scheduling plan to the scheduling plan output terminal.

[0091] The scenario-dependent intelligent scheduling optimization equipment includes servers, cloud computing devices, local computing devices, or project management platforms; the project management data source provides the set of tasks to be scheduled and the scenario-level dependencies; the resource group data source provides resource group capabilities, resource group load, team information, and working time information; the historical scheduling project data source provides the scenario-level dependencies of completed projects and historical scheduling data; the scheduling plan output terminal outputs the task start time, task completion time, resource group allocation results, task completion order, and resource group load rate; the scenario-level dependencies of completed projects include the original scenario, micro-scenario, meta-scenario, containment relationship, reference relationship, and sharing relationship in completed projects; historical scheduling data includes the task start time, task completion time, resource group allocation results, task completion order, and resource group load rate in completed projects.

[0092] Those skilled in the art will understand that the above implementation environment is only used to illustrate the implementable architecture of the embodiments of this application. The deployment method, number of data sources, number of resource groups and number of tasks of the scenario-dependent intelligent scheduling optimization equipment can be adjusted according to the actual project management environment, and the embodiments of this application do not limit this.

[0093] It should be noted that the graph attention network, GATv2, mixed-integer linear programming, Lagrange relaxation, constrained Markov decision process, proximal policy optimization algorithm, Actor-Critic architecture, and model-independent meta-learning in the embodiments of this application can all be implemented using existing algorithm frameworks. The technical improvement of the embodiments of this application does not lie in proposing graph attention network, GATv2, mixed-integer linear programming, Lagrange relaxation, constrained Markov decision process, proximal policy optimization algorithm, Actor-Critic architecture, or model-independent meta-learning itself separately, but in modeling the original scene, micro-scene, meta-scene, and inclusion, reference, and sharing relationships in the intelligent agent scene development as a heterogeneous graph, and continuously coupling the embedding vector and importance score of each node in the heterogeneous graph with the production target evaluation value and production constraint violation degree in the mixed-integer linear programming model, the reward function and cost function in the constrained Markov decision process, and the production policy update process in the proximal policy optimization algorithm.

[0094] In this embodiment, the scenario-dependent intelligent scheduling optimization device first generates a heterogeneous graph based on the scenario hierarchy dependency relationship, and then learns the embedding vector and importance score of each node based on the node type, edge type, and upstream and downstream dependency relationship of each node in the heterogeneous graph. Subsequently, the embedding vector, importance score, and scheduling constraints of each node are written into a mixed integer linear programming model to obtain the scheduling target evaluation value and the constraint violation amount corresponding to the resource constraint. Then, the scheduling target evaluation value is converted into a reward function, the constraint violation amount corresponding to the resource constraint is converted into a cost function, and a scheduling strategy that satisfies the scheduling constraints is learned through a proximal policy optimization algorithm. In this way, the scenario dependency relationship between the original scenario, micro-scenario, and meta-scenario can be passed along the process of heterogeneous graph modeling, scheduling target evaluation, constraint violation amount calculation, reward function setting, cost function setting, and scheduling strategy update.

[0095] Example 1:

[0096] Please see Figure 2 As shown, this embodiment provides a scenario-dependent intelligent scheduling optimization method, which is executed by a scenario-dependent intelligent scheduling optimization device and includes the following steps:

[0097] Step S101: Obtain the set of tasks to be scheduled, the scenario hierarchy dependency relationship, and the scheduling constraints.

[0098] In some implementation methods, the implementation steps include:

[0099] Read the set of tasks to be scheduled from at least one data source, including the project management system, task management table, requirement breakdown table, or manual configuration file. The set of tasks to be scheduled includes multiple tasks. Each task includes a task identifier, task name, task processing time, resource requirements, task priority, task scenario, task team, executable resource group, or estimated task completion time. The task identifier is used to distinguish different tasks. The task processing time is used to calculate the task completion time. The resource requirements are used to match the resource group capabilities corresponding to the executable resource group. The task priority is used to participate in the scheduling target evaluation. The task scenario is used to establish scenario-level dependencies. The task team and executable resource group are used to determine the resource group allocation results. The estimated task completion time is used to construct the state in the constrained Markov decision process.

[0100] The scenario hierarchy dependency relationships are read from at least one data source, such as the scenario configuration file, business process breakdown table, or intelligent agent scenario development platform. The scenario hierarchy dependency relationships include inclusion relationships, reference relationships, and sharing relationships. The inclusion relationship is the composition relationship between the original scenario and the micro-scenario. The inclusion relationship is used to represent that an original scenario includes at least one micro-scenario. The reference relationship is the calling relationship between a micro-scenario and the meta-scenario. The reference relationship is used to represent that when a micro-scenario is executed, it needs to call at least one meta-scenario. The sharing relationship is the relationship that the same meta-scenario is reused by multiple micro-scenarios or multiple original scenarios. The sharing relationship is used to represent that the development results corresponding to the same meta-scenario can be used by multiple micro-scenarios or multiple original scenarios.

[0101] The scheduling constraints are read from at least one of the following data sources: resource management system, project schedule requirements, team scheduling records, or constraint configuration table. The scheduling constraints include resource constraints, timing constraints, dependency constraints, and sharing constraints. Resource constraints include resource group capacity, resource group parallelism, resource group load rate limit, personnel availability time, or computing power availability time. Timing constraints include task start time, task completion time, and time limits for non-working days or project phases. Dependency constraints include the sequential execution requirements between tasks with upstream and downstream dependencies. Sharing constraints ensure that tasks corresponding to the same meta-scenario are scheduled only once.

[0102] Step S101 obtains the basic data for heterogeneous graph modeling, mixed-integer linear programming model establishment, and constrained Markov decision process modeling; the set of tasks to be scheduled is used to determine the nodes in the heterogeneous graph and the decision variables in the mixed-integer linear programming model; the scenario hierarchy dependency is used to determine the edge type and upstream and downstream dependency in the heterogeneous graph; the scheduling constraints are used to determine the constraints in the mixed-integer linear programming model and to determine the range of possible actions and cost function in the constrained Markov decision process.

[0103] Step S102: Model the set of tasks to be scheduled as a heterogeneous graph based on the scene hierarchy dependency relationship. The heterogeneous graph is used to represent the inclusion relationship, reference relationship and sharing relationship between different level scenes.

[0104] In some implementation methods, the implementation steps include:

[0105] Based on the scene hierarchy dependency relationship, the original scene, micro scene, and meta scene are determined, and original scene nodes, micro scene nodes, and meta scene nodes are generated respectively; the scene to which each task belongs in the set of tasks to be scheduled is read, and the task's scene is mapped to the original scene, micro scene, and meta scene; when the task's scene corresponds to the original scene, an original scene node is generated; when the task's scene corresponds to the micro scene, a micro scene node is generated; when the task's scene corresponds to the meta scene, a meta scene node is generated.

[0106] The original scene node includes the original scene identifier, the original scene name, the original scene priority, and the set of tasks corresponding to the original scene; the original scene identifier is used to distinguish different original scenes; the original scene name is used to record the name of the original scene in the intelligent agent scene development project; the original scene priority is determined by the project schedule requirements or task priority; the set of tasks corresponding to the original scene consists of the tasks to be scheduled in the original scene to which the task belongs.

[0107] A micro-scene node includes a micro-scene identifier, its original scene, the micro-scene processing time, and the micro-scene resource requirements. The micro-scene identifier is used to distinguish different micro-scenes. The original scene is determined by the inclusion relationship. The micro-scene processing time is obtained by summing the processing times of the tasks to be scheduled for production corresponding to the micro-scene. The micro-scene resource requirements are obtained by merging the resource requirements of the tasks to be scheduled for production corresponding to the micro-scene. When the same resource type corresponds to multiple resource requirements, the micro-scene resource requirements are the sum of the multiple resource requirements.

[0108] The meta-scenario node includes the meta-scenario identifier, the number of times the meta-scenario is reused, the number of times the meta-scenario is referenced, and the development task corresponding to the meta-scenario. The meta-scenario identifier is used to distinguish different meta-scenarios. The number of times the meta-scenario is reused is determined by the number of times the same meta-scenario is called by different micro-scenarios. The number of times the meta-scenario is referenced is determined by the number of records in the reference relationship that point to the same meta-scenario. The development task corresponding to the meta-scenario consists of the production tasks to be scheduled for the meta-scenario to which the task belongs.

[0109] Based on the inclusion relationship, generate the edge between the original scene node and the micro scene node; when the scene hierarchy dependency relationship records that an original scene includes a micro scene, generate the edge corresponding to the inclusion relationship between the corresponding original scene node and the corresponding micro scene node; the edge corresponding to the inclusion relationship is used to represent the hierarchical composition relationship of the original scene including the micro scene, and is used to determine the upstream and downstream dependency relationship between the original scene node and the micro scene node.

[0110] Edges between micro-scene nodes and meta-scene nodes are generated based on reference relationships. When a micro-scene calls a meta-scene, an edge corresponding to the reference relationship is generated between the corresponding micro-scene node and the corresponding meta-scene node. The edge corresponding to the reference relationship is used to represent the functional dependency relationship between the micro-scene and the meta-scene, and to determine the dependency relationship between the production task to be scheduled for the micro-scene and the production task to be scheduled for the meta-scene.

[0111] Edges corresponding to the sharing relationship are generated based on the sharing relationship. When two meta-scenes share the same development result, edges corresponding to the sharing relationship are generated between the nodes of the two meta-scenes. When the same meta-scene is referenced by multiple micro-scenes, edges corresponding to the sharing relationship are generated between the nodes of the multiple micro-scenes that reference the same meta-scene. When the same meta-scene is referenced by multiple original scenes, edges corresponding to the sharing relationship are generated between the nodes of the multiple original scenes that reference the same meta-scene. The edges corresponding to the sharing relationship are written into the sharing relationship edge type, and the identifier of the shared meta-scene is used as the edge attribute of the edge corresponding to the sharing relationship. The edges corresponding to the sharing relationship are used to represent that the development result corresponding to the same meta-scene is used by multiple micro-scenes or multiple original scenes, or to represent the relationship between multiple meta-scenes that share the development result.

[0112] Configure a node type for each node and an edge type for each edge. Node types include original scene node type, micro scene node type, and meta scene node type. Edge types include containment relationship edge type, reference relationship edge type, and sharing relationship edge type. Node types are used to distinguish between original scene nodes, micro scene nodes, and meta scene nodes. Edge types are used to distinguish between containment relationships, reference relationships, and sharing relationships, and are used to determine the dynamic attention coefficients later.

[0113] An initial feature matrix of nodes is generated based on the node set; each row in the initial feature matrix corresponds to one node; each row in the initial feature matrix includes one or more of the following: node type code, number of corresponding tasks, task processing time, resource requirements, task priority, number of reuses, or number of references; the node type code is determined according to the node type configuration table; the node type configuration table is pre-established by the original scene node type, micro scene node type, and meta scene node type.

[0114] The node set, edge set, initial node feature matrix, node type mapping information, and edge type mapping information are collectively defined as a heterogeneous graph. The node set includes original scene nodes, micro-scene nodes, and meta-scene nodes. The edge set includes edges corresponding to inclusion relationships, reference relationships, and sharing relationships. Edges corresponding to sharing relationships include shared relationship edges between meta-scene nodes, shared relationship edges between micro-scene nodes, or shared relationship edges between original scene nodes. The node type mapping information is used to record the correspondence between each node and its node type. The edge type mapping information is used to record the correspondence between each edge and its edge type.

[0115] In step S102, the hierarchical dependencies, cross-level references, and cross-scene sharing among the original scene, micro-scene, and meta-scene are transformed into a heterogeneous graph structure. Nodes in the heterogeneous graph structure are used to represent the original scene, micro-scene, and meta-scene. Edges in the heterogeneous graph structure are used to represent inclusion relationships, reference relationships, and sharing relationships. The heterogeneous graph structure serves as input data for subsequent learning of the embedding vectors and importance scores of each node.

[0116] Step S103: Based on the node type, edge type, and upstream and downstream dependencies of each node in the heterogeneous graph, learn the embedding vector and importance score of each node.

[0117] In some implementation methods, the implementation steps include:

[0118] Based on the original scene nodes, micro-scene nodes, meta-scene nodes, and edges of different edge types, a meta-path is generated, which is composed of alternating node types and edge types. The meta-path is used to capture the scene dependency semantics between the original scene, micro-scene, and meta-scene. The functional aggregation semantics correspond to the aggregation relationship between the meta-scene, micro-scene, and original scene. The functional reuse semantics correspond to the relationship that the same meta-scene is reused by multiple micro-scenes. The business association semantics correspond to the association relationship formed between different original scenes due to the sharing of micro-scenes or meta-scenes.

[0119] The node connection range is determined based on the meta-path to learn the embedding vectors of each node; the node connection range is the set of neighboring nodes that can be reached along the meta-path and participate in the embedding vector aggregation; for the target node, edges in the heterogeneous graph are retrieved starting from the target node according to the meta-path, and the retrieved original scene nodes, micro-scene nodes, or meta-scene nodes are added to the node connection range; the node connection range is used to limit the calculation objects of the subsequent dynamic attention coefficient.

[0120] Two nodes connected by an edge within their node connection range are designated as the source node and the target node, respectively. The source node is the node that provides the features to be aggregated, and the target node is the node whose embedding vector needs to be updated. When a target node corresponds to multiple source nodes, the dynamic attention coefficient of each source node relative to the target node is calculated, and the dynamic attention coefficients corresponding to multiple source nodes are used as normalized input.

[0121] The dynamic attention coefficients are normalized, and the source nodes are aggregated based on the normalized dynamic attention coefficients to obtain the embedding vector of the target node.

[0122] In one optional implementation, the source nodes are aggregated according to their node types to obtain type aggregation vectors; the source nodes corresponding to the original scene node types are aggregated into the original scene type aggregation vector; the source nodes corresponding to the micro scene node types are aggregated into the micro scene type aggregation vector; the source nodes corresponding to the meta scene node types are aggregated into the meta scene type aggregation vector; the type aggregation vectors are used to calculate the degree of influence of different node types on the target node embedding vector.

[0123] The downstream dependent node set and the upstream dependent node set of each node are determined based on the upstream and downstream dependencies. The downstream dependent node set is the set of subsequent nodes affected by the completion status of the current node. The upstream dependent node set is the set of nodes that impose a prerequisite constraint on the current node. The upstream and downstream dependencies are jointly determined by the edge direction, containment relationship, reference relationship, sharing relationship in the heterogeneous graph, and the dependent tasks in the set of tasks to be scheduled.

[0124] Step S103 yields the embedding vector and importance score for each node. The embedding vector represents the structural semantics of the corresponding node in the heterogeneous graph. The importance score represents the priority influence of the corresponding node in the scheduling optimization and serves as the basis for calculating the weighted sum of completion times in the subsequent mixed-integer linear programming model.

[0125] Step S104: Establish a mixed-integer linear programming model based on the embedding vector, importance score and scheduling constraints of each node. The mixed-integer linear programming model is used to evaluate the degree of violation of scheduling objectives and scheduling constraints.

[0126] In some implementation methods, the implementation steps include:

[0127] The decision variables for the mixed-integer linear programming model are determined, including task completion time and resource group load rate. In a preferred embodiment, the decision variables also include task start execution time, resource group allocation result, and task order completion relationship. The task start execution time is used to determine the starting point of the pending tasks on the time axis. The resource group allocation result is used to determine which resource group will execute the pending tasks and serves as the basis for calculating the team switching penalty. The task order completion relationship is used to determine the order execution constraints between two pending tasks. The task completion time is used to calculate the sum of the total project duration and the weighted completion time. The resource group load rate is used to calculate the resource load balancing evaluation value and the constraint violation amount corresponding to the resource constraint. The embedding vector of each node is used to determine the task processing time and resource requirement corresponding to the pending tasks. The task processing time and resource requirement serve as input parameters in the mixed-integer linear programming model to participate in the calculation of task completion time, resource group load rate, and constraint violation amount corresponding to the resource constraint.

[0128] In one alternative implementation, the decision variables are defined as follows: Indicates pending production tasks The time when the task begins execution; Indicates pending production tasks Task completion time; Indicates pending production tasks Assign to resource group When production tasks are pending Assigned to resource group hour, ,otherwise ; Indicates pending production tasks Is it a production task pending? The adjacent preceding task, when the task is to be scheduled Production tasks pending When the adjacent preceding task, ,otherwise ; Indicates resource group In the time slot Resource group load rate; It is used to determine the team switching penalty between adjacent tasks, but not to calculate the order of completion between non-adjacent tasks.

[0129] The task completion time is determined based on the task processing duration and the task start time; in one optional implementation, the task completion time is determined according to the following formula:

[0130] ;

[0131] in, Indicates pending production tasks Task processing time; the task start time is obtained by solving a mixed-integer linear programming model; the task completion time is used to subsequently calculate the total project duration and the weighted sum of completion times; task processing time Read from the set of tasks to be scheduled, or by the scenario-dependent intelligent scheduling optimization equipment based on the tasks to be scheduled. The embedding vector of the corresponding node and historical production scheduling data are determined; when the task processing time... Production tasks pending When the embedding vector of the corresponding node is determined, the task processing time is... As input parameters in a mixed-integer linear programming model, they are used to calculate the task completion time, total duration, and weighted sum of completion times.

[0132] The resource load balancing evaluation value is determined based on the resource group load rate.

[0133] The weighted sum of completion times is determined based on the importance score of each node and the task completion time.

[0134] The team switching penalty is determined based on the resource group allocation results and the preset team switching penalty rules. The method for establishing the preset team switching penalty rules includes: obtaining the team to which the task belongs, the team to which the resource group belongs, the handover relationship between adjacent tasks, and the historical cross-team handover time. When adjacent tasks belong to different resource groups or there is a cross-team handover relationship between adjacent tasks, the team switching penalty is determined based on the historical cross-team handover time, task interface complexity, or team collaboration cost. When adjacent tasks belong to the same resource group and there is no cross-team handover relationship between adjacent tasks, no team switching penalty is added, or the team switching penalty is determined according to the same-team handover rules.

[0135] Historical cross-team handover time is the average handover time or handover delay obtained from historical project scheduling data; task interface complexity is determined by the number of data interfaces, the number of deliverables, or the strength of dependencies between tasks; team collaboration cost is determined by the historical collaboration frequency between teams, the number of communication records, or the number of handover reworks; after converting historical cross-team handover time, task interface complexity, and team collaboration cost to the same numerical range, a weighted sum is obtained to obtain the team switching penalty for adjacent task execution.

[0136] The production target evaluation value is generated based on the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switchover penalty.

[0137] The constraints of the mixed-integer linear programming model are established based on resource constraints, timing constraints, dependency constraints, and sharing constraints; in an optional implementation, resource group allocation constraints are established according to the following formula:

[0138] ;

[0139] in, Indicates pending production tasks Assign to resource group ; This represents the set of resource group indexes; resource group allocation constraints are used to assign each pending production task to a resource group.

[0140] In one alternative implementation, dependency constraints are established according to the following formula:

[0141] ;

[0142] in, This represents the set of dependent task edges, which is generated from the dependent tasks in the set of tasks to be scheduled and the upstream and downstream dependencies in the heterogeneous graph; Indicates pending production tasks Production tasks pending The upstream task; dependency constraints are used to make the tasks to be scheduled pending production tasks Once completed, execution will begin.

[0143] In one alternative implementation, the resource group load rate constraint is established according to the following formula:

[0144] ;

[0145] in, Represents the set of time slot indexes; Indicates resource group In the time slot Resource group load rate; This represents the set of indexes for tasks awaiting production scheduling; Indicates pending production tasks For resource groups Resource demand; Indicates pending production tasks Is it in the time slot? By resource group implement; Indicates resource group Resource group capacity; resource group capacity is read from the resource group data source; resource requirements. Read from the set of tasks to be scheduled, or by the scenario-dependent intelligent scheduling optimization equipment based on the tasks to be scheduled. The embedding vector of the corresponding node and historical production scheduling data are determined; when resource demand... Production tasks pending When the embedding vector of the corresponding node is determined, the resource requirement As input parameters in mixed-integer linear programming models, they are used in the calculation of resource group load rate, resource load balance evaluation value, and constraint violation amount corresponding to resource constraints.

[0146] In one alternative implementation, when the set of tasks to be scheduled has already recorded tasks to be scheduled. For resource groups When determining the resource demand, the resource demand in the set of tasks to be scheduled is directly determined as follows: When no pending production tasks are recorded in the pending production task set For resource groups When the resource requirements need to be determined, or when adjustments to the resource requirements are needed based on scenario-level dependencies, the tasks to be scheduled will be... Embedding vectors and resource groups of corresponding nodes The resource group capacity data is input into the resource demand forecasting layer, and combined with the actual resource usage of similar nodes in the same resource group from historical production scheduling data, the task to be scheduled is output. For resource groups resource demand The network parameters of the resource demand prediction layer are trained from historical production scheduling data and pre-stored in the scenario-dependent intelligent scheduling optimization equipment.

[0147] In one alternative implementation, resource constraints are established according to the following formula:

[0148] ;

[0149] in, Indicates resource group The maximum load rate of the resource group; Indicates resource group In the time slot The resource constraints correspond to the constraint violations; the resource group load rate limit is read from the resource management system or constraint configuration table; the constraint violations corresponding to the resource constraints are obtained by solving the mixed-integer linear programming model, and satisfy... .

[0150] In one alternative implementation, shared constraints are established according to the following formula:

[0151] ;

[0152] in, Represents a globally unique set of meta-scenes; Representing the meta-scene The corresponding set of production tasks to be scheduled; Indicates pending production tasks Whether it is scheduled for execution depends on the production schedule. When scheduled to be executed ,otherwise Shared constraints are used to ensure that the production tasks corresponding to the same meta-scenario are scheduled only once; the globally unique meta-scenario set is generated by the shared relationships in the scenario hierarchy dependency relationship.

[0153] Determine the constraint violation amount corresponding to the resource constraint; the constraint violation amount corresponding to the resource constraint is the excess amount generated when the actual load rate of the resource group exceeds the upper limit of the resource group load rate; in an optional implementation, the constraint violation amount corresponding to the resource constraint is summarized according to the following formula:

[0154] ;

[0155] in, This represents the total number of constraint violations corresponding to resource constraints. Indicates resource group In the time slot The amount of constraint violation corresponding to the resource constraints.

[0156] By introducing the constraint violation quantities corresponding to resource constraints into the objective function of the mixed-integer linear programming model through Lagrange multipliers, the relaxed objective function is obtained.

[0157] It should be noted that the constraint violation amount corresponding to the resource constraint is the main source of the subsequent cost function; the temporal constraint, dependency constraint and shared constraint can be handled by filtering the set of available actions, action masking, constraint penalty terms or hard constraints of the mixed integer linear programming model; when the hard constraints of the mixed integer linear programming model are used for processing, the mixed integer linear programming model only outputs candidate scheduling results that satisfy the temporal constraint, dependency constraint and shared constraint.

[0158] In step S104, the mixed-integer linear programming model simultaneously provides the production scheduling target evaluation value and the constraint violation amount corresponding to the resource constraints; the production scheduling target evaluation value is used to determine the reward function in the subsequent constrained Markov decision process; the constraint violation amount corresponding to the resource constraints is used to determine the cost function in the subsequent constrained Markov decision process.

[0159] Step S105: Model the production scheduling decision process as a constrained Markov decision process, and set a reward function to characterize the production scheduling objective and a cost function to characterize the degree of violation of the production scheduling constraints in the constrained Markov decision process.

[0160] In some implementation methods, the implementation steps include:

[0161] The graph embedding matrix corresponding to the heterogeneous graph, the execution history characteristics of the set of tasks to be scheduled, the load matrix of the resource group in the future time slot, the expected completion time vector and time characteristics of each task are determined as the state in the constrained Markov decision process. The state is used to record the scheduling information that has been formed before the current scheduling decision, and serves as the input data for the near-end strategy optimization algorithm to select tasks to be executed, allocate resource groups and set start times.

[0162] The graph embedding matrix is ​​obtained by arranging the embedding vectors of each node in a preset node order; the preset node order is obtained by sorting the node identifiers and remains unchanged during the processing of the same set of tasks to be scheduled; a row in the graph embedding matrix corresponds to a node in the heterogeneous graph; the columns in the graph embedding matrix correspond to the embedding vector dimensions; the graph embedding matrix is ​​used to characterize the structural semantics of the original scene nodes, micro-scene nodes, and meta-scene nodes in the heterogeneous graph.

[0163] The execution history characteristics of the set of tasks to be scheduled include the completion percentage, actual time spent, allocated resource groups, and scheduling progress. The completion percentage is obtained by dividing the number of completed tasks by the total number of tasks in the set of tasks to be scheduled. The actual time spent is determined by the actual start time and actual completion time of the tasks to be scheduled that have been executed. The allocated resource groups are determined by the allocation results of the resource groups that have been included in the scheduling plan. The scheduling progress is obtained by dividing the processing time of completed tasks by the total processing time of all tasks.

[0164] The resource group's load matrix for future time slots is indexed by the resource group number and the future time slot number, recording the resource group load rate of each resource group within the future time slot; the future time slot is obtained by dividing the available production time range from the current time to the project end time; the granularity of the future time slot division is determined by the production time unit in the project management system; the resource group load rate is calculated from the determined production schedule, resource requirements, and resource group capacity.

[0165] The estimated completion time vector for each task includes the estimated completion time of each task in the set of tasks to be scheduled under the current scheduling status. For tasks to be scheduled that have been assigned a start time, the estimated completion time is obtained by adding the task start time and the task processing time. For tasks to be scheduled that have not been assigned a start time, the estimated completion time is obtained by adding the earliest possible start time and the task processing time. The earliest possible start time is determined by the completion time of the upstream task, the availability time of the resource group, and the timing constraints.

[0166] Time features include current time, workday marker, stage marker, and non-workday marker; the current time is determined by the system time of the scenario-dependent intelligent scheduling optimization equipment or the scheduling timeline in the project management system; the workday marker and non-workday marker are determined by the project calendar; the stage marker is determined by the project stage configuration in the project schedule requirements; time features are used to limit whether the tasks to be scheduled can start execution within the corresponding time slot.

[0167] The selection of tasks to be executed, allocation of resource groups, and setting of start times at the current moment are determined as actions in the constrained Markov decision process. The action is represented as a combination of task identifier, resource group identifier, and task start time. The task identifier is selected from the unscheduled tasks in the set of tasks to be scheduled. The resource group identifier is selected from the executable resource group corresponding to the task to be executed. The task start time is selected from the time slots that satisfy the timing constraints.

[0168] The set of actionable actions is filtered based on resource constraints, timing constraints, dependency constraints, and sharing constraints. When a pending task has an incomplete upstream task, the pending task is removed from the set of actionable actions. When a resource group does not belong to the executable resource group corresponding to the pending task, the action corresponding to the resource group is removed from the set of actionable actions. When the start time of a task falls on a non-working day or does not meet the project stage time limit, the action corresponding to the start time of the task is removed from the set of actionable actions. When a task corresponding to the same meta-scenario has already been scheduled, other tasks corresponding to the same meta-scenario are removed from the set of actionable actions.

[0169] The reward function is set based on the production scheduling target evaluation value; the production scheduling target evaluation value is output by the mixed-integer linear programming model; when the mixed-integer linear programming model optimizes by minimizing the production scheduling target evaluation value, the reward function takes the negative of the production scheduling target evaluation value; in the... At each decision-making moment, the action is executed. Then, the production scheduling target evaluation value is recalculated, and the recalculated production scheduling target evaluation value is used to determine the first... Rewards for each decision-making moment.

[0170] The cost function is set based on the constraint violation amount corresponding to the resource constraints; the constraint violation amount corresponding to the resource constraints is output by the mixed integer linear programming model; the constraint violation amount corresponding to the resource constraints is used to characterize the amount by which the resource group load rate exceeds the upper limit of the resource group load rate; in the... At the decision-making moment, after executing the action, the total constraint violation amount corresponding to the resource constraint is recalculated, and the recalculated total constraint violation amount corresponding to the resource constraint is used to determine the first decision-making moment. The cost function at each decision moment; temporal constraints, dependency constraints, and shared constraints are handled through action set filtering, action masking, or hard constraints of the mixed-integer linear programming model, and are not included in the cost function calculation.

[0171] Set the maximum allowed constraint violation amount; the maximum allowed constraint violation amount is given by the resource management system or constraint configuration table; the maximum allowed constraint violation amount is used to limit the total constraint violation amount corresponding to the resource constraint; when the resource management system or constraint configuration table does not give the maximum allowed constraint violation amount, the maximum allowed constraint violation amount is set to 0, so that the near-end strategy optimization algorithm updates the production scheduling strategy in a way that does not allow resource constraint violations.

[0172] In step S105, the production scheduling target evaluation value and the constraint violation quantity corresponding to the resource constraints output by the mixed integer linear programming model are converted into reward signals and cost signals in constraint reinforcement learning. The reward signal is used to evaluate the impact of the current action on the production scheduling target. The cost signal is used to evaluate the impact of the current action on the degree of production scheduling constraint violation. The state, action, reward function and cost function together constitute the input basis of the constrained Markov decision process.

[0173] Step S106: Use a proximal policy optimization algorithm based on reward function and cost function to learn a scheduling strategy that satisfies scheduling constraints, and generate a scheduling plan based on the scheduling strategy.

[0174] In some implementation methods, the implementation steps include:

[0175] The state in the constrained Markov decision process is input into the policy network, value network, and cost network. The policy network is used to output the action probability distribution, which includes the probability of selecting the task to be executed, the probability of allocating resources, and the probability of selecting the time to start the task. The value network is used to estimate the expected cumulative reward. The cost network is used to estimate the expected cumulative cost. The network parameters of the policy network, value network, and cost network are updated by the proximal policy optimization algorithm.

[0176] In one optional implementation, the policy network uses a Transformer encoder to process the graph embedding matrix and other state components corresponding to the heterogeneous graph, and outputs the action probability distribution; the graph embedding matrix is ​​input into the Transformer encoder to obtain the graph state encoding vector; the execution history features, the load matrix of the resource group in the future time slot, the expected completion time vector of each task and the time features are linearly mapped and concatenated with the graph state encoding vector to obtain the policy input vector; the policy network outputs the action probability distribution based on the policy input vector.

[0177] In one optional implementation, the value network estimates the expected cumulative reward based on the state components, and the cost network estimates the expected cumulative cost based on the state components; the input of the value network is the same as the input of the policy network, and the output of the value network is the estimated expected cumulative reward; the input of the cost network is the same as the input of the policy network, and the output of the cost network is the estimated expected cumulative cost; the value network and the cost network update the network parameters through the training samples corresponding to the reward function and the cost function, respectively.

[0178] The production scheduling strategy is updated based on the action probability distribution, expected cumulative reward, and expected cumulative cost; in the... At each decision moment, the policy network determines the state... Output the action probability distribution and determine the action from the set of possible actions. ; Perform actions Then, the reward function value is recalculated. and cost function value Based on the reward function value To obtain the expected cumulative reward, based on the cost function value The expected cumulative cost is obtained.

[0179] In one alternative implementation, the strategy ratio is determined according to the following formula:

[0180] ;

[0181] in, Indicates the first The strategy ratio at each decision moment; This indicates the state of the policy network after parameter updates. Down Output Action The probability of; This indicates the state of the policy network before the parameter update. Down Output Action The probability of; Represents the network parameters of the policy network.

[0182] In one alternative implementation, the reward advantage estimate and the cost advantage estimate are determined according to the following formula:

[0183] , ;

[0184] in, Indicates the first Estimation of reward advantage at each decision moment; Indicates the first Cost advantage estimation at each decision moment; Indicates the expected cumulative reward; Indicates the expected cumulative cost; Indicates the value network in state The expected cumulative reward estimate for the output; Indicates the cost network in state The expected cumulative cost estimate of the output; Network parameters representing the value network; This represents the network parameters of the cost network.

[0185] In one alternative implementation, the policy network objective function is determined according to the following formula:

[0186] ;

[0187] in, Represents the objective function of the policy network; Indicates the number of decision samples used in a single parameter update; This represents the second Lagrange multiplier in the proximal policy optimization algorithm; This represents the clipping hyperparameter; Given by the parameter configuration file; This indicates that the strategy ratio will be limited to... Within the range.

[0188] The network parameters of the policy network are updated according to the objective function of the policy network; when the cost advantage estimate after subtracting the Lagrange multiplier weighting from the reward advantage estimate corresponding to the action is positive, the output probability of the action in the corresponding state is increased; when the cost advantage estimate after subtracting the Lagrange multiplier weighting from the reward advantage estimate corresponding to the action is negative, the output probability of the action in the corresponding state is decreased; the policy ratio pruning is used to limit the magnitude of a single parameter update.

[0189] In one alternative implementation, the value network and the cost network are updated according to the following formula:

[0190] , ;

[0191] in, Represents the value network loss function; This represents the cost network loss function; Indicates the value network in state The expected cumulative reward estimate for the following output; Indicates the cost network in state The expected cumulative cost estimate of the output; Indicates the first The expected cumulative reward at each decision moment; Indicates the first The expected cumulative cost at each decision moment; the value network reduces Update network parameters; the cost network reduces Update network parameters.

[0192] The Lagrange multiplier is updated based on the cost function and the maximum allowed constraint violation. When the expected cumulative cost exceeds the maximum allowed constraint violation, the Lagrange multiplier is increased. When the expected cumulative cost does not exceed the maximum allowed constraint violation, the Lagrange multiplier is maintained or decreased. The maximum allowed constraint violation is given by the resource management system or constraint configuration table. If the resource management system or constraint configuration table does not give the maximum allowed constraint violation, the maximum allowed constraint violation is set to 0.

[0193] In one alternative implementation, the Lagrange multipliers are updated according to the following formula:

[0194] ;

[0195] in, Indicates the first The secondary parameter updates the corresponding Lagrange multipliers; Indicates the first The secondary parameter updates the corresponding Lagrange multipliers; Indicates the learning rate of the multiplier; Indicates the first The average expected cumulative cost corresponding to the parameter update; Indicates the maximum allowed constraint violation; the multiplier learning rate is given by the parameter configuration file; Used to make Lagrange multipliers non-negative.

[0196] The production scheduling strategy that satisfies the scheduling constraints is obtained by updating the network parameters of the policy network based on the reward function, cost function, and Lagrange multipliers. The production scheduling strategy is used to output the action probability distribution according to the state at each decision time and determine the task to be executed, the resource group, and the task start execution time from the set of actionable actions. When the expected cumulative cost corresponding to the production scheduling strategy does not exceed the maximum allowable constraint violation, the production scheduling strategy is determined to be a production scheduling strategy that satisfies the scheduling constraints.

[0197] Generate a production scheduling plan based on the production scheduling strategy; starting from the initial state, input the current state into the production scheduling strategy that satisfies the production scheduling constraints to obtain the action at the current decision moment; determine the tasks to be executed, resource groups, and task start times based on the actions; determine the task completion time based on the task start time and task processing time; update the execution history characteristics, the load matrix of resource groups in future time slots, the expected completion time vector and time characteristics of each task; repeat the production scheduling decision until all tasks in the set of tasks to be scheduled are arranged.

[0198] The production scheduling plan includes the start time of each task, the completion time of the task, the resource group allocation result, the task completion sequence, the resource group load rate, and the production scheduling target evaluation value. The start time and completion time of the task are used to form the production scheduling time arrangement; the resource group allocation result is used to form the resource usage arrangement; the task completion sequence is used to characterize the task execution order; the resource group load rate is used to characterize the resource usage; and the production scheduling target evaluation value is used to evaluate the production scheduling target corresponding to the production scheduling plan.

[0199] In step S106, a production scheduling plan is generated under the premise of satisfying the production scheduling constraints; the reward function is used to guide the production scheduling strategy to reduce the production scheduling target evaluation value; the cost function and Lagrange multiplier are used to limit the degree of violation of the production scheduling constraints; the near-end policy optimization algorithm is used to update the production scheduling strategy based on the action probability distribution, expected cumulative reward and expected cumulative cost.

[0200] It should be noted that in this application, the heterogeneous graph attention network does not directly output the final scheduling plan. Instead, it first converts the inclusion, reference, and sharing relationships between the original scene, micro-scene, and meta-scene into node embedding vectors and importance scores. The mixed-integer linear programming model calculates the scheduling target evaluation value and the constraint violation amount corresponding to the resource constraints based on the task start execution time, resource group allocation results, task completion time, and resource group load rate. The constrained Markov decision process breaks down the scheduling process into multiple decision moments, selecting the task to be executed, the resource group, and the task start execution time at each decision moment. The proximal policy optimization algorithm uses the scheduling target evaluation value to form a reward function and uses the constraint violation amount to form a cost function, so that the learned scheduling strategy simultaneously considers the project duration, resource balance, task priority, team switching, and resource constraints.

[0201] Example 2:

[0202] Based on Example 1, this example further illustrates the heterogeneous graph modeling and meta-path generation method.

[0203] Please see Figure 3 As shown, the scenario-dependent intelligent scheduling optimization equipment performs the following steps:

[0204] Step S201: Determine the original scenario, micro-scenario, and meta-scenario from the set of tasks to be scheduled and the scenario hierarchy dependency relationship.

[0205] In some implementations, a scenario decomposition table, a task scenario mapping table, or a business process configuration table is read. The scenario decomposition table records the hierarchical relationship between the original scenario, micro-scenario, and meta-scenario. The task scenario mapping table records the correspondence between tasks to be scheduled and the scenarios to which the tasks belong. The business process configuration table records the scenario composition corresponding to the business process. The original scenario, micro-scenario, and meta-scenario are determined based on the scenario granularity, scenario reuse identifier, and scenario parent-child relationship. When the scenario granularity is a complete business process, the corresponding scenario is determined as the original scenario. When the scenario granularity is a sub-process or sub-functional scenario, the corresponding scenario is determined as a micro-scenario. When the scenario reuse identifier indicates that the corresponding scenario is referenced by multiple micro-scenarios, the corresponding scenario is determined as the meta-scenario.

[0206] Step S202: Generate the original scene node, micro scene node, and meta scene node respectively.

[0207] In some implementations, an original scene node is generated for the original scene; the original scene node includes an original scene identifier, an original scene name, an original scene priority, and a set of tasks corresponding to the original scene; the original scene identifier is determined by the original scene number in the scene decomposition table; the original scene name is determined by the original scene name field in the scene decomposition table; the original scene priority is determined by the project schedule requirements or task priority; the set of tasks corresponding to the original scene consists of the tasks to be scheduled in the original scene to which the task belongs.

[0208] Generate micro-scene nodes for each micro-scene; each micro-scene node includes a micro-scene identifier, its original scene, micro-scene processing time, and micro-scene resource requirements; the micro-scene identifier is determined by the micro-scene number in the scene decomposition table; the original scene is determined by the parent-child relationship of the scenes; the micro-scene processing time is obtained by summing the processing times of the tasks to be scheduled for production corresponding to the micro-scene; the micro-scene resource requirements are obtained by merging the resource requirements of the tasks to be scheduled for production corresponding to the micro-scene; when the same resource type corresponds to multiple resource requirements, the micro-scene resource requirements are the sum of the multiple resource requirements.

[0209] Generate meta-scene nodes for meta-scenes; meta-scene nodes include meta-scene identifiers, meta-scene reuse counts, meta-scene reference counts, and meta-scene corresponding development tasks; the meta-scene identifier is determined by the meta-scene number in the scene decomposition table; the meta-scene reuse count is determined by the number of times the same meta-scene is called by different micro-scenes; the meta-scene reference count is determined by the number of records pointing to the same meta-scene in the reference relationship; the meta-scene corresponding development task consists of the pending production tasks of the meta-scene corresponding to the task's scene; then, write the original scene node type for the original scene node, the micro-scene node type for the micro-scene node, and the meta-scene node type for the meta-scene node.

[0210] Step S203: Generate edges of different edge types according to inclusion relationship, reference relationship and sharing relationship to obtain heterogeneous graph.

[0211] In some implementations, containment relationship edges are generated based on data records of micro-scenes contained in the original scene; the data records of micro-scenes contained in the original scene are determined by the scene parent-child relationship in the scene decomposition table; when the scene parent-child relationship records an original scene that includes a micro-scene, containment relationship edges are generated between the corresponding original scene node and the corresponding micro-scene node; the containment relationship edges are written into the containment relationship edge type; the containment relationship edges are used to characterize the composition relationship between the original scene node and the micro-scene node.

[0212] Reference edges are generated based on the data records of the micro-scene referencing the meta-scene; the data records of the micro-scene referencing the meta-scene are determined by the function call relationship in the business process configuration table or the meta-scene call record in the task scene mapping table; when the function call relationship or meta-scene call record indicates that a micro-scene calls a meta-scene, a reference edge is generated between the corresponding micro-scene node and the corresponding meta-scene node; the reference edge is written into the reference edge type; the reference edge is used to represent the call relationship between the micro-scene node and the meta-scene node.

[0213] Shared relationship edges are generated based on data records that are referenced by multiple micro-scenes or multiple original scenes for the same meta-scene. When the same meta-scene is referenced by multiple micro-scenes, shared relationship edges are generated between the nodes of the multiple micro-scenes that reference the same meta-scene. When the same meta-scene is referenced by multiple original scenes, shared relationship edges are generated between the nodes of the multiple original scenes that reference the same meta-scene. When two meta-scenes share the same development result, shared relationship edges are generated between the nodes of the two meta-scenes. The shared relationship edge is written with the shared relationship edge type. The shared meta-scene identifier, the number of times it is shared, and the identifier of the shared source node are written with the edge attribute of the shared relationship edge. Shared relationship edges are used to represent that the development result corresponding to the same meta-scene is used by multiple micro-scenes or multiple original scenes, or to represent that multiple meta-scenes have a relationship of sharing development results.

[0214] Step S204: Based on the original scene nodes, micro-scene nodes, meta-scene nodes, and edges of different edge types, generate a meta-path composed of alternating node types and edge types.

[0215] In some implementations, meta-paths are constructed according to functional aggregation semantics, functional reuse semantics, and business association semantics. The meta-paths corresponding to functional aggregation semantics are meta-scene node types, reference relationship edge types, micro-scene node types, inclusion relationship edge types, and original scene node types. Functional aggregation semantics is used to characterize the relationship between meta-scenes aggregated to the original scene via micro-scenes. The meta-paths corresponding to functional reuse semantics are meta-scene node types, reference relationship edge types, micro-scene node types, reference relationship edge types, and meta-scene node types. Functional reuse semantics is used to characterize the association relationship formed by different meta-scenes via the same micro-scene.

[0216] The meta-paths between original scenarios formed by shared nodes are generated according to the business association semantics. The meta-paths corresponding to the business association semantics are: original scenario node type, containment relationship edge type, micro-scenario node type, reference relationship edge type, meta-scenario node type, reference relationship edge type, micro-scenario node type, containment relationship edge type, and original scenario node type. The business association semantics are used to represent the association relationship formed by different original scenarios due to shared meta-scenarios. When there are shared relationship edges in the scenario hierarchy dependency relationship, the meta-paths corresponding to the business association semantics also include shared relationship edge types.

[0217] Step S205: Determine the node connection range for learning the embedding vectors of each node based on the meta-path.

[0218] In some implementations, the heterogeneous graph is traversed along the meta-path to obtain the set of aggregateable neighbor nodes corresponding to the target node. The traversal process starts from the target node and matches the nodes and edges in the heterogeneous graph in turn according to the node type and edge type recorded in the meta-path. When the matched node satisfies the node type of the corresponding position in the meta-path and the matched edge satisfies the edge type of the corresponding position in the meta-path, the matched node is added to the set of aggregateable neighbor nodes corresponding to the target node. The set of aggregateable neighbor nodes is determined as the node connection range corresponding to the target node.

[0219] Through the above steps, the heterogeneous graph preserves the direct relationship between the original scene, micro-scene, and meta-scene, and determines the node connection range across levels and scenes through meta-paths; the node connection range is used to limit the calculation object of the subsequent dynamic attention coefficient and provide structured input for the learning of the embedding vector of each node.

[0220] Example 3:

[0221] Building upon Example 2, this example further illustrates the method for learning the embedding vectors and importance scores of each node.

[0222] Please see Figure 4 As shown, the scenario-dependent intelligent scheduling optimization equipment performs the following steps:

[0223] Step S301: Select two nodes connected by an edge within the node's connection range as the source node and the target node, respectively.

[0224] In some implementations, node pairs with edge connections are selected based on the node connection range determined by the metapath and the edge set in the heterogeneous graph; the node connection range is determined by the metapath traversal result in Example 2; the edge set includes containment relationship edges, reference relationship edges, and shared relationship edges; for a node that needs to update its embedding vector, the node that needs to update its embedding vector is determined as the target node; the node within the node connection range that has an edge connection with the target node and provides features to be aggregated is determined as the source node.

[0225] Step S302: Determine the dynamic attention coefficients based on the node type of the source node, the node type of the target node, and the corresponding edge type.

[0226] In some implementations, the initial features of the source node, the initial features of the target node, and the edge type embedding are read; the initial features of the source node are determined by the row vector corresponding to the source node in the node initial feature matrix; the initial features of the target node are determined by the row vector corresponding to the target node in the node initial feature matrix; the edge type embedding is obtained by looking up the edge type mapping information in a table; then the initial features of the source node, the initial features of the target node, and the edge type embedding are input into the dynamic attention computation structure.

[0227] In one alternative implementation, the dynamic attention coefficient is determined according to the following formula:

[0228] ;

[0229] in, Indicates the source node Relative to the target node Dynamic attention coefficient; Represents the attention parameter vector; Represents a learnable linear transformation matrix; Indicates the source node The initial feature vector of the node or the embedding vector of the previous layer; Represents the target node The initial feature vector of the node or the embedding vector of the previous layer; Indicates the source node With the target node Embedding of edge types corresponding to the edge types between them; This represents a vector concatenation operation; Indicates the activation function; and It was obtained by training a heterogeneous graph attention network.

[0230] Step S303: Normalize the dynamic attention coefficients and aggregate the source nodes based on the normalized dynamic attention coefficients to obtain the embedding vector of the target node.

[0231] In some implementations, normalization is performed on multiple dynamic attention coefficients corresponding to the same target node; the normalization process applies to all source nodes within the node connection range of the target node; the normalized dynamic attention coefficients are used to represent the weights of the source node features when participating in the target node embedding vector update; then the source node features are weighted and aggregated according to the normalized dynamic attention coefficients to obtain the target node embedding vector.

[0232] In one alternative implementation, the normalized dynamic attention coefficients are determined according to the following formula:

[0233] ;

[0234] in, Indicates the source node Relative to the target node The normalized dynamic attention coefficient; Represents the target node The corresponding node connection range; Indicates the source node within the node's connection range. Relative to the target node The dynamic attention coefficient; the normalized range is the range of node connections corresponding to the same target node.

[0235] In one alternative implementation, the embedding vector of the target node is obtained according to the following formula:

[0236] ;

[0237] in, Represents the target node The embedding vector; Indicates the number of heads of attention; Given by the training profile of the heterogeneous graph attention network; Indicates the first The normalized dynamic attention coefficients corresponding to each attention head; Indicates the first The learnable linear transformation matrix corresponding to each attention head; Indicates the activation function; Indicates will The output vectors of each attention head are concatenated.

[0238] Step S304: Aggregate the source nodes according to their types to obtain type aggregation vectors.

[0239] In some implementations, the source nodes are divided into a set of original scene nodes, a set of micro-scene nodes, and a set of meta-scene nodes according to their node types. The set of original scene nodes includes source nodes of type original scene nodes. The set of micro-scene nodes includes source nodes of type micro-scene nodes. The set of meta-scene nodes includes source nodes of type meta-scene nodes. Then, weighted aggregation is performed on each type of source node to obtain a type aggregation vector.

[0240] In one alternative implementation, the type aggregation vector is determined according to the following formula:

[0241] ;

[0242] in, Represents the target node Corresponding node type Type aggregate vector; Represents the target node The node type within the node connection range is The set of source nodes; Indicates node type The corresponding learnable linear transformation matrix; Indicates the source node Relative to the target node The normalized dynamic attention coefficients.

[0243] Step S305: Determine the type-level attention weights based on the type aggregation vector; update the embedding vector of the target node based on the type-level attention weights.

[0244] In some implementations, the type aggregation vector is input into the type-level attention calculation structure to obtain the type-level attention weight; the type-level attention weight is used to represent the weight of the type aggregation vector corresponding to different node types in the target node embedding vector update process; then, the type aggregation vectors corresponding to the original scene node type, micro scene node type and meta scene node type are weighted and summed according to the type-level attention weight to obtain the updated target node embedding vector.

[0245] In one alternative implementation, the embedding vector of the target node is updated according to the following formula:

[0246] ;

[0247] in, Indicates the updated target node The embedding vector; This represents a set of node types; the set of node types includes the original scene node type, the micro-scene node type, and the meta-scene node type. Represents the target node Corresponding node type Type aggregate vector; Represents the target node Corresponding node type Type-level attention weights; type-level attention weights are obtained by normalizing the type aggregation vector input to the learnable linear layer, with the normalization range being the set of node types.

[0248] Step S306: Determine the set of downstream dependent nodes and the set of upstream dependent nodes for each node based on the upstream and downstream dependencies.

[0249] In some implementations, the downstream dependent node set and upstream dependent node set corresponding to each node are determined based on the edge direction, task completion sequence, and scene hierarchy reference relationship in the heterogeneous graph; for any node, nodes that can only be executed after any node is completed are added to the downstream dependent node set; nodes that need to be completed before any node is executed are added to the upstream dependent node set; the edge direction is determined by the scheduling dependency direction corresponding to the inclusion relationship, reference relationship, and sharing relationship; the task completion sequence is determined by the dependent tasks in the set of tasks to be scheduled.

[0250] Step S307: Generate an importance score for each node based on its embedding vector, the set of downstream dependent nodes, and the set of upstream dependent nodes.

[0251] In some implementations, the node embedding vector, the number of downstream dependent nodes, and the number of upstream dependent nodes are input into the importance score calculation structure; the node embedding vector is used to represent the structural semantics of the node in the heterogeneous graph; the number of downstream dependent nodes is used to represent the number of subsequent nodes affected after the node is completed; the number of upstream dependent nodes is used to represent the number of nodes subject to pre-restrictions; and the importance score is used to subsequently determine the weighted sum of completion times.

[0252] In one alternative implementation, the importance score for each node is generated according to the following formula:

[0253] ;

[0254] in, Indicates the source node Importance rating; Indicates the source node The embedding vector; Represents a vector of scoring parameters; Indicates the source node The set of downstream dependent nodes; Indicates the source node The set of upstream dependent nodes; This indicates the number of nodes in the downstream dependent node set; This indicates the number of nodes in the upstream dependent node set; This represents the total number of nodes in the heterogeneous graph; Indicates downstream dependency weight; The upstream dependency weights are represented by the score parameter vector, which is obtained from model training. The downstream dependency weights and upstream dependency weights are given by the parameter configuration file.

[0255] Through steps S301 to S307, the embedding vector of the target node simultaneously includes source node features, edge type information, and node type information; the importance score is generated simultaneously based on the node embedding vector, the downstream dependent node set, and the upstream dependent node set; the importance score serves as the basis for calculating the weighted sum of completion times in the subsequent mixed-integer linear programming model.

[0256] Example 4:

[0257] Based on Examples 1 to 3, this example further illustrates the method for establishing a mixed-integer linear programming model.

[0258] Please see Figure 5 As shown, the scenario-dependent intelligent scheduling optimization equipment performs the following steps:

[0259] Step S401: Determine the decision variables, which include task completion time and resource group load rate, and include the task start execution time, resource group allocation result, and task completion sequence.

[0260] In some implementations, a task start time variable is set. Resource group allocation result variables Task completion sequence relation variables Task completion time variable and resource group load rate variable ;in, and Indicates the index of tasks pending production scheduling; Represents the resource group index; Indicates the time slot index; Indicates pending production tasks The time when the task begins execution; Indicates pending production tasks Task completion time; Indicates resource group In the time slot The resource group load rate.

[0261] Will Set as a binary variable; when a production task is pending. Assigned to resource group hour, When production tasks are pending Not assigned to resource group hour, ;Will Set as a binary variable; when a production task is pending. Production tasks pending When the adjacent preceding task, When production tasks are pending Not a production schedule task When the adjacent preceding task, The task completion order relationship variable is used to determine the team switching penalty between adjacent task executions.

[0262] Step S402: Establish a mixed integer linear programming model based on decision variables, embedding vectors of each node, importance scores of each node, and production scheduling constraints.

[0263] In some implementations, the task start time, resource group allocation results, task completion order, task completion time, resource group load rate, node embedding vector, node importance score, resource constraints, timing constraints, dependency constraints, and sharing constraints are written into the mixed-integer linear programming model. The node embedding vector is used to characterize the structural semantics of the node corresponding to the task to be scheduled in the heterogeneous graph, and is used to determine the task processing time and resource requirements corresponding to the task to be scheduled. The node importance score is used to calculate the weighted sum of completion times. The scheduling constraints are used to limit the range of feasible solutions of the mixed-integer linear programming model. The task processing time determined by the node embedding vector participates in the calculation of task completion time and total project duration, and the resource requirements determined by the node embedding vector participate in the calculation of resource group load rate, resource load balancing evaluation value, and constraint violation amount corresponding to resource constraints.

[0264] In one alternative implementation, when the set of tasks to be scheduled has already recorded tasks to be scheduled. When determining the task processing time, the processing time of the tasks in the pending production task set is directly set as [the processing time]. When no pending production tasks are recorded in the pending production task set When the task processing time needs to be adjusted based on scenario hierarchy dependencies, the tasks to be scheduled should be... The embedded vector of the corresponding node is input into the processing time prediction layer, and combined with the actual task processing time of similar nodes in historical production scheduling data, the task to be scheduled is output. Task processing time The network parameters of the processing time prediction layer are trained from historical production scheduling data and pre-stored in the scenario-dependent intelligent scheduling optimization device.

[0265] Step S403: Determine the total project duration based on the task completion time.

[0266] In some implementations, the completion times of all tasks are read, and the total project duration is determined based on the maximum value among all task completion times; in an alternative implementation, the total project duration is determined according to the following formula:

[0267] ;

[0268] in, Indicates the total construction period; This represents the set of indexes for tasks awaiting production scheduling; Indicates pending production tasks The task completion time; the pending production task index set is generated from the task identifiers in the pending production task set.

[0269] Step S404: Determine the resource load balancing evaluation value based on the resource group load rate.

[0270] In some implementations, the resource group load rate of each resource group in multiple time slots is read, and a resource load balancing evaluation value is determined based on the absolute deviation of each resource group from the average load rate in the same time slot; in an optional implementation, a load deviation auxiliary variable is introduced. The time slot is determined according to the following formula. Average load rate of each resource group:

[0271] ;

[0272] in, Indicates time slot Average load rate of each resource group within the system; Represents the set of resource group indexes; Indicates resource group In the time slot The resource group load rate.

[0273] In one alternative implementation, the load deviation auxiliary variable is determined according to the following linear constraint:

[0274] , , ;

[0275] in, Indicates resource group In the time slot The corresponding load deviation auxiliary variable; the load deviation auxiliary variable is used to represent the resource group In the time slot Resource group load rate and time slot The absolute deviation between the average load rates of each resource group; the resource load balancing evaluation value is determined according to the following formula:

[0276] ;

[0277] in, This represents the resource load balancing evaluation value; This represents the time slot index set; the time slot index set is obtained by dividing the available production time range in the project schedule requirements.

[0278] Step S405: Determine the weighted sum of completion times based on the importance score of each node and the task completion time.

[0279] In some implementations, the importance score of each node is used as a weight, and the corresponding task completion time is used as the weighted object to calculate the sum of weighted completion times; in an optional implementation, the sum of weighted completion times is determined according to the following formula:

[0280] ;

[0281] in, This represents the sum of weighted completion times; Indicates pending production tasks Importance score of the corresponding node; Indicates pending production tasks The task completion time; when a pending task corresponds to multiple nodes, the importance scores of the multiple nodes are averaged, and the average value is used as the pending task completion time. Importance score for the corresponding node.

[0282] Step S406: Determine the team switching penalty based on the resource group allocation results and the preset team switching penalty rules.

[0283] In some implementations, the resource group allocation results corresponding to adjacent execution tasks are read, and the existence of cross-team switching is determined according to the preset team switching penalty rules. Adjacent execution tasks refer to the next pending production task in the same resource group scheduling timeline, which is executed by the same resource group after the previous pending production task is completed and has the smallest task start execution time. When there are multiple next pending production tasks with the same task start execution time, adjacent execution tasks are determined according to the ascending order of task identifiers.

[0284] Historical cross-team handover time is the average handover time or handover delay obtained from historical project scheduling data; task interface complexity is determined by the number of data interfaces between tasks, the number of deliverables, or the strength of dependencies; team collaboration cost is determined by the historical collaboration frequency between teams, the number of communication records, or the number of handover reworks; the team switchover penalty coefficient is determined according to the following formula:

[0285] ;

[0286] in, Indicates pending production tasks and pending production tasks Team switching penalty coefficient; This represents the normalized historical cross-team handover time. This represents the normalized task interface complexity. This represents the normalized cost of team collaboration. , and Represents the penalty weight, and satisfies The penalty weight is given by the parameter configuration file.

[0287] In one alternative implementation, the normalized historical cross-team handover time is determined according to the following formula:

[0288] ;

[0289] The normalized task interface complexity is determined using the following formula:

[0290] ;

[0291] The normalized team collaboration cost is determined using the following formula:

[0292] ;

[0293] in, Indicates pending production tasks and pending production tasks The historical handover process between teams took a long time; Indicates pending production tasks and pending production tasks The complexity of the task interface between them; Indicates pending production tasks and pending production tasks Team collaboration costs between them; and These represent the minimum and maximum values ​​of historical cross-team handover time in historical production scheduling data; and These represent the minimum and maximum values ​​of task interface complexity in historical production scheduling data, respectively. and These represent the minimum and maximum values ​​of team collaboration costs in historical production scheduling data, respectively; when the maximum and minimum values ​​are equal, the corresponding normalization result is 0.

[0294] In one alternative implementation, the team switching penalty is determined according to the following formula:

[0295] ;

[0296] in, This indicates a team switching penalty; Indicates pending production tasks Is it a production task pending? The adjacent preceding tasks; Indicates pending production tasks and pending production tasks Team switching penalty coefficient; the team switching penalty coefficient is determined by the preset team switching penalty rules; when pending production tasks and pending production tasks When the corresponding resource groups belong to the same team, .

[0297] Step S407: Generate a production scheduling target evaluation value based on the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switching penalty.

[0298] In some implementations, a multi-objective weighted approach is used to generate the production scheduling target evaluation value; in an alternative implementation, the production scheduling target evaluation value is generated according to the following formula:

[0299] ;

[0300] in, This represents the evaluation value of the production scheduling target; Indicates the total construction period; This represents the resource load balancing evaluation value; This represents the sum of weighted completion times; This indicates a team switching penalty; , , and This represents the target weight; the target weight is given in the parameter configuration file by the project schedule requirements, resource group load control requirements, task priority requirements, and team collaboration management requirements.

[0301] In one alternative implementation, the target weight is set to a non-negative number, and the target weight satisfies the following formula:

[0302] ;

[0303] in, , , and These correspond to the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switching penalty, respectively. The target weights are read through the parameter configuration file. The parameter configuration file is pre-configured by project managers according to project duration requirements, resource group load control requirements, task priority requirements, and team collaboration management requirements.

[0304] Step S408: Determine the constraint violation amount corresponding to the resource constraint, and introduce the constraint violation amount corresponding to the resource constraint into the objective function of the mixed integer linear programming model through Lagrange multipliers to obtain the relaxed objective function.

[0305] In some implementations, the constraint violation amount corresponding to the resource constraint is determined based on the difference between the actual load rate of the resource group and the upper limit of the resource group load rate; in an optional implementation, an execution allocation auxiliary variable is introduced. ;in, Indicates pending production tasks Is it in the time slot? By resource group Execution; When production is pending In the time slot By resource group During execution, When production tasks are pending In the time slot Not by resource group During execution, .

[0306] In one alternative implementation, the relationship between the execution allocation auxiliary variable and the task execution status variable, and the resource group allocation result variable is determined according to the following linear constraint:

[0307] , , ;

[0308] in, Indicates pending production tasks In the time slot Is it currently in execution status? Indicates pending production tasks Assign to resource group ; Perform allocation of auxiliary variables Task execution status variables Resource group allocation result variables All are binary variables; the above linear constraints are used to replace task execution state variables. Resource group allocation result variables The product between them.

[0309] In one alternative implementation, the resource group load rate upper limit constraint is established according to the following formula:

[0310] ;

[0311] in, Indicates resource group The maximum load rate of the resource group; Indicates resource group In the time slot The resource constraints correspond to the constraint violations; the resource group load rate limit is read from the resource management system or constraint configuration table; the constraint violations corresponding to the resource constraints are obtained by solving the mixed-integer linear programming model, and satisfy... .

[0312] In one alternative implementation, the resource group parallelism constraint is established according to the following formula:

[0313] ;

[0314] in, Indicates resource group The maximum number of tasks that can be executed in parallel within the same time slot; Read from the resource management system or constraint configuration table; perform auxiliary variable allocation. Indicates pending production tasks Is it in the time slot? By resource group Execution; resource group parallelism constraints are used to limit the resource group The number of pending production tasks to be executed within the same time slot.

[0315] In one alternative implementation, the constraint violations corresponding to the resource constraints are summarized according to the following formula:

[0316] ;

[0317] in, This represents the total number of constraint violations corresponding to resource constraints. Indicates resource group In the time slot The resource constraints are the corresponding constraint violations; the total constraint violations corresponding to the resource constraints are used to evaluate the degree of production scheduling constraint violations and serve as the basis for subsequent cost function calculations.

[0318] The constraint violation amount corresponding to the resource constraint is introduced into the objective function through the first Lagrange multiplier in the mixed-integer linear programming model; in an optional implementation, the relaxed objective function is determined according to the following formula:

[0319] ;

[0320] in, This represents the relaxed objective function; This represents the evaluation value of the production scheduling target; This represents the total number of constraint violations corresponding to resource constraints. Let represent the first Lagrange multiplier in the mixed-integer linear programming model; the initial value of the first Lagrange multiplier in the mixed-integer linear programming model is given by the parameter configuration file and satisfies . .

[0321] Step S409: Evaluate the production scheduling target based on the relaxed objective function, and evaluate the degree of production scheduling constraint violation based on the constraint violation amount corresponding to the resource constraints.

[0322] In some implementations, the relaxed objective function output value is used as the source of production scheduling target evaluation, and the constraint violation amount corresponding to the resource constraint is used as the source of the degree of production scheduling constraint violation. When the total constraint violation amount corresponding to the resource constraint is 0, the production scheduling scheme satisfies the resource constraint. When the total constraint violation amount corresponding to the resource constraint is greater than 0, the total constraint violation amount corresponding to the resource constraint is used to characterize the cumulative amount by which the resource group load rate exceeds the upper limit of the resource group load rate.

[0323] It should be noted that the constraint violation amount corresponding to the resource constraint is the main source of the subsequent cost function; the temporal constraint, dependency constraint and shared constraint are handled by filtering the set of possible actions, action masking, constraint penalty terms or hard constraints of the mixed integer linear programming model; when the hard constraints of the mixed integer linear programming model are used, the mixed integer linear programming model only outputs candidate scheduling results that satisfy the temporal constraint, dependency constraint and shared constraint.

[0324] Through steps S401 to S409, the mixed-integer linear programming model can express the production scheduling objective and resource constraint violation status in the same model; the production scheduling objective evaluation value is used to determine the reward function in the subsequent process; the constraint violation amount corresponding to the resource constraint is used to determine the cost function in the subsequent process; the relaxed objective function is used to provide the evaluation basis for the degree of production scheduling objective and production scheduling constraint violation for the near-end strategy optimization algorithm.

[0325] Example 5:

[0326] Based on Example 4, this example further illustrates the modeling method of constrained Markov decision processes and the method of setting reward and cost functions.

[0327] Please see Figure 6 As shown, the scenario-dependent intelligent scheduling optimization equipment performs the following steps:

[0328] Step S501: The graph embedding matrix corresponding to the heterogeneous graph, the execution history characteristics of the set of tasks to be scheduled, the load matrix of the resource group in the future time slot, the expected completion time vector and time characteristics of each task are determined as the state in the constrained Markov decision process.

[0329] In some implementations, the graph embedding matrix is ​​obtained by arranging the node embedding vectors output by the heterogeneous graph attention network in node order; the node order is obtained by sorting the node identifiers and remains unchanged during the processing of the same set of pending tasks; a row in the graph embedding matrix corresponds to a node in the heterogeneous graph; the columns in the graph embedding matrix correspond to the embedding vector dimensions; the graph embedding matrix is ​​used to characterize the structural semantics of the original scene nodes, micro-scene nodes, and meta-scene nodes.

[0330] In some implementations, the execution history characteristics of the set of pending production tasks include the completion percentage, execution duration, historical delay status, and actual task time. The completion percentage is obtained by dividing the number of completed pending production tasks by the total number of tasks in the set of pending production tasks. The execution duration is obtained by summing the execution durations of the pending production tasks that have been scheduled for execution. The historical delay status is obtained by comparing the actual completion time of the pending production tasks that have been scheduled for execution with the estimated completion time of the tasks. The actual task time is determined by the actual start time and actual completion time of the pending production tasks.

[0331] In some implementations, the load matrix of a resource group for future time slots includes a resource group number, a future time slot number, and a resource group load rate; the resource group number is determined by the resource group identifier in the resource group data source; the future time slot number is obtained by dividing the available production time range from the current time to the project end time; the resource group load rate is calculated from the determined production schedule, resource requirements, and resource group capacity; the load matrix of a resource group for future time slots is used to characterize the occupancy of each resource group within the subsequent production time range.

[0332] For unassigned tasks that are pending execution at the start time First, determine the production tasks to be scheduled. upstream task set When the upstream task set If empty, display the project start time. Confirmed as a production task to be scheduled upstream constraint start time When the upstream task set When not empty, according to Determine the production tasks to be scheduled The upstream constraint start time; where, Indicates upstream task Task completion time.

[0333] Then determine the production tasks to be scheduled based on the time constraints. Optional start time set ; Optional start time set The time slot is determined by a combination of project phase time limits, working day limits, non-working day limits, and task start time limits; If the time slot falls within the project phase timeframe, is a workday that allows for production scheduling, does not fall on a non-workday, and meets the task start time limit, then the time slot will be allocated. Add to optional start time set .

[0334] For tasks to be scheduled Each executable resource group From the set of optional start times Select time slots that meet the resource group availability constraints.

[0335] In one alternative implementation, according to Determine resource group The corresponding set of available start times.

[0336] in, Indicates pending production tasks In the resource group The set of available start times; Indicates resource group In the time slot Resource group load rate; Indicates pending production tasks For resource groups Resource demand; Indicates resource group Resource group capacity; Indicates resource group The maximum load rate of the resource group; Indicates pending production tasks Is it in the time slot? By resource group implement; Indicates resource group The maximum number of tasks that can be executed in parallel within the same time slot; Indicates pending production tasks The number of time slots corresponding to the task processing time.

[0337] Scenario-dependent intelligent scheduling optimization equipment according to Determine the production tasks to be scheduled The earliest possible start time; among which, Indicates pending production tasks The earliest possible start time; Indicates pending production tasks The corresponding set of executable resource groups; when the set of available start times corresponding to all executable resource groups is empty, the tasks to be scheduled will be... Mark the task as unschedulable in the current state and add it to the list of tasks awaiting scheduling. Remove it from the set of possible actions at the current decision moment.

[0338] according to Production tasks to be scheduled to start when unassigned tasks are determined The estimated completion time; among which, Indicates pending production tasks The estimated completion time under the current production schedule; the estimated completion time vector is obtained by arranging the estimated completion times of each task to be scheduled in the order of task identifiers.

[0339] In some implementations, the time features include the current time, workday marker, stage marker, and non-workday marker; the current time is determined by the system time of the scenario-dependent intelligent scheduling optimization equipment or the scheduling timeline in the project management system; the workday marker and non-workday marker are determined by the project calendar; the stage marker is determined by the project stage configuration in the project schedule requirements; the time features are used to limit whether the tasks to be scheduled can start execution within the corresponding time slot.

[0340] In one alternative implementation, the state is represented by the following formula:

[0341] ;

[0342] in, Indicates the first The state at each decision-making moment; This represents the graph embedding matrix corresponding to the heterogeneous graph; Indicates the first The historical characteristics of the execution at each decision moment; Indicates the first The load matrix of resource groups at each decision moment in future time slots; Indicates the first The estimated completion time vector of each task at each decision moment; Indicates the first The temporal characteristics of each decision moment.

[0343] Step S502, selecting the task to be executed, allocating resource groups, and setting the start time at the current moment will be determined as actions in the constrained Markov decision-making process.

[0344] In some implementations, an action is represented as an action triple consisting of a task identifier to be executed, a resource group identifier, and a task start execution time; the task identifier to be executed is selected from the tasks to be scheduled that have not yet been scheduled for execution in the set of tasks to be scheduled; the resource group identifier is selected from the executable resource group corresponding to the task to be executed; the task start execution time is selected from the time slot that satisfies the timing constraints; and the action space is a combination of the set of executable tasks, the set of available resource groups, and the set of optional start times.

[0345] In one alternative implementation, the action is represented by the following formula:

[0346] ;

[0347] in, Indicates the first Actions at each decision-making moment; Indicates the first The task to be executed at each decision-making moment; Indicates the first Resource groups allocated at each decision-making moment; Indicates tasks to be performed The time when the task begins execution.

[0348] The action space is filtered based on resource constraints, timing constraints, dependency constraints, and sharing constraints to obtain a set of actionable actions. When a pending task has an unfinished upstream task, the pending task is removed from the set of actionable actions. When a resource group does not belong to the executable resource group corresponding to the pending task, the action corresponding to the resource group is removed from the set of actionable actions. When the task start time falls on a non-working day or does not meet the project stage time limit, the action corresponding to the task start time is removed from the set of actionable actions. When a task corresponding to the same meta-scenario has already been scheduled, other tasks corresponding to the same meta-scenario are removed from the set of actionable actions.

[0349] Step S503: Determine the reward function based on the production scheduling target evaluation value.

[0350] In some implementations, the production scheduling target evaluation value output by the mixed-integer linear programming model is converted into a reward function; when the mixed-integer linear programming model optimizes by minimizing the production scheduling target evaluation value, the reward function takes the negative of the production scheduling target evaluation value; the... Actions to be executed at each decision moment Then, the production scheduling target evaluation value is recalculated, and the recalculated production scheduling target evaluation value is used to determine the first... The reward function value at each decision moment.

[0351] In one alternative implementation, the reward function is determined according to the following formula:

[0352] ;

[0353] in, Indicates the first The reward function value at each decision moment; Indicates the first Actions to be executed at each decision moment The resulting production scheduling target evaluation value; production scheduling target evaluation value The reward function value is generated by summing the total project duration, resource load balancing evaluation value, weighted completion time, and team switching penalty. This is used to enable the near-end strategy optimization algorithm to update the production scheduling strategy in the direction of reducing the evaluation value of the production scheduling target.

[0354] Step S504: Determine the cost function based on the constraint violation amount corresponding to the resource constraint.

[0355] In some implementations, the constraint violation amount corresponding to the resource constraint is converted into a cost function; the constraint violation amount corresponding to the resource constraint is used to characterize the amount by which the resource group load rate exceeds the upper limit of the resource group load rate; the first After executing the action at the first decision moment, the total constraint violation amount corresponding to the resource constraint is recalculated, and the recalculated total constraint violation amount is used to determine the first decision moment. The cost function value at each decision moment; temporal constraints, dependency constraints, and shared constraints are processed before the action enters the cost function calculation. When an action does not meet the temporal constraints, dependency constraints, or shared constraints, the action is removed from the set of actionable actions, or the policy network is prevented from outputting the action by action masking, or the corresponding candidate scheduling result is excluded by the hard constraints of the mixed integer linear programming model.

[0356] In one alternative implementation, the cost function is determined according to the following formula:

[0357] ;

[0358] in, Indicates the first The cost function value at each decision moment; Indicates the first The total constraint violation amount corresponding to the resource constraints obtained after executing an action at each decision moment; the total constraint violation amount corresponding to the resource constraints is obtained by summing the constraint violations corresponding to the resource constraints of each resource group in each time slot; the cost function value is used to make the near-end policy optimization algorithm update the scheduling strategy in the direction of limiting the degree of resource constraint violation; timing constraints, dependency constraints, and shared constraints are not included. The computational formula, temporal constraints, dependency constraints, and shared constraints are handled through action set filtering, action masking, or hard constraints of mixed-integer linear programming models.

[0359] In one alternative implementation, the expected cumulative reward and expected cumulative cost are determined according to the following formula:

[0360] , ;

[0361] in, Indicates the first The expected cumulative reward at each decision moment; Indicates the first The expected cumulative cost at each decision moment; Indicates the discount factor; This represents the final decision point in the production scheduling decision-making process; the discount factor is given by the parameter configuration file and satisfies... .

[0362] Step S505: Form a constrained Markov decision process based on the state, action, reward function, and cost function.

[0363] In some implementations, the state, action, transition probability, reward function, cost function, and discount factor are collectively defined as a constrained Markov decision process; the state space consists of a graph embedding matrix, the execution history features of the set of tasks to be scheduled, the load matrix of the resource group in the future time slot, the expected completion time vector and time features of each task; the action space consists of the task identifier, the resource group identifier, and the task start execution time; the transition probability is used to characterize the probability of entering the next state from the current state after performing an action.

[0364] In one alternative implementation, the constrained Markov decision process is defined according to the following formula:

[0365] ;

[0366] in, This represents a constrained Markov decision process. Representing the state space; Represents the action space; Indicates the transition probability; Represents the reward function; Represents the cost function; Indicates the discount factor; This indicates the maximum allowable constraint violation. The maximum allowable constraint violation is given by the resource management system or constraint configuration table. If the resource management system or constraint configuration table does not give a maximum allowable constraint violation, the maximum allowable constraint violation is set to 0.

[0367] In one alternative implementation, the transition probability is expressed by the following formula:

[0368] ;

[0369] in, Indicates the first The state at each decision-making moment; Indicates the first Actions at each decision-making moment; Indicates the first The state at the moment of decision; when the action is executed. Then the next state can be determined. hour, When performing an action The next state cannot be obtained afterward. hour, .

[0370] Through steps S501 to S505, the production scheduling decision process is transformed into a sequential decision problem solvable by constrained reinforcement learning; the state is used to represent the current production scheduling information; the action is used to represent the current production scheduling decision; the reward function is used to represent the production scheduling objective; the cost function is used to represent the degree of resource constraint violation; the temporal constraint, dependency constraint, and shared constraint are used to determine the set of possible actions, or to perform action masking, or to serve as hard constraints in the mixed-integer linear programming model; the constrained Markov decision process serves as the input basis for the proximal policy optimization algorithm to learn the production scheduling strategy.

[0371] Example 6:

[0372] Building upon Example 5, this example further illustrates a method for learning a scheduling strategy that satisfies scheduling constraints using a proximal policy optimization algorithm based on reward and cost functions.

[0373] Please see Figure 7 As shown, the scenario-dependent intelligent scheduling optimization equipment performs the following steps:

[0374] Step S601, constrain the state input policy network, value network, and cost network in the Markov decision process.

[0375] In some implementations, the policy network, value network, and cost network share a state encoding layer, a graph state encoding structure, or a feature fusion layer. The state encoding layer is used to numerically map the execution history features, the load matrix of the resource group in the future time slot, the expected completion time vector of each task, and the time features. The graph state encoding structure is used to encode the graph embedding matrix. The feature fusion layer is used to concatenate or sum the encoding results corresponding to the graph embedding matrix and the encoding results corresponding to other state components.

[0376] In some implementations, the policy network, value network, and cost network are each provided with an independent parameter layer or an independent output layer. The independent parameter layer is used to enable the policy network, value network, and cost network to learn the network parameters required for their respective outputs. The independent output layer is used to enable the policy network to output the action probability distribution, the value network to output the expected cumulative reward estimate, and the cost network to output the expected cumulative cost estimate. The inputs to the policy network, value network, and cost network are all states in a constrained Markov decision process.

[0377] Step S602: Output the action probability distribution through the policy network.

[0378] In some implementations, the policy network outputs the probability of selecting a task to be executed, the probability of allocating a resource group, and the probability of selecting a start time based on the state. The probability of selecting a task to be executed corresponds to the probability of selecting a task from the set of executable tasks. The probability of allocating a resource group corresponds to the probability of selecting a resource group from the set of available resource groups. The probability of selecting a start time corresponds to the probability of selecting the start time of a task from the set of optional start times. The policy network samples or selects the scheduling action at the current time based on the action probability distribution.

[0379] In one alternative implementation, the policy network outputs the action probability distribution according to the following formula:

[0380] ;

[0381] in, Indicates the policy network in state Down Output Action The probability of; This indicates the production scheduling action at the current moment; This indicates the task to be executed at the current moment; This indicates the resource group currently allocated; Indicates tasks to be performed The time when the task begins execution; Represents the network parameters of the policy network.

[0382] Step S603: Estimate the expected cumulative reward through the value network.

[0383] In some implementations, the value network outputs an estimated expected cumulative reward based on the state; the estimated expected cumulative reward is used to evaluate the impact of the current production scheduling state on the evaluation value of the subsequent production scheduling target; the training objective of the value network is to make the estimated expected cumulative reward close to the expected cumulative reward; the expected cumulative reward is calculated by the reward function and discount factor in Example 5.

[0384] In one alternative implementation, the value network outputs an estimate of the expected cumulative reward according to the following formula:

[0385] ;

[0386] in, Indicates the value network in state The expected cumulative reward estimate for the following output; Indicates the first The expected cumulative reward at each decision moment; This represents the network parameters of the value network; the network parameters of the value network are updated by the value network loss function.

[0387] Step S604: Estimate the expected cumulative cost using the cost network.

[0388] In some implementations, the cost network outputs an expected cumulative cost estimate based on the state; the expected cumulative cost estimate is used to evaluate the impact of the current production scheduling state on the degree of violation of subsequent production scheduling constraints; the training objective of the cost network is to make the expected cumulative cost estimate close to the expected cumulative cost; the expected cumulative cost is calculated by the cost function and discount factor in Example 5.

[0389] In one alternative implementation, the cost network outputs the expected cumulative cost estimate according to the following formula:

[0390] ;

[0391] in, Indicates the cost network in state The expected cumulative cost estimate of the output; Indicates the first The expected cumulative cost at each decision moment; This represents the network parameters of the cost network; the network parameters of the cost network are updated by the cost network loss function.

[0392] Step S605: Update the production scheduling strategy based on the action probability distribution, expected cumulative reward, and expected cumulative cost.

[0393] In some implementations, the policy network parameters are updated according to the pruning objective function of the near-end policy optimization algorithm; in the... At each decision point, read the action output by the policy network before the parameter update. The probability is calculated, and the action output by the policy network after parameter updates is read. The probability of the two probabilities is used to determine the policy ratio; the expected cumulative reward estimate and the expected cumulative cost estimate are used to determine the policy update direction.

[0394] In one alternative implementation, the strategy ratio is determined according to the following formula:

[0395] ;

[0396] in, Indicates the first The strategy ratio at each decision moment; This indicates the state of the policy network after parameter updates. Down Output Action The probability of; This indicates the state of the policy network before the parameter update. Down Output Action The probability of; This represents the network parameters of the policy network before the parameter update.

[0397] In one alternative implementation, the reward advantage estimate and the cost advantage estimate are determined according to the following formula:

[0398] , ;

[0399] in, Indicates the first Estimation of reward advantage at each decision moment; Indicates the first Cost advantage estimation at each decision moment; Indicates the first The expected cumulative reward at each decision moment; Indicates the first The expected cumulative cost at each decision moment; This represents the estimated expected cumulative reward of the value network output; This represents the expected cumulative cost estimate of the cost network output.

[0400] In one alternative implementation, the pruning objective function of the near-end policy optimization algorithm is determined according to the following formula:

[0401] ;

[0402] in, Represent the objective function for pruning; Indicates the number of decision samples used in a single parameter update; This represents the second Lagrange multiplier in the proximal policy optimization algorithm; This represents the clipping hyperparameters; the clipping hyperparameters are given by the parameter configuration file; This indicates that the strategy ratio will be limited to... Within the range.

[0403] In some implementations, decision samples are generated during the simulated scheduling process of the current project to be scheduled. The decision samples include a state, an action, a reward function value, a cost function value, and a next state. The state is determined by the graph embedding matrix, execution history features, the load matrix of the resource group in the future time slot, the expected completion time vector of each task, and time features. The action is output by the policy network from the set of actionable actions. The reward function value is determined by the scheduling target evaluation value. The cost function value is determined by the total constraint violation amount corresponding to the resource constraint. The next state is obtained by updating the scheduling information after the action is executed. The decision samples are used to update the policy network, the value network, and the cost network.

[0404] Step S606: Update the Lagrange multipliers based on the cost function and the maximum allowed constraint violation.

[0405] In some implementations, the expected cumulative cost is compared with the maximum allowable constraint violation, and the Lagrange multiplier is updated based on the comparison result; when the expected cumulative cost exceeds the maximum allowable constraint violation, the Lagrange multiplier is increased; when the expected cumulative cost does not exceed the maximum allowable constraint violation, the Lagrange multiplier is maintained or decreased; the maximum allowable constraint violation is given by the resource management system or constraint configuration table.

[0406] In one alternative implementation, the second Lagrange multiplier in the near-end policy optimization algorithm is updated according to the following formula:

[0407] ;

[0408] in, Indicates the first The second Lagrange multiplier in the proximal policy optimization algorithm corresponding to the secondary parameter update; Indicates the first The second Lagrange multiplier in the proximal policy optimization algorithm corresponding to the secondary parameter update; Indicates the learning rate of the multiplier; Indicates the first The average expected cumulative cost corresponding to the parameter update; Indicates the maximum allowed constraint violation; the multiplier learning rate is given by the parameter configuration file; This indicates a non-negative projection operation.

[0409] Step S607: Update the parameters of the policy network based on the reward function, cost function, and Lagrange multipliers to obtain a production scheduling strategy that satisfies the production scheduling constraints.

[0410] In some implementations, the reward function and the cost function weighted by Lagrange multipliers are used together as the basis for policy updates. When the degree of violation of the production scheduling constraint represented by the cost function exceeds the maximum allowable constraint violation, the Lagrange multipliers are increased, so that the constraint penalty weight corresponding to the cost advantage estimate in the pruning objective function is increased. When the degree of violation of the production scheduling constraint represented by the cost function meets the maximum allowable constraint violation, the Lagrange multipliers are maintained or decreased, so that the constraint penalty weight in the pruning objective function is maintained or decreased.

[0411] The network parameters of the strategy network are updated according to the pruning objective function, the network parameters of the value network are updated according to the value network loss function, and the network parameters of the cost network are updated according to the cost network loss function. When the expected cumulative cost corresponding to the scheduling strategy does not exceed the maximum allowable constraint violation, and the actions output by the scheduling strategy all belong to the set of actionable actions, the scheduling strategy is determined to be a scheduling strategy that satisfies the scheduling constraints. The scheduling strategy that satisfies the scheduling constraints is used to generate the scheduling plan.

[0412] Through steps S601 to S607, the production scheduling strategy update process simultaneously considers the production scheduling target and the degree of resource constraint violation; the action probability distribution is used to determine the production scheduling action; the expected cumulative reward is used to evaluate the production scheduling target; the expected cumulative cost is used to evaluate the degree of resource constraint violation; the second Lagrange multiplier in the near-end strategy optimization algorithm is used to adjust the constraint penalty weight of the cost function in the strategy update; the action set is used to restrict actions that do not meet resource constraints, temporal constraints, dependency constraints, and shared constraints from entering the production scheduling strategy output range; wherein, the constraint violation amount corresponding to the resource constraint participates in the cost function calculation, and the temporal constraints, dependency constraints, and shared constraints are processed through action set filtering, action masking, or hard constraints of the mixed integer linear programming model.

[0413] Example 7:

[0414] Based on Examples 1 to 6, this example further illustrates a method for model-independent meta-learning based on historical production schedule data before generating a production schedule plan.

[0415] Please see Figure 8 As shown, the scenario-dependent intelligent scheduling optimization equipment performs the following steps:

[0416] Step S701: Obtain historical production schedule data.

[0417] Historical production scheduling data includes the scenario hierarchy dependencies of completed projects and historical production scheduling data; the scenario hierarchy dependencies of completed projects include the original scenario, micro-scenario, meta-scenario, containment relationship, reference relationship and sharing relationship in the completed projects; the original scenario, micro-scenario and meta-scenario are used to generate the heterogeneous graph corresponding to the completed projects; the containment relationship, reference relationship and sharing relationship are used to generate the edge type and meta path corresponding to the completed projects; historical production scheduling data includes one or more of the following: task start execution time, task completion time, resource group allocation result, task completion order relationship, resource group load rate and actual production scheduling target evaluation value.

[0418] Field alignment is performed on historical project scheduling data; task identifiers, task processing durations, resource requirements, task scenarios, task teams, and executable resource groups in completed projects are converted to the same data fields as those in the current projects to be scheduled; original scenarios, micro-scenarios, meta-scenarios, inclusion relationships, reference relationships, and sharing relationships in completed projects are converted to the same scenario hierarchy dependency relationship format as those in the current projects to be scheduled; the field-aligned historical project scheduling data is used as input data for model-independent meta-learning.

[0419] Step S702: Use historical production scheduling data as a learning task to perform model-independent meta-learning to obtain meta-initialization parameters.

[0420] In some implementations, each completed project is treated as a learning task; the scenario hierarchy dependencies of completed projects and historical production scheduling data are used as inputs to the learning task; a heterogeneous graph corresponding to the learning task is established based on the scenario hierarchy dependencies of completed projects; the historical production scheduling scheme corresponding to the learning task is reconstructed based on the historical production scheduling data; the production scheduling strategy loss function or the production scheduling target loss function is used as the learning task loss; the production scheduling strategy loss function is determined by the action probabilities in the historical production scheduling scheme and the action probabilities output by the strategy network; the production scheduling target loss function is determined by the actual production scheduling target evaluation value and the production scheduling target evaluation value generated by the strategy network.

[0421] In one alternative implementation, model-independent meta-learning is performed according to the following formula:

[0422] , ;

[0423] in, Indicates the first One learning task; Indicates the number of learning tasks; Indicates the initialization parameters; Indicates the first The adaptation parameters corresponding to each learning task; This represents the updated meta-initialization parameters; Indicates the first The training loss corresponding to each learning task; Indicates the first The validation loss corresponding to each learning task; Indicates the inner loop learning rate; This represents the outer loop learning rate; the inner loop learning rate and outer loop learning rate are given by the parameter configuration file.

[0424] Step S703: Initialize the near-end strategy optimization algorithm based on the meta-initialization parameters, and obtain the production scheduling strategy through gradient update.

[0425] In some implementations, meta-initialization parameters are loaded into the policy network, value network, and cost network; when the meta-initialization parameters include shared state coding layer parameters, the shared state coding layer parameters are loaded into the shared state coding layers of the policy network, value network, and cost network; when the meta-initialization parameters include independent output layer parameters, the independent output layer parameters are loaded into the action probability distribution output layer of the policy network, the expected cumulative reward output layer of the value network, and the expected cumulative cost output layer of the cost network, respectively.

[0426] In some implementations, project-level gradient updates are performed for currently pending production projects. The inner loop is updated step by step; the state, action, reward function and cost function of the current project to be scheduled are sampled by the near-end policy optimization algorithm during the simulated scheduling process of the current project to be scheduled, or are obtained by migrating the historical project data after field alignment; the simulated scheduling process starts from the initial state of the current project to be scheduled, selects an action from the set of available actions according to the current scheduling strategy, and calculates the reward function value and cost function value after the action is executed; This indicates the number of inner loop update steps; the number of inner loop update steps is given by the parameter configuration file; each inner loop update updates the network parameters according to the pruning objective function, value network loss function, and cost network loss function in Example 6; completing project-level gradient updates or After the inner ring is updated, a production scheduling strategy adapted to the current projects to be scheduled is obtained.

[0427] Step S704: Run the adapted production scheduling strategy to generate a production scheduling plan.

[0428] In some implementations, the status corresponding to the current project to be scheduled is input into the adapted scheduling strategy, and the tasks to be executed, resource group allocation results, and start times are output. The tasks to be executed are determined from the set of executable tasks of the current project to be scheduled. The resource group allocation results are determined from the set of available resource groups corresponding to the tasks to be executed. The start time is determined from the set of optional start times that meet the timing constraints. Then, the task completion time is determined according to the task processing time, and the status corresponding to the current project to be scheduled is updated.

[0429] In some implementations, the scheduling decision is repeatedly executed according to the adapted scheduling strategy until all the tasks in the current pending scheduling project are arranged; after each scheduling decision, the execution history characteristics, the load matrix of the resource group in the future time slot, the expected completion time vector and time characteristics of each task are updated; the generated scheduling plan includes the task start execution time, task completion time, resource group allocation result, task completion sequence relationship, resource group load rate and scheduling target evaluation value.

[0430] Through steps S701 to S704, the meta-initialization parameters are determined using the scenario hierarchy dependency relationship of completed projects and historical production scheduling data; the meta-initialization parameters are used to initialize the policy network, value network and cost network in the near-end policy optimization algorithm; the adapted production scheduling strategy is used to generate the production scheduling plan for the current projects to be scheduled.

[0431] Example 8:

[0432] Based on Examples 1 to 7, this example provides an overall example to illustrate the complete execution process of the scenario-dependent intelligent scheduling optimization method in the same project.

[0433] In some implementations, a set of tasks to be scheduled, scenario-level dependencies, and scheduling constraints are obtained from an intelligent agent scenario development project. The set of tasks to be scheduled includes login intent recognition tasks, order query tasks, payment status confirmation tasks, message generation tasks, and exception transfer to human assistant tasks. Scenario-level dependencies include after-sales service original scenario, order processing original scenario, identity verification micro-scenario, order query micro-scenario, payment confirmation micro-scenario, intent recognition meta-scenario, message generation meta-scenario, and transfer to human assistant meta-scenario. Scheduling constraints include resource constraints, timing constraints, dependency constraints, and sharing constraints. Resource groups include algorithm resource groups, backend resource groups, and test resource groups.

[0434] From the set of pending production tasks and the dependency relationships of the scenario hierarchy, the original scenario, micro-scenario, and meta-scenario are determined; the original scenario of after-sales service and the original scenario of order processing are determined as the original scenario; the micro-scenario of identity verification, the micro-scenario of order query, and the micro-scenario of payment confirmation are determined as micro-scenarios; the meta-scenario of intent recognition, the meta-scenario of message generation, and the meta-scenario of transferring to human agent are determined as meta-scenarios; among them, the meta-scenario of intent recognition is referenced by the micro-scenario of identity verification and the micro-scenario of order query; the meta-scenario of message generation is referenced by the micro-scenario of order query and the micro-scenario of payment confirmation.

[0435] Generate original scene nodes, micro-scene nodes, and meta-scene nodes based on the original scene, micro-scenes, and meta-scenes; generate inclusion relationship edges based on the relationship between the identity verification micro-scene and the order query micro-scene in the after-sales service original scene; generate inclusion relationship edges based on the relationship between the order query micro-scene and the payment confirmation micro-scene in the order processing original scene; generate reference relationship edges based on the relationship between the identity verification micro-scene referencing the intent recognition meta-scene, the order query micro-scene referencing the intent recognition meta-scene and the message generation meta-scene, and the payment confirmation micro-scene referencing the message generation meta-scene; generate shared relationship edges based on the relationship between the intent recognition meta-scene and the message generation meta-scene being referenced by multiple micro-scenes; where both the identity verification micro-scene and the order query micro-scene reference the intent recognition meta-scene, a shared relationship edge is generated between the identity verification micro-scene node and the order query micro-scene node; where both the order query micro-scene and the payment confirmation micro-scene reference the message generation meta-scene, a shared relationship edge is generated between the order query micro-scene node and the payment confirmation micro-scene node; the shared relationship edge is written to the shared relationship edge type, and the shared meta-scene identifier is used as the edge attribute of the shared relationship edge.

[0436] The embedding vectors of each node are learned based on metapaths and node connection ranges. Metapaths include meta-scene node types, reference relationship edge types, micro-scene node types, containment relationship edge types, and original scene node types. The node connection range is determined by the set of aggregateable neighbor nodes obtained by traversing the heterogeneous graph along the metapath. For each target node, dynamic attention coefficients are calculated based on source node features, target node features, and edge type embeddings. The source node features are aggregated based on the normalized dynamic attention coefficients to obtain the embedding vector of the target node.

[0437] Importance scores for each node are generated based on the downstream dependent node set and the upstream dependent node set. For intent recognition meta-scenario nodes, the downstream dependent node set includes identity verification micro-scenario nodes and order query micro-scenario nodes. For message generation meta-scenario nodes, the downstream dependent node set includes order query micro-scenario nodes and payment confirmation micro-scenario nodes. For order query micro-scenario nodes, the upstream dependent node set includes intent recognition meta-scenario nodes. The number of nodes in the downstream dependent node set, the number of nodes in the upstream dependent node set, and the node embedding vector are used together to generate the importance scores.

[0438] A mixed-integer linear programming model is established based on task completion time, resource group load rate, task start time, resource group allocation results, task completion sequence, importance score of each node, and scheduling constraints. The login intent recognition task is assigned to the algorithm resource group, the order query task to the backend resource group, the payment status confirmation task to the backend resource group, the message generation task to the algorithm resource group, and the exception transfer to manual task to the testing resource group. Task completion time is determined based on task start time and processing time. Resource group load rate is determined based on resource requirements and resource group capacity. Total project duration is determined based on task completion time. Resource load balancing evaluation value is determined based on resource group load rate. Weighted completion time sum is determined based on importance score and task completion time. Team switching penalty is determined based on resource group allocation results and preset team switching penalty rules.

[0439] The scheduling target evaluation value is generated based on the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switchover penalty; the constraint violation amount corresponding to the resource constraint is determined based on the difference between the actual load rate of the resource group and the upper limit of the resource group load rate; when the actual load rate of the backend resource group in a certain future time slot exceeds the upper limit of the backend resource group load rate, the excess part is determined as the constraint violation amount corresponding to the resource constraint of the backend resource group in the corresponding future time slot; the constraint violation amounts corresponding to the resource constraints of each resource group in each future time slot are summed to obtain the total constraint violation amount corresponding to the resource constraint.

[0440] The graph embedding matrix, execution history features, resource group load matrix for future time slots, estimated completion time vector, and time features are defined as states. The graph embedding matrix is ​​obtained by arranging the embedding vectors of each node according to the node identifier. The execution history features record the number of tasks scheduled, the completion rate, and the allocated resource groups. The resource group load matrix for future time slots records the resource group load rate of the algorithm resource group, backend resource group, and test resource group in the future time slot. The estimated completion time vector records the estimated completion time of each task to be scheduled in the current scheduling state. The time features record the current time, working day marker, stage marker, and non-working day marker.

[0441] The actions are defined as selecting a task to be executed, allocating a resource group, and setting a start time. At one decision moment, the action can be to select an order query task, allocate a backend resource group, and set the start time of the order query task. At another decision moment, the action can be to select a message generation task, allocate an algorithm resource group, and set the start time of the message generation task. The actions are determined from the set of actionable actions. The set of actionable actions is obtained by filtering resource constraints, timing constraints, dependency constraints, and sharing constraints.

[0442] Set a reward function and a cost function; the reward function is determined by the production scheduling target evaluation value; when the mixed-integer linear programming model optimizes by minimizing the production scheduling target evaluation value, the reward function takes the negative of the production scheduling target evaluation value; the cost function is determined by the total constraint violation amount corresponding to the resource constraints; when the total constraint violation amount corresponding to the resource constraints is 0, the cost function value is 0; when the total constraint violation amount corresponding to the resource constraints is greater than 0, the cost function value is equal to the total constraint violation amount corresponding to the resource constraints.

[0443] The scheduling strategy is updated using a proximal policy optimization algorithm based on reward and cost functions. The state is input into the policy network, value network, and cost network. The policy network outputs the probability of selecting the task to be executed, the probability of allocating the resource group, and the probability of selecting the start time. The value network outputs the expected cumulative reward estimate. The cost network outputs the expected cumulative cost estimate. The network parameters of the policy network are updated based on the reward advantage estimate, cost advantage estimate, and Lagrange multipliers.

[0444] When the number of constraints violated in relation to resource constraints exceeds the maximum allowed number of constraints violated, update the Lagrange multipliers and increase the constraint penalty weight; when the number of constraints violated in relation to resource constraints does not exceed the maximum allowed number of constraints violated, maintain or decrease the constraint penalty weight; when the expected cumulative cost of the scheduling strategy does not exceed the maximum allowed number of constraints violated, and all actions output by the scheduling strategy belong to the set of actionable actions, the scheduling strategy is determined to be a scheduling strategy that satisfies the scheduling constraints.

[0445] Based on the scheduling strategy, the output includes the task start time, task completion time, resource group allocation results, task order of completion, and resource group load rate. The task start time is used to determine the starting point of each task to be scheduled on the scheduling timeline. The task completion time is determined by the task start time and the task processing time. The resource group allocation results are used to determine the resource group corresponding to each task to be scheduled. The task order of completion is used to determine the execution order among the tasks to be scheduled. The resource group load rate is used to determine the occupancy of the algorithm resource group, backend resource group, and test resource group in the future time slot. The above outputs are summarized into a scheduling plan.

[0446] Example 8 illustrates the sequential relationship between heterogeneous graph modeling, embedding vector learning, importance scoring, mixed-integer linear programming model, constrained Markov decision process, and proximal policy optimization algorithm in this application's embodiments. The set of tasks to be scheduled and the scenario hierarchy dependency relationship are first transformed into a heterogeneous graph. After embedding vector learning and importance scoring, the heterogeneous graph is input into the mixed-integer linear programming model. The mixed-integer linear programming model outputs the scheduling target evaluation value and the constraint violation amount corresponding to the resource constraints. The scheduling target evaluation value and the constraint violation amount corresponding to the resource constraints are then input into the constrained Markov decision process and the proximal policy optimization algorithm to finally generate a scheduling plan.

[0447] Example 9:

[0448] Please see Figure 9 As shown, this embodiment provides a scenario-dependent intelligent scheduling optimization system. Since the scenario-dependent intelligent scheduling optimization system uses the scenario-dependent intelligent scheduling optimization method in embodiments 1 to 8, it also has the same or corresponding technical effects as the scenario-dependent intelligent scheduling optimization method, which will not be described in detail here.

[0449] The scenario-dependent intelligent scheduling optimization system includes a data acquisition module, a heterogeneous graph modeling module, a node learning module, a model building module, a process modeling module, and a solution generation module.

[0450] The data acquisition module is used to acquire the set of tasks to be scheduled, the scenario hierarchy dependencies, and the scheduling constraints. The data acquisition module reads the set of tasks to be scheduled from the project management system, task management table, requirement breakdown table, or manual configuration file; reads the scenario hierarchy dependencies from the scenario configuration file, business process breakdown table, or intelligent agent scenario development platform; and reads the scheduling constraints from the resource management system, project schedule requirements, team scheduling records, or constraint configuration table.

[0451] The heterogeneous graph modeling module is used to model the set of tasks to be scheduled into a heterogeneous graph based on the scene hierarchy dependency relationship; the heterogeneous graph is used to represent the inclusion relationship, reference relationship and sharing relationship between different level scenes; the heterogeneous graph modeling module determines the original scene, micro scene and meta scene based on the scene hierarchy dependency relationship; generates a node set based on the original scene, micro scene and meta scene; generates an edge set based on the inclusion relationship, reference relationship and sharing relationship; and the node set, edge set, node initial feature matrix, node type mapping information and edge type mapping information are jointly determined as the heterogeneous graph.

[0452] The node learning module is used to learn the embedding vector and importance score of each node based on the node type, edge type, and upstream and downstream dependencies of each node in the heterogeneous graph. The node learning module determines the node connection range based on the meta-path; determines the dynamic attention coefficient based on the node type of the source node, the node type of the target node, and the corresponding edge type; normalizes the dynamic attention coefficient; aggregates the source nodes based on the normalized dynamic attention coefficient to obtain the embedding vector of the target node; and generates the importance score of each node based on the embedding vector of each node, the set of downstream dependent nodes, and the set of upstream dependent nodes.

[0453] The model building module is used to establish a mixed-integer linear programming model based on the embedding vectors, importance scores, and scheduling constraints of each node. The mixed-integer linear programming model is used to evaluate the scheduling objectives and the degree of violation of scheduling constraints. The model building module determines the task completion time, resource group load rate, task start execution time, resource group allocation results, and task completion order. It generates the scheduling objective evaluation value based on the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switchover penalty. It determines the constraint violation amount corresponding to the resource constraints based on the resource constraints. The constraint violation amount corresponding to the resource constraints is introduced into the objective function of the mixed-integer linear programming model through Lagrange multipliers to obtain the relaxed objective function.

[0454] The process modeling module models the scheduling decision-making process as a constrained Markov decision process, and sets a reward function to represent the scheduling objective and a cost function to represent the degree of constraint violation in the constrained Markov decision process. The process modeling module determines the state by the graph embedding matrix, execution history features, the load matrix of resource groups in future time slots, the expected completion time vector of each task, and time features; it determines the action by selecting the task to be executed at the current moment, allocating resource groups, and setting the start time; it determines the reward function based on the evaluation value of the scheduling objective; and it determines the cost function based on the constraint violation amount corresponding to the resource constraint.

[0455] The scheme generation module is used to learn a scheduling strategy that satisfies scheduling constraints using a proximal policy optimization algorithm based on reward and cost functions, and generate a scheduling scheme based on the scheduling strategy. The scheme generation module takes the state input of the constrained Markov decision process into the policy network, value network, and cost network; outputs the action probability distribution through the policy network; estimates the expected cumulative reward through the value network; estimates the expected cumulative cost through the cost network; updates the scheduling strategy based on the action probability distribution, expected cumulative reward, and expected cumulative cost; and generates a scheduling scheme based on the scheduling strategy that satisfies the scheduling constraints.

[0456] In one possible implementation, the heterogeneous graph modeling module includes a node generation unit, an edge generation unit, a meta-path generation unit, and a node connection range determination unit.

[0457] The node generation unit is used to determine the original scenario, micro-scenario, and meta-scenario from the set of tasks to be scheduled and the scenario hierarchy dependency relationship, and generate original scenario nodes, micro-scenario nodes, and meta-scenario nodes respectively. The node generation unit writes the original scenario identifier, original scenario name, original scenario priority, and the original scenario corresponding task set for the original scenario node; writes the micro-scenario identifier, the original scenario to which it belongs, the micro-scenario processing time, and the micro-scenario resource requirements for the micro-scenario node; and writes the meta-scenario identifier, the number of times the meta-scenario is reused, the number of times the meta-scenario is referenced, and the development task corresponding to the meta-scenario for the meta-scenario node.

[0458] The edge generation unit generates edges of different types based on inclusion, reference, and sharing relationships to obtain a heterogeneous graph. The edge generation unit generates inclusion relationship edges based on data records of micro-scenes included in the original scene; it generates reference relationship edges based on data records of meta-scenes referenced by micro-scenes; it generates sharing relationship edges based on data records of the same meta-scene referenced by multiple micro-scenes or multiple original scenes; when the same meta-scene is referenced by multiple micro-scenes, sharing relationship edges are generated between the nodes of the multiple micro-scenes referencing the same meta-scene; when the same meta-scene is referenced by multiple original scenes, sharing relationship edges are generated between the nodes of the multiple original scenes referencing the same meta-scene; when two meta-scenes share the same development result, sharing relationship edges are generated between the nodes of the two meta-scenes; and the inclusion relationship edges, reference relationship edges, and sharing relationship edges are written into the edge set.

[0459] The meta-path generation unit is used to generate meta-paths composed of alternating node types and edge types based on original scene nodes, micro-scene nodes, meta-scene nodes, and edges of different edge types. The meta-path generation unit constructs meta-paths according to functional aggregation semantics, functional reuse semantics, and business association semantics. Functional aggregation semantics corresponds to meta-scene node types, reference relationship edge types, micro-scene node types, containment relationship edge types, and original scene node types. Functional reuse semantics corresponds to meta-scene node types, reference relationship edge types, micro-scene node types, reference relationship edge types, and meta-scene node types. Business association semantics corresponds to meta-paths formed between original scene nodes via shared nodes.

[0460] The node connection range determination unit is used to determine the node connection range for learning the embedding vectors of each node based on the meta-path; the node connection range determination unit traverses the heterogeneous graph along the meta-path; adds neighboring nodes that can be reached along the meta-path and participate in the embedding vector aggregation to the set of aggregable neighboring nodes; and determines the set of aggregable neighboring nodes as the node connection range.

[0461] In one possible implementation, the node learning module includes a dynamic attention coefficient determination unit, an embedding vector generation unit, a type-level update unit, and an importance score generation unit.

[0462] The dynamic attention coefficient determination unit is used to determine the dynamic attention coefficient based on the node type of the source node, the node type of the target node, and the corresponding edge type. The dynamic attention coefficient determination unit reads the initial features of the source node, the initial features of the target node, and the edge type embedding. The initial features of the source node, the initial features of the target node, and the edge type embedding are input into the dynamic attention calculation structure to obtain the dynamic attention coefficient of the source node relative to the target node.

[0463] The embedding vector generation unit is used to normalize the dynamic attention coefficients and aggregate the source nodes based on the normalized dynamic attention coefficients to obtain the embedding vector of the target node; the embedding vector generation unit performs normalization processing on multiple dynamic attention coefficients corresponding to the same target node; and performs weighted aggregation on the source node features according to the normalized dynamic attention coefficients to obtain the embedding vector of the target node.

[0464] The type-level update unit is used to aggregate source nodes according to node type to obtain type aggregation vectors; determine type-level attention weights based on type aggregation vectors; update the embedding vector of the target node according to type-level attention weights; the type-level update unit divides source nodes according to original scene node type, micro-scene node type and meta-scene node type; performs aggregation for each type of source node; and performs weighted fusion of type aggregation vectors corresponding to different node types according to type-level attention weights.

[0465] The importance score generation unit is used to determine the set of downstream dependent nodes and the set of upstream dependent nodes for each node based on upstream and downstream dependencies; it generates the importance score for each node based on the embedding vector of each node, the set of downstream dependent nodes, and the set of upstream dependent nodes; the importance score generation unit determines the set of downstream dependent nodes and the set of upstream dependent nodes based on the edge direction, task completion sequence, and scene hierarchy reference relationship in the heterogeneous graph; the node embedding vector, the number of downstream dependent nodes, and the number of upstream dependent nodes are input into the importance score calculation structure to obtain the importance score.

[0466] In one possible implementation, the model building module includes a decision variable determination unit, a production scheduling target evaluation unit, a production scheduling constraint processing unit, and a relaxation processing unit.

[0467] The decision variable determination unit is used to determine the task completion time and resource group load rate, and to determine the task start time, resource group allocation result, and task completion sequence. The decision variable determination unit sets the task start time variable, resource group allocation result variable, task completion sequence variable, task completion time variable, and resource group load rate variable. The resource group allocation result variable and the task completion sequence variable are binary variables.

[0468] The production scheduling target evaluation unit is used to determine the total project duration based on task completion time, determine the resource load balancing evaluation value based on resource group load rate, determine the weighted sum of completion time based on the importance score of each node and task completion time, determine the team switching penalty based on resource group allocation results and preset team switching penalty rules, and generate the production scheduling target evaluation value. The preset team switching penalty rules are established based on the team to which the task belongs, the team to which the resource group belongs, the handover relationship between adjacent tasks, and the historical cross-team handover time.

[0469] The scheduling constraint processing unit is used to establish resource constraints, timing constraints, dependency constraints, and sharing constraints. Resource constraints are used to limit the capacity of resource groups, the parallelism of resource groups, the upper limit of resource group load rate, and the available time of personnel or computing power. Timing constraints are used to limit the start time of task execution, the completion time of task, and the time limit of non-working days or project stages. Dependency constraints are used to limit the sequential execution requirements between tasks with upstream and downstream dependencies. Sharing constraints are used to ensure that tasks corresponding to the same meta-scenario are scheduled only once.

[0470] The relaxation processing unit determines the constraint violation amount corresponding to the resource constraint. It introduces the constraint violation amount of the resource constraint into the objective function of the mixed-integer linear programming model through Lagrange multipliers to obtain the relaxed objective function. The relaxation processing unit determines the constraint violation amount corresponding to the resource constraint based on the difference between the actual load rate of the resource group and the upper limit of the resource group's load rate. It sums the constraint violations corresponding to the resource constraints of each resource group in each time slot to obtain the total constraint violation amount corresponding to the resource constraint. Finally, it multiplies the total constraint violation amount corresponding to the resource constraint by Lagrange multipliers and adds it to the production scheduling target evaluation value to obtain the relaxed objective function.

[0471] In one possible implementation, the process modeling module includes a state determination unit, an action determination unit, and a function setting unit.

[0472] The state determination unit is used to determine the graph embedding matrix, execution history features, resource group load matrix for future time slots, expected completion time vector of each task, and time features as the state. The graph embedding matrix is ​​obtained by arranging the embedding vectors of each node in node order. The execution history features include the completed percentage, the executed duration, the historical delay status, and the actual time consumed by the task. The resource group load matrix for future time slots includes the resource group number, the future time slot number, and the resource group load rate. The expected completion time vector of each task includes the expected completion time of each task to be scheduled in the current scheduling state. The time features include the current time, working day marker, stage marker, and non-working day marker.

[0473] The action determination unit is used to determine the selection of tasks to be executed, the allocation of resource groups, and the setting of start times as actions at the current moment; the action determination unit determines the action triple consisting of the task identifier, the resource group identifier, and the task start time as an action; the action space is determined by the combination of the set of executable tasks, the set of available resource groups, and the set of optional start times; the set of actionable actions is obtained by filtering resource constraints, timing constraints, dependency constraints, and sharing constraints.

[0474] The function setting unit is used to determine the reward function based on the production scheduling target evaluation value and the cost function based on the constraint violation amount corresponding to the resource constraints. When the mixed integer linear programming model optimizes by minimizing the production scheduling target evaluation value, the function setting unit determines the negative of the production scheduling target evaluation value as the reward function value and the total constraint violation amount corresponding to the resource constraints as the cost function value. The reward function, cost function, state, and action are used together to construct a constrained Markov decision process.

[0475] In one possible implementation, the scheme generation module includes a network input unit, an action output unit, a reward estimation unit, a cost estimation unit, a policy update unit, a Lagrange multiplier update unit, and a production scheduling scheme output unit.

[0476] The network input unit is used to input the state of the constrained Markov decision process into the policy network, value network, and cost network; the network input unit inputs the graph embedding matrix, execution history features, resource group load matrix for future time slots, expected completion time vectors of each task, and time features into the shared state encoding layer, graph state encoding structure, or feature fusion layer; then the encoding results are input into the policy network, value network, and cost network.

[0477] The action output unit is used to output the action probability distribution through the policy network; the action output unit outputs the probability of selecting the task to be executed, the probability of allocating the resource group, and the probability of selecting the start time according to the state; it samples or selects the scheduling action at the current moment from the set of actionable actions according to the action probability distribution; the scheduling action includes the task identifier, the resource group identifier, and the task start execution time.

[0478] The reward estimation unit is used to estimate the expected cumulative reward through the value network; the reward estimation unit outputs the expected cumulative reward estimate based on the state; the expected cumulative reward is calculated by the reward function and the discount factor; the network parameters of the value network are updated based on the difference between the expected cumulative reward estimate and the expected cumulative reward.

[0479] The cost estimation unit is used to estimate the expected cumulative cost through the cost network; the cost estimation unit outputs the expected cumulative cost estimate based on the state; the expected cumulative cost is calculated by the cost function and the discount factor; the network parameters of the cost network are updated based on the difference between the expected cumulative cost estimate and the expected cumulative cost.

[0480] The strategy update unit is used to update the production scheduling strategy based on the action probability distribution, expected cumulative reward, and expected cumulative cost; the strategy update unit determines the strategy ratio based on the action probabilities before and after the parameter update; determines the reward advantage estimate based on the expected cumulative reward and the value network output; determines the cost advantage estimate based on the expected cumulative cost and the cost network output; and updates the network parameters of the strategy network according to the pruning objective function.

[0481] The Lagrange multiplier update unit is used to update the Lagrange multipliers according to the cost function and the maximum allowed constraint violation. The Lagrange multiplier update unit compares the average expected cumulative cost with the maximum allowed constraint violation. When the average expected cumulative cost exceeds the maximum allowed constraint violation, the Lagrange multiplier is increased. When the average expected cumulative cost does not exceed the maximum allowed constraint violation, the Lagrange multiplier is kept constant or decreased. After the Lagrange multiplier undergoes a non-negative projection operation, it participates in the network parameter update of the policy network.

[0482] The production scheduling output unit is used to generate a production scheduling plan based on a production scheduling strategy that meets the production scheduling constraints. Starting from the initial state, the production scheduling output unit inputs the current state into a production scheduling strategy that meets the production scheduling constraints; based on the production scheduling strategy that meets the production scheduling constraints, it outputs the tasks to be executed, resource groups, and task start times; it determines the task completion time based on the task start time and task processing time; it updates the state and repeats the production scheduling decision until all tasks in the set of tasks to be scheduled are arranged; the production scheduling plan includes the task start time, task completion time, resource group allocation results, task priority relationship, resource group load rate, and production scheduling target evaluation value.

[0483] In one possible implementation, the scenario-dependent intelligent scheduling optimization system also includes a meta-learning module.

[0484] The meta-learning module is used to acquire historical project scheduling data, which includes the scenario hierarchy dependencies of completed projects and historical scheduling data. The historical project scheduling data is used as a learning task to perform model-independent meta-learning to obtain meta-initialization parameters. Based on the meta-initialization parameters, the near-end policy optimization algorithm is initialized, and the scheduling policy is obtained through gradient updates. The meta-learning module loads the meta-initialization parameters into the policy network, value network, and cost network. For the current project to be scheduled, project-level gradient updates or inner-loop updates are performed to obtain a scheduling policy adapted to the current project to be scheduled.

[0485] It should be noted that the scenario-dependent intelligent scheduling optimization system provided in this embodiment is only illustrated by the division of each functional module. In engineering implementation, the functions can be assigned to different functional modules according to the operating environment. That is, the internal structure of the scenario-dependent intelligent scheduling optimization system is divided into different functional modules to complete all or part of the functions described above. The scenario-dependent intelligent scheduling optimization system provided in this embodiment and the scenario-dependent intelligent scheduling optimization method embodiment belong to the same concept. For details of the specific implementation process, please refer to the method embodiment, which will not be repeated here.

[0486] The above description is merely a preferred embodiment of this application, and the scope of protection of this application shall be determined by the scope defined in the claims. Any improvements and modifications made by those skilled in the art without departing from the essence of the technical solution of this application shall fall within the scope of protection of this application. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of this application shall also be considered within the scope of protection of this application.

Claims

1. A scenario-dependent intelligent scheduling optimization method, characterized in that, include: Obtain the set of tasks to be scheduled, the scenario hierarchy dependencies, and the scheduling constraints; Based on the scenario hierarchy dependency, the set of tasks to be scheduled is modeled as a heterogeneous graph. The heterogeneous graph is used to represent the inclusion relationship, reference relationship and sharing relationship between scenarios at different levels. Methods for modeling the set of tasks to be scheduled as a heterogeneous graph based on the scene hierarchy dependency include: The original scene, micro scene, and meta scene are determined from the set of tasks to be scheduled and the scene hierarchy dependency relationship, and the original scene node, micro scene node, and meta scene node are generated respectively. Different edge types are generated based on inclusion, reference, and sharing relationships to obtain a heterogeneous graph. Based on the node type, edge type, and upstream and downstream dependencies of each node in the heterogeneous graph, the embedding vector and importance score of each node are learned. A mixed-integer linear programming model is established based on the embedding vectors, importance scores, and scheduling constraints of each node. This model is used to evaluate the degree of violation of scheduling objectives and constraints. Specifically, the mixed-integer linear programming model generates scheduling objective evaluation values, including: The total project duration is determined based on the task completion time. Determine the resource load balancing evaluation value based on the resource group load rate; The weighted sum of completion times is determined based on the importance score of each node and the task completion time. The team switching penalty is determined based on the resource group allocation results and the preset team switching penalty rules. The production scheduling target evaluation value is generated based on the total project duration, resource load balancing evaluation value, weighted completion time sum, and team switchover penalty. The production scheduling decision process is modeled as a constrained Markov decision process, and a reward function to characterize the production scheduling objective and a cost function to characterize the degree of violation of the production scheduling constraints are set in the constrained Markov decision process. A proximal policy optimization algorithm based on reward and cost functions is used to learn scheduling strategies that satisfy scheduling constraints, and a scheduling plan is generated based on the scheduling strategies. The scheduling constraints include resource constraints, timing constraints, dependency constraints, and sharing constraints. The sharing constraints are used to ensure that tasks corresponding to the same meta-scenario are scheduled only once.

2. The scenario-dependent intelligent scheduling optimization method according to claim 1, characterized in that, After obtaining the heterogeneous graph, a meta-path is generated based on the original scene nodes, micro-scene nodes, meta-scene nodes, and edges of different edge types. The node connection range for learning the embedding vectors of each node is determined based on the meta-path.

3. The scenario-dependent intelligent scheduling optimization method according to claim 2, characterized in that, Methods for learning the embedding vectors of each node include: Two nodes connected by an edge within the node's connection range are designated as the source node and the target node, respectively. The dynamic attention coefficient is determined based on the node type of the source node, the node type of the target node, and the corresponding edge type. The dynamic attention coefficients are normalized, and the source nodes are aggregated based on the normalized dynamic attention coefficients to obtain the embedding vector of the target node.

4. The scenario-dependent intelligent scheduling optimization method according to claim 3, characterized in that, Other methods for obtaining the embedding vector of the target node include: Aggregate the source nodes according to their node types to obtain a type aggregation vector; Determine type-level attention weights based on the type aggregation vector; Update the embedding vector of the target node based on the type-level attention weights.

5. The scenario-dependent intelligent scheduling optimization method according to claim 1, characterized in that, Methods for learning the importance scoring of each node include: Determine the set of downstream dependent nodes and the set of upstream dependent nodes for each node based on the upstream and downstream dependencies; An importance score for each node is generated based on its embedding vector, the set of downstream dependent nodes, and the set of upstream dependent nodes.

6. The scenario-dependent intelligent scheduling optimization method according to claim 1, characterized in that, Methods for establishing mixed-integer linear programming models include: Determine the decision variables, which include the task start time, resource group allocation results, task completion order, task completion time, and resource group load rate. A mixed-integer linear programming model is established based on decision variables, embedding vectors of each node, importance scores of each node, and production scheduling constraints.

7. The scenario-dependent intelligent scheduling optimization method according to claim 1, characterized in that, Methods for establishing mixed-integer linear programming models also include: Determine the constraint violation amount corresponding to the resource constraint, and introduce the constraint violation amount into the objective function of the mixed integer linear programming model through Lagrange multipliers to obtain the relaxed objective function. The production scheduling objective is evaluated based on the relaxed objective function, and the degree of constraint violation is evaluated based on the amount of constraint violation.

8. The scenario-dependent intelligent scheduling optimization method according to claim 1, characterized in that, Methods for modeling the production scheduling decision process as a constrained Markov decision process include: The graph embedding matrix corresponding to the heterogeneous graph, the execution history characteristics of the set of tasks to be scheduled, the load matrix of the resource group in the future time slot, the expected completion time vector and time characteristics of each task are determined as the state in the constrained Markov decision process. The actions of selecting tasks to be executed, allocating resource groups, and setting start times at the current moment will be determined as constraints in the Markov decision-making process.

9. The scenario-dependent intelligent scheduling optimization method according to claim 7, characterized in that, Production scheduling targets include production scheduling target evaluation values, and the degree of violation of production scheduling constraints includes the amount of constraint violation corresponding to resource constraints; The reward function is determined based on the production scheduling target evaluation value, while the cost function is determined based on the constraint violation amount corresponding to the resource constraints.

10. The scenario-dependent intelligent scheduling optimization method according to claim 8, characterized in that, Methods for learning scheduling strategies that satisfy scheduling constraints using proximal policy optimization algorithms based on reward and cost functions include: The state inputs in the constrained Markov decision process are the policy network, value network, and cost network; The action probability distribution is output through the policy network; Estimate expected cumulative rewards through value networks; Estimate the expected cumulative cost using a cost network; The production scheduling strategy is updated based on the action probability distribution, expected cumulative reward, and expected cumulative cost.

11. The scenario-dependent intelligent scheduling optimization method according to claim 10, characterized in that, Methods for updating production scheduling strategies include: Update the Lagrange multipliers in the near-end policy optimization algorithm based on the cost function and the maximum allowed constraint violation. Based on the reward function, cost function, and updated Lagrange multipliers, the parameters of the policy network are updated to obtain a production scheduling policy that satisfies the production scheduling constraints.

12. The scenario-dependent intelligent scheduling optimization method according to claim 1, characterized in that, Before generating the production schedule, the following steps are also included: Obtain historical production schedule data, which includes the scenario hierarchy dependencies of completed projects and historical production schedule data. Historical production scheduling data is used as a learning task to perform model-independent meta-learning and obtain meta-initialization parameters. The near-end strategy optimization algorithm is initialized based on the meta-initialization parameters, and the production scheduling strategy is obtained through gradient update.

13. A scenario-dependent intelligent scheduling optimization system, characterized in that, include: The data acquisition module is used to acquire the set of tasks to be scheduled, the scenario hierarchy dependencies, and scheduling constraints. The heterogeneous graph modeling module is used to model the set of tasks to be scheduled as a heterogeneous graph based on the scene hierarchy dependency relationship. The heterogeneous graph is used to represent the inclusion relationship, reference relationship and sharing relationship between different levels of scenes. The method for modeling a set of tasks to be scheduled as a heterogeneous graph based on scene hierarchy dependencies includes: determining the original scene, micro scene, and meta scene from the set of tasks to be scheduled and scene hierarchy dependencies, and generating original scene nodes, micro scene nodes, and meta scene nodes respectively; generating edges of different edge types based on inclusion relationships, reference relationships, and sharing relationships to obtain a heterogeneous graph; The node learning module is used to learn the embedding vector and importance score of each node based on the node type, edge type and upstream and downstream dependencies of each node in the heterogeneous graph. The model building module is used to build a mixed-integer linear programming model based on the embedding vectors, importance scores, and scheduling constraints of each node. This mixed-integer linear programming model is used to evaluate the degree of violation of scheduling objectives and constraints. Specifically, the mixed-integer linear programming model generates scheduling objective evaluation values, including: determining the total project duration based on task completion times; determining the resource load balancing evaluation value based on resource group load rates; determining the weighted sum of completion times based on the importance scores of each node and task completion times; determining the team switching penalty based on resource group allocation results and preset team switching penalty rules; and generating the scheduling objective evaluation value based on the total project duration, resource load balancing evaluation value, weighted sum of completion times, and team switching penalty. The process modeling module is used to model the production scheduling decision process as a constrained Markov decision process, and to set a reward function to characterize the production scheduling objective and a cost function to characterize the degree of violation of the production scheduling constraints in the constrained Markov decision process. The scheme generation module is used to learn a scheduling strategy that satisfies scheduling constraints by using a proximal policy optimization algorithm based on reward function and cost function, and generate a scheduling scheme based on the scheduling strategy. The scheduling constraints include resource constraints, timing constraints, dependency constraints and sharing constraints. The sharing constraints are used to ensure that tasks corresponding to the same meta-scenario are scheduled only once.

Images