Multi-target production scheduling method and system based on reinforcement learning

Abstract

The invention provides a multi-target production scheduling method and system based on reinforcement learning, and the method comprises the steps: carrying out the matching analysis based on a task demand list and an equipment capability baseline, and obtaining a scheduling constraint space containing a task execution dependency relationship and an equipment operation state boundary condition; performing structured feature extraction on the scheduling constraint space through a graph neural network, generating a production scheduling association graph containing node features and edge features, and calling a multi-target reinforcement learning strategy model to perform strategy iteration optimization on the production scheduling association graph, obtaining a scheduling strategy parameter set fusing the task completion timeliness and the resource utilization balance degree; and generating an equipment task time sequence allocation scheme. According to the invention, the overall effectiveness and reliability of production scheduling can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing, and in particular to a multi-objective production scheduling method and system based on reinforcement learning. Background Art

[0002] With the development of intelligent manufacturing technology, production efficiency can be improved and production costs can be reduced by rationally arranging the execution sequence of production tasks and resource allocation plans. At present, scheduling plans are usually generated by establishing a matching relationship between tasks and equipment based on mathematical programming models or heuristic algorithms. Traditional production scheduling methods have obvious shortcomings in dealing with complex production constraints and multi-objective optimization problems. It is difficult to effectively integrate the task execution dependency and the equipment operation status boundary conditions, resulting in poor adaptability and robustness of the scheduling plan. At the same time, the existing methods do not fully extract the structured features in the production system and cannot accurately capture the complex relationship between task priority and equipment resource sharing. In addition, when faced with the multi-objective optimization of task completion time and resource utilization balance, traditional methods find it difficult to achieve a dynamic balance of multiple goals in a dynamically changing production environment, affecting the overall effect and reliability of production scheduling. Summary of the Invention

[0003] The present invention provides a multi-objective production scheduling method and system based on reinforcement learning.

[0004] In the first aspect, an embodiment of the present invention provides a multi-objective production scheduling method based on reinforcement learning, the method comprising: performing a matching analysis based on a task requirement list and an equipment capability baseline to obtain a scheduling constraint space including task execution dependencies and equipment operating status boundary conditions; performing structured feature extraction on the scheduling constraint space through a graph neural network to generate a production scheduling association graph including node features and edge features, the node features corresponding to task attributes and equipment attributes, and the edge features corresponding to priority relationships between tasks and resource sharing relationships between equipment; calling a multi-objective reinforcement learning strategy model to perform strategy iterative optimization on the production scheduling association graph to obtain a set of scheduling strategy parameters that integrates task completion timeliness and resource utilization balance; generating an equipment task timing allocation plan based on the scheduling strategy parameter set, the equipment task timing allocation plan being used to indicate the task execution sequence and resource allocation rules of each production equipment within a continuous production period.

[0005] In a second aspect, an embodiment of the present invention provides a computer system, comprising: a memory storing a computer program; and a processor for loading the computer program to implement the multi-objective production scheduling method based on reinforcement learning as described above.

[0006] The multi-objective production scheduling method based on reinforcement learning provided by the present invention constructs a scheduling constraint space based on matching analysis of task requirement lists and equipment capability baselines. It can integrate task execution dependencies and equipment operating status boundary conditions into a unified multi-dimensional constraint model, avoiding the lack of synergy caused by the separate treatment of task and equipment constraints in traditional scheduling methods, and improving the comprehensiveness and accuracy of scheduling constraint modeling. Through graph neural network, structured feature extraction is performed on the scheduling constraint space to generate a production scheduling association graph. Task attributes, equipment attributes, and the relationship between them can be converted into explicit node features and edge features, breaking through the limitation of traditional unstructured feature representation that is difficult to capture implicit associations. It enables the production scheduling system to more accurately perceive the complex relationship between task priority and equipment resource sharing. The multi-objective reinforcement learning strategy model is used to iteratively optimize the production scheduling association graph, which can dynamically balance task completion timeliness and resource utilization balance, so that the generated scheduling strategy parameter set can better adapt to real-time production status changes. Finally, the equipment task timing allocation plan generated based on the scheduling strategy parameter set can more accurately indicate the task execution order and resource allocation rules of each production equipment within the continuous production period, effectively improving the overall effectiveness and reliability of production scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a flowchart of a multi-objective production scheduling method based on reinforcement learning provided by an embodiment of the present invention.

[0008] Figure 2 It is a schematic diagram of the composition of a computer system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] See also Figure 1 , Figure 1 A flowchart of a multi-objective production scheduling method based on reinforcement learning provided in an embodiment of the present invention. The method can be executed by a computer system and may include the following steps: Step S100: Perform matching analysis based on the task requirement list and the equipment capability baseline to obtain a scheduling constraint space including task execution dependencies and equipment operating status boundary conditions.

[0010] In a production scheduling scenario, a task requirement list is a document that records the various requirements of production tasks. It contains specific attribute information for each task, such as the task duration requirement, resource consumption type, and quantity requirement. This information is the basic basis for arranging production tasks. The equipment capability baseline is a collection of operating parameters and capabilities that describe the production equipment. It covers data such as the equipment's processing power, resource supply type, and continuous operating time, reflecting the equipment's actual working capabilities during the production process. Matching analysis is the process of matching and adapting the tasks in the task requirement list with the equipment in the equipment capability baseline. The purpose is to identify the equipment that can be used for each task and the range of tasks that the equipment can carry. The scheduling constraint space is a multi-dimensional constraint system constructed based on task execution dependencies and equipment operating status boundary conditions. It is used to limit the range of feasible solutions for production scheduling and ensure that the scheduling results meet the actual requirements of production.

[0011] As an implementation manner, step S100 may specifically include the following steps S110 to S160: Step S110: parse the task attribute information in the task requirement list, extract the duration requirements, resource consumption types and quantity requirements of each task through semantic analysis, generate a task attribute feature vector, and map the task attribute feature vector into a fixed-dimensional numerical vector through the embedding layer.

[0012] The task attribute information in the task requirements list is a collection of information describing the specific characteristics of each task, documenting the various requirements of the task in text or other forms. Semantic parsing is a common natural language processing technology used to understand and analyze semantic information in text. Through semantic parsing, key task attribute information can be extracted from the task requirements list, such as duration requirements, resource consumption types, and quantity requirements. Duration requirements refer to the time required to complete a task, which can be determined based on factors such as task complexity and process flow. Resource consumption types and quantity requirements specify the types and quantities of resources required during task execution.

[0013] The task attribute feature vector is a vector that integrates and represents the extracted task attribute information, converting various attribute information of the task into the form of a mathematical vector. The embedding layer can map discrete text information or other non-numerical information into a numerical vector of fixed dimension.

[0014] Step S120: Collect the device operating parameters in the device capability baseline, obtain the device processing capability, resource supply type and continuous operation time data through the parameter extraction algorithm, and construct a device capability feature matrix. The rows and columns of the device capability feature matrix correspond to the device and resource types respectively.

[0015] The device operating parameters in the device capability baseline are indicators that describe the various states and performance of the device during operation. These parameters can be obtained through the device's sensors, monitoring systems, or other data collection methods. In this step, parameter extraction algorithms (such as the chi-square test and information gain) are used to accurately obtain the device's processing capacity, resource supply type, and continuous operation time data. Processing capacity refers to the amount of tasks or workload that the device can complete per unit time, reflecting the device's operating efficiency. Resource supply type and continuous operation time data clearly define the types of resources the device can provide and the length of time the device can operate continuously.

[0016] The device capability matrix is ​​a two-dimensional matrix with rows corresponding to different devices and columns corresponding to different resource types. Each element in the matrix represents the device's supply capability or related parameters for that resource type.

[0017] Step S130: Perform compatibility matching analysis on the task attribute feature vector and the device capability feature matrix, calculate the matching degree between the task and the device through feature space mapping, and determine the set of candidate devices that each task can adapt to and the range of task types that the device can carry.

[0018] Compatibility matching analysis is the process of comparing and matching task attribute feature vectors with device capability feature matrices to identify compatibility relationships between tasks and devices. Task attribute feature vectors contain various task attribute information, while device capability feature matrices contain various device operating parameters and capability information. Feature space mapping maps the task attribute feature vectors and device capability feature matrix from their original data space to a new feature space. This new feature space facilitates the calculation of task-device compatibility. Feature space mapping can employ linear or nonlinear transformations, such as principal component analysis (PCA) and kernel principal component analysis (KPCA). Compatibility is a measure of the degree of compatibility between tasks and devices, reflecting the likelihood that the task can be successfully completed on the device. The candidate device set is a set of devices that are selected for each task based on the compatibility. The range of tasks a device can handle refers to the range of types and sizes of tasks that the device can handle, determined by factors such as the device's processing capabilities and resource availability. Similarity calculation methods, such as cosine similarity and Euclidean distance, can be used to calculate task-device compatibility. For example, the cosine similarity between a task attribute feature vector and a row vector in a device capability feature matrix is ​​calculated to measure the matching degree. The closer the cosine similarity value is to 1, the higher the matching degree between the task and the device. When determining the set of candidate devices that can be adapted for each task, devices with a matching degree above a certain threshold are selected as candidates based on the calculated matching degree. When determining the range of task types that a device can handle, factors such as the device's processing power and resource availability can be used to determine the range of tasks that the device can handle.

[0019] Step S140: Analyze the process dependency between different tasks and generate a directed graph of task execution dependency. The nodes of the directed graph represent tasks, and the directed edges represent dependency directions.

[0020] Process dependencies refer to the sequential order and logical connections between tasks due to production process requirements. For example, in a production process, if Task A cannot begin until Task B is completed, this is a process dependency. A directed task execution dependency graph is a graphical representation that visually illustrates the dependencies between tasks. The nodes in the graph represent tasks, with each node corresponding to a specific task. Directed edges represent the direction of the dependency. Directed edges from one node to another indicate that the previous task is a predecessor of the next task, meaning that the next task cannot begin until the previous task is completed.

[0021] As an implementation manner, step S14 may specifically include the following steps S141 to S146: Step S141: Extract the process route description information of each task in the task requirement list, identify the direct and indirect dependencies between tasks through natural language understanding, and generate preliminary dependency identification results, where the direct dependency is reflected as the logical sequence of process connection, and the indirect dependency is reflected as the association of shared process conditions.

[0022] The process route description information is the text information in the task requirement list that describes the task production process flow, which includes the various processes of the task and the sequence and logical relationship between the processes. A direct dependency relationship refers to the existence of a clear logical sequence of process connections between tasks. For example, Task B can only be started after Task A is completed. This relationship is direct and clear. An indirect dependency relationship refers to the association between tasks through sharing certain process conditions. For example, if Task A and Task B both need to be carried out under specific temperature conditions, then there is an indirect dependency relationship between Task A and Task B. The preliminary dependency identification result is the result obtained after preliminary analysis and judgment of the dependency relationship between tasks through natural language understanding, which provides basic information for the subsequent generation of a directed graph of task execution dependencies.

[0023] Step S142: Based on the direct dependency relationship in the preliminary dependency identification result, a directed connection edge is established between the corresponding task nodes, and the edge eigenvalue is calculated through historical process data. The edge eigenvalue reflects the conversion time consumption characteristics of the two tasks, and an initial directed graph structure including the direct connection edge and the edge eigenvalue is generated.

[0024] In a directed graph of task execution dependencies, directed edges represent direct dependencies between tasks, pointing from one task node to another and clarifying the order in which tasks are executed. Historical process data is a database that records various process information during the production process, including data such as task transition times and resource consumption. The edge eigenvalue is a numerical value on a directed edge that reflects the transition time between two tasks, such as the time required to transition from Task A to Task B.

[0025] In an embodiment of the present invention, directed connection edges are established between corresponding task nodes in a directed graph of task execution dependencies based on direct dependencies in the preliminary identification results of dependencies. For example, if the preliminary identification results indicate that task A is a predecessor task of task B, then a directed edge is drawn between the node corresponding to task A and the node corresponding to task B. When calculating edge eigenvalues, the conversion time data between tasks is extracted from historical process data. For example, by querying historical process data, the average time interval between the completion of task A and the start of task B is counted, and this average time interval is used as the edge eigenvalue. When generating the initial directed graph structure, the directed connection edges and edge eigenvalues ​​are added to the graph to form an initial directed graph containing direct connection edges and edge eigenvalues.

[0026] Step S143: Based on the indirect dependency relationships in the preliminary dependency identification results, combined with the edge eigenvalues ​​of the initial directed graph structure, the dependency transfer path between non-adjacent process tasks is calculated through the transitive closure algorithm. The path is described by the intermediate task node sequence and the accumulated edge eigenvalues, and the indirect connection path is added to the initial directed graph structure.

[0027] Indirect dependencies refer to dependencies between tasks that exist through sharing certain process conditions or intermediate tasks. A dependency transfer path refers to a path from one task node to another non-adjacent task node, which is described by the sequence of intermediate task nodes and the cumulative edge eigenvalue. The sequence of intermediate task nodes refers to the order of the intermediate task nodes passed through in the path, and the cumulative edge eigenvalue refers to the sum of all edge eigenvalues ​​in the path, reflecting the total conversion time from one task to another non-adjacent task. When using the transitive closure algorithm, the Floyd-Warshall algorithm can be adopted. The edge eigenvalues ​​in the initial directed graph structure are used as elements of the adjacency matrix of the graph. For nodes that are not directly connected, the elements in the adjacency matrix are set to infinity. Then, the adjacency matrix is ​​iteratively updated using the Floyd-Warshall algorithm to calculate the shortest path and cumulative edge eigenvalue between any two nodes.

[0028] Step S144: For the task execution dependency directed graph containing direct connection edges and indirect connection paths, prioritize each task node according to the importance of the task in the production process. The importance is calculated through the process impact weight, which is determined based on the constraint strength of the task on subsequent processes. Mark the critical task nodes and non-critical task nodes in the task execution dependency directed graph.

[0029] The importance of a task in the production process refers to the degree of influence of the task on the entire production process, reflecting the criticality of the task in the production process. The process impact weight is an indicator that measures the strength of the constraints of a task on subsequent processes, and is determined based on the degree of influence of the task on the resource requirements, time requirements, etc. of the subsequent processes. Priority sorting is the process of arranging task nodes from high to low according to their importance, with the aim of giving priority to tasks with high importance in production scheduling. In an embodiment of the present invention, the analytic hierarchy process (AHP) can be used to calculate the process impact weight. First, determine the constraints of the task on the subsequent processes, such as resource requirements, time requirements, etc. Then construct a judgment matrix to compare the degree of influence of different tasks on each constraint factor, and obtain the process impact weight of each task by calculating the eigenvector of the judgment matrix.

[0030] Step S145: Perform loop detection on the task execution dependency directed graph after marking the key nodes, and use the graph traversal algorithm to traverse all paths in the directed graph. If there is a circular dependency, start the dependency adjustment mechanism and optimize the dependency edges that cause the cycle based on the process constraint rules to make the directed graph loop-free.

[0031] Loop detection is the process of examining a directed graph of task execution dependencies to determine whether there are any circular dependencies, such as by using a depth-first search algorithm. A circular dependency is a closed loop between tasks. A graph traversal algorithm is used to traverse all nodes and edges in a graph. This algorithm can traverse all paths in a directed graph to identify possible circular dependencies.

[0032] The dependency adjustment mechanism adjusts and optimizes the dependencies between tasks after discovering a circular dependency. Process constraint rules are established based on the actual requirements of the production process and specify the dependencies and execution order between tasks. Optimizing circular dependency edges involves modifying or deleting edges that form circular dependencies based on the process constraint rules, eliminating any loops in the directed graph.

[0033] Step S146: Based on the optimized acyclic directed graph, add a time window attribute to each task node. The time window attribute is calculated through the completion time of all predecessor tasks and the start time of all successor tasks to generate a complete task execution dependency directed graph data structure including node priority, edge eigenvalues ​​and time window attributes.

[0034] The time window attribute refers to the time range within which a task node is available for production scheduling. It is determined by the completion time of all predecessor tasks and the start time of all successor tasks. A predecessor task is a task that must be completed before a task, and a successor task is a task that must start after a task.

[0035] In this embodiment of the present invention, when calculating the time window attribute, the predecessor and successor tasks of each task must first be determined. For each task node, the maximum completion time of all its predecessor tasks is found, which is used as the earliest start time of the task; the minimum start time of all its successor tasks is found, which is used as the latest finish time of the task. The time window attribute is the time period between the earliest start time and the latest finish time of the task.

[0036] When generating the complete directed graph data structure for task execution dependencies, node priorities, edge eigenvalues, and time window attributes are added to the optimized acyclic directed graph. Node priorities reflect the importance of tasks in the production process, edge eigenvalues ​​reflect the time-consuming nature of task transitions, and time window attributes constrain the execution time range of tasks.

[0037] Step S150: combining the equipment continuous operation time data with the task duration requirement, calculating the task processing capacity of the equipment within the unit production cycle through sequential logic, and determining the equipment operation status boundary conditions.

[0038] Equipment continuous operation time data is the data in the equipment capability baseline that records the length of time the equipment can operate continuously. Task duration requirements are the information in the task requirements list that records the time required to complete each task. Temporal logic is a method of calculation and reasoning based on time sequence and logical relationships. In this step, temporal logic can be used to calculate the task processing capacity of the equipment within a unit production cycle. Task processing capacity refers to the number and scale of tasks that the equipment can handle within a unit production cycle. Equipment operating status boundary conditions refer to the working range and limitations of the equipment under different times and conditions, and are determined by factors such as the equipment's processing capacity, resource supply type, and continuous operation time.

[0039] Step S160: integrating the task execution dependency directed graph and the device operation status boundary conditions, and constructing a scheduling constraint space through a constraint integration algorithm. The scheduling constraint space stores the associated constraint information of tasks and devices based on a multi-dimensional tensor structure.

[0040] The constraint integration algorithm is used to integrate and process different types of constraints. In this step, the constraint integration algorithm integrates task execution dependencies and device operating state boundary conditions into the scheduling constraint space. A multidimensional tensor structure is a high-dimensional data structure that can store and represent complex multidimensional data. In this step, the scheduling constraint space stores the associated constraints between tasks and devices based on the multidimensional tensor structure. The different dimensions of the multidimensional tensor can represent different constraint factors such as tasks, devices, time, and resources.

[0041] In an embodiment of the present invention, when performing constraint integration, the task execution order and time constraint information in the task execution dependency directed graph must first be converted into the form of a tensor. For each task node in the task execution dependency directed graph, its time window attributes and priority information can be represented as a dimension of the tensor. Then, the information such as the device task processing capacity and resource supply restrictions in the device operation status boundary conditions are also represented as the corresponding dimensions of the tensor. Through the constraint integration algorithm, the tensor information of the task execution dependency and the device operation status boundary conditions is merged and processed, and finally, a scheduling constraint space based on a multi-dimensional tensor structure is formed.

[0042] Step S200: Structured features of the scheduling constraint space are extracted through a graph neural network to generate a production scheduling association graph containing node features and edge features. Node features correspond to task attributes and equipment attributes, and edge features correspond to priority relationships between tasks and resource sharing relationships between equipment.

[0043] In embodiments of the present invention, when using graph neural networks for structured feature extraction, models such as graph convolutional networks (GCNs) and graph attention networks (GATs) can be employed. First, the attribute information of task nodes and device nodes in the scheduling constraint space is used as input to the graph neural network. The graph neural network then updates and aggregates node and edge features through a message passing mechanism. During message passing, a node receives information from its neighboring nodes and performs a weighted sum based on edge features to update its own feature representation. Ultimately, through multiple iterations and updates of the graph neural network, a production scheduling association graph containing node and edge features is generated.

[0044] As an implementation manner, step S200 may specifically include the following steps S210 to S260: Step S210: Encode the task nodes and device nodes in the scheduling constraint space to generate an initial node feature vector. The task node feature vector is generated by integrating the task type, resource requirements and time window data. The device node feature vector is generated by integrating the device type, resource supply and operating status data. The dimensions of the task node feature vector and the device node feature vector are consistent.

[0045] Task type is the classification information of the task, such as production tasks, inspection tasks, etc. Resource requirements are the type and quantity of resources required for the execution of the task. Time window data is information such as the earliest start time and the latest completion time of the task. The task node feature vector is a vector generated by combining the task type, resource requirements and time window data, reflecting the basic attributes and constraints of the task. Equipment type is the classification information of the equipment, such as processing equipment, transportation equipment, etc. Resource supply is the type and quantity of resources that the equipment can provide. Operation status data is the current operation status and performance indicators of the equipment, such as the equipment's processing capacity, resource supply type and continuous operation time. The equipment node feature vector is a vector generated by combining the equipment type, resource supply and operation status data, reflecting the basic attributes and capabilities of the equipment.

[0046] In embodiments of the present invention, encoding methods such as one-hot encoding and embedded encoding can be used. When generating a task node feature vector, the encoding vectors for the task type, resource requirements, and time window data are concatenated in a specific order to form a comprehensive task node feature vector. When generating a device node feature vector, the encoding vectors for the device type, resource supply, and operating status data are concatenated in a specific order to form a comprehensive device node feature vector.

[0047] Step S220: Based on the edge eigenvalues ​​of the task execution dependency directed graph, the task dependency edge eigenvalues ​​of the connection relationships between task nodes are comprehensively calculated by means of dependency strength, process switching cost, and parallel execution possibility.

[0048] The edge eigenvalues ​​of a directed graph of task execution dependencies reflect the time characteristics of transitions between tasks. Dependency strength is a metric that measures the closeness of dependencies between tasks, reflecting the degree to which one task depends on another. Process switching cost refers to the expense incurred when switching from one task to another, including resource adjustment time and equipment preparation time. Parallel execution possibility refers to the possibility of executing two tasks simultaneously, which is influenced by factors such as process compatibility.

[0049] As an implementation manner, step S220 may specifically include the following steps S221 to S224: Step S221: extract edge eigenvalue basic data from the task execution dependency directed graph, the edge eigenvalue basic data includes process association type information of the task node pair, determine the closeness level of the process association type through process logic analysis, and generate a dependency strength quantification value based on the closeness level.

[0050] Edge eigenvalue basic data is the basic information contained in the eigenvalues ​​of edges in a directed graph of task execution dependencies. It includes information on the process relationship type between task nodes, such as whether the tasks have a direct process connection or share process conditions. The closeness level categorizes the closeness of the process relationship between tasks, for example, strong, medium, or weak. The dependency strength quantification value is a numerical value generated based on the closeness level, representing the dependency strength in a quantitative form. In embodiments of the present invention, process logic analysis can utilize methods such as rule matching and machine learning classification. For example, the process relationship type information "Task A is a predecessor process of Task B, and the output of Task A directly serves as the input of Task B" can be determined to be a strong relationship; "Task A and Task B both need to be performed in the same process environment, but there is no direct process connection" can be determined to be a medium relationship; and "Task A and Task B only have occasional indirect connections in the production process" can be determined to be a weak relationship.

[0051] When generating dependency strength quantification values, each level of closeness can be assigned a numerical value.

[0052] Step S222: Based on the historical process conversion records in the edge eigenvalue basic data, extract the resource adjustment time and equipment preparation operation time of the task node in the conversion process, perform time series statistical analysis on the time consumption data, and generate a quantitative value of the process switching cost that reflects the conversion efficiency.

[0053] The historical process conversion record is a document that records the process conversion process between pairs of task nodes in the basic data of the edge eigenvalues ​​in the directed graph of task execution dependencies, and contains detailed information on each process conversion, such as resource adjustment time and equipment preparation operation time. Resource adjustment time refers to the time required to adjust resource supply and allocation when switching from one task to another. Equipment preparation operation time refers to the time required to prepare and adjust the equipment when switching from one task to another. In the embodiment of the present invention, when performing time series statistical analysis, statistical indicators such as mean, variance, and median can be used.

[0054] Step S223: Based on the parallel execution case records in the edge eigenvalue basic data, the frequency of scenarios in which task nodes can be started simultaneously in historical production is counted, and combined with the process compatibility assessment results, a parallel execution possibility quantitative value reflecting the feasibility of synchronous execution is generated.

[0055] Parallel execution case records are documents documenting instances in which task node pairs were simultaneously started in historical production, based on the edge feature value data within the directed task execution dependency graph. Scenario frequency refers to the number of times a task node pair was simultaneously started in historical production. Process compatibility assessment results assess whether a task node pair can be executed simultaneously under the same process conditions, which can be determined through process analysis and experimental verification.

[0056] The quantified value of the parallel execution possibility is a numerical value that reflects the feasibility of synchronous execution of task nodes. It is generated by counting the frequency of scenarios and combining the results of process compatibility assessment.

[0057] Step S224: Input the quantified value of dependency strength, the quantified value of process switching cost and the quantified value of parallel execution possibility into the feature fusion module, dynamically allocate fusion weights based on the degree of influence of each quantified value on the production scheduling target, and perform weighted sum operation to generate the task dependency edge feature value of the connection relationship between task nodes.

[0058] The feature fusion module is a neural network module used to fuse and integrate different types of features. The quantified values ​​of dependency strength, process switching cost, and parallel execution possibility describe the characteristics of the connection relationship between task nodes from different perspectives. Production scheduling objectives refer to the goals to be achieved during the production scheduling process, such as task completion timeliness and balanced resource utilization.

[0059] Dynamically assigning fusion weights is the process of adjusting the weights of each quantized value in real time based on its impact on production scheduling objectives. A weighted summation operation multiplies each quantized value by its corresponding fusion weight and then adds the results. This weighted summation allows different types of quantized values ​​to be fused into a comprehensive task-dependent edge feature value.

[0060] In the embodiment of the present invention, the feature fusion module may adopt a fully connected layer structure. The input of the fully connected layer is the quantized value of the dependency strength, the quantized value of the process switching cost, and the quantized value of the parallel execution possibility, and the output is the feature value of the task dependency edge.

[0061] An adaptive weight allocation algorithm can be used to dynamically assign fusion weights. First, based on historical production data and experience, the initial impact of each quantified value on the production scheduling objective is determined. For example, dependency strength has a greater impact on task completion time, process switching costs have a greater impact on resource utilization balance, and parallel execution potential has a greater impact on improving production efficiency. Then, during training, the fusion weights of each quantified value are adjusted in real time based on model feedback and actual scheduling results. Finally, the quantified values ​​of dependency strength, process switching costs, and parallel execution potential are multiplied by their corresponding fusion weights, and the results are added together to obtain the task dependency edge feature values ​​that represent the connections between task nodes.

[0062] Step S230: Based on the device resource sharing capability data, the device sharing edge eigenvalue of the connection relationship between the device nodes is comprehensively calculated through resource transmission efficiency, cooperative working coefficient and conflict avoidance parameter.

[0063] Device resource sharing capability data is a dataset that records resource sharing between devices. It includes information such as inter-device resource transfer records, collaborative production cases, and historical resource conflicts. Resource transfer efficiency refers to the efficiency of resource transfer between devices, reflecting the speed and accuracy of resource transfer between devices. The interoperability coefficient is an indicator of the interoperability between devices, reflecting the stability and efficiency of collaboration when devices work together to complete tasks. The conflict avoidance parameter describes the device's ability and strategy to handle resource competition and conflicts.

[0064] The device shared edge eigenvalue is a feature vector calculated by comprehensively calculating the resource transmission efficiency, cooperative working coefficient and conflict avoidance parameter, which describes the characteristics of the connection relationship between device nodes.

[0065] As an implementation manner, step S230 may specifically include the following steps S231 to S234: Step S231: Extract inter-device resource transmission records from the device resource sharing capability data. The resource transmission records include the transmission resource type, single transmission time, and number of successful transmissions. The ratio of the actual transmission resource amount to the theoretical transmission capacity per unit time is calculated to generate a quantitative value of resource transmission efficiency.

[0066] Inter-device resource transfer records are documents within the device resource sharing capability data that record the resource transfer process between devices. They include information such as the resource type, single transfer duration, and number of successful transfers. The resource type refers to the type of resource transferred between devices, such as raw materials or energy. The single transfer duration refers to the time spent on each resource transfer. The number of successful transfers refers to the number of successful resource transfers between devices.

[0067] The resource transmission efficiency quantification value reflects the efficiency of resource transmission between devices and is generated by calculating the ratio of the actual transmission resource volume to the theoretical transmission capacity per unit time.

[0068] In an embodiment of the present invention, when calculating the actual amount of resources transmitted per unit time, the length of the unit time is first determined, for example, one hour. Then, the transmission records within the unit time are filtered out from the inter-device resource transmission records, and the total amount of actual resources transmitted is calculated. Theoretical transmission capacity refers to the amount of resources that a device can transmit per unit time under ideal conditions and can be determined based on the device's transmission capabilities and design parameters. For example, if the theoretical transmission capacity of device A is 10 units of raw material X per hour, then the theoretical transmission capacity of device A is 10 units. Finally, the actual amount of resources transmitted per unit time is divided by the theoretical transmission capacity to obtain a quantitative value of resource transmission efficiency.

[0069] Step S232: Based on the collaborative production cases in the equipment resource sharing capability data, the completion quality scores and task cycle compliance rates of the tasks jointly participated by the two devices are extracted, and the quality scores and cycle compliance rates are integrated according to the process requirement weights to generate a quantitative value of the collaborative work coefficient that reflects the stability of the collaboration.

[0070] Collaborative production cases are documents within the equipment resource sharing capability data that record instances of equipment participating in joint tasks. The completion quality score evaluates the quality of the equipment's collaborative task completion and can be determined through product inspections, customer feedback, and other methods. The task cycle compliance rate refers to the percentage of tasks completed by equipment within the specified task cycle, reflecting the equipment's efficiency and ability to complete tasks on time.

[0071] The process requirement weight is the weight assigned to the completion quality score and task cycle compliance rate based on the production process requirements, reflecting the production process's emphasis on quality and efficiency. The collaborative working coefficient quantitative value reflects the stability of collaboration between equipment. It is generated by combining the completion quality score and task cycle compliance rate according to the process requirement weight.

[0072] Step S233: Based on the resource conflict history records in the device resource sharing capability data, the production interruption duration caused by resource competition and the effectiveness score of the conflict resolution measures are counted, and combined with the interruption frequency analysis results, a conflict avoidance parameter quantitative value reflecting the conflict handling capability is generated.

[0073] Resource conflict history records resource contention and conflicts between devices within the device's resource sharing capability data. This includes information about the duration of production interruptions caused by resource contention and conflict resolution measures. Production interruption duration refers to the length of time production was halted due to resource contention. The conflict resolution effectiveness score evaluates the effectiveness of the measures taken to resolve resource conflicts. This score can be determined based on factors such as production recovery and loss assessment.

[0074] The interruption frequency analysis results analyze the frequency of production interruptions caused by resource conflicts, reflecting the frequency of resource conflicts. The conflict avoidance parameter quantification value reflects the device's conflict handling capabilities. It is generated by combining the duration of production interruptions, the effectiveness score of conflict resolution measures, and the interruption frequency analysis results.

[0075] In the embodiment of the present invention, when counting the duration of production interruptions, each production interruption event caused by a resource conflict is screened from the resource conflict history records, the start time and end time of the interruption are recorded, and the total duration of the interruption is calculated.

[0076] Step S234: Input the quantified value of resource transmission efficiency, the quantified value of collaborative working coefficient and the quantified value of conflict avoidance parameter into the feature fusion module, dynamically allocate fusion weights based on the degree of influence of each quantified value on the smoothness of resource allocation, and perform weighted sum operation to generate the device shared edge feature value of the connection relationship between device nodes.

[0077] The quantified values ​​of resource transfer efficiency, collaborative working coefficient, and conflict avoidance parameter describe the characteristics of the connection relationship between device nodes from different perspectives. Resource allocation fluency refers to the smoothness of resource allocation and transfer between devices and is an important goal in production scheduling.

[0078] Dynamically assigning fusion weights involves adjusting the weights of each quantized value in real time based on its impact on resource allocation smoothness. A weighted summation operation multiplies each quantized value by its corresponding fusion weight and then adds the results. This weighted summation allows different quantized values ​​to be combined into a comprehensive device-shared edge feature value.

[0079] In an embodiment of the present invention, the feature fusion module can adopt a fully connected layer structure, the input is the quantized value of resource transmission efficiency, the quantized value of collaborative working coefficient and the quantized value of conflict avoidance parameter, and the output is the device shared edge feature value. When allocating fusion weights, an adaptive weight allocation algorithm can be used to determine the initial impact of each quantized value on the smoothness of resource allocation based on historical production data and experience. For example, resource transmission efficiency has a greater impact on the speed of resource allocation, the collaborative working coefficient has a greater impact on the collaborative stability between devices, and the conflict avoidance parameter has a greater impact on avoiding resource conflicts and ensuring production continuity. Then, during the training process, the fusion weights of each quantized value are adjusted in real time based on the feedback from the model and the actual scheduling results.

[0080] Step S240: Initialize the message passing parameters of the graph neural network, and input the task node feature vector, device node feature vector, task dependency edge feature value and device sharing edge feature value into the message passing layer of the graph neural network.

[0081] The message passing parameters of graph neural networks are the parameters used for the message passing process in graph neural networks, including node feature conversion matrices, edge feature weights, etc. The task node feature vector is a feature vector generated by comprehensively considering the task type, resource requirements, and time window data, and describes the attribute information of the task. The device node feature vector is a feature vector generated by comprehensively considering the device type, resource supply, and operating status data, and describes the attribute information of the device. The task dependency edge eigenvalue is a feature vector calculated by comprehensively considering the dependency strength, process switching cost, and parallel execution possibility, and describes the characteristics of the connection relationship between task nodes. The device sharing edge eigenvalue is a feature vector calculated by comprehensively considering the resource transmission efficiency, collaborative work coefficient, and conflict avoidance parameter, and describes the characteristics of the connection relationship between device nodes.

[0082] In an embodiment of the present invention, a random initialization method can be used to initialize the message passing parameters of the graph neural network. For example, the node feature conversion matrix can be initialized as a random matrix, and the edge feature weight can be initialized as a random vector.

[0083] When task node feature vectors, device node feature vectors, task dependency edge feature values, and device shared edge feature values ​​are input into the message passing layer of the graph neural network, the message passing layer processes this input information based on the message passing parameters. During message passing, a node receives information from its neighboring nodes, adjusts the neighboring node weights based on the edge feature values, and then updates its own feature representation.

[0084] Step S250: In the message passing layer, the feature representation of each node is updated by aggregating the neighbor node information of each node. The neighbor information aggregation of the task node uses the task dependency edge eigenvalue to adjust the neighbor node weight. The neighbor information aggregation of the device node uses the device sharing edge eigenvalue to adjust the neighbor node weight. The message passing is iterated to fuse the neighborhood information.

[0085] The message passing layer in a graph neural network is used to transfer information between nodes and update node feature representations. Aggregation is the process of collecting and integrating information about a node's neighbor nodes. Neighbor node information refers to the feature information and edge feature information of nodes adjacent to a node.

[0086] Neighbor information aggregation for task nodes refers to the process by which task nodes collect and integrate information about their neighboring task nodes. During this process, the eigenvalues ​​of task dependency edges are used to adjust the weights of neighbor nodes, so that neighbors with close dependencies on the task node have a greater influence on its feature updates. Neighbor information aggregation for device nodes refers to the process by which device nodes collect and integrate information about their neighboring device nodes. During this process, the eigenvalues ​​of device sharing edges are used to adjust the weights of neighbor nodes, so that neighbors with close resource sharing relationships with the device node have a greater influence on its feature updates.

[0087] Message passing iteration refers to the process of multiple message passing. Through multiple iterations, neighborhood information can be continuously integrated to make the feature representation of the node more comprehensive and accurate.

[0088] As an implementation manner, step S250 may specifically include the following steps S251 to S254: Step S251: In each iteration of message passing, the neighbor node set of each node is sampled. The task node selects the neighbor node with the highest contribution based on the task dependency edge eigenvalue, and the device node selects the neighbor node with the highest contribution based on the device sharing edge eigenvalue.

[0089] In each round of message passing, in order to reduce the amount of computation and improve efficiency, the set of neighbor nodes of each node can be sampled. The contribution refers to the degree of influence of the neighbor node on the feature update of the current node.

[0090] Task nodes select the most contributing neighbor node based on the task dependency edge eigenvalue. This is because the task dependency edge eigenvalue reflects the closeness of the dependency relationship between tasks. The closer the dependency, the greater the contribution of the neighbor node to the feature update of the current task node. Device nodes select the most contributing neighbor node based on the device sharing edge eigenvalue. This is because the device sharing edge eigenvalue reflects the closeness of the resource sharing relationship between devices. The closer the resource sharing relationship, the greater the contribution of the neighbor node to the feature update of the current device node.

[0091] In an embodiment of the present invention, a probability-based sampling method can be used for sampling. For a task node, the sum of the task dependency edge eigenvalues ​​of each neighboring task node is first calculated. Then, the probability of each neighboring task node being selected is determined based on the ratio of the task dependency edge eigenvalues. Then, random sampling is performed based on the probability to select the neighboring node with the highest contribution.

[0092] For device nodes, the same method is used to calculate the sum of the device shared edge eigenvalues ​​of each neighboring device node. The probability of each neighboring device node being selected is determined based on the proportion of the device shared edge eigenvalues. Then, random sampling is performed to select the neighboring node with the highest contribution.

[0093] Step S252: Perform weighted summation on the selected neighbor node features to obtain a weighted summation result, wherein the weight value of the task node neighbor is calculated by the task dependency edge eigenvalue, and the weight value of the device node neighbor is calculated by the device sharing edge eigenvalue. The edge eigenvalue is positively correlated with the corresponding weight value of the neighbor node.

[0094] After each node determines its selected neighbor nodes, it performs a weighted sum of their features. The weights are calculated based on the edge eigenvalues, which reflect the closeness of the relationship between nodes. A larger edge eigenvalue indicates a closer relationship and a greater influence of the neighbor node on the current node, thus increasing the weight. For task nodes, the weights of their neighbor nodes are calculated using the task-dependent edge eigenvalues. For device nodes, the weights of their neighbor nodes are calculated using the device-shared edge eigenvalues.

[0095] Step S253: concatenate the weighted summation result with the feature vector of the current node, map it to the intermediate feature space through a linear transformation layer, perform nonlinear activation processing on the feature vector of the intermediate feature space, perform layer normalization on the activated feature vector to eliminate the dimensional difference between different feature dimensions, and the normalization operation is performed on the feature channel dimension.

[0096] The linear transformation layer is a neural network layer that maps the concatenated feature vectors to an intermediate feature space through matrix multiplication. The dimensions of this intermediate feature space can be adjusted as needed. Nonlinear activation processing performs nonlinear transformations on the feature vectors in the intermediate feature space. Nonlinear activation functions, such as ReLU and Sigmoid, can enhance the model's expressive power.

[0097] Step S254: Perform a residual connection on the normalized feature vector and the original feature vector of the current node to generate an updated node feature representation.

[0098] The role of the residual connection is to alleviate the gradient vanishing problem, so that the model can better learn the changes in features and improve the training effect and performance of the model.

[0099] In an embodiment of the present invention, the layer-normalized feature vector is added element-by-element to the original feature vector of the current node. Through residual connection, the information of the original feature vector is retained, and the information of the neighboring nodes and the processed feature information are added, so that the updated node feature representation is more comprehensive and accurate.

[0100] Step S260: Integrate the updated node features, task dependency edge feature values, and device sharing edge feature values, and generate a production scheduling association graph through a graph construction algorithm. The production scheduling association graph is stored in a heterogeneous graph structure to distinguish the type attributes of task nodes and device nodes and the corresponding edge feature types.

[0101] Fusion is the process of integrating and unifying updated node features, task dependency edge feature values, and device shared edge feature values. The goal is to fuse different types of feature information into a single graph. The graph construction algorithm is used to construct the production scheduling association graph, generating the graph's nodes and edges based on the fused feature information.

[0102] The production scheduling association graph is a heterogeneous graph consisting of task nodes, device nodes, and edges between them. This heterogeneous graph structure distinguishes the type attributes of task and device nodes and the corresponding edge feature types, enabling the graph to more accurately represent the various relationships and information in production scheduling. In an embodiment of the present invention, during fusion, the updated node features, task dependency edge feature values, and device shared edge feature values ​​are first collated and organized. The updated node features can be used as the node attributes of the graph, while the task dependency edge feature values ​​and device shared edge feature values ​​can be used as the edge attributes of the graph. Next, a graph construction algorithm (such as a graph generation algorithm) is used to generate the production scheduling association graph. Based on the relationships between task nodes and device nodes, an adjacency matrix is ​​constructed. The elements of the adjacency matrix represent the existence of edges between nodes and the edge feature values. Finally, the node and edge attributes are added to the graph to form the production scheduling association graph. The production scheduling association graph is stored using a heterogeneous graph structure, distinguishing the type attributes of task and device nodes and the corresponding edge feature types by different node and edge types.

[0103] Step S300: Calling a multi-objective reinforcement learning strategy model to perform strategy iteration optimization on the production scheduling association map to obtain a scheduling strategy parameter set that integrates task completion timeliness and resource utilization balance.

[0104] The multi-objective reinforcement learning policy model is a reinforcement learning model used to solve multi-objective optimization problems. It can find the optimal scheduling strategy through continuous policy iteration while considering multiple objectives. The production scheduling association graph is a heterogeneous graph that contains task nodes, equipment nodes, and the relationships between them. It stores various information related to production scheduling. Policy iteration optimization refers to the process of the model continuously trying different scheduling strategies, adjusting the strategy based on environmental feedback, and gradually finding the optimal strategy.

[0105] Task completion efficiency refers to the efficiency with which tasks are completed within the specified timeframe, reflecting the timeliness of production scheduling. Resource utilization balance refers to the degree of balance in the distribution of resources across different tasks and equipment, reflecting the effective use of resources. The scheduling policy parameter set is a set of parameters used to determine the production scheduling strategy, encompassing information such as the task-equipment allocation plan and task execution time.

[0106] As an implementation manner, step S300 may specifically include the following steps S310 to S380: Step S310: Constructing a network structure of a multi-objective reinforcement learning strategy model, wherein the network structure includes a graph feature extraction module, an action generation module, and a value evaluation module.

[0107] The network structure of the multi-objective reinforcement learning strategy model is composed of multiple modules, which work together to realize the processing of the production scheduling association graph and the generation of strategies. The graph feature extraction module is used to extract useful feature information from the production scheduling association graph, and can convert the information of nodes and edges in the graph into feature vectors. The action generation module generates a candidate scheduling action set based on the feature vectors extracted by the graph feature extraction module. The candidate scheduling action set contains multiple task equipment allocation schemes. The value evaluation module is used to evaluate the value of each candidate scheduling action and evaluate the quality of the action based on the execution results of the candidate scheduling action and the feedback from the environment. In an embodiment of the present invention, when constructing the graph feature extraction module, a model such as a graph convolutional network (GCN) or a graph attention network (GAT) can be used to automatically learn the feature representation of nodes and edges in the graph, and extract the global feature information of the graph through multi-layer convolution or attention mechanism.

[0108] The action generation module can adopt a multi-layer perceptron (MLP) structure, receiving the feature vectors output by the graph feature extraction module and generating a set of candidate scheduling actions through multiple fully connected layers and activation functions. The value assessment module can also adopt a multi-layer perceptron structure, receiving the feature vectors output by the graph feature extraction module and the candidate scheduling actions generated by the action generation module, and evaluating the value of the candidate scheduling actions through multiple fully connected layers and activation functions.

[0109] Step S320: Initialize the parameters of the network structure, including the node feature conversion matrix of the graph feature extraction module, the strategy weight vector of the action generation module, and the value function parameters of the value evaluation module. Pre-train the initial parameters using historical production scheduling case data to generate a pre-trained parameter set.

[0110] The parameters of the network structure are the parameters that the model needs to learn during training and determine its performance and behavior. The node feature conversion matrix of the graph feature extraction module is used to convert and extract node features from the graph. The policy weight vector of the action generation module is used to generate a set of candidate scheduling actions. The value function parameters of the value assessment module are used to evaluate the value of candidate scheduling actions.

[0111] Historical production scheduling case data contains past production scheduling information, such as task equipment allocation plans, task execution times, task completion timeliness, and resource utilization balance. Pre-training allows the model to learn some basic patterns and rules before formal training, improving training efficiency and performance.

[0112] When initializing the network structure's parameters, random initialization can be used. Then, historical production scheduling case data is used to pre-train the initial parameters. This pre-training process employs supervised learning, using the task and equipment allocation schemes from the historical production scheduling case data as labels and the feature vectors extracted by the graph feature extraction module as input. The model parameters are updated by minimizing the error between the predicted results and the labels. After multiple iterations of training, a pre-trained parameter set is generated.

[0113] Step S330: Input the production scheduling association graph into the graph feature extraction module, aggregate the neighborhood features of task nodes and device nodes through multi-layer graph convolution operations, update the node feature representation, and generate a graph feature vector containing global association information. When task nodes are aggregated, the neighbor weights are adjusted based on the task dependency edge feature values, and when device nodes are aggregated, the neighbor weights are adjusted based on the device sharing edge feature values.

[0114] Multi-layer graph convolution is the core operation in the graph feature extraction module. By performing multiple convolution operations, we aggregate the neighborhood features of task and device nodes, allowing the node feature representation to include more global context. Neighborhood features refer to the feature information of nodes adjacent to a particular node. Aggregating neighborhood features makes the node feature representation more comprehensive and accurate.

[0115] The task dependency edge eigenvalue reflects the closeness of the dependency relationship between task nodes. When task nodes are aggregated, the neighbor weights are adjusted based on the task dependency edge eigenvalue, so that neighbor nodes with close dependency relationships with task nodes have a greater influence on their feature updates. The device sharing edge eigenvalue reflects the closeness of the resource sharing relationship between device nodes. When device nodes are aggregated, the neighbor weights are adjusted based on the device sharing edge eigenvalue, so that neighbor nodes with close resource sharing relationships with device nodes have a greater influence on their feature updates.

[0116] As an implementation manner, step S330 may specifically include the following steps S331 to S336: Step S331: Extract the task node feature matrix, equipment node feature matrix, task dependency edge feature matrix and equipment sharing edge feature matrix from the production scheduling association graph. The rows of the task node feature matrix correspond to task nodes, and the columns correspond to task types, resource requirements and time window characteristics; the rows of the equipment node feature matrix correspond to equipment nodes, and the columns correspond to equipment types, resource supply and operating status characteristics.

[0117] The task node feature matrix is ​​a two-dimensional matrix whose rows correspond to the task nodes in the production scheduling association map, and whose columns correspond to various attribute characteristics of the tasks, such as task type, resource requirements, and time window characteristics. The device node feature matrix is ​​also a two-dimensional matrix whose rows correspond to the device nodes in the production scheduling association map, and whose columns correspond to various attribute characteristics of the devices, such as device type, resource supply, and operating status characteristics.

[0118] The task dependency edge feature matrix is ​​a two-dimensional matrix whose rows and columns correspond to task nodes, and the elements in the matrix represent the dependency edge eigenvalues ​​between task nodes. The device shared edge feature matrix is ​​also a two-dimensional matrix whose rows and columns correspond to device nodes, and the elements in the matrix represent the shared edge eigenvalues ​​between device nodes.

[0119] Step S332: Initialize the graph convolution layer parameters of the graph feature extraction module, including the task node convolution kernel weight, the device node convolution kernel weight and the bias vector. The dimension of the task node convolution kernel weight matches the column dimension of the task node feature matrix, and the dimension of the device node convolution kernel weight matches the column dimension of the device node feature matrix.

[0120] The graph convolution layer parameters are used for graph convolution operations in the graph feature extraction module. They include the task node convolution kernel weights, the device node convolution kernel weights, and the bias vector. The task node convolution kernel weights are used to perform convolution operations on the features of the task node, the device node convolution kernel weights are used to perform convolution operations on the features of the device node, and the bias vector is used to add a constant term after the convolution operation to increase the expressiveness of the model.

[0121] The dimensions of the task node convolution kernel weights match the column dimensions of the task node feature matrix to ensure that the convolution operation can correctly extract and transform the features of the task node. The dimensions of the device node convolution kernel weights match the column dimensions of the device node feature matrix to ensure that the convolution operation can correctly extract and transform the features of the device node.

[0122] In embodiments of the present invention, random initialization can be used when initializing graph convolution layer parameters. For example, the task node convolution kernel weights can be initialized as a random matrix, where the number of rows and columns of the matrix is ​​determined by the column dimension of the task node feature matrix and the design of the graph convolution layer. The device node convolution kernel weights can also be initialized as a random matrix, where the number of rows and columns of the matrix is ​​determined by the column dimension of the device node feature matrix and the design of the graph convolution layer.

[0123] The bias vector can be initialized to a random vector whose dimension is determined by the output dimension of the graph convolution layer. The initialized graph convolution layer parameters will be used in subsequent graph convolution operations. These parameters are continuously updated through training, allowing the model to better extract feature information from the graph.

[0124] Step S333: In the first layer of graph convolution operation, convolution calculation is performed on the task node feature matrix, and the convolution kernel dynamically adjusts the weight based on the task dependency edge feature matrix, aggregates the neighborhood task features of the task node, and generates the first layer feature matrix of the task node; convolution calculation is performed on the device node feature matrix, and the convolution kernel dynamically adjusts the weight based on the device shared edge feature matrix, aggregates the neighborhood device features of the device node, and generates the first layer feature matrix of the device node.

[0125] In the first layer of graph convolution, convolution calculations are performed on the task node feature matrix and the device node feature matrix. Convolution calculation involves convolving the convolution kernel with the node feature matrix. This operation aggregates the neighborhood features of the node, allowing the node's feature representation to include more neighborhood information.

[0126] The convolution kernel dynamically adjusts weights based on the task dependency edge feature matrix to adjust the aggregate weights of neighboring task features according to the closeness of the dependencies between task nodes. Similarly, the convolution kernel dynamically adjusts weights based on the device sharing edge feature matrix to adjust the aggregate weights of neighboring device features according to the closeness of the resource sharing relationship between device nodes.

[0127] In an embodiment of the present invention, when performing a convolution calculation on a task node feature matrix, the weight of the convolution kernel is first dynamically adjusted based on the task dependency edge feature matrix. Then, the adjusted convolution kernel is convolved with the task node feature matrix to aggregate the neighborhood task features of the task node and generate the first-layer feature matrix of the task node. For the device node feature matrix, the same method is used to dynamically adjust the weight of the convolution kernel based on the device shared edge feature matrix. The adjusted convolution kernel is then convolved with the device node feature matrix to aggregate the neighborhood device features of the device node and generate the first-layer feature matrix of the device node.

[0128] Step S334: In the second-layer graph convolution operation, the first-layer feature matrix of the task node and the first-layer feature matrix of the device node are interactively convolved, the task node features and the associated device node features are fused according to the weights of the device shared edge feature matrix, and the device node features and the associated task node features are fused according to the weights of the task dependent edge feature matrix to generate the second-layer feature matrix of the task node and the second-layer feature matrix of the device node.

[0129] In the second-layer graph convolution operation, interactive convolution is performed on the first-layer feature matrix of the task nodes and the first-layer feature matrix of the device nodes. Interactive convolution is the process of fusing the features of the task nodes and the device nodes. Through interactive convolution, the feature representation of the task nodes can include information about the associated device nodes, and the feature representation of the device nodes can include information about the associated task nodes.

[0130] The weighted fusion of task node features and associated device node features based on the device sharing edge feature matrix adjusts the impact of device node features on task node features based on the closeness of resource sharing between device nodes. The weighted fusion of device node features and associated task node features based on the task dependency edge feature matrix adjusts the impact of task node features on device node features based on the closeness of dependency between task nodes.

[0131] In an embodiment of the present invention, when performing interactive convolution, for a task node, the feature vector of the associated device node is multiplied by its weight according to the weight of the device-sharing edge feature matrix, and the result is then fused with the feature vector of the task node. For a device node, the same method is used: the feature vector of the associated task node is multiplied by its weight according to the weight of the task-dependency edge feature matrix, and the result is then fused with the feature vector of the device node. Through interactive convolution, the second-layer feature matrix of the task node and the second-layer feature matrix of the device node are generated.

[0132] Step S335: Repeat the graph convolution operation for a preset number of layers, and each layer performs feature aggregation based on the task node feature matrix, device node feature matrix and corresponding edge feature matrix of the previous layer, and finally generates the task node final feature matrix and the device node final feature matrix.

[0133] Repeating the graph convolution operation for a preset number of layers allows the model to more deeply learn the feature information in the production scheduling correlation graph. Through multiple feature aggregations, the node feature representation can contain more global correlation information. Each layer of graph convolution operation aggregates features based on the task node feature matrix, device node feature matrix, and corresponding edge feature matrix of the previous layer, which continuously updates the node feature representation.

[0134] In an embodiment of the present invention, the preset number of layers can be determined based on the design of the model and actual needs. In the third-layer graph convolution operation, the same method is used to perform convolution calculations on the second-layer feature matrix of the task node and the second-layer feature matrix of the device node. The convolution kernel dynamically adjusts the weights based on the task dependency edge feature matrix and the device sharing edge feature matrix, aggregates the neighborhood features of the task node and the device node, and generates the third-layer feature matrix of the task node and the third-layer feature matrix of the device node. Repeat this process until the preset number of layers is reached. Finally, the final feature matrix of the task node and the final feature matrix of the device node are generated. These two matrices contain the feature information of the task node and the device node after multiple feature aggregations.

[0135] Step S336: Concatenate the final feature matrix of the task nodes and the final feature matrix of the device nodes row by row to generate a graph feature vector containing global association information of all nodes. The dimension of the graph feature vector is the sum of the number of task nodes and the number of device nodes multiplied by the node feature dimension.

[0136] The dimension of the graph feature vector is the sum of the number of task nodes and the number of device nodes multiplied by the node feature dimension, because the graph feature vector contains the feature information of all task nodes and device nodes. In an embodiment of the present invention, when splicing, each row of the final feature matrix of the task nodes is arranged in sequence, and then each row of the final feature matrix of the device nodes is arranged in sequence behind the final feature matrix of the task nodes to form a new matrix. This new matrix is ​​expanded row by row to obtain the graph feature vector.

[0137] Step S340: Input the graph feature vector into the action generation module, and generate a candidate scheduling action set through the multi-task output layer. The candidate scheduling action set includes multiple task equipment allocation schemes. Each task equipment allocation scheme includes a task identifier, a corresponding equipment identifier, and a recommended execution time slice. The number of task equipment allocation schemes is consistent with the number of task nodes in the production scheduling association graph.

[0138] The action generation module is a module in the multi-objective reinforcement learning strategy model used to generate candidate scheduling action sets. It receives the graph feature vector as input and generates multiple task device allocation plans through the multi-task output layer. The multi-task output layer is a neural network layer that can simultaneously output allocation plans for multiple tasks.

[0139] The candidate scheduling action set is a set of possible scheduling actions. Each task-device allocation solution includes a task ID, a corresponding device ID, and a recommended execution time slice. The task ID uniquely identifies a task, the corresponding device ID indicates the device to which the task is assigned, and the recommended execution time slice indicates the recommended execution time range for the task.

[0140] In this embodiment of the present invention, after the graph feature vector is input into the action generation module, the multi-task output layer in the action generation module generates a set of candidate scheduling actions based on the graph feature vector. The multi-task output layer can adopt a multi-layer perceptron structure, generating a task device allocation plan through multiple fully connected layers and activation functions.

[0141] Step S350: A probability sampling strategy is used to select the current scheduling action from the candidate scheduling action set. The probability sampling strategy is implemented by dynamically adjusting the ratio of exploration probability to utilization probability. The exploration probability is used to select untried task equipment allocation schemes, and the utilization probability is used to select task equipment allocation schemes with higher estimated historical value.

[0142] The probabilistic sampling strategy is used to select the current scheduling action from a set of candidate scheduling actions. It balances the exploration of new scheduling actions with the utilization of existing experience by dynamically adjusting the ratio of exploration probability to utilization probability. The exploration probability refers to the probability of selecting an untried task and equipment allocation solution. By exploring new solutions, better scheduling strategies can be discovered. The utilization probability refers to the probability of selecting a task and equipment allocation solution with a high estimated historical value. By leveraging existing experience, the selected solution is guaranteed to have a certain degree of reliability.

[0143] In an embodiment of the present invention, the ratio of the exploration probability to the utilization probability can be dynamically adjusted based on the model's training progress and feedback from the environment. In the early stages of model training, the exploration probability can be set higher so that the model can try more scheduling actions and discover potential optimal strategies. As training progresses, the utilization probability can be gradually increased so that the model can use existing experience to select better scheduling actions. When selecting the current scheduling action from the candidate scheduling action set through a probabilistic sampling strategy, a random number is generated based on the exploration probability and the utilization probability, and the size of the random number is used to determine whether to select an untried task equipment allocation plan or a task equipment allocation plan with a higher estimated historical value.

[0144] Step S360: Update the production scheduling association map based on the task equipment allocation plan in the current scheduling action, adjust the execution status characteristics of the corresponding task nodes and the load status characteristics of the equipment nodes, and generate an updated production scheduling association map.

[0145] The task-device allocation plan for the current scheduling action contains information such as the task ID, the corresponding device ID, and the recommended execution time. This information can be used to update the production scheduling association map. Updating the production scheduling association map reflects the execution results of the current scheduling action, ensuring that the information in the map is consistent with the actual production scheduling situation.

[0146] Adjusting the execution status characteristics of the corresponding task node means updating the execution status of the task node, such as the task's start time, end time, and completion progress, based on the recommended execution time slice in the task device allocation plan. Adjusting the load status characteristics of the device node means updating the load status of the device node, such as the device's resource usage and working hours, based on the task allocation in the task device allocation plan.

[0147] In an embodiment of the present invention, when updating, for task nodes, the time window attributes and execution status characteristics of the task nodes are updated based on the recommended execution time slice in the task device allocation plan. For device nodes, the load status characteristics of the device nodes are updated based on the task allocation in the task device allocation plan. For example, if task A is assigned to device B for execution, the resource usage and working hours of device B are increased. Ultimately, an updated production scheduling association map is generated. The updated map reflects the execution results of the current scheduling action, providing new information for subsequent policy iterative optimization.

[0148] Step S370: Input the updated production scheduling association graph into the graph feature extraction module, generate a new graph feature vector as the next state feature, and calculate the immediate reward signal of the current scheduling action. The immediate reward signal is calculated based on the task completion time index and the resource utilization balance index.

[0149] The updated production scheduling association graph is input into the graph feature extraction module to obtain an updated graph feature vector. The graph feature vector contains global association information for all nodes in the production scheduling association graph and can be used as the next-state feature for subsequent policy iteration optimization. The immediate reward signal is an evaluation metric for the current scheduling action, reflecting the quality of the current scheduling action. The immediate reward signal is calculated based on a combination of the task completion timeliness index and the resource utilization balance index. The task completion timeliness index reflects the completion of the task within the specified time, while the resource utilization balance index reflects the balanced distribution of resources across different tasks and equipment.

[0150] In an embodiment of the present invention, after the updated production scheduling association graph is input into the graph feature extraction module, the graph feature extraction module generates a new graph feature vector as the next state feature according to the method of the previous steps S331-S336.

[0151] When calculating the immediate reward signal, it's necessary to separately calculate the task completion timeliness metric and the resource utilization balance metric. The task completion timeliness metric can be calculated by comparing the actual task completion time with the planned completion time. For example, if the task is completed early, a positive reward can be given, while if the task is completed late, a negative reward can be given. The resource utilization balance metric can be calculated by calculating the variance in resource allocation across different tasks and devices. A smaller variance indicates a more balanced resource allocation, resulting in a higher reward.

[0152] Then, the task completion timeliness index and resource utilization balance index are weighted and fused according to the production goal priority to generate an immediate reward signal for the current scheduling action. The production goal priority can be determined based on the timeliness weight and balance weight in the production plan. For example, if the timeliness weight is 0.6 and the balance weight is 0.4, then the task completion timeliness index is multiplied by 0.6 and the resource utilization balance index is multiplied by 0.4. The results are then added together to generate the immediate reward signal.

[0153] As an embodiment, in step S370, calculating the immediate reward signal of the current scheduling action may specifically include the following steps S371 to S376: Step S371: Extract the execution status features of all task nodes from the updated production scheduling association map. The execution status features include the current execution progress and estimated completion time of the task. Identify the critical path task nodes based on the task execution dependency directed graph. The critical path task nodes are the sequence of task nodes that affect the overall production cycle.

[0154] Execution status features describe the current execution status of a task, including its current progress and estimated completion time. Extracting the execution status features of all task nodes from the updated production scheduling association graph allows us to understand the current execution status of each task. The task execution dependency directed graph, a graph describing the dependencies between tasks, can be used to identify critical path task nodes. Critical path task nodes are sequences of task nodes that affect the overall production cycle in production scheduling; delays in these task nodes can extend the entire production cycle.

[0155] In an embodiment of the present invention, when extracting the execution status features of the task nodes from the updated production scheduling association graph, all task nodes in the graph are traversed to obtain the time window attributes and execution status information of each task node, and these information are used as execution status features.

[0156] To identify critical path task nodes, use the Critical Path Method (CPM). By calculating each task's earliest start time, earliest finish time, latest start time, and latest finish time, you can identify the sequence of task nodes that impact the overall production cycle. For example, starting from the start node of a directed graph of task execution dependencies, calculate each task's earliest start time and earliest finish time based on the task's dependencies and durations. Then, starting from the graph's end node, calculate the latest start time and latest finish time for each task in reverse order. Task nodes with the same earliest start time and latest finish time are considered critical path task nodes.

[0157] Step S372: Calculate the completion time deviation of the critical path task nodes. The completion time deviation of each critical path task node is the difference between the estimated completion time and the planned completion time. Sum the completion time deviations of all critical path task nodes to generate a total time deviation value.

[0158] Completion time variance is a metric used to measure whether critical path tasks are completed on time. It reflects the difference between the actual completion time and the planned completion time. The completion time variance for each critical path task node is the difference between the estimated completion time and the planned completion time. A positive difference indicates a delay in the task completion; a negative difference indicates an early completion.

[0159] The total timeliness deviation value is the sum of the completion timeliness deviations of all critical path task nodes, reflecting the overall situation of the entire production scheduling in terms of task completion timeliness.

[0160] In this embodiment of the present invention, when calculating the timeliness deviation of a critical path task node, the estimated completion time of each critical path task node is first obtained from the execution status characteristics, and the planned completion time of each critical path task node is obtained from the task backlog. Then, the difference between the estimated completion time and the planned completion time of each critical path task node is calculated to obtain the timeliness deviation of each critical path task node. The timeliness deviations of all critical path task nodes are summed to obtain a total timeliness deviation value.

[0161] Step S373: Calculate the task completion time reward component based on the total time deviation value. When the total time deviation value is negative, the reward component is positively correlated with the absolute value of the deviation value; when the total time deviation value is positive, the reward component is negatively correlated with the absolute value of the deviation value; when the total time deviation value is zero, the reward component is zero.

[0162] The timeliness reward component of a task is the portion of the immediate reward signal related to the timeliness of task completion. It is calculated based on the total timeliness deviation. When the total timeliness deviation is negative, the task is completed ahead of schedule, and the reward component is positively correlated with the absolute value of the deviation. That is, the greater the time of early completion, the higher the reward component. When the total timeliness deviation is positive, the task is completed late, and the reward component is negatively correlated with the absolute value of the deviation. That is, the greater the time of delayed completion, the lower the reward component. When the total timeliness deviation is zero, the task is completed on time, and the reward component is zero.

[0163] In this embodiment of the present invention, when calculating the time-limited reward component for task completion, a linear function can be used to represent the relationship between the reward component and the total time-limited deviation value. For example, when the total time-limited deviation value is negative, the reward component = k × |total time-limited deviation value|, where k is a positive constant and |total time-limited deviation value| represents the absolute value of the total time-limited deviation value; when the total time-limited deviation value is positive, the reward component = -k × |total time-limited deviation value|; and when the total time-limited deviation value is zero, the reward component = 0.

[0164] Step S374: Extract the load status characteristics of all device nodes from the updated production scheduling association map. The load status characteristics include the current resource occupancy rate and remaining resource capacity of the device. Calculate the mean and dispersion of the resource occupancy rate of all device nodes. The dispersion degree is represented by the sum of the squares of the deviations between the resource occupancy rate and the mean.

[0165] Load status characteristics describe the current load of a device node and include information such as the device's current resource utilization and remaining resource capacity. The mean resource utilization is the average of the resource utilization rates of all device nodes, reflecting the overall utilization of device resources. The dispersion level measures the uniformity of the distribution of device resource utilization. The dispersion level is expressed as the sum of squared deviations between the resource utilization rate and the mean. A larger sum of squared deviations indicates a more uneven distribution of device resource utilization.

[0166] In an embodiment of the present invention, when extracting the load status characteristics of the device nodes from the updated production scheduling association map, all device nodes in the map are traversed to obtain the resource occupancy and remaining resource capacity information of each device node, and use this information as the load status characteristics.

[0167] To calculate the mean resource utilization, add the resource utilization of all device nodes and divide it by the number of device nodes to get the mean. To calculate the dispersion, first calculate the deviation of each device node's resource utilization from the mean, then square the deviation, and finally sum all the squared deviations.

[0168] Step S375: Calculate the resource utilization balance reward component based on the degree of dispersion. When the degree of dispersion is less than the preset balance standard, the reward component is negatively correlated with the degree of dispersion; when the degree of dispersion is greater than the preset balance standard, the reward component is positively correlated with the degree of dispersion.

[0169] The resource utilization balance reward component is the portion of the immediate reward signal that is related to resource utilization balance and is calculated based on the degree of dispersion. The preset balance standard is a pre-set threshold used to measure whether the distribution of device resource utilization is uniform.

[0170] When the degree of dispersion is less than the preset equilibrium standard, it means that the distribution of device resource occupancy rate is relatively uniform, and the reward component is negatively correlated with the degree of dispersion, that is, the smaller the degree of dispersion, the higher the reward component; when the degree of dispersion is greater than the preset equilibrium standard, it means that the distribution of device resource occupancy rate is uneven, and the reward component is positively correlated with the degree of dispersion, that is, the greater the degree of dispersion, the lower the reward component.

[0171] In this embodiment of the present invention, when calculating the resource utilization balance reward component, a piecewise function can be used to represent the relationship between the reward component and the degree of dispersion. For example, when the degree of dispersion is less than the preset balance standard, the reward component = k1 × (preset balance standard - degree of dispersion), where k1 is a positive constant; when the degree of dispersion is greater than the preset balance standard, the reward component = -k2 × (degree of dispersion - preset balance standard), where k2 is a positive constant.

[0172] Step S376: The task completion time reward component and the resource utilization balance reward component are weighted and integrated according to the production target priority. The priority is determined by the time weight and balance weight in the production plan, and an immediate reward signal for the current scheduling action is generated. The immediate reward signal is used to update the value function parameters of the value evaluation module.

[0173] The purpose of weighting the reward components for timeliness and resource balance based on production goal priority is to combine both objectives, generating a comprehensive reward signal. Production goal priority is determined by the timeliness weight and balance weight in the production plan. The timeliness weight and balance weight respectively represent the importance of timeliness and resource balance in production scheduling.

[0174] The immediate reward signal is an evaluation metric for the current scheduling action and is used to update the value function parameters of the value assessment module. The value function parameters of the value assessment module determine the model's assessment of the value of different scheduling actions. Feedback from the immediate reward signal adjusts the value function parameters, enabling the model to better learn the optimal scheduling strategy.

[0175] In an embodiment of the present invention, when performing weighted fusion, the task completion time reward component is multiplied by the time weight, and the resource utilization balance reward component is multiplied by the balance weight, and then the two results are added to obtain an immediate reward signal. When the immediate reward signal is used to update the value function parameters of the value assessment module, a gradient descent algorithm can be used. The gradient descent algorithm is an algorithm for optimizing model parameters. It calculates the gradient of the value function with respect to the parameters, and then updates the parameters in the opposite direction of the gradient, so that the value of the value function gradually decreases. By continuously using the immediate reward signal to update the value function parameters, the model can continuously learn and optimize to find the optimal scheduling strategy.

[0176] Step S380: The experience samples including the current graph feature vector, the current scheduling action, the immediate reward signal and the next state feature are stored in the experience replay pool. When the number of experience samples stored in the experience replay pool reaches the preset capacity, batch experience samples are randomly sampled to train the network structure parameters. The node feature conversion matrix of the graph feature extraction module, the strategy weight vector of the action generation module and the value function parameters of the value evaluation module are optimized through the back propagation algorithm. The training is iterated until the value estimation value output by the value evaluation module converges, and a set of scheduling strategy parameters that integrates the task completion time and resource utilization balance is output.

[0177] The experience replay pool is a buffer for storing experience samples. Experience samples contain information such as the current graph feature vector, the current scheduling action, the immediate reward signal, and the next state characteristics. Storing experience samples in the experience replay pool allows the model to reuse these samples during training, improving training efficiency. When the number of experience samples stored in the experience replay pool reaches the preset capacity, batches of experience samples are randomly sampled for training. Random sampling can avoid correlation between samples, allowing the model to learn different scheduling situations more comprehensively. Iterative training until the value estimate output by the value evaluation module converges is to enable the model to learn the optimal scheduling strategy. Convergence of the value estimate means that the model's value assessment of different scheduling actions has stabilized, and the model has now learned a better scheduling strategy.

[0178] In an embodiment of the present invention, when storing experience samples in the experience replay pool, the current graph feature vector, the current scheduling action, the immediate reward signal, and the next state feature are stored in sequence as a sample, and then the sample is added to the experience replay pool. When the number of experience samples stored in the experience replay pool reaches the preset capacity, batch experience samples are randomly sampled from the experience replay pool. When the backpropagation algorithm is used to optimize the network structure parameters, the sampled batch experience samples are input into the model, the error between the model output and the immediate reward signal is calculated, and then the gradient of each parameter is calculated by the backpropagation algorithm. The node feature conversion matrix of the graph feature extraction module, the strategy weight vector of the action generation module, and the value function parameters of the value evaluation module are updated according to the gradient. This process is repeated and training is continuously iterated until the value estimate output by the value evaluation module converges. Finally, a set of scheduling policy parameters that integrates the task completion timeliness and resource utilization balance is output. This parameter set contains information such as the optimal task equipment allocation plan and task execution time.

[0179] Step S400: generating a device task timing allocation plan based on a scheduling policy parameter set, wherein the device task timing allocation plan is used to indicate the task execution sequence and resource allocation rules of each production device within a continuous production period.

[0180] The scheduling policy parameter set is derived from a multi-objective reinforcement learning strategy model through policy iteration optimization. It includes information such as the task-device allocation plan and task execution time. The equipment-task timing allocation plan is a specific production scheduling plan generated based on the scheduling policy parameter set. It specifies the task execution order and resource allocation rules for each production equipment within the continuous production period.

[0181] As an implementation manner, step S400 may specifically include the following steps S410 to S460: Step S410: Parse the scheduling policy parameter set to extract the task-device allocation priority sequence, resource configuration rules, and time sequence partitioning parameters.

[0182] The scheduling policy parameter set contains information such as the task-device allocation plan and task execution time. Parsing the scheduling policy parameter set is to extract useful information from it to generate a time-series allocation plan for device tasks. The task-device allocation priority sequence refers to the priority order in which tasks are assigned to devices, which determines the order in which tasks are assigned to devices. Resource allocation rules are the rules for allocating and using resources during the production process, specifying resource allocation methods and usage restrictions. Timing partitioning parameters are the parameters used to divide production time periods, determining the length and division method of production time periods.

[0183] In embodiments of the present invention, when parsing a scheduling policy parameter set, the parameter set needs to be organized and analyzed. The scheduling policy parameter set can be represented as a data structure, such as a dictionary or list. Task-device allocation priority sequences, resource configuration rules, and time partitioning parameters can then be extracted based on the data structure's keys or indexes.

[0184] Step S420: Based on the task-device allocation priority sequence and the task execution dependency directed graph, a task timing constraint network is constructed, in which nodes represent tasks, directed edges represent execution order constraints between tasks, and edge features represent minimum time intervals.

[0185] In the embodiment of the present invention, when constructing a task timing constraint network, the task allocation order is first determined based on the task-device allocation priority sequence, and then the dependency relationships and time constraint information between tasks are extracted from the task execution dependency directed graph.

[0186] For each task node in the task execution dependency directed graph, its inbound and outbound edges in the task timing constraint network are determined according to its dependency and priority.

[0187] The edge feature represents the minimum time interval, which can be determined based on the duration and dependencies of the tasks. Finally, a task timing constraint network is constructed, which provides important constraints for subsequent task allocation and time scheduling.

[0188] Step S430: performing resource allocation processing on each production device according to resource configuration rules and device operation status boundary conditions, determining the resource supply status of the device in each sequential unit, and ensuring that resource allocation satisfies the device capacity constraints.

[0189] Resource allocation rules define how resources are allocated and used during the production process, specifying resource allocation methods and usage restrictions. Equipment operating state boundary conditions define the operating range and limitations of equipment under different times and conditions, encompassing information such as the equipment's processing capacity, resource supply type, and continuous operating time.

[0190] Resource allocation is the process of allocating resources to production equipment based on resource configuration rules and equipment operating status boundary conditions. Determining the resource availability of a device within each sequential unit ensures that the device has sufficient resources to complete its tasks within each sequential unit. Resource allocation must meet equipment capacity constraints, meaning that resource allocation cannot exceed the device's maximum processing capacity and resource availability.

[0191] In this embodiment of the present invention, resource allocation is first performed based on resource configuration rules to determine the resource allocation method. For example, resource configuration rules may dictate that resources be allocated based on task priority or equipment requirements. Then, resources are allocated to each production device based on the device's operating status boundary conditions. For each sequential unit, resources are allocated based on the device's resource requirements and resource supply capacity. Finally, the resource supply status of the device within each sequential unit is determined.

[0192] Step S440: Combine the task timing constraint network and the device resource allocation results, generate an initial device task timing allocation plan through a greedy algorithm, and allocate devices and timing units to tasks from high to low priority to ensure that the resource requirements and time window constraints of the tasks are met.

[0193] In the embodiment of the present invention, when generating the initial device task timing allocation plan, the tasks are first sorted from high to low according to the task-device allocation priority sequence. Then, the device and timing unit are allocated to each task in turn.

[0194] For each task, the range of sequential units in which the task can be executed is determined based on the execution order constraints and time window constraints in the task's timing constraint network. Then, based on the device resource allocation results, a device with sufficient resources that can meet the task's requirements is selected within the range of executable sequential units. This process is repeated, assigning devices and sequential units to each task until all tasks are assigned. Finally, an initial device-task timing allocation plan is generated. This initial plan assigns devices and sequential units to tasks in descending order of priority, ensuring that the task's resource requirements and time window constraints are met.

[0195] Step S450: Conflict detection and optimization processing are performed on the initial device task timing allocation plan, time overlap conflicts and resource competition conflicts are identified, and conflicts are resolved by adjusting the task execution order or timing units.

[0196] The initial device task scheduling plan may contain issues such as time overlap and resource contention. Time overlap refers to the situation where multiple tasks are assigned to the same device within the same sequential unit. Resource contention refers to the situation where the same resource is requested by multiple tasks within the same sequential unit. Conflict detection and optimization is the process of inspecting and adjusting the initial device task scheduling plan. The goal is to identify and resolve conflicts within the plan, improving its feasibility and effectiveness. By adjusting the task execution order or sequential units, conflicts can be resolved, ensuring smooth task execution.

[0197] Step S460: Evaluate the performance indicators of the optimized timing allocation plan, which include task delay characteristics and device load balancing characteristics. If the indicators meet the preset requirements, it is determined as the final plan and a device task timing allocation plan is generated.

[0198] Performance metrics are key indicators for evaluating the quality of optimized solutions. Task latency reflects whether tasks can be completed on time, while device load balancing reflects whether device resources are being utilized effectively. By evaluating these metrics, we can determine whether the solution meets actual production needs.

[0199] In an embodiment of the present invention, task delay characteristics can be evaluated by calculating indicators such as the average delay time and maximum delay time of all tasks. Device load balancing characteristics can be evaluated by calculating the variance of the device's resource utilization within each sequential unit. The smaller the variance, the more balanced the device load. Preset requirements are set based on production goals and actual conditions. When the optimized solution's task delay characteristics and device load balancing characteristics meet the preset requirements, the solution is determined to be the final solution.

[0200] When generating a time-series plan for equipment tasks, the final plan's task allocation information, equipment resource utilization, and timing unit arrangements are organized into a detailed document or data structure to guide actual production scheduling. This plan defines the task execution sequence and resource allocation rules for each piece of production equipment within a continuous production period, providing strong support for efficient production operations.

[0201] See also Figure 2 , Figure 2This is a schematic diagram of the structure of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 may be connected via a bus or other means. The processor 101 (also known as the Central Processing Unit (CPU)) is the computing and control core of the computer system, capable of parsing various instructions within the computer system and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 101. The communication interface 102 may also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system for storing programs and data. It is understood that the memory 103 herein may include both the built-in memory of the computer system and, of course, the extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system, but this is not limited to this in the present invention.

[0202] In one embodiment, the processor 101 executes the multi-objective production scheduling method based on reinforcement learning provided in the above embodiment of the present invention by running the computer program in the memory 103 .

Claims

1. A multi-objective production scheduling method based on reinforcement learning, characterized in that: The method comprises: Based on the matching analysis of the task requirement list and the equipment capability baseline, a scheduling constraint space including task execution dependencies and equipment operating status boundary conditions is obtained; A graph neural network is used to extract structured features from the scheduling constraint space to generate a production scheduling association graph containing node features and edge features. The node features correspond to task attributes and equipment attributes, and the edge features correspond to priority relationships between tasks and resource sharing relationships between equipment. Calling a multi-objective reinforcement learning strategy model to perform strategy iteration optimization on the production scheduling association map to obtain a scheduling strategy parameter set that integrates task completion timeliness and resource utilization balance; A device task timing allocation plan is generated based on the scheduling strategy parameter set, and the device task timing allocation plan is used to indicate the task execution order and resource allocation rules of each production device within a continuous production period.

2. The method according to claim 1, characterized in that The matching analysis based on the task requirement list and the equipment capability baseline is performed to obtain a scheduling constraint space containing task execution dependencies and equipment operating status boundary conditions, including: Parsing the task attribute information in the task requirement list, extracting the duration requirement, resource consumption type and quantity requirement of each task through semantic parsing, generating a task attribute feature vector, and mapping the task attribute feature vector into a fixed-dimensional numerical vector through an embedding layer; Collecting device operating parameters in the device capability baseline, obtaining the device's processing capability, resource supply type, and continuous operation time data through a parameter extraction algorithm, and constructing a device capability feature matrix, wherein rows and columns of the device capability feature matrix correspond to device and resource types, respectively; Perform compatibility matching analysis on the task attribute feature vector and the device capability feature matrix, calculate the matching degree between the task and the device through feature space mapping, and determine the set of candidate devices that can be adapted for each task and the range of task types that the device can carry; Analyze the process dependencies between different tasks and generate a directed graph of task execution dependencies, where nodes of the directed graph represent tasks and directed edges represent dependency directions; Combining the equipment's continuous operating time data with the task duration requirements, the task processing capacity of the equipment within a unit production cycle is calculated using sequential logic to determine the equipment's operating status boundary conditions. The task execution dependency directed graph and the device operation status boundary conditions are integrated, and a scheduling constraint space is constructed through a constraint integration algorithm. The scheduling constraint space stores the associated constraint information of tasks and devices based on a multi-dimensional tensor structure.

3. The method according to claim 2, characterized in that The step of analyzing the process dependencies between different tasks and generating a directed graph of task execution dependencies includes: Extracting process route description information for each task in the task requirement list, identifying direct and indirect dependencies between tasks through natural language understanding, and generating preliminary dependency identification results, wherein direct dependencies are reflected as the logical sequence of process connections, and indirect dependencies are reflected as the association of shared process conditions; Based on the direct dependency relationship in the preliminary dependency relationship identification result, a directed connection edge is established between the corresponding task nodes, and the edge eigenvalue is calculated using historical process data. The edge eigenvalue reflects the time consumption characteristics of the conversion between the two tasks, and an initial directed graph structure including the direct connection edge and the edge eigenvalue is generated; Based on the indirect dependency relationships in the preliminary dependency relationship identification results, combined with the edge eigenvalues ​​of the initial directed graph structure, the dependency transfer paths between non-adjacent process tasks are calculated using a transitive closure algorithm. The paths are described by the intermediate task node sequence and the accumulated edge eigenvalues, and the indirect connection paths are added to the initial directed graph structure. For a task execution dependency directed graph including direct connection edges and indirect connection paths, prioritize each task node according to the importance of the task in the production process, and mark critical task nodes and non-critical task nodes in the task execution dependency directed graph; Perform loop detection on the task execution dependency directed graph after marking the key nodes, traverse all paths in the directed graph using a graph traversal algorithm, and activate a dependency adjustment mechanism if a cyclic dependency exists. Optimize the dependency edges that cause the cycle based on process constraint rules to make the directed graph acyclic; Based on the optimized acyclic directed graph, a time window attribute is added to each task node to generate a complete task execution dependency directed graph data structure including node priority, edge eigenvalue and time window attributes.

4. The method according to claim 1, wherein The structured feature extraction of the scheduling constraint space by the graph neural network to generate a production scheduling association graph containing node features and edge features includes: Encoding the task nodes and device nodes in the scheduling constraint space to generate an initial node feature vector, integrating the task type, resource demand and time window data to generate a task node feature vector, and integrating the device type, resource supply and operating status data to generate a device node feature vector, wherein the dimensions of the task node feature vector and the device node feature vector are consistent; Based on the edge eigenvalues ​​of the directed graph of task execution dependencies, the task dependency edge eigenvalues ​​of the connection relationships between task nodes are comprehensively calculated by taking dependency strength, process switching cost, and parallel execution possibility into consideration; Based on the device resource sharing capability data, the device sharing edge eigenvalues ​​of the connection relationship between device nodes are comprehensively calculated through resource transmission efficiency, collaborative work coefficient and conflict avoidance parameters; Initialize the message passing parameters of the graph neural network and input the task node feature vector, device node feature vector, task dependency edge feature value, and device shared edge feature value into the message passing layer of the graph neural network; In the message passing layer, the feature representation of each node is updated by aggregating the neighbor node information of each node. The neighbor information aggregation of the task node uses the task dependency edge feature value to adjust the neighbor node weight. The neighbor information aggregation of the device node uses the device shared edge feature value to adjust the neighbor node weight. The message passing is iteratively performed to fuse the neighborhood information. The updated node features, task dependency edge feature values, and equipment sharing edge feature values ​​are integrated to generate a production scheduling association graph through a graph construction algorithm.

5. The method according to claim 4, characterized in that In the message passing layer, the feature representation of each node is updated by aggregating the neighbor node information of each node, including: In each iteration of message passing, the neighbor node set of each node is sampled. The task node selects the neighbor node with the highest contribution based on the eigenvalue of the task dependency edge, and the device node selects the neighbor node with the highest contribution based on the eigenvalue of the device sharing edge. Perform weighted summation on the selected neighbor node features to obtain a weighted summation result. The weight value of the task node neighbor is calculated by the task dependency edge eigenvalue, and the weight value of the device node neighbor is calculated by the device sharing edge eigenvalue. The edge eigenvalue is positively correlated with the corresponding weight value of the neighbor node. The weighted summation result is concatenated with the feature vector of the current node, mapped to the intermediate feature space through a linear transformation layer, and the feature vector of the intermediate feature space is nonlinearly activated. The activated feature vector is layer-normalized to eliminate the dimensional difference between different feature dimensions. The normalization operation is performed on the feature channel dimension. The normalized feature vector is residually connected with the original feature vector of the current node to generate an updated node feature representation.

6. The method according to claim 4, characterized in that The edge eigenvalues ​​of the task execution dependency directed graph are based on the task dependency edge eigenvalues, which are calculated by comprehensively calculating the connection relationship between task nodes through dependency strength, process switching cost, and parallel execution possibility, including: Extracting edge eigenvalue basic data from the task execution dependency directed graph, the edge eigenvalue basic data including process association type information of task node pairs, determining the closeness level of the process association type through process logic analysis, and generating a dependency strength quantization value based on the closeness level; Based on the historical process conversion records in the edge eigenvalue basic data, the resource adjustment time and equipment preparation operation time of the task node in the conversion process are extracted. Time series statistical analysis is performed on the time consumption data to generate a quantitative value of the process switching cost that reflects the conversion efficiency. Based on the parallel execution case records in the edge eigenvalue basic data, the frequency of scenarios in which task nodes can be started simultaneously in historical production is counted. Combined with the process compatibility assessment results, a parallel execution possibility quantitative value reflecting the feasibility of synchronous execution is generated; The quantified values ​​of dependency strength, process switching cost, and parallel execution possibility are input into the feature fusion module. Based on the degree of influence of each quantified value on the production scheduling target, the fusion weight is dynamically assigned. The weighted sum operation is performed to generate the task dependency edge feature value of the connection relationship between task nodes. The device resource sharing capability data is based on the device sharing edge feature value of the connection relationship between the device nodes, which is calculated by comprehensively calculating the resource transmission efficiency, the cooperative working coefficient and the conflict avoidance parameter, including: Extract inter-device resource transmission records from the device resource sharing capability data. The resource transmission records include the transmission resource type, single transmission time, and number of successful transmissions. Calculate the ratio of the actual transmission resource amount to the theoretical transmission capacity per unit time to generate a quantitative value of resource transmission efficiency. Based on collaborative production cases from the equipment resource sharing capability data, the completion quality scores and task cycle compliance rates of tasks jointly participated by two pieces of equipment are extracted. The quality scores and cycle compliance rates are then integrated according to the process requirements weights to generate a quantitative value of the collaborative work coefficient that reflects the stability of the collaboration. Based on the resource conflict history in the equipment resource sharing capability data, the duration of production interruptions caused by resource competition and the effectiveness score of conflict resolution measures are calculated. Combined with the interruption frequency analysis results, a quantitative value of the conflict avoidance parameter reflecting the conflict handling capability is generated; The quantified values ​​of resource transmission efficiency, collaborative working coefficient and conflict avoidance parameter are input into the feature fusion module. The fusion weight is dynamically allocated based on the degree of influence of each quantified value on the smoothness of resource allocation. The weighted sum operation is performed to generate the device shared edge feature value of the connection relationship between device nodes.

7. The method according to claim 1, characterized in that The multi-objective reinforcement learning strategy model is called to perform strategy iteration optimization on the production scheduling association map to obtain a scheduling strategy parameter set that integrates task completion timeliness and resource utilization balance, including: Constructing a network structure of a multi-objective reinforcement learning strategy model, wherein the network structure includes a graph feature extraction module, an action generation module, and a value assessment module; Initialize the parameters of the network structure, including the node feature conversion matrix of the graph feature extraction module, the strategy weight vector of the action generation module, and the value function parameters of the value assessment module. Pre-train the initial parameters using historical production scheduling case data to generate a pre-trained parameter set. The production scheduling association graph is input into the graph feature extraction module. The neighborhood features of task nodes and device nodes are aggregated through multi-layer graph convolution operations. The node feature representation is updated to generate a graph feature vector containing global association information. When task nodes are aggregated, the neighbor weights are adjusted based on the feature values ​​of task dependency edges. When device nodes are aggregated, the neighbor weights are adjusted based on the feature values ​​of device shared edges. The graph feature vector is input into the action generation module, and a candidate scheduling action set is generated through the multi-task output layer. The candidate scheduling action set includes multiple task equipment allocation schemes. Each task equipment allocation scheme includes a task identifier, a corresponding equipment identifier, and a recommended execution time slice. The number of task equipment allocation schemes is consistent with the number of task nodes in the production scheduling association graph. A probabilistic sampling strategy is used to select the current scheduling action from the candidate scheduling action set. The probabilistic sampling strategy is implemented by dynamically adjusting the ratio of exploration probability to utilization probability. The exploration probability is used to select untried task equipment allocation schemes, while the utilization probability is used to select task equipment allocation schemes with higher estimated historical value. Update the production scheduling association map based on the task equipment allocation plan in the current scheduling action, adjust the execution status characteristics of the corresponding task nodes and the load status characteristics of the equipment nodes, and generate an updated production scheduling association map; The updated production scheduling association graph is input into the graph feature extraction module to generate a new graph feature vector as the next state feature. The immediate reward signal of the current scheduling action is calculated based on the comprehensive calculation of the task completion time index and the resource utilization balance index. The experience samples containing the current graph feature vector, current scheduling action, immediate reward signal and next state features are stored in the experience replay pool. When the number of experience samples stored in the experience replay pool reaches the preset capacity, batch experience samples are randomly sampled to train the network structure parameters. The node feature conversion matrix of the graph feature extraction module, the strategy weight vector of the action generation module and the value function parameters of the value evaluation module are optimized through the back propagation algorithm. The training is iterated until the value estimation value output by the value evaluation module converges, and a set of scheduling strategy parameters that integrates the task completion time and resource utilization balance is output.

8. The method according to claim 7, characterized in that The production scheduling association graph is input into the graph feature extraction module, and the neighborhood features of the task nodes and the equipment nodes are aggregated through multi-layer graph convolution operations, the node feature representation is updated, and a graph feature vector containing global association information is generated, including: Extract the task node feature matrix, equipment node feature matrix, task dependency edge feature matrix, and equipment sharing edge feature matrix from the production scheduling association graph. The rows of the task node feature matrix correspond to task nodes, and the columns correspond to task type, resource demand, and time window features. The rows of the equipment node feature matrix correspond to equipment nodes, and the columns correspond to equipment type, resource supply, and operating status features. Initialize the graph convolution layer parameters of the graph feature extraction module, including the task node convolution kernel weights, device node convolution kernel weights, and bias vectors. The dimensions of the task node convolution kernel weights match the column dimensions of the task node feature matrix, and the dimensions of the device node convolution kernel weights match the column dimensions of the device node feature matrix. In the first-layer graph convolution operation, convolution calculation is performed on the task node feature matrix. The convolution kernel dynamically adjusts the weight based on the task dependency edge feature matrix, aggregates the neighborhood task features of the task node, and generates the first-layer feature matrix of the task node. Convolution calculation is performed on the device node feature matrix. The convolution kernel dynamically adjusts the weight based on the device shared edge feature matrix, aggregates the neighborhood device features of the device node, and generates the first-layer feature matrix of the device node. In the second-layer graph convolution operation, the first-layer feature matrix of the task node is interactively convolved with the first-layer feature matrix of the device node. The task node features and the associated device node features are fused according to the weights of the device shared edge feature matrix. The device node features and the associated task node features are fused according to the weights of the task dependent edge feature matrix to generate the second-layer feature matrix of the task node and the second-layer feature matrix of the device node. Repeat the graph convolution operation for a preset number of layers. Each layer aggregates features based on the task node feature matrix, device node feature matrix, and corresponding edge feature matrix of the previous layer, and finally generates the final feature matrix of the task node and the final feature matrix of the device node. The final feature matrix of the task nodes and the final feature matrix of the device nodes are concatenated row by row to generate a graph feature vector containing the global association information of all nodes. The dimension of the graph feature vector is the sum of the number of task nodes and the number of device nodes multiplied by the node feature dimension.

9. The method according to claim 7, characterized in that The calculation of the immediate reward signal for the current scheduling action includes: Extract the execution status features of all task nodes from the updated production scheduling association graph. The execution status features include the current execution progress and estimated completion time of the task. Based on the directed graph of task execution dependencies, identify the critical path task nodes. The critical path task nodes are the sequence of task nodes that affect the overall production cycle. Calculate the completion time deviation of the critical path task nodes. The completion time deviation of each critical path task node is the difference between the estimated completion time and the planned completion time. Sum the completion time deviations of all critical path task nodes to generate the total time deviation value. The reward component for task completion time is calculated based on the total time deviation value. When the total time deviation value is negative, the reward component is positively correlated with the absolute value of the deviation value; when the total time deviation value is positive, the reward component is negatively correlated with the absolute value of the deviation value; when the total time deviation value is zero, the reward component is zero. Extract the load status features of all device nodes from the updated production scheduling association graph. The load status features include the current resource occupancy rate and remaining resource capacity of the device. Calculate the mean and dispersion of the resource occupancy rates of all device nodes. The dispersion is represented by the sum of squared deviations between the resource occupancy rate and the mean. The resource utilization balance reward component is calculated based on the degree of dispersion. When the degree of dispersion is less than the preset balance standard, the reward component is negatively correlated with the degree of dispersion; when the degree of dispersion is greater than the preset balance standard, the reward component is positively correlated with the degree of dispersion. The task completion time reward component and the resource utilization balance reward component are weighted and fused according to the production target priority. The priority is determined by the time weight and balance weight in the production plan, and an immediate reward signal for the current scheduling action is generated. The immediate reward signal is used to update the value function parameters of the value evaluation module.

10. A computer system, characterized in that: include: a memory storing a computer program; A processor, configured to load the computer program to implement the multi-objective production scheduling method based on reinforcement learning as described in any one of claims 1 to 9.

Images