Graph neural network based dynamic scheduling method for dual-resource flexible job shop

Abstract

The embodiment of the application discloses a double-resource flexible job shop dynamic scheduling method based on a graph neural network, and relates to the technical field of double-resource constraint workshop production scheduling. The specific method comprises the following steps: taking the minimization of the maximum completion time and the total cost as an optimization target, establishing an integer programming model of a flexible job shop dynamic scheduling problem considering human-machine double-resource constraints of worker fatigue and proficiency; according to the characteristics of the double-resource constraints, constructing a heterogeneous disjunctive graph at the current scheduling time; using a new type of heterogeneous graph neural network, obtaining a global state embedding vector fused with original features; based on the global state embedding vector, constructing a reinforcement learning environment model; using a multi-PPO algorithm to train the environment model to generate an optimized scheduling model; and outputting a scheduling scheme by using the optimized scheduling model. The scheduling method provided by the application realizes end-to-end fast solving of the double-resource constraint scheduling problem.

Description

Technical Field

[0001] This invention relates to the field of production scheduling technology for dual-resource-constrained workshops, and in particular to a dynamic scheduling method for dual-resource flexible workshops based on graph neural networks. Background Technology

[0002] The Flexible Job-shop Scheduling Problem (FJSP) plays a crucial role in modern manufacturing. In classic FJSP, operations are typically ordered with the objective of minimizing certain metrics, and a processing machine is selected for each operation from a set of candidate equipment. In recent years, with the development of high-end manufacturing industries such as equipment manufacturing, automobile manufacturing, and aerospace, human factors and labor scheduling in manufacturing systems have received widespread attention. The Dual-Resource Constrained Flexible Job-shop Scheduling Problem (DRC-FJSP) considers both the flexibility and constraints of machines and workers, narrowing the gap between scheduling models and actual production. However, in actual production, uncertain events such as machine failures, job insertions, and random workpiece arrivals frequently occur, leading to deviations between planning and execution, rendering the original solution ineffective. Therefore, based on DRC-FJSP, the Dual-Resource Constrained Dynamic Flexible Job-shop Scheduling Problem (DRC-DFJSP) is proposed.

[0003] To meet production demands, companies typically provide cross-training for workers, creating multi-skilled workers with varying skill sets and proficiency levels. When allocating tasks, fully considering these skill differences and allocating personnel rationally can effectively reduce production costs, improve efficiency, and enhance product quality. However, worker fatigue is unavoidable in this model, and long-term fatigue accumulation can damage worker health, reduce work efficiency, and degrade the performance of the production system. Therefore, how to integrate the impact of worker proficiency and fatigue on process processing time into the DRC-DFJSP, and how to handle equipment allocation and worker assignment for each process when dynamic scheduling issues arise, has become a pressing problem in actual production.

[0004] Currently, there are two main categories of methods for solving dynamic scheduling problems: predictive reactive scheduling (rescheduling) and fully reactive scheduling (real-time scheduling). Predictive reactive scheduling decomposes the dynamic problem into a series of static subproblems and then uses metaheuristic algorithms to generate a scheduling scheme. While predictive reactive scheduling can generate high-quality scheduling schemes, it struggles to meet real-time requirements. Fully reactive scheduling methods do not generate complete scheduling schemes; instead, they make decisions immediately based on real-time information when dynamic events occur. Priority scheduling rules (PDRs) are typically used, but efficient PDRs rely heavily on human experience and require extensive expertise.

[0005] In summary, relying on human management to achieve dynamic scheduling is no longer sufficient to meet current production demands. It is necessary to develop corresponding dynamic scheduling schemes based on virtual workshop technologies to improve end-to-end rapid processing in dual-resource-constrained scheduling problems. Summary of the Invention

[0006] The embodiments of the present invention provide a dynamic scheduling method for a dual-resource flexible job shop based on graph neural networks, which can improve the end-to-end processing speed in dual-resource constrained scheduling problems.

[0007] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0008] In a first aspect, the method provided by embodiments of the present invention includes:

[0009] A dynamic scheduling method for flexible job shops based on graph neural networks is proposed, applied to a virtual workshop within a dynamic scheduling system. The dynamic scheduling system also includes a physical workshop and a virtual-physical interaction module. Through the virtual-physical interaction module, the dynamic scheduling system uses virtual control of the physical workshop, bidirectional interaction, and closed-loop iteration. The virtual workshop completes resource scheduling based on real-time production information from the physical workshop, and the physical workshop performs production activities according to the scheduling instructions issued by the virtual workshop. The method includes:

[0010] Step 1: Establish and train a dynamic scheduling model for a flexible workshop with dual resources, taking into account worker fatigue and skill level. In some scenarios, this model can also be called an "integer programming model for the dynamic scheduling problem of a flexible workshop with dual resource constraints".

[0011] Step 2: Using real-time production factor data, such as real-time data on production factors like workpieces, machines, and workers, construct a heterogeneous disjunctive diagram of the Dynamic Flexible Workshop (DRC-DFJSP) under dual resource constraints to characterize the actual production status of the workshop.

[0012] Step 3: Construct a novel heterogeneous graph neural network (NHGNN) and extract features from the heterogeneous disjunctive graph to obtain the global state embedding vector;

[0013] Step 4: Input the global state embedding vector into the dual-resource flexible workshop dynamic scheduling model and record the output results. The multi-action strategy network outputs the scheduling actions of process selection, machine and worker allocation until a complete scheduling scheme is generated.

[0014] When a dynamic event occurs, steps 2 through 4 are repeated until the production task is completed. For example, the dynamic event includes: a new workpiece arriving randomly, wherein the arrival time of the new workpiece approximately follows a Poisson distribution; the arriving new workpiece is dynamically scheduled along with unprocessed processes, while processes in progress cannot be terminated, and already processed processes are no longer scheduled.

[0015] Specifically, in the dual-resource flexible workshop dynamic scheduling model, the impact of worker fatigue and skill level on process processing time is expressed as follows: Where, p ijkr For worker W r In machine M k Upper processing step O ij Actual processing time, p ijk For O ij In M k The preset processing time on the screen FL is a parameter representing the degree of influence of fatigue on processing time. r (t) represents W at time t. r Fatigue parameters, PL ijr For W r Execute O ij The proficiency parameter, i, is the skill level of the i-th workpiece J. i The index j is the workpiece J. i The j-th process O ij The index, k is the kth machine M k The index, r, is the r-th worker W. r The index.

[0016] The types of real-time production factor data include: workpiece real-time data, machine real-time data, worker real-time data, workpiece-machine associated real-time data, and workpiece-worker associated real-time data. Specifically, the workpiece real-time data includes: the scheduling status of each process, the estimated processing time and actual processing time of each process, the number of candidate machines for each process, the number of candidate workers for each process, the estimated start time and actual start time of each process, the number of unscheduled processes for the workpiece, and the estimated completion time and actual completion time of the workpiece. The machine real-time data includes: machine unit time processing cost, the number of processes the machine can process, machine available time, and machine utilization rate. The worker real-time data includes: worker fatigue level, the number of processes the worker can process, worker available time, and worker utilization rate. The workpiece-machine associated real-time data includes: the preset processing time of each process on the machine. The workpiece-worker associated real-time data includes: the worker's proficiency in processing each process and the worker's unit time processing cost for each process.

[0017] The construction of a dual-resource-constrained dynamic flexible job-shop scheduling problem (or dual-resource-constrained dynamic flexible job-shop scheduling problem) is used to characterize the actual production state of the workshop. The problem (abbreviated as DRC-DFJSP) involves a heterogeneous disjunctive graph, including: constructing process nodes, where the original vector of the process node is the workpiece real-time data; constructing machine nodes, where the original vector of the machine node is the machine real-time data; constructing worker nodes, where the original vector of the worker node is the worker real-time data; adding conjunctive arcs between process nodes to constrain the processing order between processes of the same workpiece; adding OM arcs between process nodes and their compatible machine nodes, where the original vector of the OM arc is the workpiece-machine associated real-time data; adding OW arcs between process nodes and their compatible worker nodes, where the original vector of the OW arc is the workpiece-worker associated real-time data; and then continuously updating the heterogeneous disjunctive graph, including: when a process is scheduled, only the arcs between this process and its assigned machine and worker are retained, and then the other OM arcs and OW arcs of this process are removed; repeating the process of updating the heterogeneous disjunctive graph until all processes are scheduled, generating a scheduling scheme graph.

[0018] In this embodiment, step 3 includes: constructing an embedding layer, in which machine nodes, worker nodes and operation nodes are embedded sequentially;

[0019] The NHGNN model is constructed by stacking N embedding layers. The final embedding vectors of three types of nodes are obtained through the NHGNN model, where N is a positive integer. Then, the average value of the three types of embedding vectors is calculated and concatenated to obtain the global state embedding vector.

[0020] During machine node embedding, worker nodes in the second-order neighborhood are embedded into operator nodes in the first-order neighborhood, and operator nodes in the first-order neighborhood are embedded into machine nodes that serve as target nodes. Similarly, during worker node embedding, machine nodes in the second-order neighborhood are embedded into operator nodes in the first-order neighborhood, and operator nodes in the first-order neighborhood are embedded into human nodes that serve as target nodes. During operator node embedding, the vectors of each node are processed and concatenated using six MLPs. Each node's vector includes: the original vector of an operator node, the original vectors of the preceding and succeeding operator nodes, the embedding vectors of the machine nodes adjacent to the operator node, and the embedding vectors of the worker nodes adjacent to the operator node. For example:

[0021] Machine node embedding:

[0022] (1) Embed worker nodes (second-order neighborhoods) into operator nodes (first-order neighborhoods):

[0023] Concatenate and map the original vectors of worker nodes and OW arcs to d dimensions; map the original vectors of operator nodes to d dimensions; calculate and normalize the attention coefficients of worker nodes to operator nodes; aggregate worker node features to obtain operator node embedding vectors.

[0024] (2) Embed the operator node (first-order neighborhood) into the machine node (target node):

[0025] The operator node embedding vector and the original OM arc vector are concatenated and mapped to d dimensions; the original machine node vector is mapped to d dimensions; the attention coefficient of the operator node to the machine node is calculated and normalized; the operator node features are aggregated to complete the machine node embedding.

[0026] Worker node embedding:

[0027] (1) Embed the machine node (second-order neighborhood) into the operator node (first-order neighborhood):

[0028] The original vectors of machine nodes and OM arcs are concatenated and mapped to d dimensions; the original vectors of operation nodes are mapped to d dimensions; the attention coefficients of machine nodes to operation nodes are calculated and normalized; the machine node features are aggregated to obtain the operation node embedding vector.

[0029] (2) Embed the operator node (first-order neighborhood) into the worker node (target node):

[0030] The operator node embedding vector and the original OW arc vector are concatenated and mapped to d dimensions; the original worker node vector is mapped to d dimensions; the attention coefficients of the operator node to the worker node are calculated and normalized; the operator node features are aggregated to complete the worker node embedding.

[0031] Operation node embedding: Six MLPs are used to process and concatenate the original vectors of the preceding and following operation nodes, as well as the embedding vectors of the operation nodes themselves and their adjacent machine and worker nodes, to obtain their embedding vectors.

[0032] The NHGNN model is constructed by stacking N embedding layers to obtain the final embedding vectors of three types of nodes.

[0033] Calculate the average value of the three types of embedding vectors and concatenate them to obtain the global state embedding vector.

[0034] Furthermore, it also includes: training the dual-resource flexible workshop dynamic scheduling model using the MPPO (multi-action proximal policy optimization) algorithm; wherein, based on the features of the embedding vectors corresponding to feasible process actions and the global state embedding vectors, a process Actor network is constructed; based on the features of the embedding vectors corresponding to feasible machine actions and the global state embedding vectors, a machine Actor network is constructed; based on the features of the embedding vectors corresponding to feasible worker actions and the global state embedding vectors, a worker Actor network is constructed; and based on the features of the global state embedding vectors, a Critic network is constructed.

[0035] The MPPO algorithm requires first constructing an environment model based on a Markov Decision Process (MDP). The specific steps are as follows:

[0036] State: At time step t, state s t The proposed heterogeneous disjunction graph represents the state of DRC-DFJSP, encompassing all static and dynamic features of DRC-DFJSP.

[0037] Action: At time step t, action a t From process action a o ∈A o Machine action a m ∈A m and worker actions a w ∈A w Composition, A o It is a set of optional processes, A m It is action a o A set of compatible machines, A w It is action a o A compatible set of workers;

[0038] Transition: At time step t, the agent transitions based on state s t Take action a t Interacting with the environment to transition to a new state. t+1And based on action a t Update the heterogeneous extraction graph;

[0039] Reward: Maximize the completion time C of both optimization objectives max And total cost L c The immediate rewards are defined as r1 and r2, and the weighted reward is r. t =ω1r1+ω1r2, satisfying ω1+ω2=1, ω1 and ω2 are weighting coefficients and ω1, ω2≥0.1;

[0040] Strategy: Multi-Action Strategy π θ (a o a m a w |s t ) By process strategy Machine strategy and worker strategy composition.

[0041] The policy network is pre-trained offline using the multi-action proximal policy optimization (MPPO) algorithm. The specific steps are as follows:

[0042] (1) Training the Actor Network

[0043] Each Actor network is trained to maximize its respective loss function. Among them, c p =1 and c e =0.01 represents the truncation hyperparameter and entropy coefficient hyperparameter, respectively; o is the subscript of the relevant parameters of the process strategy; m is the subscript of the relevant parameters of the machine strategy; and w is the subscript of the relevant parameters of the worker strategy. It is about tailoring the proxy target. The entropy targets are as follows:

[0044]

[0045] in, It is a target strategy With the behavioral strategy used for sampling The probability ratio, ∈ (0, 1), is the relevant parameter subscript (o, m, w) of the cutting parameter h, which takes the process strategy, machine strategy, and worker strategy, respectively, representing the actions corresponding to the three strategies.

[0046] (2) Update the Critic network

[0047] In this paper, the Critic network and the Actor network share the same NHGNN model. φ (st Using a d with two d φ Hidden layers of dimensionality and MLP with tanh activation φ To accept the global state embedding vector g extracted by the NHGNN model t It also outputs a scalar to estimate the cumulative return at a given point:

[0048] v φ (s t ) = MLP φ (g t )

[0049] The Critic network aims to minimize the mean squared error (MSE) L. MSE (φ) is updated:

[0050] L MSE (φ)=E t [MSE(r t v φ (s t ))]

[0051] Initialize MPPO algorithm parameters, the Actor interacts with the environment to schedule each instance, and collects experience quadruples (s t a t r t s t+1 The quadruple is stored in the experience replay pool until all tasks have been scheduled, and Et is the expected value of future cumulative rewards. Φ It's a Critic network. Then, the probability ratio is calculated. Target Actor Error and Critic error L MSE (φ), and update the network using the gradient ascent method. Finally, the target Actor parameters are... Copy to behavior Actor parameter Complete one iteration, repeat the iteration until the convergence criterion is met, complete the training, and obtain the optimized scheduling model.

[0052] An optimization scheduling model based on the MPPO algorithm is used for workshop production optimization scheduling. The optimization scheduling model employs an offline training mode. In actual production, the global state embedding vector is input into the MPPO algorithm-based optimization scheduling model, and the multi-action policy network outputs scheduling actions for process selection, machine and worker allocation, until a complete scheduling scheme is generated.

[0053] Secondly, the method provided in the embodiments of the present invention can be executed by a dual-resource flexible workshop dynamic scheduling system based on graph neural networks, the system comprising:

[0054] The virtual-physical interaction module is used for two-way interaction between the physical workshop and the virtual workshop;

[0055] The physical workshop is used to execute scheduling plans and collect real-time data to support dynamic scheduling of the virtual workshop. The physical workshop includes: a data acquisition module for collecting real-time data of workshop production elements; and a manufacturing execution module for executing scheduling plans to organize actual production.

[0056] The virtual workshop is used to respond to dynamic events and formulate scheduling schemes to guide actual production in the physical workshop. The virtual workshop includes: a state representation module for constructing a heterogeneous disjunctive graph of DRC-DFJSP; a feature extraction module for constructing NHGNN to extract global state embedding vectors from the heterogeneous disjunctive graph; an environment model module for constructing a reinforcement learning environment model of DRC-DFJSP; a scheduling model module for training a scheduling model using the MPPO algorithm to generate an optimized scheduling model; and an optimized scheduling module for generating a scheduling scheme using the optimized scheduling model.

[0057] This invention provides a dynamic scheduling method for dual-resource flexible workshops based on graph neural networks. It constructs a DRC-DFJSP integer programming model considering worker skill and fatigue. Addressing the characteristics of dual-resource constraint problems, it proposes a heterogeneous disjunctive graph to represent scheduling states, designs a novel heterogeneous graph neural network for efficient state feature extraction, and proposes multi-action proximal policy optimization (MPPO). This enables end-to-end solving of DRC-DFJSP problems of varying scales, overcoming the shortcomings of existing dual-resource manufacturing systems where problem models differ from actual production and scheduling schemes lack practicality. This significantly improves the end-to-end processing speed in dual-resource constraint scheduling problems. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the dual-resource flexible workshop scheduling problem provided in an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a digital twin-enhanced dual-resource flexible workshop dynamic scheduling system provided in an embodiment of the present invention;

[0061] Figure 3This is a schematic diagram of the heterogeneous extraction of DRC-DFJSP provided in an embodiment of the present invention;

[0062] Figure 4 A schematic diagram of the global state embedding process based on NHGNN provided for an embodiment of the present invention;

[0063] Figure 5 This is a schematic diagram of the multi-Actor-Critic network structure provided in an embodiment of the present invention;

[0064] Figure 6 This is an example of a new workpiece arrival scheduling Gantt chart with the machine as the vertical axis, provided as an embodiment of the present invention; wherein, Figure 6 (a) is a graph showing the results obtained based on PPO without improvement. Figure 6 (b) is a graph showing the results obtained based on MPPO.

[0065] Figure 7 This is an example of a Gantt chart for scheduling new workpiece arrivals, with the worker as the vertical axis. Figure 7 (a) is a graph showing the results obtained based on PPO without improvement. Figure 7 (b) is a graph showing the results obtained based on MPPO. Detailed Implementation

[0066] In recent years, reinforcement learning has provided a novel approach to solving scheduling problems with dynamic events. Reinforcement learning models learn through interaction with the environment, requiring no pre-collected data and capable of estimating future state values, thus effectively addressing various dynamic events in real-world production workshops. Although more and more scholars are using reinforcement learning methods to solve shop floor scheduling problems, current research mainly focuses on Free-Site Scheduling (FJSP), with applications in the DRC-DFJSP field remaining rare. Reinforcement learning offers a new approach to shop floor scheduling problems in dynamic environments. By interacting with the environment and learning in real-time to optimize scheduling strategies, it effectively addresses various dynamic events in actual production. However, traditional reinforcement learning-based scheduling methods cannot effectively handle large state spaces, and the trained strategies are often only applicable to single-scale instances. Graph neural networks, on the other hand, can handle graphs of different sizes and effectively extract deep-level feature information, overcoming the limitations of matrix representation. Therefore, this invention employs a novel heterogeneous graph neural network for feature extraction in a dual-resource manufacturing system and uses reinforcement learning to train the scheduling model, enabling it to handle scheduling problems of different scales and exhibiting excellent generalization capabilities.

[0067] Example 1

[0068] This invention provides a dynamic scheduling method for dual-resource flexible job shops based on graph neural networks, the method comprising:

[0069] Step 1: Consider an integer programming model for the dynamic scheduling problem of flexible workshops under dual resource constraints, taking into account worker fatigue and skill level.

[0070] In step 1, the schematic diagram of the dynamic scheduling problem of the dual-resource-constrained flexible job shop is as follows: Figure 1 The diagram shows n workpieces J, which are processed by l workers W on m machines M. Workpiece J i ∈J includes n i An operation following a precedence relationship, n i This represents the i-th operation of the n-th workpiece, constituting J. i Operation set O i Process O ij The processing is carried out by its compatible machine set One of the machines M k And one worker W from the compatible worker set r To facilitate execution, this problem model is decomposed into three sub-tasks: (1) arranging the processing sequence of each process; (2) assigning machines to each process; and (3) assigning workers to each process. The problem constraints are as follows:

[0071] (1) At time point 0, all machines, workers and work tasks are available and the workers' fatigue value is 0.

[0072] (2) Each machine and worker can only process one process at a time;

[0073] (3) The processing time for each process is known, depending on the machine and the worker.

[0074] (4) There are priority constraints between the various processes of the same task, and only one process can be processed at a time;

[0075] (5) The process must not be interrupted during processing;

[0076] (6) Ignore machine malfunctions and non-machine processing operations;

[0077] (7) Auxiliary time (such as the time for transporting the workpiece) should be included in the processing time.

[0078] (8) All tasks have the same priority.

[0079] In step 1, considering worker fatigue and skill level specifically means:

[0080] Workers' skill levels are categorized into three levels: Basic I, Intermediate II, and Expert III, based on their work processes. Workers exhibit heterogeneity in skill combinations and proficiency; for the same work process, the same worker may have the same proficiency level, but different workers may have different levels; conversely, the same worker may have varying levels of proficiency across different work processes. Worker W r Processing step Oij Proficiency is determined by parameter PL ijr ={p I p II p III The description states that the corresponding processing cost per unit time is determined by the parameter WC. ijr ={[a I b I ], [a II b II ], [a III b III [Description]. Furthermore, fatigue among multi-skilled workers is unavoidable. Therefore, considering the impact of worker fatigue and skill level, the actual processing time for each process is as follows:

[0081]

[0082] Where, p ijkr For worker W r In machine M k Upper processing step O ij Actual processing time, p ijk For O ij In M k The preset processing time on the screen FL is a parameter representing the degree of influence of fatigue on processing time. r (t) represents W at time t. r Fatigue level, PL ijr For W r Processing O ij Proficiency level.

[0083] In step 1, the model optimization scheduling objective is:

[0084] Traditional dynamic scheduling algorithms often optimize for a single objective. This invention optimizes for both maximum completion time and total cost simultaneously, achieving multi-objective optimization. The objective function is defined as follows:

[0085] (1) Minimize maximum completion time: To improve the production efficiency of the workshop, it is necessary to shorten the production cycle of orders. Minimize the maximum completion time of the workpiece as the first objective function f1, as shown below:

[0086]

[0087] Among them, C i Indicates the completion time of workpiece i;

[0088] (2) Minimize total cost: To improve the company's market competitiveness, it is necessary to control the total production cost, i.e., the total cost of machines and workers. Minimize it as the second objective function f2, as shown below:

[0089]

[0090] Among them, MC k Indicates machine M k The unit time processing cost, WC ijr Worker W r Processing step O ij The unit time cost, taking its proficiency PL ijr The corresponding interval, x ijkr As a binary variable, when process O ij In machine M k Worker W r p is 1 during processing and 0 otherwise. ijkr For process O ij The actual processing time.

[0091] Step 2: Based on real-time data of production factors such as workpieces, machines, and workers, construct a DRC-DFJSP heterogeneous extraction diagram that represents the actual production status of the workshop.

[0092] In step 2, the real-time data of production factors specifically refers to:

[0093] (1) Real-time data of workpieces: scheduling status of each process, (estimated / actual) processing time of each process, number of candidate machines for each process, number of candidate workers for each process, (estimated / actual) start time of each process, number of workpieces without scheduled processes, and (estimated / actual) completion time of workpieces.

[0094] (2) Real-time machine data: machine unit time processing cost, number of machine processing steps, machine available time, machine utilization rate;

[0095] (3) Real-time worker data: worker fatigue level, number of processes that a worker can process, worker available time, and worker utilization rate;

[0096] (4) Workpiece-machine real-time data: preset processing time of each process on the machine;

[0097] (5) Workpiece-worker real-time data: worker's proficiency in each process and the unit time processing cost of each process.

[0098] In step 2, the DRC-DFJSP heterogeneous extraction diagram is specifically as follows:

[0099] DRC-DFJSP requires not only selecting the machines for the processing steps but also the workers operating those machines. To more intuitively represent the real-time production status, this invention proposes a heterogeneous disjunctive map H for DRC-DFJSP. t =(O tM, W, C, E t Add machine nodes and worker nodes and expand edge types. During dynamic scheduling, each state s t Both can be represented as H t O t With E t Dynamic changes, such as Figure 3 The diagram shown is a heterogeneous disjunction graph of a 3×3×2 DRC-DFJSP instance. The specific construction method is as follows:

[0100] (1) Construct the initial heterogeneous dissociation graph:

[0101] The initial heterogeneous dissociation diagram H0 at time 0 is as follows: Figure 3 As shown in (a): Construct process nodes, with the original vector being the real-time data of the workpiece. Let n be the total number of workpieces J, and i be the i-th workpiece J. i index; n i It is workpiece J i The total number of processes O included; j is the number of workpieces J. i The j-th process O ij The index; construct machine nodes, with the original vector being real-time machine data. Let m be the total number of machines M, and k be the number of the r-th machine M. k The index is used to construct worker nodes, with the original vector representing real-time worker data. Let l be the set of worker nodes, where l is the total number of workers W; and r is the r-th worker W. r The index. Add conjunctive arcs between operations to constrain the processing sequence between operations on the same workpiece, C = { <O ij O ij+1 >|1≤i≤n,1≤j≤n i -1) is the conjunctive arc set; OM arcs are added between the process and its compatible machine, the original vector is the workpiece-machine association real-time data, E O-M ∈E represents the set of undirected arcs between the work process and available machines; OW arcs are added between the work process and its compatible workers, with the original vector being the real-time work-worker correlation data, E O-W Let E be the set of undirected arcs between processes and available workers. The original vectors of various nodes and edges are shown in Table 1.

[0102] Table 1 Original Vectors

[0103]

[0104] (2) Update the heterogeneous dissociation graph:

[0105] If at time t, process O 11 With O 21Scheduled, updated heterogeneous disjunction graph H t As shown in 0(b), specifically: when a process is scheduled, only the process node O is retained. ij and the machine node M it was assigned k The arc between Worker Node W r The arc between Remove other OM and OW arcs in this process;

[0106] If a new job arrives at time t+1, the H of the heterogeneous disjunction graph will be updated. t+1 As shown in 0(c), specifically: upon arrival of one or more new jobs, the new jobs are... All processes included O ij Add to H t In the middle. And each process node O ij With its machine candidate set M ij All machine nodes M k Connect to generate arc Its worker candidate set W ij All worker nodes W r Connect to generate arc

[0107] Repeat this process until all processes are scheduled, generating a scheduling scheme diagram H as shown in 0(d). n .

[0108] Step 3: Construct a novel heterogeneous graph neural network (NHGNN) to extract features from the heterogeneous disjunctive graph constructed in Step 2 to obtain the global state embedding vector.

[0109] End-to-end reinforcement learning methods require extracting effective information from underlying raw features to achieve efficient agent decision-making. This invention proposes an NHGNN for extracting information from heterogeneous graphs H... t Extracting the original features and obtaining the global state embedding vector involves the following steps:

[0110] Building the embedding layer: This involves embedding machine nodes, worker nodes, and operation nodes in three stages. Taking a 2×3×2 instance as an example, the embedding process is as follows: Figure 4 As shown:

[0111] (1) Stage 1 machine node embedding, such as Figure 4 As shown in (a):

[0112] Step 1: Worker Node W r via edge With process node O ijConnect them, through cascading operations, W r The original vector zr is expanded to And through linear transformation Map it to d dimensions, and O ij Original vector x ij Through linear transformation Mapped to d dimensions. W r , for O ij The attention coefficient can be calculated as follows:

[0113]

[0114] Where, a∈R 2d N t (O ij _W) is O ij Adjacent W r The set of O. ij The self-attention coefficient can be calculated as follows:

[0115] e ij =LeakyReLU(a T [W O x ij ||W O x ij ])

[0116] The attention coefficient (α) is obtained by normalization using the softmax function. ijr α ij And use the sigmoid function σ to process node W. r and O ij The aggregation features are as follows:

[0117]

[0118] Obtain process node O that incorporates worker characteristics ijw Embedded vector x ijw ∈R d ;

[0119] Step 2: Integrating worker characteristics into process node O ijw via edge With machine node M k Connected, O ijw For point M k The attention coefficient can be calculated as follows:

[0120]

[0121] Wherein, linear matrix N t (M k _O) is Mk Adjacent operation node O ijw Set, u ijk Represents OM arc The original vector of M. k The self-attention coefficient can be calculated as follows:

[0122] e k =LeakyReLU(a T [W M y k ||W M y k ])

[0123] Normalization yields the attention coefficient (a) ijk α k ), Aggregation node O ijw and node M k The characteristics and process are as follows:

[0124]

[0125] Obtain machine node M k Embedding vector y′ k ∈R d ;

[0126] (2) In the second stage, worker node embedding, similar to machine nodes, worker nodes also integrate their second-order and first-order neighborhood feature information through two steps, such as... Figure 4 As shown in (b):

[0127] Step 1: Calculate machine node M k For process node O ij Attention coefficient f ijk and O ij Self-attention coefficient f ij Normalization yields (β) ijk ,β ij ), Aggregation node M k and node O ij Based on the characteristics, process node O that integrates machine segment characteristics is obtained. ijm Embedded vector x ijm ∈R d The process is as follows:

[0128]

[0129] Wherein, linear matrix N t (O ij _M) is O ij neighboring M k The set of O. ijThe self-attention coefficient can be calculated as follows:

[0130] f ij =LeakyReLU(a T [W O x ij ||W O x ij ])

[0131] Normalization process yields (β) ijk ,β ij Aggregation node M k and node O ij Features:

[0132]

[0133] Among them, R d β represents the dimension of the embedded vector after embedding. ijk It is a fusion machine node M k For process node O ij Attention coefficient f ijk The result after normalization; β ij It is process node O ij Self-attention coefficient f ij The result after normalization.

[0134] Step 2: Calculate the process node O that incorporates machine features. ijm For worker node W r Attention coefficient f ijr and W r Self-attention coefficient f r Normalization yields (β) ijr ,β r ), Aggregation node O ijm and node W r Obtain worker node W r Embedding vector z′ r ∈R d The process is as follows:

[0135]

[0136] Wherein, linear matrix N t (W r _O) is W r Adjacent process nodes O ijm Set. W r The self-attention coefficient can be calculated as follows:

[0137] f r =LeakyReLU(a T [WW z r ||W W z r ])

[0138] Normalization process yields (β) ijr ,β r Aggregation node O ijm and node W r Features:

[0139]

[0140] Where, β ijr It is a process node O that integrates machine characteristics. ijm For worker node W r Attention coefficient f ijr The result after normalization; β r It is worker node W r Self-attention coefficient f r The result after normalization. ijr This indicates the OW arc. The original vector.

[0141] (3) The third stage involves embedding process nodes. Building upon the embedding of machine nodes and worker nodes, process node embedding is performed, such as... Figure 4 As shown in (c):

[0142] Define 6 MLPs For the eigenvector x respectively ij-1 x ij x ij+1 y′ k and z′ r Processing is performed. Each MLP has a d-dimensional output, two d... ω The hidden layer and exponential linear unit (ELU) activation are performed. Then, the results are concatenated and subjected to ELU activation and MLP processing. Finally, node O is obtained. ij Embedded vector x′ ij ∈R d The embedding process is as follows:

[0143]

[0144] Where, x ij-1 x ij x ij+1 These are nodes O ij-1 O ij and O ij+1 The original vector. It is node O ij All adjacent machine nodes M′ kEmbedding vector y′ k The sum and average. It is node O ij All adjacent worker nodes W′ r Embedding vector z′ r The average of the sums.

[0145] Building the NHGNN model: The NHGNN model is constructed by stacking N embedding layers with identical structures but independently trainable parameters, to obtain the final embedding vectors. and Pooling three types of embedding vectors and concatenating them in parallel yields a heterogeneous disjunctive graph H. t global state embedding vector h t ∈R 3d The process is as follows:

[0146]

[0147] Among them, |O|, |M|, and |W| are H t The number of process nodes, machine nodes, and worker nodes.

[0148] Step 4: Input the global state embedding vector obtained in Step 3 into the scheduling model trained by the multi-action proximal policy optimization (MPPO) algorithm. The multi-action policy network outputs the scheduling actions for process selection, machine and worker allocation, until a complete scheduling scheme is generated.

[0149] In step 4, the environment model of the multi-action proximal policy optimization algorithm is specifically as follows:

[0150] Constructing an environment model based on MDP specifically includes:

[0151] State: At time step t, state s t The proposed heterogeneous disjunction graph represents the state of DRC-DFJSP, encompassing all static and dynamic features of DRC-DFJSP.

[0152] Action: At time step t, action a t (a o a m a w ) by process action a o ∈A o Machine action a m ∈A m and worker actions a w ∈A w Composition, A o It is a set of optional processes, A m It is action ao A set of compatible machines, A w It is action a o A compatible set of workers;

[0153] Transition: At time step t, the agent transitions based on state s t Take action a t Interacting with the environment to transition to a new state. t+1 And based on action a t Update heterogeneous dissociation graph H t+1 ;

[0154] Reward: r1 is the immediate reward for the maximum completion time, calculated based on the average machine utilization. n is the total number of workpieces J; m is the total number of machines M; n i It is workpiece J i The total number of processes O included; 1 is the total number of workers W; x ijkr Process O ij Whether by worker W r In machine M k If the processing is performed on the above, the value is 1; otherwise, it is 0. ijkr Process O ij Worker W r In machine M k The processing time for the above processing. Where CT(t) is the maximum actual completion time at time step t, and r1 takes the following values:

[0155]

[0156] r2 represents the immediate reward for total cost, and is used to calculate the total processing time for workers and machines. With total cost L c The ratio between (t) The values ​​of r2 are as follows:

[0157]

[0158] The weighted reward is r t =ω1r1+ω1r2, satisfying ω1+ω2=1 and ω1, ω2≥0.1. Each ω i After training for a certain number of times, save the ω. i The corresponding model parameters. Then, select ω with the smallest Euclidean distance. i+1 Continue training, repeating until all agent parameters corresponding to all weight vectors are saved;

[0159] Strategy: Multi-Action Strategy π θ (a o a m a w |st ) By process strategy Machine strategy and worker strategy composition;

[0160] In step 4, the scheduling model specifically refers to:

[0161] The scheduling model consists of three Actor networks and one Critic network, and the overall network structure is as follows: Figure 6 As shown:

[0162] In state st, each feasible process action a is recorded. o ∈A o corresponding embedding vector and global state embedding features h t Cascade, and use two d θ Hidden layers of dimensionality and tanh activation Processing, we get a o The probability distribution of choice P(a) o s t Normalization yields the process steps.

[0163] In state s t And select process action a o Under the condition that each feasible machine action a m ∈A m corresponding embedding vector and h t Cascade, and use two d θ Hidden layers of dimensionality and tanh activation Processing, we get a m The probability distribution of choice P(a) m s t _a o Normalization yields the machine policy.

[0164] In state s t And select process action a o and machine action a m Under the condition that each feasible worker action a w ∈A w corresponding embedding vector and h t Cascade, and use two d θ Hidden layers of dimensionality and tanh activation Processing, we get a w The probability distribution of choice P(a) w s t _a o _am Normalization yields worker strategies

[0165] Using two d φ Hidden layers of dimensionality and MLP with tanh activation φ Process h t , obtain the Critic network v φ .

[0166] In step 4, the multi-action proximal policy optimization (MPPO) algorithm is specifically as follows:

[0167] Proximal policy optimization (PPO) is an advanced policy gradient method within an Actor-Critic framework, widely used for both discrete and continuous control tasks. However, DRC-DFJSP is a multi-action task. To address this characteristic, this invention proposes a multi-action proximal policy optimization (MPPO) algorithm. The MPPO algorithm is used to pre-train a scheduling model offline. The specific steps are as follows:

[0168] Calculate the advantage function Where γ is the discount factor, t is the current time step, and v φ (s t ) is the Critic network based on state s t The obtained value function value is T << |O|;

[0169] In each iteration of the training process, the three Actors interact with the environment to collect experience quadruples (s t a t r t s t+1 The Actor network is trained and the Critic network is updated using the collected samples.

[0170] (1) Training the Actor Network

[0171] Each Actor network is trained to maximize its respective loss function. Among them, c p =1 and c e =0.01, representing the truncation hyperparameter and the entropy coefficient hyperparameter, respectively. It is about tailoring the proxy target. The entropy targets are as follows:

[0172]

[0173] in, It is a target strategy With the behavioral strategy used for sampling The probability ratio, ∈ (0, 1) is the clipping parameter.

[0174] (2) Update the Critic network

[0175] The Critic network aims to minimize the mean squared error (MSE) L. MSE (φ) is updated:

[0176] L MSE (φ)=E t [MSE(r t v φ (s t ))]

[0177] In step 4, the scheduling model trained by the multi-action proximal policy optimization (MPPO) algorithm specifically refers to:

[0178] Initialize the MPPO algorithm parameters and randomly generate B instances. The Actor interacts with the environment to schedule each instance and collects experience quadruples (s). t a t r t s t+1 The quadruple is stored in the experience replay pool until all tasks have been scheduled. Then, the probability ratio is calculated. Target Actor Error and Critic error L MSE (φ), and update the network E using the gradient ascent method. u Next. Finally, the target Actor parameters... Copy to behavior Actor parameter Complete one iteration. Training has a total of E iterations. t Each iteration e val Validation is performed on an independent validation set each time, e. rep Replace a batch of B instances each time until the convergence criterion is met, complete the training, and obtain the optimized scheduling model.

[0179] In step 4, the multi-action strategy network outputs scheduling actions for process selection, machine and worker allocation, until a complete scheduling scheme is generated, specifically as follows:

[0180] An optimization scheduling model based on the MPPO algorithm is used for workshop production optimization scheduling. In actual production, the global state is embedded into a vector and input into the optimization scheduling model trained offline by the MPPO algorithm. The multi-action strategy network outputs the scheduling actions of process selection, machine and worker allocation, until a complete scheduling scheme is generated.

[0181] Step 5: When a dynamic event occurs, repeat steps 2-4 until the production task is completed.

[0182] In step 5, the dynamic event specifically refers to:

[0183] Dynamic events involve the random arrival of new workpieces, with arrival times approximately following a Poisson distribution. New workpieces are dynamically scheduled by mixing them with unprocessed processes. Processes in progress cannot be terminated, and already processed processes are no longer scheduled.

[0184] Example 2

[0185] A dynamic scheduling system for a dual-resource flexible job shop based on graph neural networks, the system comprising:

[0186] The virtual-physical interaction module is used for bidirectional interaction between the physical workshop and the virtual workshop. In this embodiment, the physical workshop transmits real-time data of production factors to the virtual workshop through the virtual interaction module, and the virtual workshop sends production optimization scheduling plans to the physical workshop through the virtual interaction module.

[0187] The physical workshop is used to execute scheduling schemes and collect real-time data to support dynamic scheduling of the virtual workshop, including:

[0188] The data acquisition module is used to collect real-time data on production elements in the workshop.

[0189] The real-time data of production factors collected are specifically as follows:

[0190] Real-time data for each workpiece: scheduling status of each process, (estimated / actual) processing time of each process, number of candidate machines for each process, number of candidate workers for each process, (estimated / actual) start time of each process, number of unscheduled processes for the workpiece, and (estimated / actual) completion time of the workpiece.

[0191] Real-time machine data: machine processing cost per unit time, number of processes that the machine can process, machine available time, and machine utilization rate;

[0192] Real-time worker data: worker fatigue level, number of processes a worker can complete, worker available time, and worker utilization rate;

[0193] Workpiece-machine real-time data: preset processing time for each process on the machine;

[0194] Workpiece-worker real-time data: worker proficiency in each process and processing cost per unit time for each process.

[0195] The manufacturing execution module is used to organize production by executing the scheduling plan. In this embodiment, the manufacturing execution module carries out production according to the scheduling plan issued by the virtual workshop.

[0196] Virtual workshops are used to respond to dynamic events and develop scheduling plans to guide actual production in the physical workshop, including:

[0197] The state characterization module is used to construct the heterogeneous disjunction graph of DRC-DFJSP.

[0198] DRC-DFJSP requires not only selecting the machines for the processing steps but also the workers operating those machines. To more intuitively represent the real-time production status, this invention proposes a heterogeneous disjunctive map H for DRC-DFJSP. t =(O t M, W, C, E t Add machine nodes and worker nodes and expand edge types. During dynamic scheduling, each state s t Both can be represented as H t O t With E t Dynamic changes such as Figure 3 The image shown is a 3×3×2 DRC-DFJSP instance.

[0199] (1) Construct the initial heterogeneous dissociation graph:

[0200] The initial heterogeneous dissociation diagram H0 at time 0 is as follows: Figure 3 As shown in (a): Construct process nodes, with the original vector being the real-time data of the workpiece. This represents a set of process nodes; machine nodes are constructed, with the original vectors representing real-time machine data. For the machine node set, construct worker nodes; the original vectors represent real-time worker data. For the worker node set; add conjunctive arcs between operations to constrain the processing sequence between operations on the same workpiece, C = { <O ij O ij+1 >||1≤i≤n,1≤j≤n i -1} is the conjunctive arc set; OM arcs are added between the process and its compatible machine, the original vector is the workpiece-machine association real-time data, E O-M ∈E represents the set of undirected arcs between the work process and available machines; OW arcs are added between the work process and its compatible workers, with the original vector being the real-time work-worker correlation data, E O-W Let E be the set of undirected arcs between processes and available workers. The original vectors of various nodes and edges are shown in Table 1.

[0201] Table 1 Original Vectors

[0202]

[0203] (2) Update the heterogeneous dissociation graph:

[0204] If at time t, process O 11 With O 21 Scheduled, updated heterogeneous disjunction graph H t As shown in 0(b), specifically: when a process is scheduled, only the process node O is retained. ij and the machine node M it was assigned k The arc between Worker Node W r The arc between Remove other OM and OW arcs in this process;

[0205] If a new job arrives at time t+1, the H of the heterogeneous disjunction graph will be updated. t+1 As shown in 0(c), specifically: upon arrival of one or more new jobs, the new jobs are... All processes included O ij Add to H t In the middle. And each process node O ij With its machine candidate set M ij All machine nodes M k Connect to generate arc Its worker candidate set W ij All worker nodes W r Connect to generate arc

[0206] The feature extraction module is used to construct NHGNN to extract global state embedding vectors from heterogeneous disjunctive graphs.

[0207] Building the embedding layer: This involves embedding machine nodes, worker nodes, and operation nodes in three stages. Taking a 2×3×2 instance as an example, the embedding process is as follows: Figure 4 As shown:

[0208] (1) Stage 1 machine node embedding, such as Figure 4 As shown in (a):

[0209] Step 1: Worker Node W r via edge With process node O ij Connect them, through cascading operations, W r Original vector z r Expand to And through linear transformation Map it to d dimensions, and O ijOriginal vector x ij Through linear transformation Mapped to d dimensions. W r For O ij The attention coefficient can be calculated as follows:

[0210]

[0211] Where, a∈R 2d N t (O ij _W) is O ij W r Set. O ij The self-attention coefficient can be calculated as follows:

[0212] e ij =LeakyReLU(a T [W O x ij ||W O x ij ])

[0213] The attention coefficient (α) is obtained by normalization using the softmax function. ijr α ij And use the sigmoid function σ to process node W. r and O ij The aggregation features are as follows:

[0214]

[0215] Obtain process node O that incorporates worker characteristics ijw Embedded vector x ijw ∈R d ;

[0216] Step 2: Integrating worker characteristics into process node O ijw via edge With machine node M k Connected, O ijw For point M k The attention coefficient can be calculated as follows:

[0217]

[0218] Wherein, linear matrix N t (M k _O) is M k Adjacent operation node O ijw Set. M k The self-attention coefficient can be calculated as follows:

[0219] e k=LeakyReLU(a T [W M y k ||W M y k ])

[0220] Normalization yields the attention coefficient (α) ijk α k ), Aggregation node O ijw and node M k The characteristics and process are as follows:

[0221]

[0222] Obtain machine node M k Embedding vector y′ k ∈R d ;

[0223] (2) In the second stage, worker node embedding, similar to machine nodes, worker nodes also integrate their second-order and first-order neighborhood feature information through two steps, such as... Figure 4 As shown in (b):

[0224] Step 1: Calculate machine node M k For process node O ij Attention coefficient f ijk and O ij Self-attention coefficient f ij Normalization yields (β) ijk ,β ij ), Aggregation node M k Nodes and points O ij Based on the characteristics, process node O that integrates machine segment characteristics is obtained. ijm Embedded vector x ijm ∈R d The process is as follows:

[0225]

[0226] Wherein, linear matrix N t (O ij _M) is O ij neighboring M k A set;

[0227] Step 2: Calculate the process node O that incorporates machine features. ijm For worker node W r Attention coefficient f ijr and W r Self-attention coefficient f r Normalization yields (β) ijr ,βr ), Aggregation node O ijm and node W r Obtain worker node W r Embedding vector z′ r ∈R d The process is as follows:

[0228]

[0229] Where the linear matrix N t (W r _O) is W r O ijm A set of.

[0230] (3) The third stage involves embedding process nodes. Building upon the embedding of machine nodes and worker nodes, process node embedding is performed, such as... Figure 4 As shown in (c):

[0231] Define 6 MLPs For the eigenvector x respectively ij-1 x ij x ij+1 y′ k and z′ r Processing is performed. Each MLP has a d-dimensional output, two d... ω The hidden layer and exponential linear unit (ELU) activation are performed. Then, the results are concatenated and subjected to ELU activation and MLP processing. Finally, node O is obtained. ij Embedded vector x′ ij ∈R d The embedding process is as follows:

[0232]

[0233] Where, x ij-1 x ij x ij+1 These are nodes O ij-1 O ij and O ij+1 The original vector. It is node O ij All adjacent machine nodes M′ r Embedding vector y′ k The sum and average. It is node O ij All adjacent worker nodes W′ r Embedding vector z′ r The average of the sums.

[0234] Building the NHGNN model: The NHGNN model is constructed by stacking N embedding layers with identical structures but independently trainable parameters, to obtain the final embedding vectors. and Pooling three types of embedding vectors and concatenating them in parallel yields a heterogeneous disjunctive graph H. t global state embedding vector h t ∈R 3d The process is as follows:

[0235]

[0236] Among them, |O|, |M|, and |W| are H t The number of process nodes, machine nodes, and worker nodes.

[0237] The environment model module is used to construct the reinforcement learning environment model for DRC-DFJSP.

[0238] Constructing a reinforcement learning environment model based on MDP specifically includes:

[0239] State: At time step t, state s t The proposed heterogeneous disjunction graph represents the state of DRC-DFJSP, encompassing all static and dynamic features of DRC-DFJSP.

[0240] Action: At time step t, action a t (a o a m a w ) by process action a o ∈A o Machine action a m ∈A m and worker actions a w ∈A w Composition, A o It is a set of optional processes, A m It is action a o A set of compatible machines, A w It is action a o A compatible set of workers;

[0241] Transition: At time step t, the agent transitions based on state s t Take action a t Interacting with the environment to transition to a new state. t+1 And based on action a t Update heterogeneous dissociation graph H t+1 ;

[0242] Reward: r1 is the immediate reward for the maximum completion time, calculated based on the average machine utilization. Where CT(t) is the maximum actual completion value at time step t, and r1 takes the following values:

[0243]

[0244] r2 represents the immediate reward for total cost, and is used to calculate the total processing time for workers and machines. With total cost L c The ratio between (t) The values ​​of r2 are as follows:

[0245]

[0246] The weighted reward is r t =ω1r1+ω1r2, satisfying ω1+ω2=1 and ω1, ω2≥0.1. Each ω i After training for a certain number of times, save the ω. i The corresponding model parameters. Then, select ω with the smallest Euclidean distance. i+1 Continue training, repeating until all agent parameters corresponding to all weight vectors are saved;

[0247] Strategy: Multi-Action Strategy π θ (a o a m a w |s t ) By process strategy Machine strategy and worker strategy composition;

[0248] The scheduling model module is used to train a scheduling model using the MPPO algorithm and generate an optimized scheduling model.

[0249] Scheduling model:

[0250] The scheduling model consists of three Actor networks and one Critic network, and the overall network structure is as follows: Figure 6 As shown:

[0251] In state s t Each feasible process action a o ∈A o corresponding embedding vector and global state embedding features h t Cascade, and use two d θ Hidden layers of dimensionality and tanh activation Processing, we get a o The probability distribution of choice P(a) o s t Normalization yields the process steps.

[0252] In state s t And select process action a o Under the condition that each feasible machine action a m ∈A m corresponding embedding vector and h t Cascade, and use two d θ Hidden layers of dimensionality and tanh activation Processing, we get a m The probability distribution of choice P(a) m s t _a o Normalization yields the machine policy.

[0253] In state s t And select process action a o and machine action a m Under the condition that each feasible worker action a w ∈A w corresponding embedding vector and h t Cascade, and use two d θ Hidden layers of dimensionality and tanh activation Processing, we get a w The probability distribution of choice P(a) w s t _a o _a m Normalization yields worker strategies

[0254] Using two d φ Hidden layers of dimensionality and MLP with tanh activation φ Process h t , obtain the Critic network v φ .

[0255] MPPO algorithm:

[0256] Proximal policy optimization (PPO) is an advanced policy gradient method within an Actor-Critic framework, widely used for both discrete and continuous control tasks. However, DRC-DFJSP is a multi-action task, making direct application of the PPO algorithm inapplicable. This invention proposes a multi-action proximal policy optimization (MPPO) algorithm, utilizing offline pre-training of the scheduling model. The specific steps are as follows:

[0257] Calculate the advantage function Where γ is the discount factor, t is the current time step, and v φ (s t ) is the Critic network based on state s t The obtained value function value is T << |O|;

[0258] In each iteration of the training process, the three Actors interact with the environment to collect experience quadruples (s t a t r t s t+1 The Actor network is trained and the Critic network is updated using the collected samples.

[0259] (1) Training the Actor Network

[0260] Each Actor network is trained to maximize its respective loss function. Among them, c p =1 and c e =0.01, representing the truncation hyperparameter and the entropy coefficient hyperparameter, respectively. It is about tailoring the proxy target. The entropy targets are as follows:

[0261]

[0262] in, It is a target strategy With the behavioral strategy used for sampling The probability ratio, ∈ (0, 1) is the clipping parameter.

[0263] (2) Update the Critic network

[0264] The Critic network aims to minimize the mean squared error (MSE) L. MSE (φ) is updated:

[0265] L MSE (φ)=E t [MSE(r t v φ (s t ))]

[0266] MPPO algorithm offline training scheduling model:

[0267] Initialize the MPPO algorithm parameters and randomly generate B instances. The Actor interacts with the environment to schedule each instance and collects experience quadruples (s). t a t rt s t+1 The quadruple is stored in the experience replay pool until all tasks have been scheduled. Then, the probability ratio is calculated. Target Actor Error and Critic error L MSE (φ), and update the network E using the gradient ascent method. u Next. Finally, the target Actor parameters... Copy to behavior Actor parameter Complete one iteration. Training has a total of E iterations. t Each iteration e val Validation is performed on an independent validation set each time, e. rep Replace a batch of B instances each time until the convergence criterion is met, complete the training, and obtain the optimized scheduling model.

[0268] The optimized scheduling module is used to generate scheduling schemes using the optimized scheduling model.

[0269] An optimization scheduling model based on the MPPO algorithm is used for workshop production optimization scheduling. In actual production, the global state is embedded into a vector and input into the optimization scheduling model trained offline by the MPPO algorithm. The multi-action strategy network outputs the scheduling actions of process selection, machine and worker allocation, until a complete scheduling scheme is generated.

[0270] When dynamic events occur in actual production, an optimized scheduling model is used to generate a scheduling scheme, which is then issued to the manufacturing execution module for production until completion. In the actual production data of an aerospace equipment manufacturing company, the company has 6 machines and 5 workers operating them. The initial number of parts processed is 10, and subsequent additions of 10 parts are also made. With the feature extraction process consistent, scheduling solutions are obtained using both the unmodified PPO method and the MPPO method, as shown below. Figure 6 , 7 The results in the figure show that the white boxes represent the original workpieces in the workshop, and the black boxes represent newly arrived workpieces. In comparison, the improved MPPO algorithm has a completion time of 214 hours and a cost of 2375 yuan, while the unimproved PPO algorithm has a completion time of 246 hours and a cost of 2485 yuan. Through feature extraction and optimized MPPO scheduling, production efficiency was significantly improved, and both completion time and cost decreased, with completion time reduced by 13.20% and cost reduced by 4.42%.

[0271] Compared with existing technologies, the advantages of this invention are as follows: This invention solves the problems of low accuracy of problem models, poor practicality of scheduling schemes, and difficulty in handling dynamic events in existing dual-resource manufacturing system scheduling methods. Because there is still a lack of research considering worker characteristics, the problem model differs from actual production, resulting in scheduling schemes that still lack practicality. Although traditional scheduling methods can obtain exact solutions, their computational complexity is high, and the solution time increases exponentially, making it impossible to find satisfactory solutions for large-scale dynamic instances in a finite time to respond to dynamic events.

[0272] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A dynamic scheduling method for dual-resource flexible job shops based on graph neural networks, characterized in that, include: Step 1: Establish and train a dynamic scheduling model for a flexible workshop with dual resources, taking into account worker fatigue and skill level; Step 2: Using real-time production factor data, construct a heterogeneous disjunctive diagram of a dynamic flexible work workshop under dual resource constraints to characterize the actual production status of the workshop. Step 3: Use a novel heterogeneous graph neural network (NHGNN) to extract features from the heterogeneous disjunctive graph and obtain the global state embedding vector; Step 4: Input the global state embedding vector into the dual-resource flexible workshop dynamic scheduling model and record the output results; When a dynamic event occurs, repeat steps 2 to 4 until the production task is completed. Step 3 includes: constructing an embedding layer, embedding machine nodes, worker nodes and operation nodes in sequence in the embedding layer; constructing an NHGNN model by stacking N embedding layers, obtaining the final embedding vectors of the three types of nodes through the NHGNN model, where N is a positive integer, then calculating the average value of the three types of embedding vectors and concatenating them to obtain the global state embedding vector. During the machine node embedding process, worker nodes in the second-order neighborhood are embedded into operation nodes in the first-order neighborhood, and operation nodes in the first-order neighborhood are embedded into machine nodes that serve as target nodes. During the worker node embedding process, machine nodes in the second-order neighborhood are embedded into operation nodes in the first-order neighborhood, and operation nodes in the first-order neighborhood are embedded into worker nodes that serve as target nodes. During the operation node embedding process, the vectors of each node are processed and concatenated through various MLPs. The vectors of each node include: the original vector of an operation node, the original vectors of the preceding and following operation nodes of this operation node, the embedding vector of the machine node adjacent to this operation node, and the embedding vector of the worker node adjacent to this operation node. It also includes: training the dual-resource flexible workshop dynamic scheduling model using the Multi-Action Proximal Policy Optimization (MPPO) algorithm; wherein, based on the features of the embedding vectors corresponding to feasible process actions and the global state embedding vectors, a process Actor network is constructed; based on the features of the embedding vectors corresponding to feasible machine actions and the global state embedding vectors, a machine Actor network is constructed; based on the features of the embedding vectors corresponding to feasible worker actions and the global state embedding vectors, a worker Actor network is constructed; and based on the features of the global state embedding vectors, a Critic network is constructed.

2. The method according to claim 1, characterized in that, In the dynamic scheduling model of the dual-resource flexible workshop, the impact of worker fatigue and skill level on process processing time is expressed as follows: ,in, For workers In the machine Upper processing steps The actual processing time for exist The preset processing time on the screen This parameter represents the degree of influence of fatigue on processing time. for time Fatigue parameters, for implement The proficiency parameter, It is the first workpiece index, It is a workpiece The Steps index, It is the first machine index, It is the first worker The index.

3. The method according to claim 1, characterized in that, The types of real-time production factor data include: real-time workpiece data, real-time machine data, real-time worker data, real-time workpiece-machine associated data, and real-time workpiece-worker associated data. The real-time worker data includes: worker fatigue level, number of processes a worker can process, available worker time, and worker utilization rate; the real-time workpiece-machine association data includes: the preset processing time of each process on the machine; and the real-time workpiece-worker association data includes: the worker's proficiency in processing each process and the processing cost per unit time for each process.

4. The method according to claim 3, characterized in that, The construction of the heterogeneous disjunctive graph of the dynamic flexible workshop under dual resource constraints, used to characterize the actual production state of the workshop, includes: Construct process nodes, where the original vector of each process node is the real-time data of the workpiece; Construct machine nodes, where the original vector of the machine node is real-time machine data; Construct worker nodes, where the original vector of each worker node is real-time worker data; Add conjunctive arcs between process nodes to constrain the processing sequence between processes of the same workpiece; Add between process nodes and their compatible machine nodes Arc, the The original vector of the arc is the workpiece-machine association real-time data; Add between process nodes and their compatible worker nodes Arc, the The original vector of the arc is the workpiece-worker correlation real-time data; Then, the heterogeneous disjunction graph was continuously updated, including: when a process is scheduled, only the arcs between this process and the machine it is assigned to, and between workers, are retained, and then the other arcs between this process are removed. arc and Arc; Repeat the process of updating the heterogeneous disjunction graph until all processes are scheduled, generating a scheduling scheme graph.

5. The method according to claim 1, characterized in that, The dynamic events include: new workpieces arriving randomly, wherein the arrival time of the new workpieces approximately follows a Poisson distribution; the arriving new workpieces are dynamically scheduled together with unprocessed processes, while processes in progress cannot be terminated and already processed processes are no longer scheduled.

6. The method according to any one of claims 1-5, characterized in that, The method is executed by a dual-resource flexible job shop dynamic scheduling system based on graph neural networks, the system comprising: The virtual-physical interaction module is used for two-way interaction between the physical workshop and the virtual workshop; The physical workshop is used to execute scheduling plans and collect real-time data to support dynamic scheduling of the virtual workshop. The physical workshop includes: a data acquisition module for collecting real-time data of workshop production elements; and a manufacturing execution module for executing scheduling plans to organize actual production. The virtual workshop is used to respond to dynamic events and formulate scheduling schemes to guide actual production in the physical workshop. The virtual workshop includes: a state representation module for constructing a heterogeneous disjunctive graph of the dynamic flexible work workshop; a feature extraction module for constructing an NHGNN to extract global state embedding vectors from the heterogeneous disjunctive graph; an environment model module for constructing a reinforcement learning environment model of the dynamic flexible work workshop; a scheduling model module for training a scheduling model using the MPPO algorithm to generate an optimized scheduling model; and an optimized scheduling module for generating a scheduling scheme using the optimized scheduling model.

Images