Multi-view graph neural network-based replacement flow shop scheduling method and system

Abstract

The invention belongs to the technical field of flow shop scheduling, and discloses a replacement flow shop scheduling method and system based on a multi-view graph neural network, and the method comprises the steps: defining a problem, and constructing a target function and a constraint condition; the method comprises the steps of obtaining a time feature vector of a problem based on the same job node, constructing a node sparse graph, obtaining node matrix embedding based on the time feature vector and the node sparse graph, aggregating the node matrix embedding, obtaining global node embedding, and calculating graph embedding according to the global node embedding; then, node context embedding is constructed, node probability distribution is calculated according to the node context embedding, and jobs are selected according to the node probability distribution until all the jobs are completed; and optimizing model parameters to obtain a final scheduling scheme. According to the method, initial setting time is considered, a multi-feature representation method is provided, the feature information of the problem is extracted and embedded by using a multi-view graph neural network, and stability and high efficiency can be kept on problems of different scales.

Description

Technical Field

[0001] The present invention relates to the technical field of flow shop scheduling, and in particular to a replacement flow shop scheduling method and system based on a multi-view graph neural network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Scheduling is a key issue in modern manufacturing. Effective scheduling methods can improve production efficiency and resource utilization. The permutation flow shop scheduling problem (PFSP) is a classic scheduling problem and one of the most comprehensively studied scheduling problems. The problem is to obtain the best job processing sequence within the scheduling rules to minimize the maximum completion time (makespan), and the same job sequence is followed on all machines. Solving the PFSP with only one machine is simple, however, the PFSP with more than two machines has been proven to be an NP-hard combinatorial optimization problem. With the improvement of economy and production technology, the requirements for manufacturing are getting higher and higher. In the actual production process, it is usually necessary to consider a setup time, which is the time when the machine is maintained or the mold is replaced after the processing operation. When this time depends on the current processing job and the previous job, it is the sequence-dependent setup time (SDST).

[0004] The permutation flow shop scheduling problem with sequence-dependent setup time (PFSP-SDST) is the same as PFSP, usually with maximum completion time, total flow time, and total weighted time as the objectives. In order to solve this problem, a large number of algorithms have been proposed. The traditional methods used to solve this problem can be summarized into three categories: exact methods, heuristic methods, and meta-heuristic methods. Exact methods are used to solve this problem, which include mixed integer programming (MILP) models, complete enumeration, and branch-and-bound algorithms. When solving large-scale problems, the computational time of exact methods is often unacceptable, and it is difficult to obtain the optimal solution. Researchers prefer heuristic methods because they are simpler and faster. For example, the classic Nawaz-Enscore-Ham (NEH) heuristic is competitive in minimizing the maximum completion time. In complex practical problems, in order to avoid heuristic methods from falling into local optimality, many meta-heuristic methods have been proposed, which are dedicated to solving PFSP with various constraints. Classic meta-heuristic methods include genetic algorithms (GA), simulated annealing algorithms (SA), and ant colony optimization (ACO) algorithms.

[0005] With the development of machine learning, many researchers have tried to apply learning-based methods to PFSP, which is a novel and challenging field. Learning-based methods eliminate the overhead of manually designing scheduling algorithms to some extent. For PFSP, learning-based methods are often composed of an encoding-decoding module and a training module, using different networks as encoders and decoders, and reinforcement learning (RL) or deep reinforcement learning (DRL) algorithms for training. RL converges linear functions through step-by-step training. Pointer networks (PN) and recurrent neural networks (RNN) are used to solve problems. Graph neural networks have powerful feature information extraction capabilities and can embed information efficiently. Some studies use graph neural networks (GNNs) as encoders and show good performance. The application of DRL scheduling problems often uses deep neural networks (DNNs) to parameterize strategies, continuously optimize the decision-making process, and gradually obtain high-quality solutions. Generally speaking, this type of method embeds the characteristics of the problem through GNNs, and then the policy network constructs a scheduling solution based on the embedded data. Recently, DRL-based methods have been increasingly applied to PFSP and have made varying degrees of progress. However, current research often targets simple PFSPs and rarely considers other constraints of the problem, such as setup time. However, setup time is unavoidable in many actual production environments, such as the process of processing cakes of different sizes requires switching between different molds. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a replacement flow shop scheduling method and system based on a multi-view graph neural network, takes into account the constraints of setting time, proposes a multi-feature representation method, and uses a multi-view graph neural network to extract and embed the feature information of the problem, which can maintain stability and efficiency on problems of different scales.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a replacement flow shop scheduling method based on a multi-view graph neural network, comprising the following steps:

[0009] Define the permutation flow shop scheduling problem with sequence-dependent setup times and construct the desired objective function in the actual production process and the constraints that satisfy the problem;

[0010] Based on the same job node, extract features of the problem to obtain time feature vectors and time feature matrices, build a node sparse graph based on the time feature matrix, input the time feature vectors and node sparse graph into the graph attention neural network to obtain node matrix embedding, aggregate the node matrix embedding to obtain global node embedding, and calculate graph embedding based on the global node embedding;

[0011] Build node context embedding based on global node embedding and graph embedding, calculate node probability distribution based on node context embedding, and select jobs based on node probability distribution until all jobs are completed;

[0012] Define the loss function, optimize the model parameters, and obtain the final scheduling plan.

[0013] As an optional implementation, the time feature vector includes a processing time feature vector, a sequence-related setting time feature vector and an initial setting time feature vector;

[0014] The time feature matrix includes a processing time feature matrix, a sequence-related setting time feature matrix and an initial setting time feature matrix.

[0015] As an optional implementation, the node matrix embeddings are aggregated to obtain a global node embedding, specifically: the weights of different node matrix embeddings are learned using an attention method, and the node matrix embeddings are integrated into a global node embedding according to the learned weight values.

[0016] As an optional implementation, the desired objective function in the actual production process is: the job sequence with the minimum maximum completion time.

[0017] As an optional implementation, the constraints that satisfy the problem are:

[0018]

[0019] C max ≥C m,j ,j∈{1,2,…,n}

[0020]

[0021] C i,j ≥0,i∈{1,2,…,m},j∈{1,2,…,n}

[0022] Among them, z j,k If artifact j comes before artifact k, it is 1, otherwise it is 0. j and k represent the indices of the artifacts, n represents the number of artifacts, and C max represents the maximum completion time, C i,j represents the completion time of job j on machine i, x i,j represents the processing time of job j on machine i, s i,j,k It represents the setup time to switch from processing job j to processing job k on machine i.

[0023] As an optional implementation, the PPO algorithm is used to train the model. The PPO algorithm adopts an actor-critic structure, calculates the entropy loss function of the actor network and the mean square error loss of the critic network respectively, and uses the two to calculate the total loss function.

[0024] In a second aspect, the present invention provides a replacement flow shop scheduling system based on a multi-view graph neural network, comprising:

[0025] A model building module is configured to: define a permutation flow shop scheduling problem with sequence-dependent setup times, and construct an objective function desired in an actual production process and constraints that satisfy the problem;

[0026] The multi-view representation module is configured to: extract features of the problem based on the same job node, obtain a time feature vector and a time feature matrix, construct a node sparse graph based on the time feature matrix, input the time feature vector and the node sparse graph into the graph attention neural network, obtain node matrix embedding, aggregate the node matrix embedding, obtain a global node embedding, and calculate the graph embedding based on the global node embedding;

[0027] The decision module is configured to: construct node context embedding based on global node embedding and graph embedding, calculate node probability distribution according to the node context embedding, and select jobs according to the node probability distribution until all jobs are completed;

[0028] The model training module is configured to: define the loss function, optimize the model parameters, and obtain the final scheduling solution.

[0029] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is completed.

[0030] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0031] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present disclosure proposes a replacement flow shop scheduling method and system based on a multi-view graph neural network, which represents the processing time, sequence-related setup time and initial setup time of the PFSP-SDST problem as different features, and represents them from different angles. They have the same set of job nodes, and a disjunctive graph is constructed for them to assist multi-layer GNN embedding information. A new MGNN is proposed for PFSP-SDST to handle multi-feature representation. MGNN can be divided into single-view embedding and multi-view aggregation. In the single-view embedding part, GAT is improved to consider the edge information of the node. In the multi-view aggregation part, the embedded information of different features is aggregated into a global node representation, and a more accurate feature representation is used in the decision-making stage. The entire solution process is formulated as an MDP, and the method can maintain stability and efficiency on problems of different scales.

[0034] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0036] Figure 1 A data processing flow chart of a replacement flow shop scheduling method based on a multi-view graph neural network provided in Example 1 of the present invention;

[0037] Figure 2 A framework diagram of a replacement flow shop scheduling method based on a multi-view graph neural network provided in Example 1 of the present invention;

[0038] Figure 3 A Gantt chart of an example of the permutation flow shop scheduling problem of the present invention;

[0039] Figure 4 A schematic diagram of a Markov decision process provided in Example 1 of the present invention;

[0040] Figure 5 A schematic diagram of constructing a node sparse graph based on a time feature matrix provided in Example 1 of the present invention;

[0041] Figure 6 A schematic diagram of obtaining global node embedding based on a time feature matrix provided in Example 1 of the present invention;

[0042] Figure 7The graphs are loss function graphs under different parameter values ​​during the model training process of the present invention, wherein (a) is a graph showing the effect of the shear coefficient on the loss function, and (b) is a graph showing the effect of the learning rate on the loss function;

[0043] Figure 8 The convergence curve of the deep reinforcement learning (DRL) method provided in Example 1 of the present invention;

[0044] Fig. 9 are the gap curves of different methods;

[0045] Fig.10 The gap value of different methods in large-scale generation of datasets;

[0046] Fig.11 It is the gap value of different methods on the benchmark dataset, where (a) is the box plot of different methods on the SDST10 dataset, and (b) is the box plot of different methods on the SDST10 dataset. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0049] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0050] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0051] Example 1

[0052] like Figure 1-2 As shown, this embodiment provides a replacement flow shop scheduling method based on a multi-view graph neural network, comprising the following steps:

[0053] S1 defines the permutation flow shop scheduling problem with sequence-dependent setup times and constructs the desired objective function in the actual production process and the constraints that satisfy the problem;

[0054] S2 extracts features of the problem based on the same job node, obtains the time feature vector and the time feature matrix, builds a node sparse graph based on the time feature matrix, inputs the time feature vector and the node sparse graph into the graph attention neural network, obtains the node matrix embedding, aggregates the node matrix embedding, obtains the global node embedding, and calculates the graph embedding based on the global node embedding;

[0055] S3 builds node context embedding based on global node embedding and graph embedding, calculates node probability distribution based on node context embedding, and selects jobs based on node probability distribution until all jobs are completed;

[0056] S4 defines the loss function, optimizes the model parameters, and obtains the final scheduling plan.

[0057] The permutation flow shop scheduling problem (PFSP) is a classic scheduling problem with wide applications in manufacturing systems. A large number of exact and approximate algorithms have been proposed to solve this problem. With the development of machine learning technology, more and more studies have used learning-based methods to solve PFSP. However, few studies consider other constraints of the problem, which makes it difficult to represent and extract features of the problem. In order to fill this gap, this paper studies the permutation flow shop scheduling problem with sequence-dependent setup time (PFSP-SDST) and proposes a deep learning (DRL)-based framework to solve the problem. First, PFSP-SDST is represented as a Markov decision process (MDP), and the scheduling of jobs is regarded as a state. Secondly, a multi-feature representation method is proposed for PFSP-SDST, and a multi-view graph neural network (MGNN) is used to extract and embed feature information of the problem. Finally, extensive experiments show that the DRL method outperforms heuristic methods and existing RL-based methods on generated datasets and benchmark datasets, and maintains stability on problems of different scales.

[0058] The symbols used in this disclosure have the following meanings:

[0059] m represents the number of machines; n represents the number of workpieces; M represents the set of machines, M = {1, 2, ..., m}; J represents the set of workpieces, J = {1, 2, ..., n}; i represents the index of the machine, i∈M = {1, 2, ..., m}; j, k both represent the index of the workpiece, j, k∈J = {1, 2, ..., n}; X n×m represents the processing time matrix; S m×n×n represents the sequence-dependent setting time matrix; S n×1represents the initial setting time matrix; x i,j represents the processing time of job j on machine i, x i,j ∈X n×m ;s i,j,k represents the setup time from processing job j on machine i to processing job k, s i,j,k ∈S m×n×n ; represents the initial setup time; τ represents the processing sequence of the workpieces; C i,j represents the completion time of job j on machine i; C max Indicates the maximum completion time; z j,k It means that if workpiece j is before workpiece k, it is 1; otherwise it is 0 (j≠k).

[0060] The permutation flow shop scheduling problem is a set of jobs J = {1, 2, ..., n} that are processed on a set of machines M = {1, 2, ..., m}. During this process, the jobs are processed in the scheduling order τ = {τ 1 ,τ 2 ,…,τ n} are processed on the machine in sequence. The goal of this problem is to find the maximum completion time (makesapn, C max )Minimize the maximum completion time minimize(C max The permutation flow shop problem with sequence-dependent scheduling time is based on PFSP and takes into account the setup time of the machine before processing the job. The problem can be expressed as F m |permu,s ijk |C max The processing time of job j on machine i is x i,j ∈X n×m , machine i is processing the processing sequence τ = {τ 1 ,τ 2 ,…,τ n The setup time before the first job in} is called the initial setup time Machine i needs to consider the setup time s before it can finish job j and continue to process the next job k i,j,k ∈S m×n×n This processing process is serial, and job j can only be processed on the next machine i after it is processed on the previous machine i-1. Therefore, the maximum completion time of this problem is the completion time of the last job on the last machine. Tables 1 and 2 show examples of PFSP-SDST with three machines and four jobs. Figure 3 As shown, a Gantt chart with τ = [2, 3, 4, 1] and makespan = 23 is shown.

[0061] Table 1 Processing time and initial setup time (3 machines and 4 workpieces)

[0062]

[0063] Table 2 Sequence-dependent setup times (3 machines and 4 workpieces)

[0064]

[0065] Maximum completion time C of PFSP-SDST max =max(C i,j ),i∈{1,2,…,m},j∈{1,2,…,n} can be calculated by the following formula:

[0066]

[0067] C 1,j =C 1,j-1 +x 1,j +s 1,j-1,j ,j∈{1,2,…,n}(2)

[0068]

[0069] C i,j =max(C i-1,j ,C i,j-1 +s i,j-1,j )+x i,j ,i∈{1,2,…,m},j∈{1,2,…,n}(4)

[0070] Formula (1) calculates the time it takes to complete the first job on the first machine, taking into account the initial setup time before the first job is processed. The processing time of the remaining jobs on a machine can be calculated using Formula (2), which takes into account the sequence-related setup time. Formula (3) calculates the time it takes to complete the first job on the remaining machines, taking into account the initial setup time before the first job is processed. Formula (4) calculates the time it takes to complete the remaining jobs on the remaining machines.

[0071] Establishing a correct and effective mathematical model is crucial to solving optimization problems. This paper proposes a mixed integer linear model (MILP) of PFSP-SDST, the goal of which is to minimize C max :

[0072] Min(C max )(5)

[0073] constraint:

[0074]

[0075] C max ≥C m,j ,j∈{1,2,…,n}(8)

[0076]

[0077] C i,j ≥0,i∈{1,2,…,m},j∈{1,2,…,n}(12)

[0078] Among them, z j,k If artifact j comes before artifact k, it is 1, otherwise it is 0. j and k represent the indices of the artifacts, n represents the number of artifacts, and C max represents the maximum completion time, C i,j represents the completion time of job j on machine i, x i,j represents the processing time of job j on machine i, s i,j,k It represents the setup time to switch from processing job j to processing job k on machine i.

[0079] Formula (6) and Formula (7) ensure that each workpiece must and can only be assigned to one machine, and each machine can only process one job. Formula (8) ensures that the maximum completion time is not less than the completion time of the workpiece on the last machine. Formula (9) shows that the completion time of the first job in the processing sequence needs to take into account the initial setup time. Formula (10) For each machine i and workpiece j, the completion time of workpiece j on machine i is at least equal to the completion time of the previous workpiece on the machine plus the processing time and setup time of the current workpiece. Formula (11) For each machine i and workpiece j, the completion time of workpiece j on machine i is at least equal to its completion time on the previous machine plus the processing time and setup time of the current workpiece. Formula (12) ensures that the completion time of any workpiece on any machine is non-negative.

[0080] In order to effectively solve this problem, a deep learning framework based on MGNN is proposed, such as Figure 2 Specifically, PFSP-SDST is regarded as a multi-feature problem, and a multi-view graph neural network (MGNN) is designed to process multiple features. The attention-based decision network generates the action probability distribution, and the PPO algorithm is used to train the network structure.

[0081] Deep reinforcement learning tasks can usually be modeled as a Markov decision process, which can be described as a five-tuple M = {S, A, T, R, π}. Among them, S is the state set, A is the action set, T is the state transition probability, R is the reward function, and π is the policy network. This is a process in which an agent interacts with the environment. The agent selects an action based on the current state, and the environment feeds back the reward function and the next state based on the selected action. This process is as follows: Figure 4 As shown. The problem of solving a PFSP with n jobs can be viewed as an n-step decision process. Here, the solution process is regarded as the environment, the scheduling method is regarded as the agent, and the specific scheduling process is as follows. At decision step t, the agent observes the state of the environment and takes action to select the next job from the unscheduled job set. Then, it proceeds to the next decision step until all jobs are scheduled. The specific definitions of state, action, reward, transition probability and strategy are as follows:

[0082] State: At step t, the set of all scheduled and unscheduled jobs constitutes state s t ∈S t At t = 0, all jobs are in the unscheduled job set, and at t = n, all jobs are in the scheduled job set, which means that the current scheduling decision process is completed.

[0083] Action: Action t ∈A t From the state t The next job index to be processed is selected in the job list. To prevent duplicate operations and ensure that jobs are selected in the unscheduled job set, a mask mechanism is applied to select eligible jobs.

[0084] Transfer: passing through state s t and action a t After that, the environment will go to the next state s t+1 , which is the processing completion time of the current job in this problem.

[0085] Reward: In the current state t Next, the agent chooses action a t The final benefit is reflected by the reward function, and the total reward function can guide the agent to learn better strategies. In this paper, reward is defined as state s t In and t+1 Makespan of the partial schedule between, r(s t |a t |s t+1 )=C max (s t )-C max (s t+1 ).

[0086] Policy: Strategy π θ Defines each state s t For action set A t The probability, here π θ is parameterized as a neural network, p = π θ (s t |a t ) represents the probability distribution of the strategy output job at decision step t, and θ represents the weight of the network.

[0087] In this decision process, the state set changes from the unscheduled state to the scheduled state, and finally to the final arrangement of the job τ = [τ 0 ,τ 1 ,…,τ n-1 ].

[0088] In the classic shop scheduling problem, disjunctive graphs are often used to intuitively represent the solution process of the scheduling problem, and good results have been achieved. Due to the characteristics of PFSP, Makespan depends on the processing order of the jobs. Therefore, compared with the disjunctive graph, the k-nearest neighbor graph (a sparse graph based on KNN technology) can represent PFSP more simply and effectively. However, when considering other constraints of the problem, a single graph structure often cannot represent complex features. The constraints of the problem are often diverse. When converting them into features, they often face dimensional inconsistencies, and simple linear transformations will inevitably damage the feature information. Therefore, this paper solves the above problems by constructing a multi-view structure. First, different characteristics of the problem are regarded as different features, and they have the same set of nodes, that is, job j. They are processing time features, Sequence Correlation Setting Time Features Initial setup time characteristics Then, if Figure 5 As shown in Figure 1, a sparse graph based on KNN is constructed. In order to improve efficiency, KD tree is used to construct the sparse graph. The characteristics of the problem are represented by different views. The structure is represented as G = {V, E 1 ,E 2 ,…,E ε}, where V represents the node, E represents the view, and ε represents the number of views.

[0089] In terms of processing complex features, multi-view graph neural networks can generate more discerning and expressive node representations, which helps the model perform better in the following tasks. Here, the encoding process can be divided into two parts: single-view embedding and multi-view aggregation, such as Figure 6As shown in the figure, each view is first embedded by a graph attention neural network (GAT) and a set of view-specific node representations are learned; then, an attention-based method is used to learn the weights of different views; finally, each view is integrated into the final node embedding according to the learned weights as the input of the decoder.

[0090] A single view is embedded and calculated through a multi-layer GAT, and the edge features are considered in this disclosure. GAT can fully consider the relationship between data and automatically learn the importance between nodes. Its effectiveness has been proven in previous work. The feature matrix X of PFSP-SDST n×m , S m×n×n , and their corresponding sparse graph structures as input, and the view E is obtained through GAT p , E s and E si First, the linear projection of the feature matrix is ​​calculated as the initial embedding, that is, the embedding of the 0th layer:

[0091]

[0092] in, They represent the processing time node features, the sequence-related setting time node features and the initial setting time node features, respectively, through W x ∈R d×n , W s ∈R d×(n×m) , W si ∈R d Perform linear projections, d = 128 is the embedding dimension of the network layer, Indicates the edge features corresponding to the three node features. For the convenience of representation, in subsequent calculations, represents the feature representation of the node at layer l, represents the edge feature of the node at the lth layer, l∈{1,2,…,n}. Then, the attention coefficient of node j and node k at the lth layer is calculated as follows:

[0093]

[0094] Among them, σ l represents the activation function LeakyReLU, a T ∈R 2d and W l ∈R d is a learnable weight vector, T represents transposition, and || represents a connection operation. Multi-head attention (MHA) is applied to the attention coefficient obtained from the above formula, and then residual connections are applied between every two layers:

[0095]

[0096] W δ is a learnable weight vector, BN l represents batch normalization, δ=8 in the MHA layer, the head dimension is 16, and FF is a feed-forward layer. After N layers of GAT, the embedding of the view β of node j is obtained

[0097] For each view, representing different feature relationships between nodes, they have different models of network parameters. In order to constrain the network parameters, the L2 norm is applied for regularization. The L2 norm is first applied to each view, and then they are added together to constrain the network parameters between the two views:

[0098]

[0099] Among them, w j , w k represents the weight matrix of view j and view k, and β represents the number of views.

[0100] In order to effectively aggregate the information of each view, an attention-based aggregation method is applied to each view. Considering the different importance of each view, the attention mechanism is used to learn and assign weights to them. First, the different views of the node are represented as Connect in its column direction as c j , where j = {1, 2, …, n} represents a node and β∈{1, 2, …, Β} represents a view.

[0101]

[0102] Among them, θ j,β represents the weight of view β of node j, Represents a set of learnable weights for view β. Then, the global node representation is obtained through the weight coefficients of different views:

[0103]

[0104] Finally, according to the global node representation h j Compute graph embedding

[0105] Decision process: In the MGNN stage, we get global node embedding and graph embedding. In the decision stage, as a downstream task, we apply an MHA to calculate the probability of the node. Decision making is the last part of the neural network and the key part of MDP. Here, the final output is the probability distribution of the node, which is used for action a. t In status tFirst, we use the output of MGNN to construct the context embedding of the node:

[0106]

[0107] Among them, h ( ' c) is a three-dimensional vector, h (g) For graph embedding, is the embedding of the first job, represents the embedding of the previous job, and [·,·,·] is a horizontal concatenation operation. Then, MHA with M=8 head dimension=16 is used for further refinement:

[0108]

[0109] Then, the query vector Q and the key vector K are used to calculate the attention coefficient u of the decoder layer at the decision step t (c)j :

[0110]

[0111] Where r = 10, the logarithmic value used for tanh function calculation is kept within [-r, r], W Q and W K is a learnable parameter, u t represents the set of unscheduled jobs at decision step t. Set the logarithmic value of scheduled jobs to -∞ to prevent duplicate scheduling. Finally, the probability of the node is calculated by SoftMax

[0112]

[0113] After the above steps, the probability distribution vector of the node is obtained, and the next job is selected according to the node distribution vector using a certain strategy (sampling or greedy). At the same time, the scheduled job and unscheduled job set are updated.

[0114] Training process: Among the learning-based methods, RL and IL are very popular methods. However, the RL method needs to be trained from scratch and converges slowly during training. The IL method learns excellent expert solutions and can converge quickly, but the results are also limited by the expert solutions, and it is difficult to obtain a solution that is better than the expert solution. In addition, finding an efficient expert method is also a challenge. In this paper, the PPO algorithm is used for training. PPO adopts an actor-critic structure and is an algorithm based on policy gradients. Policy network π θ As an actor, critic It consists of three one-dimensional convolutional layers and is used to predict the state s tThe value v(s t ), Represents network parameters. The input dimension of Critic is d, the output dimension is 1, and the node embedding of input MGNN is h j , output the predicted value The training process is shown in Algorithm 1. I iterations of training are performed. During this process, the batch size is 32. After each batch is processed, the actor network is updated. The probability ratio of the updated strategy to the old strategy is:

[0115]

[0116] Calculate the loss of the actor network and apply a clipping function to limit the ratio to a reasonable range:

[0117] Loss clip(θ) =E[min{r t(θ) A t ,clip(r t(θ) ),1-ε,1+ε)A t}] (27)

[0118] Loss E(θ) =Entropy(π θ (a t |s t )) (28)

[0119]

[0120] Where, ε = 0.2, A t is the advantage function, through the cumulative reward R t and the critic's prediction value get, Calculate the entropy loss function Loss of the actor network E(θ) And the mean square error loss of the critic network And use them to update the total loss function of the actor-critic model w p ,w v ,w e There are three weights that coordinate each function to achieve gradient descent and continuously reduce the loss function. For each batch, the memory is initialized. During training, the strategy is verified on a set of independent validation examples every 16 iterations.

[0121] Experiments: This paper compares with several types of optimization methods from various perspectives, including heuristic methods, meta-heuristic methods, and learning-based methods.

[0122] The experiments were conducted on the generated dataset and the public dataset. The generated dataset was randomly generated, with the processing time uniformly distributed between 1 and 11, and the initial setup time and sequence-related setup time uniformly distributed between 1 and 4. SDST10, SDST50, and SDST100 in Taillard were selected as public datasets. Since the dataset does not contain the initial setup time, the initial setup time of the same range was added. The number of machines was 5, 10, and 20, and the number of jobs was 10, 20, 30, and 50, respectively. For experiments of different scales, 1280 examples were used for training and 128 examples were used for verification.

[0123] The specific parameter settings in the experiment are shown in Table 3.

[0124] Table 3 Symbols and values ​​of network parameters

[0125]

[0126]

[0127] Three types of optimization methods are used to compare with the proposed algorithm, including heuristic methods, meta-heuristic methods and learning-based methods. Among them, the heuristic methods include the longest processing time rule (LPT), the shortest processing time rule (SPT), Palmer, Grasp and NEH; the meta-heuristic methods include GVNS and IG; the popular RL method and IL method are selected based on learning. For the proposed method, two different selection strategies are compared: greedy and sampling. In addition, this paper also compares with Gurobi. Gurobi is an exact solver, and its time limit is set to 1800s and 2400s here. The optimal solution obtained by Gurobi is used to evaluate the proposed method and all comparison schemes. All methods are carried out in the same environment.

[0128] The specific implementation of the contrasting method is as follows: Plamer is modified to adapt to PFSP-SDST, and the process of calculating weights takes into account SDST and sets a weight for it to balance the weights. Since Plamer does not need to perform complex insertion, replacement and other operations, it runs the fastest and is therefore used as the initial solution for GVNS. In the solution process of GVNS, Tabu and VND are used for local search, where the number of iterations of Tabu search is set to 100. The IG algorithm uses NEH to generate the initial solution. The RL method is an actor-critic (AC) network, in which RNN is used for encoding. The IL-based method is behavioral cloning (BC), with GAT as the encoder. The learning-based method uses the same hyperparameters as the proposed method.

[0129] The experimental results are compared from three aspects: average makespan, Gap and average running time. The lower the better. Gap is calculated by the optimal solution and the current solution, as shown in formula (31):

[0130]

[0131] The total running time of the IL-based method includes the time to generate expert solutions, training time, and verification time.

[0132] The experimental results were analyzed and compared:

[0133] Sensitivity analysis: Correct and effective hyperparameters are very important for the policy network. This embodiment experimentally analyzes the hyperparameters used. For the policy network disclosed in this invention, the learning rate lr and the clipping coefficient (Clipping coefficient, cl) are relatively critical parameters. In order to find the most effective parameter values, lr is set to 1e-4, 2e-4, 3e-4 and 4e-4, and cl is set to 0.1, 0.2 and 0.3 for combined experimental verification. Fix one of the parameters to observe the situation of another parameter. Here, the impact of the parameters on the network is judged by the convergence of the loss. Faster and better convergence is a reflection of network performance. The experiment was trained on 1280 instances with 10 jobs and 5 machines, and verified on 128 instances of the same scale. When the lr value is 3e-4, the experimental results are as follows: Figure 7 As shown in (a), it can be seen that the value of clip has a great impact on loss. When cl = 0.2, loss can converge faster and better. When cl is 0.2, the change of lr will also lead to the change of loss. Figure 7As shown in (b), when lr is 3e-4, the loss converges better. This shows that lr = 3e-1 and cl = 0.2 are more effective parameter values ​​for the policy network, so this parameter value is used in all subsequent experiments.

[0134] Ablation study: In order to verify the effectiveness of MGNN and PPO, an ablation experiment was conducted. Different training methods and neural network structures were used for testing. In this experiment, the very popular RL-based AC method and IL-based BC method were selected, and the neural networks selected RNN, GAT and the proposed MGNN for comparison. All methods were trained on 1280 instances with 20 jobs and 5 machines, and verified on 128 instances of the same scale. The performance of several frameworks is comprehensively compared from four aspects: Makespan, gap, running time and number of network parameters. The results are shown in Table 4. Under the sampling selection strategy, the average gap of the MGNN and PPO frameworks is 4.76% lower than that of other frameworks; under the greedy selection strategy, the average gap is 3.66% lower than that of other frameworks. From Fig. 9 It can be seen that MGNN shows better performance than other networks, and PPO is also more effective than other training methods. Although the network structure of the proposed framework is more complex and the network parameters are slightly more than other frameworks, considering that PPO is an offline method, the running time is acceptable. The convergence curve of the DRL method is shown in Figure 8 As shown, it can be seen that this method can converge quickly and stably.

[0135] Table 4 Ablation experiment results

[0136]

[0137]

[0138] Generated Datasets: Comparative experiments were conducted on generated datasets. The generated datasets were divided into small-scale examples and large-scale examples. The number of jobs in the small-scale examples was 10, 15, and 20, and the number of machines was 5 and 10, respectively; the number of jobs in the large-scale examples was 30 and 50, and the number of machines was 10 and 20, respectively. Here, three different types of methods were compared, including heuristic methods, metaheuristic methods, and learning-based methods. In addition, the running results of the Gurobi solver were compared, and the maximum running time of Gurobi was limited to 2400s. Table 5 shows the running results of the small-scale examples.

[0139] Among the heuristic methods, NEH shows better performance. The IG algorithm can often obtain suboptimal solutions, even better than Gurobi's solution, but it takes a considerable amount of time. Compared with other learning-based frameworks, the framework proposed in this paper shows better performance and is better than heuristic methods such as LPT, SPT, Palmer and Grasp. In addition, the framework performs better on large-scale datasets. As shown in Table 6, as the problem size increases, the average gap value is reduced by 1.11% compared with the average gap value of small-scale datasets. From Fig.10 It can be seen that the DRL method has better results on large-scale problems. From the perspective of running time, the running time of heuristic methods and metaheuristic methods increases exponentially, while the running time of DRL increases steadily within an acceptable range as the problem size increases. The analysis of experimental results shows that the DRL method has certain advantages in solving PFSP-SDT, and MGNN can effectively provide high-quality problem information.

[0140] Table 5 Results of different methods on small-scale generated datasets

[0141]

[0142]

[0143] Table 6 Results of different methods on large-scale generated datasets

[0144]

[0145] Benchmark datasets: Generalization is a key indicator for evaluating an algorithm or structure. In order to verify the generalization performance of the proposed DRL method, this section compares the performance of this method with other methods on public datasets. The experiments were conducted on the benchmark datasets Tallid SDST10 and SDST50, whose setup time ratios are 10% and 50% of the processing time, respectively. Since the dataset does not include the initial setup time, we extended it on its basis and added the same proportion of initial setup time as its setup time. For all experimental results, the algorithms are still compared in terms of makespan, gap, and running time.

[0146] The results on SDST10 show that the proposed DRL method is better than the heuristic method and other learning-based methods. The box plot is shown in Fig.11(a). Compared with the two scheduling rules LPT and SPT, the average gap of the DRL method is 5.92% smaller, and 1.52% lower than the average gap of the best performing heuristic method Grasp. Among all learning-based methods, DRL shows the best performance. The average gap of DRL is 3.61% lower than that of RL methods and 2.27% lower than that of IL methods. Sampling strategies can often explore more possibilities, but this will also take longer. DRL with a sampling strategy takes longer to run than DRL with a greedy strategy, but the average gap is only reduced by 0.35%. The average running time of the DRL method is much less than that of iterative local search (ILS), GRASP, GVNS and IG. Even on the largest 50x20 example, the average running time of the DRL method is only 30.26s. DRL with a greedy strategy is faster, and the average gap is still better than the heuristic method. Learning-based methods usually take less computing time. Among learning-based methods, DRL with a greedy strategy has similar running time to other methods.

[0147] As the proportion of setup time increases, the gap value of the DRL method on SDST50 increases slightly, but the trend of the gap value is almost the same as that on SDST10. Compared with the heuristic method, the DRL method shows a smaller gap value and average running time, as shown in the box plot. Fig.11 (b) shows. The average gap values ​​of the DRL method using the sampling strategy and the greedy strategy are 6.36% and 4.47% lower than those of the heuristic method, respectively. Compared with other learning-based methods, the average gap value of the DRL method is 4.28% lower. From the perspective of average running time, the DRL method is far superior to Gurobi, ILS, Grasp and meta-heuristic methods. Compared with other learning-based methods, the DRL method using the greedy strategy consumes less average running time.

[0148] By comparing the running results, the proposed method shows better performance and stability on different data sets.

[0149] The permutation flow shop scheduling problem with sequence-dependent setup time (PFSP-SDST) is a classic combinatorial optimization problem. Effectively solving this problem is very important for actual manufacturing. In recent years, more and more works have used learning-based methods to solve optimization problems. This paper proposes a DRL method based on MGNN to solve PFSP-SDST. Different from other works, this method represents PFSP-SDST from different perspectives, and represents the processing time, SDST and initial setup time of the problem as different features, which have the same set of job nodes. In addition, a new MGNN is proposed for PFSP-SDST to handle multi-feature representation, aggregate the embedded information of different features into a global node representation, and use more accurate feature representation in the decision stage. The solution process is formulated as an MDP, where MGNN and the decision network constitute a policy network, and PFSP-SDST is regarded as an environment. Through experimental verification, this method has good performance on the generated data set benchmark dataset, outperforming other learning-based methods and some heuristic methods, and showing stability on instances of different sizes. In future work, the network structure will be further optimized to achieve the purpose of lightweight, which is very important for reducing network complexity and computation time. The method will be applied to more complex scheduling problems and extended to a multi-objective approach to solve multi-objective PFSP.

[0150] Example 2

[0151] This embodiment provides a replacement flow shop scheduling system based on a multi-view graph neural network, including:

[0152] A model building module is configured to: define a permutation flow shop scheduling problem with sequence-dependent setup times, and construct an objective function desired in an actual production process and constraints that satisfy the problem;

[0153] The multi-view representation module is configured to: extract features of the problem based on the same job node, obtain a time feature vector and a time feature matrix, construct a node sparse graph based on the time feature matrix, input the time feature vector and the node sparse graph into the graph attention neural network, obtain node matrix embedding, aggregate the node matrix embedding, obtain a global node embedding, and calculate the graph embedding based on the global node embedding;

[0154] The decision module is configured to: construct node context embedding based on global node embedding and graph embedding, calculate node probability distribution according to the node context embedding, and select jobs according to the node probability distribution until all jobs are completed;

[0155] The model training module is configured to: define the loss function, optimize the model parameters, and obtain the final scheduling solution.

[0156] It should be noted that the above modules correspond to the steps described in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0157] In further embodiments, there is also provided:

[0158] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, it will not be described in detail here.

[0159] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0160] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0161] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in Example 1 is completed.

[0162] The method in Example 1 can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it is not described in detail here.

[0163] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in embodiment 1 is implemented.

[0164] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the process / method as described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided between program modules as needed. Machine executable instructions for program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0165] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the computer or other programmable data processing device, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer or completely on a remote computer or server.

[0166] In the context of the present invention, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, and the like. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, and the like.

[0167] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0168] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A permutation flow shop scheduling method based on multi-view graph neural network, characterized in that: The following steps are involved: Define the permutation flow shop scheduling problem with sequence-dependent setup times and construct the desired objective function in the actual production process and the constraints that satisfy the problem; Based on the same job node, extract features of the problem to obtain time feature vectors and time feature matrices, build a node sparse graph based on the time feature matrix, input the time feature vectors and node sparse graph into the graph attention neural network to obtain node matrix embedding, aggregate the node matrix embedding to obtain global node embedding, and calculate graph embedding based on the global node embedding; Build node context embedding based on global node embedding and graph embedding, calculate node probability distribution based on node context embedding, and select jobs based on node probability distribution until all jobs are completed; Define the loss function, optimize the model parameters, and obtain the final scheduling plan.

2. The permutation flow shop scheduling method based on multi-view graph neural network according to claim 1, characterized in that: The time feature vectors include a processing time feature vector, a sequence-related setting time feature vector and an initial setting time feature vector; The time feature matrix includes a processing time feature matrix, a sequence-related setting time feature matrix and an initial setting time feature matrix.

3. The permutation flow shop scheduling method based on multi-view graph neural network according to claim 1, characterized in that: The node matrix embeddings are aggregated to obtain the global node embedding. Specifically, the weights of different node matrix embeddings are learned using the attention method, and the node matrix embeddings are integrated into the global node embedding according to the learned weight values.

4. The permutation flow shop scheduling method based on multi-view graph neural network according to claim 1, characterized in that: The desired objective function in the actual production process is: the job sequence with the minimum maximum completion time.

5. The permutation flow shop scheduling method based on multi-view graph neural network according to claim 1, characterized in that: The constraints that satisfy the problem are: C max ≥C m,j ,j∈{1,2,…,n} C i,j ≥0,i∈{1,2,…,m},j∈{1,2,…,n} Among them, z j,k If artifact j comes before artifact k, it is 1, otherwise it is 0. j and k represent the indices of the artifacts, n represents the number of artifacts, and C max represents the maximum completion time, C i,j represents the completion time of job j on machine i, x i,j represents the processing time of job j on machine i, s i,j,k It represents the setup time to switch from processing job j to processing job k on machine i.

6. The permutation flow shop scheduling method based on multi-view graph neural network according to claim 1, characterized in that: The PPO algorithm is used to train the model. The PPO algorithm adopts the actor-critic structure, calculates the entropy loss function of the actor network and the mean square error loss of the critic network respectively, and uses the two to calculate the total loss function.

7. A permutation flow shop scheduling system based on multi-view graph neural network, characterized in that: include: A model building module is configured to: define a permutation flow shop scheduling problem with sequence-dependent setup times, and construct an objective function desired in an actual production process and constraints that satisfy the problem; The multi-view representation module is configured to: extract features of the problem based on the same job node, obtain a time feature vector and a time feature matrix, construct a node sparse graph based on the time feature matrix, input the time feature vector and the node sparse graph into the graph attention neural network, obtain node matrix embedding, aggregate the node matrix embedding, obtain a global node embedding, and calculate the graph embedding based on the global node embedding; The decision module is configured to: construct node context embedding based on global node embedding and graph embedding, calculate node probability distribution according to the node context embedding, and select jobs according to the node probability distribution until all jobs are completed; The model training module is configured to: define the loss function, optimize the model parameters, and obtain the final scheduling solution.

8. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images