Control method for workpiece machining scheduling based on multi-dimensional edge weighted graph attention reinforcement learning
Through the method of attention-reinforced learning based on multi-dimensional edge power chart, the problem of scheduling difficulty under large-scale, multi-variety, and small-scale production orders is solved, and efficient and fast workpiece processing and scheduling is achieved, which is suitable for production with high delivery time pressure. environment.
Patent Information
- Application Number
- CN202310752942.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-25
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-06-25
Smart Images

Figure CN116757425B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent optimization scheduling technology, and in particular to a control method for workpiece processing scheduling based on multi-dimensional edge weight graph attention reinforcement learning. Background Art
[0002] In today's society, customer needs are becoming increasingly diversified and personalized. Competition between companies is no longer limited to product prices and quality, but is carried out in terms of quickly and accurately responding to customer needs and providing customized services. The production orders of enterprises are gradually showing the characteristics of large-scale, multi-variety, and small batches. Among them, large-scale, multi-variety, and small-batch production orders have brought the following challenges to the production scheduling of enterprises: First, the difficulty of production line planning has increased. The production of multi-variety orders requires discrete manufacturers to switch equipment on the production line according to order requirements to ensure that the requirements of various orders are met; second, the pressure on delivery deadlines has increased. Due to the large number of orders and different delivery deadlines for different orders, the difficulty of production scheduling has increased, and the delivery deadlines are more likely to become chaotic.
[0003] This problem is called "Permutation Flow Shop with Sequence Dependent Set-up Times (PFSP-SDST)". The PFSP-SDST scheduling problem refers to the situation in which different products in a production line need different equipment switching times when passing through different equipment, and these switching times are related to the relationship between the previous product and the next product. The goal of this scheduling problem is to improve indicators such as production efficiency and delivery efficiency as much as possible while satisfying constraints. The existing solutions can be roughly divided into two categories: one is the exact algorithm, which can use the column generation algorithm to solve the PFSP-SDST problem with the goal of minimizing the total weighted delay. First, the column generation algorithm is used to solve the continuous relaxation model of the set partitioning formula obtained by Dantzig-Wolfe decomposition. Then, the pricing subproblem of column generation is decomposed into job-level subproblems. Finally, a cut generation algorithm is developed for the column generation algorithm. Lagrangian relaxation and cut generation techniques can also be applied to solve the PFSP-SDST problem. The original problem is decomposed into single job-level sub-problems that can be effectively solved by dynamic programming; the other category is heuristic algorithms, which use an adaptive hybrid genetic algorithm to solve the PFSP-SDST problem of minimizing completion time and total weighted delay, and propose a new discrete artificial bee colony algorithm based on decomposition by considering the PFSP-SDST problem of minimizing completion time, and an enhanced migratory bird optimization algorithm to solve the PFSP-SDST problem of minimizing completion time.
[0004] In the actual production process, companies usually have strict requirements on the execution time of algorithms. On the one hand, the algorithm’s execution time is limited due to the tight order delivery time and complex and time-consuming production process; on the other hand, the algorithm needs to respond quickly to external changes, considering the uncertainty of supplier delivery, customer emergency orders and temporary order returns. For precise algorithms, when the problem scale increases to a certain extent, it is impossible to provide a high-quality solution within a limited time. For heuristic algorithms, although a feasible production plan can be given in a relatively short time, there is still much room for improvement in the solution. Summary of the invention
[0005] The present invention provides a control method for workpiece processing scheduling based on multi-dimensional edge weighted graph attention reinforcement learning, which is used to solve the defect that high-quality solutions cannot be given within a limited time in the prior art, thereby obtaining a high-efficiency and high-quality scheduling solution.
[0006] The present invention provides a control method for workpiece processing scheduling based on multi-dimensional edge weight graph attention reinforcement learning, comprising:
[0007] Based on the workpiece processing target, a multi-dimensional edge weight feature directed graph is constructed;
[0008] Constructing a scheduling model, inputting the multi-dimensional edge weight feature directed graph into the scheduling model, generating a pre-processing sequence set, and optimizing the scheduling model based on an offline reinforcement learning algorithm;
[0009] Utilizing a hybrid beam search and priority scheduling rule algorithm, the pre-processing sequence set is optimized to obtain a production processing sequence set;
[0010] Based on the production processing sequence set and the workpiece processing target, a workshop scheduling processing plan is obtained.
[0011] In a possible implementation, the workpiece processing target includes problem parameters and an objective function;
[0012] The problem parameters include workpiece processing time, workpiece switching time, and workpiece delivery time;
[0013] The objective function includes the completion time and the delay time of the workpiece.
[0014] In a possible implementation manner, the multi-dimensional edge-weighted directed graph includes nodes and directed edges, the nodes represent workpieces, and are represented by the workpiece processing time and the workpiece delivery time;
[0015] The directed edges represent the workpiece processing sequence, which is represented by the workpiece switching time.
[0016] In a possible implementation, the scheduling model includes an encoder and a decoder;
[0017] The encoder is constructed using a graph attention mechanism of multi-dimensional edge weight features to generate final node features and global features of the directed graph;
[0018] The decoder generates a set of pre-processed sequences based on the global features of the directed graph using Markov decision making.
[0019] In a possible implementation, the encoder includes:
[0020] Calculate the importance of the features of adjacent node j to node i, and use the activation function to perform nonlinear transformation on the importance value;
[0021] Based on the importance value, the normalization algorithm is used to obtain the attention weight of the neighboring node j on the new feature generated by node i;
[0022] After weighted averaging the adjacent features of node i according to the attention weight, a new feature of node i is obtained by nonlinear transformation;
[0023] The new features of node i are learned in multiple rounds to obtain the final node features and the global features of the directed graph.
[0024] In a possible implementation, optimizing the scheduling model based on an offline reinforcement learning algorithm includes:
[0025] Obtain several problem instances;
[0026] Inputting the problem instance into the scheduling model, generating a probability distribution of a processing sequence, and generating a first processing sequence by random sampling and a second processing sequence by greedy search;
[0027] Calculating a first objective function value of the problem instance in the first processing sequence, and calculating a second objective function value of the problem instance in the second processing sequence;
[0028] Using the REINFORCE algorithm, the gradient of the scheduling model parameters is calculated;
[0029] Based on the gradient of the scheduling model parameters, the scheduling model parameters are updated using a gradient descent algorithm to obtain a minimum objective function value.
[0030] In a possible implementation, judging whether the scheduling model needs to be updated according to a paired sample one-sided test method specifically includes:
[0031] Obtain several test problem instances;
[0032] Input the inspection problem instance into the current scheduling model, generate the current processing sequence, and obtain the current objective function value, input the inspection problem instance into the updated scheduling model, generate the updated processing sequence, and obtain the updated objective function value;
[0033] Based on the current objective function value and the updated objective function value, a sample difference value, a sample difference value mean value and a sample difference value standard deviation value are obtained;
[0034] Determining a significance level based on the sample mean difference;
[0035] Constructing statistics based on the sample difference mean;
[0036] Based on the constructed statistic, obtaining a probability value of the statistic;
[0037] When the probability value is less than the significance level value, the update scheduling model is updated; when the probability value is not less than the significance level value, the update scheduling model is not updated.
[0038] In a possible implementation, the use of a hybrid beam search and a priority scheduling rule algorithm to optimize the pre-processing sequence set to obtain a production processing sequence set includes:
[0039] S31, initializing a set of empty sequences and initializing sub-algorithm weights;
[0040] S32, obtaining a new local sequence based on the set;
[0041] S33, assigning the sub-algorithm weight to the new local sequence;
[0042] S34, executing steps S32-S33 until a complete sequence is generated;
[0043] S35, calculating the objective function value of the complete sequence.
[0044] In a possible implementation manner, obtaining a new local sequence based on the set includes:
[0045] Use beam search decoding to train the scheduling model and obtain the k new local sequences with the highest probability, denoted as P1;
[0046] By optimizing the scheduling rule with the minimum switching time, k new local sequences are obtained, denoted as P2;
[0047] Using the earliest delivery first scheduling rule, k new local sequences are obtained, denoted as P3.
[0048] In a possible implementation, assigning the sub-algorithm weight to the new local sequence includes:
[0049] According to the sub-algorithm weight w1, w1*k local sequences are extracted from P1 according to the probability distribution, denoted as
[0050] Extract w2*K local sequences from P2 according to uniform distribution, denoted as
[0051] Extract w3*K local sequences from P3 according to uniform distribution, denoted as
[0052] make
[0053] The present invention provides a control method for workpiece processing scheduling based on multi-dimensional edge weighted graph attention reinforcement learning, which solves the scheduling problem with a variable number of workpieces by integrating the representation method of the multi-dimensional edge weighted feature directed graph; utilizes the graph attention mechanism encoder of the multi-dimensional edge weighted feature to effectively learn the implicit features of the scheduling problem; adopts the context feature decoding method of the inherited feature to solve the problem characteristics of the scheduling problem; and obtains the optimal scheduling plan by integrating the priority scheduling rule and the beam search decoding hybrid reinforcement learning algorithm, thereby obtaining a high-efficiency and high-quality scheduling plan. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0055] Figure 1 It is one of the flow charts of the control method of workpiece processing scheduling based on multi-dimensional edge weight graph attention reinforcement learning provided by the present invention;
[0056] Figure 2 It is a schematic diagram of a representation example structure of a multi-dimensional edge weight feature directed graph provided by the present invention;
[0057] Figure 3 It is a schematic diagram of the global feature flow of calculating a directed graph provided by the present invention;
[0058] Figure 4 It is a schematic diagram of the structure of the encoder provided by the present invention;
[0059] Figure 5 It is a schematic diagram of the encoder calculation flow provided by the present invention;
[0060] Figure 6 It is a schematic diagram of the calculation flow of the decoder provided by the present invention;
[0061] Figure 7 It is one of the flow diagrams of optimizing the scheduling model based on the offline reinforcement learning algorithm provided by the present invention;
[0062] Figure 8 This is the second flow chart of optimizing the scheduling model based on the offline reinforcement learning algorithm provided by the present invention;
[0063] Fig. 9 It is a schematic diagram of a flow chart of updating encoder-decoder parameters provided by the present invention;
[0064] Fig.10 It is one of the flow diagrams of optimizing the pre-processing sequence set by using the hybrid beam search and priority scheduling rule provided by the present invention;
[0065] Fig.11 This is the second flow chart of optimizing the pre-processing sequence set by using the hybrid beam search and priority scheduling rules provided by the present invention. DETAILED DESCRIPTION
[0066] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0067] Combine the following Figure 1-Figure 11 The present invention describes a control method for workpiece processing scheduling based on multi-dimensional edge weight graph attention reinforcement learning.
[0068] To facilitate understanding of the technical solution of the embodiment of the present invention, assume that a batch of workpieces need to be processed by multiple machines on an assembly line in sequence, where N is the set of workpieces, M is the set of machines, and p is j,m is the processing time of workpiece j∈N on machine m∈M, s i,j,m is the switching time of machine m∈M from processing workpiece i∈N to processing workpiece j∈N, d j is the delivery time of workpiece j∈N. The method described in this embodiment is used to determine the optimal processing sequence of the completion time and the delay time of the workpiece.
[0069] like Figure 1 As shown, S1. Based on the workpiece processing target, a multi-dimensional edge weight feature directed graph is constructed.
[0070] In step S1 , the workpiece processing target includes problem parameters and an objective function. The problem parameters include workpiece processing time, workpiece switching time, and workpiece delivery time. The objective function includes workpiece completion time and delay time.
[0071] In step S1, the multi-dimensional edge weight feature directed graph consists of |N| nodes and |N|*(|N|-1) directed edges, where the nodes represent workpieces and the features of the nodes are the workpiece processing time p of the workpiece on each machine. j,m and the delivery time d of the workpiece j The two parts represent the workpiece processing sequence, and the directed edge is characterized by the workpiece switching time s generated by processing adjacent workpieces. i,j,m Therefore, the original node feature is [p j,m ,d j ], the original directed edge feature (edge feature) is [s i,j,m ]. Figure 2 This is a representation example of a multi-dimensional edge weight feature directed graph constructed when the number of workpieces |N|=4.
[0072] By constructing a multi-dimensional edge-weight feature directed graph, it is possible to keep the node feature dimension and edge feature dimension unchanged while the number of nodes changes, which has higher applicability and scalability.
[0073] S2. Build a scheduling model, input the multi-dimensional edge weight feature directed graph into the scheduling model, generate a pre-processing sequence set, and optimize the scheduling model based on the REINFORCE (offline reinforcement learning) algorithm.
[0074] In step S2, the scheduling model includes an encoder and a decoder, wherein the encoder is constructed by using a graph attention mechanism that integrates multi-dimensional edge weight features. The encoder includes an input mapping layer and L feature learning layers. Each feature learning layer is composed of four modules: a graph attention mechanism module (EGAT) that integrates multi-dimensional edge weight features, a residual connection module (Skip-connection), a feedforward neural network module (Feed Forward) and a batch normalization module (BatchNorm).
[0075] like Figure 3-5 As shown in the figure, the input data of the encoder is a representation instance based on a multi-dimensional edge weight feature directed graph, and the output data is the feature representation of each node and the feature representation of the entire scheduling problem. When the number of machines |M| takes a large value, the composition of the node features will be extremely unbalanced, and the processing time feature dimension is |M|, while the delivery time feature dimension is 1.
[0076] The input mapping layer in the encoder performs a preliminary transformation on the original node features and original edge features in the multi-dimensional edge weight feature directed graph to obtain the mapped node features and mapped edge features after the input mapping layer. By mapping the processing time features and delivery time features respectively, and concatenating the mapped processing time features and delivery time features as the features of the mapping node, the specific expression is:
[0077]
[0078]
[0079] in, is the feature of node i after the input mapping layer;
[0080] [] stands for splicing;
[0081] W1, W2, and W3 are parameter matrices;
[0082] p i,m is the processing time of workpiece j on machine m;
[0083] d i is the delivery time of workpiece i;
[0084] is the edge feature after the input mapping layer;
[0085] s i,j,m It is the switching time from machine m processing workpiece i to machine j processing workpiece.
[0086] In the feature learning layer, graph attention (EGAT) that integrates multi-dimensional edge weight features is used to effectively learn the implicit features in the input data and obtain the global features of the directed graph.
[0087] Furthermore, the global features of the directed graph are calculated, including:
[0088] S21. Calculate the importance e of the features of adjacent node j to node i ij , using LeakyReLU (activation function) to adjust the importance of e ij Perform a nonlinear transformation:
[0089]
[0090] Where W is the parameter matrix.
[0091] S22. Based on importance ij , using softmax (normalization algorithm), we get the attention weight (contribution) α of the neighboring node j to the new feature generated by node i i,j :
[0092]
[0093] Where exp is the exponential function.
[0094] S23, the adjacent features of node i are calculated according to the attention weight α i,j After weighted averaging, the new feature of node i is obtained by using nonlinear transformation σ
[0095]
[0096] S24. Perform multiple rounds of learning on the new features of node i to obtain the final node features and the global features of the directed graph.
[0097] Calculate the attention weights k times, because it is difficult to capture all the feature information of the neighbors by calculating the attention only once:
[0098]
[0099] Furthermore, the overall calculation process of the feature learning layer is as follows:
[0100]
[0101]
[0102]
[0103] In the formula, is the final node feature after multiple rounds of learning;
[0104] l∈L represents the lth feature learning layer;
[0105] BN l represents the batch normalization module of the lth feature learning layer;
[0106] EGAT l The graph attention mechanism module representing the lth feature learning layer that integrates multi-dimensional edge weight features;
[0107] Represents the features of node i after l-1 layers of feature learning;
[0108] Represents directed edge features (remains unchanged during the learning process);
[0109] FF l Represents the forward propagation module of the lth feature learning layer;
[0110] Represents the global features of the directed graph after L layers of feature learning;
[0111] Represents the final feature of node i after L layers of feature learning.
[0112] like Figure 6 As shown in the figure, the decoding process of the decoder is to use Markov decision to gradually generate a pre-processed sequence set π = {π1,…,π t ,…π |N|}, π t ∈N, the decoding process uses the tuple {X t ,A,P,R}.
[0113] Among them, the state space X t Represents the state of each stage, which is composed of the features of the problem described in the current stage. In the present invention, the state of each stage includes node features, graph features and decoding context features, where node features and graph features are the output of the encoder, respectively. Decoding context features It is a feature based on inherited properties.
[0114] Since the switching time of the selected workpiece in this stage depends on the property of the workpiece selected in the previous stage, the features of the workpiece selected in the previous stage are used as decoding context features at each decoding moment. For example, in the initial stage, a learnable parameter vector v is set f Instead of selecting the features of the artifact in the previous stage, the specific formula is as follows: where t represents the number of decoding steps.
[0115]
[0116] In the formula, t represents the number of decoding steps;
[0117] Represents the node features corresponding to the node selected in the decoding process of step t-1;
[0118] The action space A represents the set of selectable actions at each stage. In order to avoid generating infeasible processing sequences, an action mask based on an exponential function is introduced to cover the actions that do not meet the requirements to simplify the action space;
[0119] The state transition P is used to express the probability of transitioning from the current state to another state. In the present invention, the state transition is deterministic, and random factors such as workshop machine failure or temporary damage to the workpiece are not considered;
[0120] The reward function R represents the job sorting problem. The decision of the present invention is to determine the value, so the reward is a function of the final sort π.
[0121] The decoder includes a first-layer multi-head masked attention mechanism and a second-layer single-head masked attention mechanism. The first-layer multi-head masked attention mechanism contains P attention calculations, which converts the current state The nonlinear mapping is The second layer is a single-head mask attention mechanism that calculates the mapped state With each node feature The attention weight is regarded as the probability of the workpiece being selected in the current stage.
[0122] The specific calculation process of decoding step t is as follows:
[0123] (1) Calculate the state vector under the p-th attention The query vector q sp , Node Features The key vector k ip Sum value vector v ip :
[0124]
[0125]
[0126]
[0127] In the formula, and is the parameter matrix.
[0128] (2) Calculate the state vector under the p-th attention Compatibility with the i-th node
[0129]
[0130] Where, d k It is k ip Dimensions;
[0131] Represents the query vector q sp The transposed vector of ;
[0132] i represents a node or artifact;
[0133] π t′ represents the artifact selected by the decoding process at step t′;
[0134] represents any t step;
[0135] (3) Calculate the attention weight under the pth attention
[0136]
[0137] (4) Calculate the original state vector under the p-th attention The representation vector h sp :
[0138]
[0139] (5) Aggregate the original state vector under P attentions The representation vector h sp :
[0140]
[0141] In the formula, Represents the state vector after learning;
[0142] W out is the parameter matrix;
[0143] (6) Calculation The query vector The key vector k i ;
[0144]
[0145]
[0146] Where W Q and W k is the parameter matrix;
[0147] (7) Calculation and Compatibility
[0148]
[0149] In the formula, C is a fixed constant;
[0150] tanh is the hyperbolic tangent function;
[0151] Represents the query vector The transposed vector of ;
[0152] d k It is k i Dimensions;
[0153] (8) Calculate the probability p of each node being selected i ;
[0154]
[0155] In the formula, π t Represents the artifact selected by the decoding process at step t.
[0156] like Figure 7-9 As shown, the scheduling model is optimized based on the offline reinforcement learning algorithm. The specific optimization process is as follows:
[0157] S211. Obtain several problem instances;
[0158] S212, inputting the problem instance into the scheduling model, generating a probability distribution of a processing sequence, and generating a first processing sequence by random sampling and a second processing sequence by greedy search;
[0159] S213, calculating a first objective function value of the problem instance in the first processing sequence, and calculating a second objective function value of the problem instance in the second processing sequence;
[0160] S214. Calculate the gradient of the scheduling model parameters using the REINFORCE algorithm;
[0161] S215. Based on the gradient of the scheduling model parameters, the scheduling model parameters are updated using a gradient descent algorithm to obtain a minimum objective function value.
[0162] Specifically, from a series of instances whose problem parameters obey a certain distribution, a number of problem instances S = {s1,…s i ,…};
[0163] Problem instances i Input to the encoder-decoder (scheduling model) to obtain the probability distribution p of the processing sequence θ (π|s i ), generate the first processing sequence π by random sampling i And generate the second processing sequence through greedy search Two different algorithms are used for calculation, making the offline reinforcement learning algorithm more robust and effective.
[0164] Computational Problem Examples i In the first processing sequence, π i The first objective function value L(π i |s i ), in the second processing sequence The second objective function value when
[0165] Calculate the gradient of the encoder-decoder parameter θ, using the gradient update formula of the REINFORCE algorithm with the baseline as follows;
[0166]
[0167] The encoder-decoder parameters θ are updated using the gradient descent algorithm to minimize the objective function value, which is expressed as:
[0168]
[0169] Where θ′ represents the candidate encoder-decoder parameters;
[0170] lr is the learning rate;
[0171] like Fig. 9 As shown, a paired sample one-sided t-test is used to determine whether the parameters of the encoder-decoder need to be updated, including:
[0172] S2151. Obtain several test problem instances;
[0173] S2152, input the inspection problem instance into the current scheduling model, generate the current processing sequence, and obtain the current objective function value, input the inspection problem instance into the updated scheduling model, generate the updated processing sequence, and obtain the updated objective function value;
[0174] S2153, based on the current objective function value and the updated objective function value, obtaining a sample difference, a sample difference mean, and a sample difference standard deviation;
[0175] S2154. Determine a significance level based on the sample difference mean;
[0176] S2155. Construct statistics based on the sample difference mean;
[0177] S2156. Based on the constructed statistic, obtain a probability value of the statistic;
[0178] S2157. When the probability value is less than the significance level value, the update scheduling model is updated; when the probability value is not less than the significance level value, the update scheduling model is not updated.
[0179] Specifically, a number of test problem instances are randomly selected from a series of instances whose problem parameters obey a certain distribution.
[0180] The problem instance will be tested Input to the encoder-decoder with parameters θ and θ′ respectively, and generate the current processing sequence through greedy search and And calculate the current objective function value and update the objective function value
[0181] Calculate sample difference Sample mean difference and the sample standard deviation
[0182] Determine the null hypothesis Alternative hypothesis Determine the significance level α;
[0183] Calculate the construction statistic t value,
[0184] Calculate the probability value P of the statistic r , if P r If P is less than the significance level α, the update scheduling model is updated, that is, the parameter θ←θ′ is updated; if P r If it is not less than the significance level α, the update scheduling model will not be updated.
[0185] S3. Use hybrid beam search and priority scheduling rules to optimize the pre-processing sequence set to obtain the production processing sequence set.
[0186] like Fig.10 and Fig.11 As shown, in step S3, the specific optimization process includes:
[0187] S31, initializing a set of empty sequences and initializing sub-algorithm weights;
[0188] S32, obtaining a new local sequence based on the set;
[0189] S33, assigning the sub-algorithm weight to the new local sequence;
[0190] S34, executing steps S32-S33 until a complete sequence is generated;
[0191] S35, calculating the objective function value of the complete sequence.
[0192] Specifically, a set P containing k empty sequences is initialized to store the pre-processed sequence set; initialization sub-algorithm weights w1, w2, w3 are set, where w1+w2+w3=1;
[0193] For the local sequences in the set P: the trained encoder-decoder model is decoded through beam search to obtain k new local sequences with the highest probability, denoted as P1; the minimum switching time priority scheduling rule is used to obtain k new local sequences, denoted as P2; the earliest delivery priority scheduling rule is used to obtain k new local sequences, denoted as P3;
[0194] According to the sub-algorithm weights w1, w2, w3, w1*k local sequences are extracted from P1 according to the probability distribution, denoted as Extract w2*k local sequences from P2 according to uniform distribution, denoted as Extract w3*k local sequences from P3 according to uniform distribution, denoted as make
[0195] Repeat steps S32-S33 until a complete sequence is generated;
[0196] Calculate the set P = {π 1 ,…,π k ,…,π K}, and select the sequence with the smallest objective function value as the solution.
[0197] further, is the kth processing sequence in the set P, L k is the objective function value of processing sequence k, is the completion time of processing sequence k, T k is the delay time of processing sequence k, is the completion time of the tth job in processing sequence k, is the delay time of the tth workpiece in processing sequence k, is the completion time of the tth workpiece in processing sequence k on machine m; is the delivery time of the tth job in processing sequence k.
[0198]
[0199]
[0200]
[0201]
[0202]
[0203]
[0204] In the formula, is the processing time of the tth workpiece in processing sequence K on machine m;
[0205] is the switching time of the Kth workpiece in the processing sequence on machine m.
[0206] S4. Obtain a workshop scheduling and processing plan based on the production and processing sequence set and the workpiece processing target.
[0207] Specifically, when the objective function is the main factor, the final scheduling solution is selected from the production and processing sequence set according to the delay time of the workpiece.
[0208] Based on the permutation flow workpiece processing target considering the equipment switching time, the present invention proposes a representation method based on a directed graph of multi-dimensional edge weight features to solve the scheduling problem with a variable number of workpieces; by designing a graph attention mechanism encoder that integrates multi-dimensional edge weight features, the implicit features of the permutation flow workpiece processing target considering the equipment switching time are effectively learned; and in view of the problem characteristics of the objective function being related to the processing workpiece sequence, a context feature decoding method based on inheritance characteristics is designed, and a hybrid reinforcement learning algorithm that integrates priority scheduling rules and beam search decoding is proposed for optimization, thereby realizing an accurate algorithm that can respond quickly to external changes, and can provide a feasible production plan in a relatively short time.
[0209] The hybrid reinforcement learning algorithm provided by the present invention, which integrates the priority scheduling rule and beam search decoding, can efficiently solve a type of permutation flow workpiece processing target that takes into account the equipment switching time. In terms of time, the algorithm provided by the present invention is significantly better than commercial solvers and genetic algorithms; in terms of solution quality, it is significantly better than the genetic algorithm and close to the solution obtained by running the commercial solver for 600 seconds. Table 1 gives the solution time and solution values of different algorithms under three problem scales with the number of workpieces N = 15, 25 and 35, and gives the solution gap with the target value obtained by running Gurobi for 600 seconds as the baseline; DRL_Greedy represents the traditional deep reinforcement learning algorithm using greedy search; DRL_Hybrid represents the hybrid reinforcement learning algorithm integrating the priority scheduling rule and beam search decoding, Gurobi1, Gurobi2, Gurobi3 respectively represent the use of Gurobi solvers under different time constraints, and GA represents the genetic algorithm.
[0210] Table 1 Analysis of the performance of the algorithm of the present invention under different problem scales
[0211]
[0212] The hybrid reinforcement learning algorithm proposed by the present invention, which integrates priority scheduling rules and beam search decoding, can generalize and solve instances of different problem sizes (only when the number of workpieces changes) and has good solution quality. The present invention sets up three experiments to verify whether the trained model can achieve good performance when the number of nodes changes. The experiments are respectively to solve the problem of 25 workpieces with the model trained with 15 workpieces, to solve the problem of 35 workpieces with the model trained with 25 workpieces, and to solve the problem of 50 workpieces with the model trained with 35 workpieces.
[0213] Table 2 Generalization performance analysis of the algorithm of the present invention under different problem scales
[0214]
[0215]
[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A control method for workpiece processing scheduling based on multi-dimensional edge weight graph attention reinforcement learning, characterized in that: include: Based on the workpiece processing target, a multi-dimensional edge weight feature directed graph is constructed; Constructing a scheduling model, inputting the multi-dimensional edge weight feature directed graph into the scheduling model, generating a pre-processing sequence set, and optimizing the scheduling model based on an offline reinforcement learning algorithm; Utilizing a hybrid beam search and a priority scheduling rule algorithm, the pre-processing sequence set is optimized to obtain a production processing sequence set; Based on the production processing sequence set and the workpiece processing target, a workshop scheduling processing plan is obtained; The workpiece processing target includes problem parameters and target function; The problem parameters include workpiece processing time, workpiece switching time, and workpiece delivery time; The objective function includes the completion time and delay time of the workpiece; The multi-dimensional edge-weighted directed graph includes nodes and directed edges, wherein the nodes represent workpieces and are represented by the workpiece processing time and the workpiece delivery time; The directed edges represent the workpiece processing sequence, which is represented by the workpiece switching time; The scheduling model includes an encoder and a decoder; The encoder is constructed using a graph attention mechanism of multi-dimensional edge weight features to generate final node features and global features of the directed graph; The decoder generates a set of pre-processed sequences based on the global features of the directed graph using Markov decision making; The encoder comprises: Calculate the importance of the features of adjacent node j to node i, and use the activation function to perform nonlinear transformation on the importance value; Based on the importance value, the normalization algorithm is used to obtain the attention weight of the neighboring node j on the new feature generated by node i; After weighted averaging the adjacent features of node i according to the attention weight, a new feature of node i is obtained by nonlinear transformation; Perform multiple rounds of learning on the new features of node i to obtain the final node features and the global features of the directed graph; The scheduling model is optimized based on an offline reinforcement learning algorithm, including: Obtain several problem instances; Inputting the problem instance into the scheduling model, generating a probability distribution of a processing sequence, and generating a first processing sequence by random sampling and a second processing sequence by greedy search; Calculating a first objective function value of the problem instance in the first processing sequence, and calculating a second objective function value of the problem instance in the second processing sequence; Using the REINFORCE algorithm, the gradient of the scheduling model parameters is calculated; Based on the gradient of the scheduling model parameters, the scheduling model parameters are updated using a gradient descent algorithm to obtain a minimum objective function value; The method of optimizing the pre-processing sequence set by using a hybrid beam search and a priority scheduling rule algorithm to obtain a production processing sequence set includes: S31, initializing a set of empty sequences and initializing sub-algorithm weights; S32, obtaining a new local sequence based on the set; S33, assigning the sub-algorithm weight to the new local sequence; S34, executing steps S32-S33 until a complete sequence is generated; S35, calculating the objective function value of the complete sequence; The obtaining of a new local sequence based on the set comprises: Use beam search decoding to train the scheduling model and obtain the k new local sequences with the highest probability, denoted as P1; By optimizing the scheduling rule with the minimum switching time, k new local sequences are obtained, denoted as P2; Using the earliest delivery priority scheduling rule, k new local sequences are obtained, denoted as P3; The step of assigning the sub-algorithm weight to the new local sequence comprises: According to the sub-algorithm weights w1, w2, w3, w1*k local sequences are extracted from P1 according to the probability distribution, denoted as Extract w2*K local sequences from P2 according to uniform distribution, denoted as Extract w3*K local sequences from P3 according to uniform distribution, denoted as make 2. The control method for workpiece processing scheduling based on multi-dimensional edge weight graph attention reinforcement learning according to claim 1 is characterized in that: According to the paired sample one-sided test method, whether the scheduling model needs to be updated is judged, specifically including: Obtain several test problem instances; Input the inspection problem instance into the current scheduling model, generate the current processing sequence, and obtain the current objective function value, input the inspection problem instance into the updated scheduling model, generate the updated processing sequence, and obtain the updated objective function value; Based on the current objective function value and the updated objective function value, a sample difference value, a sample difference value mean value and a sample difference value standard deviation value are obtained; Determining a significance level based on the sample mean difference; Constructing statistics based on the sample difference mean; Based on the constructed statistic, obtaining a probability value of the statistic; When the probability value is less than the significance level value, the update scheduling model is updated; when the probability value is not less than the significance level value, the update scheduling model is not updated.
