Fuzzy shop scheduling method and system based on deep reinforcement learning and graph neural network
Through the deep reinforcement learning and graph neural network methods, the complexity and fuzziness of the fuzzy workshop scheduling problem are solved, efficient scheduling decisions are achieved, production efficiency and generalization performance are improved, and it is suitable for large-scale and complex practical applications.
Patent Information
- Application Number
- CN202410978531.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing technology is difficult to effectively solve the complexity and ambiguity of fuzzy workshop scheduling problems, especially when solving large-scale problems, the calculation complexity is high and the solution efficiency is low, and the actual needs cannot be met.
Using a method based on deep reinforcement learning and graph neural network, the graph neural network model is constructed, the features in the fuzzy workshop scheduling problem are extracted and transformed into Markov decision-making process. The decision-making is optimized using the Actor-Critic model and the near-end strategy optimization algorithm. Network, realize fuzzy workshop scheduling decisions.
It realizes efficient solution to fuzzy workshop scheduling problems, improves production efficiency, has good generalization performance, and can achieve good results in large-scale and complex practical applications.
Smart Images

Figure CN118521130B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of fuzzy shop scheduling, and specifically to a fuzzy shop scheduling method and system based on deep reinforcement learning and graph neural network. Background Art
[0002] The workshop production scheduling problem is one of the important problems in production management, and the fuzzy workshop scheduling problem takes into account the uncertainty in the production process. The solution of the fuzzy workshop scheduling problem is of great significance for improving production efficiency and reducing production costs. However, due to the complexity and ambiguity of the problem itself, it is difficult to solve, and the current traditional methods are often difficult to solve effectively. Especially in solving large-scale problems, traditional methods cannot meet actual needs due to their high computational complexity and low solution efficiency.
[0003] In the existing research on deep reinforcement learning and graph neural networks, although general workshop scheduling problems can be handled, it is difficult to directly apply existing methods to solve fuzzy workshop scheduling problems due to the complexity brought by fuzziness. Especially in feature extraction and scheduling decisions, fuzziness increases the difficulty of the problem. Therefore, how to effectively handle fuzzy workshop scheduling problems and solve them by leveraging the advantages of deep reinforcement learning and graph neural networks is an important challenge in the current technological development. Summary of the invention
[0004] The purpose of the present invention is to provide a fuzzy workshop scheduling method and system based on deep reinforcement learning and graph neural network to make up for the shortcomings of the prior art.
[0005] By combining the advantages of deep reinforcement learning and the characteristics of graph neural networks, the present invention makes full use of the graph structure information in the scheduling problem, can effectively extract and utilize the features in the fuzzy shop scheduling problem, and thus achieves an efficient solution to the fuzzy shop scheduling problem. In addition, the method can also adaptively handle scheduling problems of different scales, has good generalization performance, and can achieve good results in large-scale and complex practical applications.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] A fuzzy shop scheduling method based on deep reinforcement learning and graph neural network includes the following specific steps:
[0008] S10: Modeling the fuzzy shop scheduling problem to obtain a fuzzy shop scheduling mathematical model;
[0009] S20: randomly generating fuzzy workshop scheduling data according to the workshop scheduling data;
[0010] S30: Build a graph neural network model and use it to extract features;
[0011] S40: converting the fuzzy workshop scheduling mathematical model into a Markov decision process, completing the design of the state space, action space and reward function, so as to construct a simulation scheduling environment;
[0012] S50: Construct the strategy network and value network to obtain the Actor-Critic model;
[0013] S60: The features extracted by the graph neural network model are used as the input of the Actor-Critic model. The Proximal Policy Optimization (PPO) algorithm is used to optimize the Actor-Critic network model to obtain a fuzzy shop scheduling decision model.
[0014] S70: The fuzzy job shop scheduling decision model gives the scheduling action to be taken in the next step according to the fuzzy job shop scheduling scenario, and completes the solution of the fuzzy job shop scheduling problem.
[0015] Furthermore, the step S10 is specifically as follows:
[0016] S11: Determine the definition of the fuzzy scheduling problem, based on the shop floor scheduling problem, where the duration and completion time of the task are fuzzy and are represented using triangular fuzzy numbers;
[0017] S12: Determine the objective function of the model. In the fuzzy shop scheduling problem, the objective function is to minimize the maximum fuzzy completion time. ;
[0018] S13: Describe the constraints of the fuzzy scheduling problem. In the fuzzy scheduling problem, each process can only be performed on a fixed machine, the working time of each machine cannot overlap, and the processes of each task must be completed in a fixed order;
[0019] S14: The duration of each process and the completion time of the task are expressed by triangular fuzzy numbers, and the fuzziness is introduced into the scheduling problem to fuzzify the constraints.
[0020] Furthermore, the step S20 is specifically as follows:
[0021] S21: Determine the common problem scale of the fuzzy shop floor scheduling problem, the number of tasks j and the number of machines m, and the time s required for the process of each task;
[0022] S22: For each process of each task, generate a real number to represent the time required;
[0023] S23, use triangular fuzzy numbers to represent the time required for the process, fuzzify the generated real number, generate its left and right boundaries, and assume that To represent randomly generated fuzzy numbers, we have:
[0024] (9);
[0025] S24. The generated fuzzy shop scheduling data is saved as a data set for use in subsequent fuzzy shop scheduling problem solving.
[0026] Furthermore, the step S30 is specifically as follows:
[0027] S31: The graph neural network model is a multi-channel graph convolutional neural network, and the three channels contain two graph convolutional layers with a dimension of 64. Relu is used as the activation function between the two graph convolutional layers;
[0028] S32: Define a parallel disjunctive graph to represent the fuzzy shop scheduling problem:
[0029] (1);
[0030] Among them, V is the vertex set in the disjunctive graph, representing all process nodes, as well as the nodes representing the start and end of scheduling; A is a set of directed arcs, and D is a disjunctive arc;
[0031] S33: Designing a multi-channel graph convolutional neural network to extract disjunctive graphs The node features in the multi-channel convolution operation are expressed as:
[0032] (2);
[0033] Where A represents the adjacency matrix of the graph, It means to find the inverse of the degree matrix and take the square root, c represents the channel, and each channel is subjected to k layers of graph convolution operations to obtain the channel feature vector , and each channel contains a weight matrix , each channel will use the mean pooling operation to extract the global features of the channel corresponding map before output. The mean pooling operation is defined as:
[0034] (3);
[0035] Then concatenate the pooled global features and the original channel features as the output of the channel:
[0036] (4);
[0037] Finally, the output vector of all channels Connect as the output of the network :
[0038] (5).
[0039] Further, the S40 includes the following:
[0040] State design, defining a state space that can represent all possible shop scheduling states;
[0041] Action design, defining all possible scheduling actions and assigning a process to a machine for execution;
[0042] Reward function design, based on the estimated completion time of fuzzy shop scheduling, a reward function is given .
[0043] Furthermore, the S40 specifically includes the following steps:
[0044] S41: Node feature design and scheduling state design. At each scheduling time step t, the state is the disjunction graph of the current state:
[0045] (6);
[0046] Among them, the initial state is the original disjunctive graph of the fuzzy shop scheduling task, and the final state It is the solution after all process scheduling is completed, that is, the solution graph of deconstruction is completed;
[0047] S42: Action selection, the reinforcement learning model directly selects the process execution, action Set as the legal process that can be selected at the current scheduling time step t. As the process is gradually executed, the set of selected processes The scale is getting smaller and smaller;
[0048] S43: Reward design: After executing the selected process, the increment of the fuzzy completion time is calculated as the reward item, which is:
[0049] (7);
[0050] S44: Simulation scheduling environment:
[0051] Based on the MDP design results and the mathematical model of the fuzzy shop scheduling problem, a fuzzy scheduling simulation environment is constructed. The environment loads fuzzy scheduling tasks, the reinforcement learning model outputs scheduling actions, the scheduling environment executes scheduling actions, and completes state transfer to give rewards corresponding to the actions. The reinforcement learning model uses the experience generated by continuous interaction with the simulation scheduling environment to optimize the network model.
[0052] Furthermore, the step S50 is specifically as follows:
[0053] S51, Policy Network Depend on The network weight parameter is , and has 2 hidden layers with a dimension of 64, uses tanh as the activation function, and the output dimension is the number of processes; after the embedding vector of the multi-channel graph convolutional neural network is input into the policy network, the policy network outputs the priority vector of the action, which is a scalar matrix specifically:
[0054] (8);
[0055] Generate a mask vector m based on the scheduling tags of all processes, and apply the mask to the policy network output to obtain the modified priority matrix , as follows:
[0056] , (9);
[0057] use Function processing priority matrix Get the probability distribution of process selection:
[0058] (10);
[0059] S52, Value Network The value network adopts a hidden layer architecture similar to the policy network, which shares a multi-channel graph neural network with the policy network. The weight matrix is , with 2 hidden layers of dimension 32, using ReLU as the activation function. The value network input is the global features of the graph obtained after global mean pooling , which is obtained by concatenating the global features of each channel:
[0060] (11);
[0061] The output of the value network is the value of the current state and the state-action pair, which is specifically expressed as:
[0062] (12).
[0063] Furthermore, the step S60 is specifically as follows:
[0064] S61: Initialize the parameters of the Actor and Critic networks;
[0065] S62: read in a set of manually generated fuzzy shop scheduling numbers and use the Actor network to make decisions to select scheduling actions;
[0066] S63: According to the action taken and the data set, combined with the current environment state, a new state and reward are obtained, and the target value is calculated using the reward and the output of the value network in the new state to update the value network;
[0067] S64: Use the output of the value network in the new state and the actual reward to calculate the advantage function, and use the advantage function to update the policy network;
[0068] S65: Repeat the above steps until the policy network and the value network converge, and save the optimal decision network to solve the fuzzy workshop scheduling problem.
[0069] Furthermore, the step S70 is specifically as follows:
[0070] S71: Selecting the optimal fuzzy workshop scheduling decision model according to the current fuzzy workshop scheduling scenario;
[0071] S72: Use The function outputs the probability of each action in the scheduling process, and samples from the action probability distribution to obtain the scheduling action sequence; the actual fuzzy scheduling production task is loaded into the simulation scheduling environment, and the fuzzy workshop scheduling decision model uses a greedy strategy to gradually select the action with the highest probability. ,Right now:
[0072] (13);
[0073] Perform actual scheduling based on the probability distribution of the model output;
[0074] S73: Save and evaluate the scheduling order output by the model, and retain the scheduling time of the model.
[0075] A fuzzy shop scheduling system based on deep reinforcement learning and graph neural network, including:
[0076] Fuzzy scheduling mathematical modeling module, used to establish and save the mathematical model of fuzzy scheduling problems to ensure the constraints and fuzziness of fuzzy problems;
[0077] The fuzzy scheduling data generation module is used to generate artificial fuzzy shop scheduling data to provide enough data for the deep reinforcement learning algorithm to learn;
[0078] Markov decision model module, used to communicate fuzzy shop scheduling problems with deep reinforcement learning algorithms;
[0079] Graph neural network module, used to extract state change features during the scheduling process;
[0080] Decision and value network module, used to establish decision and optimize decision network models;
[0081] The proximal strategy optimization algorithm module is used to optimize the Actor-Critic network model and obtain the fuzzy shop scheduling decision model;
[0082] The policy network evaluation module is used to evaluate the solution performance under actual fuzzy shop scheduling application scenarios and data.
[0083] Compared with the prior art, the present invention has the following beneficial effects:
[0084] When faced with large-scale and complex problems, traditional optimization algorithms often fail to meet actual needs due to their high computational complexity and low solution efficiency.
[0085] The present invention adopts a fuzzy workshop scheduling method and system of deep reinforcement learning and graph neural network, which can effectively handle the complexity and ambiguity of fuzzy workshop scheduling problems; through a three-channel graph neural network, the state change characteristics in the scheduling process are extracted, which solves the difficulties caused by ambiguity to feature extraction and graph information embedding; through a proximal strategy optimization algorithm, the optimal decision network is obtained, and the efficient solution to the fuzzy workshop scheduling problem is achieved, thereby improving production efficiency; through deep reinforcement learning, the scheduling strategy network can be adjusted according to the rewards during training to obtain the optimal decision network, thereby achieving efficient solution to the fuzzy workshop scheduling problem. Due to the end-to-end capability of deep reinforcement learning technology, when dealing with scheduling problems of different scales, better generalization performance is obtained, and good results can be achieved in large-scale and complex practical applications, realizing a comprehensive and efficient solution to the fuzzy workshop scheduling problem. The present invention has strong practicality and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0086] Figure 1 This is a flow chart of the fuzzy shop scheduling method based on deep reinforcement learning and graph neural network provided by the present invention.
[0087] Figure 2 This is a schematic diagram of the decomposition graph structure provided by the present invention.
[0088] Figure 3 A block diagram of a solution to a fuzzy scheduling task instance provided by the present invention.
[0089] Figure 4 A Gantt chart is solved for the fuzzy scheduling task instance provided by the present invention.
[0090] Figure 5 This is a structural block diagram of the fuzzy shop scheduling system based on deep reinforcement learning and graph neural network provided by the present invention. DETAILED DESCRIPTION
[0091] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0092] Embodiment 1:
[0093] like Figure 1 As shown in the figure, a method for solving the fuzzy shop scheduling problem based on deep reinforcement learning and graph neural network specifically includes the following steps:
[0094] S10, modeling the fuzzy shop scheduling problem to obtain a fuzzy shop scheduling mathematical model;
[0095] S11. The objective function of the fuzzy shop scheduling problem is to minimize the maximum fuzzy completion time, which is mathematically expressed as ,in The total time required from the start to the completion of the scheduling process is the maximum completion time. By modeling the constraints in the scheduling problem, the resource allocation problem in the fuzzy scheduling problem can be quantified. In the fuzzy workshop scheduling problem, triangular fuzzy numbers are used. The mathematical definition of triangular fuzzy numbers is as follows:
[0096] A fuzzy number is a fuzzy set on a given domain U, which refers to any , there is a number Correspondingly, is called the membership degree of x to U, μ is called the membership degree of x, which is also a fuzzy number. Triangular fuzzy number is a special fuzzy number. is a triangular fuzzy number with a membership function The expression is as follows:
[0097] (14);
[0098] In the fuzzy scheduling problem, since the duration of the process is fuzzy, the start time of the process is also fuzzy. Its three components are respectively To represent, j is the job number, m is the machine number, so each process can be represented by the interval To indicate that , in the formula is the duration of the process of job j on machine m. A quantity.
[0099] Since a machine can only run one process at a time, the machine has a unary resource constraint. The machine m can be defined as a sequence variable In the context of fuzzy scheduling, each machine needs to be defined as three machines that follow the same processing order, using three sequence variables To express. Include all intervals ,in , all tasks should be in the same order in and For processing job j and There are two possibilities for the required machine m:
[0100] (15);
[0101] or:
[0102] (16);
[0103] The calculation formula for the maximum completion time constraint is as follows:
[0104] (17);
[0105] Where J represents the set of all processes, Represents triangular fuzzy number The subscript of is the start time of job j on machine m Quantity, is the duration of the process of job j on machine m. Quantity, is the last machine that processes job j.
[0106] To ensure that operations on a machine do not overlap, there are:
[0107] (18);
[0108] To maintain the integrity of the problem, that is, to schedule tasks in the same order for each fuzzy component, we have:
[0109] (19);
[0110] Ensure that the job priority constraints remain unchanged through mathematical constraints:
[0111] (20);
[0112] In the scheduling process, in order to ensure that each start time is a correct triangular fuzzy number, we have:
[0113] (twenty one).
[0114] S20, randomly generating fuzzy workshop scheduling data according to the workshop scheduling data;
[0115] S21. Determine the scale of the problem.
[0116] S22. For each process of each task, generate a real number to represent the approximate time required.
[0117] S23, use triangular fuzzy numbers to represent the time required for the process, fuzzify the generated real number, generate its left and right boundaries, and assume that To represent randomly generated fuzzy numbers, we have:
[0118] (twenty two);
[0119] in Set to 0.85, Set to 1.3.
[0120] S24. The generated fuzzy shop scheduling data is saved as a data set for use in subsequent fuzzy shop scheduling problem solving.
[0121] S30. Design a graph neural network model. Use a decomposition graph to describe the state changes in the fuzzy scheduling process, and use the graph neural network to extract features;
[0122] According to the characteristics of triangular fuzzy numbers, isomorphic disjunctive graphs are constructed for the left bound, peak value and boundedness of triangular fuzzy numbers to describe the fuzzy scheduling process. According to the mathematical model constraints of S10, three isomorphic disjunctive graphs are constructed to represent the fuzzy scheduling problem. The problem is solved by determining the direction of the disjunctive arc. The feature extraction of isomorphic disjunctive graphs is realized by using a multi-channel graph convolutional neural network, as follows:
[0123] S31. Define a parallel disjunctive graph:
[0124] (twenty three);
[0125] Represents the fuzzy shop scheduling problem. Where V is the vertex set in the disjunctive graph, representing all process nodes, as well as the nodes representing the start and end of scheduling. Due to the fuzziness of the problem, the characteristics of process nodes in different parallel disjunctive graphs are different. A is a set of directed arcs, representing the priority constraints between processes of the same job. D is a disjunctive arc, which undirectedly connects a pair of processes that need to be processed by the same machine.
[0126] Solving the fuzzy job shop scheduling problem based on the parallel disjunctive graph is to find a feasible solution. ,according to Determine the process scheduling sequence, and then determine the direction of the disjunctive arc in the parallel disjunctive graph, complete the graph deconstruction, and obtain the subgraph is the solution. Denotes the Hamiltonian connection of disjunctive arcs in D. According to By forward propagation, the fuzzy start time and end time of all processes can be obtained, and then the maximum fuzzy completion time can be obtained. .
[0127] like Figure 2 As shown, the fuzzy scheduling parallel solution is obtained according to the scheduling order Next, complete the deconstruction of the disjunctive graph.
[0128] S32. Design a multi-channel graph convolutional neural network to extract disjunctive graphs The node features in are used for subsequent reinforcement learning models.
[0129] In the multi-channel graph convolutional neural network, channels are set for each of the three components of the triangular fuzzy number to complete the node embedding of the parallel disjunctive graph. The multi-channel convolution operation is expressed as:
[0130] (twenty four);
[0131] Where A represents the adjacency matrix of the graph, It means to find the inverse of the degree matrix and take the square root. c represents the channel. After each channel undergoes k layers of graph convolution operations, the channel feature vector is obtained. , and each channel contains a weight matrix Each channel will use the mean pooling operation to extract the global features of the channel corresponding map before output. The mean pooling operation is defined as:
[0132] (25);
[0133] Then the pooled global features and the original channel features are concatenated as the output of the channel.
[0134] (26);
[0135] Finally, the output vector of all channels Connect as the output of the network :
[0136] (27).
[0137] Each channel in the multi-channel graph convolutional neural network contains two graph convolution layers with a dimension of 64, and Relu is used as the activation function between the two graph convolution layers.
[0138] S40, converting the fuzzy workshop scheduling mathematical model into a Markov decision process, completing the design of the state space, action space and reward function, and building a simulation scheduling environment.
[0139] First, based on the fuzzy shop scheduling mathematical model, the mathematical constraints in the scheduling process are determined and then transformed into a Markov decision process. The key elements of the constructed Markov decision process include:
[0140] S41. Node feature design and scheduling status design:
[0141] At each scheduling time step t, the state is the disjunction graph of the current state:
[0142] (28);
[0143] Therefore, the initial state is the original disjunctive graph of the fuzzy shop scheduling task. And the final state It is the solution after all process scheduling is completed, that is, the solution graph of the deconstruction is completed.
[0144] The characteristics of each process node in the parallel extraction graph include the following features:
[0145] 1) Scheduling flag: It is represented by binary code, which is 1 when the process is scheduled, otherwise it is 0;
[0146] 2) Processing time: The operation time of each disjunctive graph corresponding to the triangular fuzzy number.
[0147] 3) Expected completion time of the process: The earliest expected completion time of the process under each disjunction diagram.
[0148] 4) Machine idle time: The possible idle time of the machine before scheduling the process.
[0149] S42, action selection:
[0150] The reinforcement learning model directly selects the process execution and action Set to the legal process that can be selected at the current scheduling time step t. The selected process should meet the priority constraints of the job. As the process is gradually executed, the set of selectable processes The scale is getting smaller and smaller.
[0151] S43, Reward Design:
[0152] After executing the selected process, the increment of the maximum fuzzy completion time is calculated as the reward item. That is:
[0153] (29).
[0154] S44, simulation scheduling environment:
[0155] Based on the MDP design results and the mathematical model of the fuzzy shop scheduling problem, a fuzzy scheduling simulation scheduling environment is constructed. The environment loads fuzzy scheduling tasks, the reinforcement learning model outputs scheduling actions, the scheduling environment executes scheduling actions, and completes state transfer to give rewards corresponding to the actions. The reinforcement learning model uses the experience generated by continuous interaction with the simulation scheduling environment to optimize the network model.
[0156] S50, construct a policy network and a value network to obtain an Actor-Critic model; define a network for inputting environmental information in the current state, outputting a probability distribution, and selecting the next action through the probability distribution. This network is the Actor in the Actor-Critic model, that is, the decision network; define a network for evaluating the value of selecting an action in a given state. The value means the expected future return of selecting this action. This network is the Critic in the Actor-Critic model, that is, the value network; combine the decision network and the value network to obtain the Actor-Critic network model. The present invention constructs a reinforcement learning model based on the Actor-Critic architecture, so it is necessary to construct a corresponding policy network and value network .
[0157] S51, Policy Network Depend on The network weight parameter is , and has 2 hidden layers with a dimension of 64, uses tanh as the activation function, and the output dimension is the number of processes. After the embedding vector of the multi-channel graph convolutional neural network is input into the policy network, the policy network outputs the priority vector of the action, which is a scalar matrix specifically:
[0158] (30).
[0159] Then, a mask vector m is generated based on the scheduling tags of all processes to mask the processes that have been scheduled, setting the legal scheduling action to 1 and the illegal scheduling action to negative infinity. Applying the mask to the output of the policy network yields the modified priority matrix , as follows:
[0160] (31);
[0161] use Function processing priority matrix Get the probability distribution of process selection:
[0162] (32).
[0163] S52, Value Network The value network adopts a hidden layer architecture similar to the policy network, which shares a multi-channel graph neural network with the policy network. The weight matrix is , with 2 hidden layers of dimension 32, using ReLU as the activation function. The value network input is the global features of the graph obtained after global mean pooling , which is obtained by concatenating the global features of each channel:
[0164] (33).
[0165] The output of the value network is the value of the current state and the state-action pair, which is specifically expressed as:
[0166] (34).
[0167] S60. Use the proximal policy optimization algorithm (PPO) to optimize the Actor-Critic network model and obtain the fuzzy workshop scheduling decision model.
[0168] The present invention selects the widely used PPO-clip algorithm that limits the difference between new and old strategies by clipping as the reinforcement learning model training algorithm. and value network The whole as a reinforcement learning network The parameters are recorded as According to importance sampling, the network will be optimized and the network applied to the environment Synchronize after a certain number of iterations, with a total of Episodes iterations. In each iteration, a fuzzy scheduling task instance is randomly generated in the simulation environment and Max_step is solved. Max_step is determined according to the instance size. During the solution process, the experience data obtained from the interaction is stored in the experience pool. Then, the data collected in the experience pool is used to update the reinforcement learning model Epochs times.
[0169] Adam was used as the network optimizer for training, with the total number of iterations set to Episodes=10000, the number of network updates per round to Epochs=3, the reward discount factor γ=1.0, and the learning rate In the PPO training algorithm parameters, set the shear coefficient , square error loss coefficient and entropy gain coefficient .
[0170] The iteration interval is set to 50 rounds, and the performance of the reinforcement learning strategy model is verified using a validation set constructed using 100 fuzzy scheduling instances.
[0171] S70, loading the actual fuzzy scheduling production task into the simulation scheduling environment, using the trained reinforcement learning scheduling decision model, selecting the scheduling process, and finally obtaining the scheduling plan.
[0172] S71. Read instance information, including the number of jobs in the instance and the number of machines required, that is, the number of processes for each job.
[0173] S72. Based on the information of the instance, use it as the input of the corresponding trained optimal decision network model, use the softmax function to output the probability of each action in the scheduling process, and sample from the action probability distribution to obtain the scheduling action sequence. Load the actual fuzzy scheduling production task into the simulation scheduling environment, and the fuzzy workshop scheduling decision model uses a greedy strategy to gradually select the action with the highest probability. ,Right now:
[0174] (35).
[0175] S73. Complete the scheduling of all processes step by step, obtain a complete scheduling result, calculate the fuzzy completion time of the solution, and output the time spent in the scheduling process and the scheduling diagram. Figure 3 A solution diagram for a fuzzy scheduling task instance is given. Figure 4 A Gantt chart for solving a fuzzy scheduling task instance is given.
[0176] Embodiment 2:
[0177] like Figure 5 As shown, a fuzzy workshop scheduling system based on deep reinforcement learning and graph neural network can specifically include a fuzzy scheduling mathematical modeling module 10, a fuzzy scheduling data generation module 20, a graph neural network module 30, a Markov decision model module 40, a decision and value network module 50, a proximal strategy optimization algorithm module 60, and a strategy network evaluation module 70:
[0178] Fuzzy scheduling mathematical modeling module 10: used to establish and save the mathematical model of the fuzzy scheduling problem to ensure the constraints and fuzziness of the fuzzy problem;
[0179] Fuzzy scheduling data generation module 20: used to generate artificial fuzzy shop scheduling data to provide sufficient data for deep reinforcement learning algorithm to learn;
[0180] Figure neural network module 30: used to extract state change features during the scheduling process;
[0181] Markov decision model module 40: used to communicate fuzzy shop scheduling problems and deep reinforcement learning algorithms;
[0182] Decision and value network module 50: used to establish decision and optimize decision network model;
[0183] Proximal strategy optimization algorithm module 60: used to optimize the Actor-Critic network model to obtain a fuzzy workshop scheduling decision model;
[0184] Strategy network evaluation module 70: used to evaluate the solution status under actual fuzzy shop scheduling application scenarios and data.
[0185] Embodiment 3:
[0186] Before this method was proposed, mathematical programming methods were often used to obtain the optimal solution to the fuzzy workshop scheduling problem. This method can find the optimal solution to a small-scale problem within a limited time (6 hours). However, when the problem scale is slightly larger, the time spent by the mathematical programming method is extremely long, far exceeding the acceptable range in practical applications. Secondly, heuristic algorithms are also widely used in this problem. They can always get a feasible solution to the problem within an acceptable time, but in larger-scale problems, the time spent by the heuristic method will also become a problem in the application. The scheduling rule method is also often used to compare with the deep reinforcement learning method. The solution efficiency of this method is considerable, but the effect is average.
[0187] In order to verify the effect of the fuzzy shop scheduling problem proposed in this invention, experiments were conducted on the datasets Fisher and Thomson's FT, Adams et al.'s ABZ, Applegate and Cook's ORB, and Taillard's Ta, which are commonly used baselines for fuzzy shop scheduling problems. Minimizing the maximum fuzzy completion time and the time spent in the scheduling process is used as the evaluation index.
[0188] Since the scale of the problem varies, the difficulty of a fuzzy scheduling instance is affected by two factors: the scheduling scale of the problem and the hardness of the instance. To express.
[0189] In order to better demonstrate the efficiency of various solution methods, the present invention proposes a formula for measuring the efficiency of problem solving: , where time is the time spent solving the fuzzy workshop scheduling instance. It can be obtained that the higher the efficiency of a solution method, the higher the value of this indicator. The average solution efficiency of the present invention is 170.85, the average solution efficiency of the best existing heuristic algorithm is 126.54, and the average solution efficiency of the best mathematical programming method is 1.40. Tables 1 and 2 record the average results of this method in minimizing the maximum fuzzy completion time and scheduling time, as well as the effect of this problem under the scheduling rule method.
[0190] Table 1
[0191] .
[0192] Table 2
[0193] .
[0194] In addition, this method trains models of different sizes, and each model can be applied to solve problems of all sizes. Table 3 records the performance of the decision model trained by this method with a data scale of 20 times 20 on data sets of different sizes.
[0195] Table 3
[0196] .
[0197] It can be seen from the above table that the present invention can obtain a feasible solution that minimizes the maximum fuzzy completion time while greatly reducing the time spent, thereby effectively improving the scheduling efficiency.
[0198] The present invention provides a method and system for solving fuzzy workshop scheduling problems based on deep reinforcement learning and graph neural networks, which can effectively handle the complexity and fuzziness of fuzzy workshop scheduling problems, and solve the problems of high computational complexity and low solution efficiency of traditional optimization algorithms when facing large-scale and complex problems. It improves the production efficiency of fuzzy workshop scheduling problems, and can also generalize and solve fuzzy workshop scheduling problems of various scales and ultra-large scales, and can achieve good results in complex practical applications.
[0199] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A fuzzy shop scheduling method based on deep reinforcement learning and graph neural network, It is characterized in that The specific steps include: S10: Modeling the fuzzy shop scheduling problem to obtain a fuzzy shop scheduling mathematical model; S20: randomly generating fuzzy workshop scheduling data according to the workshop scheduling data; S30: construct a graph neural network model, and use the graph neural network model to extract features; the specific details of step S30 are as follows: S31: The graph neural network model is a multi-channel graph convolutional neural network, and the three channels contain two graph convolutional layers with a dimension of 64. Relu is used as the activation function between the two graph convolutional layers; S32: Define a parallel disjunctive graph to represent the fuzzy shop scheduling problem: G l (σ)=(V, A∪D), l=1, 2, 3 (10) Where, the vertex set in the disjunctive graph represents all process nodes, A is a set of directed arcs, and Z is a disjunctive arc; S33: Design a multi-channel graph convolutional neural network to extract the disjunctive graph G l The node features in the multi-channel convolution operation are expressed as: Where A represents the adjacency matrix of the graph, D -1 / 2 It means to find the inverse of the degree matrix and take the square root, c represents the channel, and each channel is subjected to k layers of graph convolution operations to obtain the channel feature vector At the same time, each channel contains a weight matrix Each channel will use the mean pooling operation to extract the global features of the channel corresponding map before output. The mean pooling operation is defined as: Then concatenate the pooled global features and the original channel features as the output of the channel: Finally, the output vector H′ of all channels c Connect them together as the output H′ of the network: H′=Concat(H′ 1 ,H′ 2 ,...,H′ C ) (14); S40: converting the fuzzy workshop scheduling mathematical model into a Markov decision process, completing the design of the state space, action space and reward function, so as to construct a simulation scheduling environment; S50: Construct the strategy network and value network to obtain the Actor-Critic model; S60: The features extracted by the graph neural network model are used as the input of the Actor-Critic model, and the Proximal Policy Optimization (PPO) algorithm is used to optimize the Actor-Critic network model to obtain the fuzzy shop scheduling decision model. S70: The fuzzy job shop scheduling decision model gives the scheduling action to be taken in the next step according to the fuzzy job shop scheduling scenario, and completes the solution of the fuzzy job shop scheduling problem.
2. The fuzzy shop scheduling method according to claim 1, It is characterized in that The step S10 is specifically as follows: S11: Determine the definition of the fuzzy scheduling problem, based on the shop floor scheduling problem, where the duration and completion time of the task are fuzzy and are represented using triangular fuzzy numbers; S12: Determine the objective function of the model. In the fuzzy shop scheduling problem, the objective function is to minimize the maximum fuzzy completion time. S13: Describe the constraints of the fuzzy scheduling problem; S14: The duration of each process and the completion time of the task are expressed by triangular fuzzy numbers, and the fuzziness is introduced into the scheduling problem to fuzzify the constraints.
3. The fuzzy shop scheduling method according to claim 1, It is characterized in that The step S20 is specifically as follows: S21: Determine the common problem scale of the fuzzy shop floor scheduling problem, the number of tasks j and the number of machines m, and the time s required for the process of each task; S22: For each process of each task, generate a real number to represent the time required; S23: Use triangular fuzzy numbers to represent the time required for the process, fuzzify the generated real number, and generate its left and right boundaries. Assume that (x l ,x,x r ) to represent the randomly generated fuzzy number, then: S24: Save the generated fuzzy shop scheduling data as a data set.
4. The fuzzy shop scheduling method according to claim 1, It is characterized in that The S40 includes the following: (1) State design, defining a state space that can represent all possible shop scheduling states; (2) Action design, defining all possible scheduling actions and assigning a process to a machine for execution; (3) Reward function design: Based on the estimated completion time of fuzzy shop scheduling, a reward function is given. R(a t ,s t )=Δ(C max )=C max (t)-C max (t-1)。 5. The fuzzy shop scheduling method according to claim 4, It is characterized in that The S40 specifically includes the following steps: S41: Node feature design and scheduling state design. At each scheduling time step t, state s t is the disjunction graph of the current state: s t =[G 1 (σ(t)), G 2 (σ(t)), G 3 (σ(t))]=(V,A∪R(σ(t))) (15); Among them, the initial state s 0 is the original disjunctive graph of the fuzzy shop scheduling task, and the final state s T It is the solution after all process scheduling is completed, that is, the solution graph of deconstruction is completed; S42: Action selection, the reinforcement learning model directly selects the process to execute, action a t ∈A t Set as the legal process that can be selected at the current scheduling time step t. As the process is gradually executed, the selected process set A t The scale is getting smaller and smaller; S43: Reward design: After executing the selected process, the increment of the fuzzy completion time is calculated as the reward item, which is: R(a t ,s t )=Δ(C max )=C max (t)-C max (t-1) (16); S44: Simulation scheduling environment: Based on the MDP design results and the mathematical model of the fuzzy shop scheduling problem, a fuzzy scheduling simulation environment is constructed. The environment loads fuzzy scheduling tasks, the reinforcement learning model outputs scheduling actions, the scheduling environment executes scheduling actions, and completes state transfer to give rewards corresponding to the actions. The reinforcement learning model uses the experience generated by continuous interaction with the simulation scheduling environment to optimize the network model.
6. The fuzzy shop scheduling method according to claim 1, It is characterized in that The step S50 is specifically as follows: S51: Policy Network π θ Depend on The network weight parameter is θ π , and has 2 hidden layers with a dimension of 64, uses tanh as the activation function, and the output dimension is the number of processes; after the embedding vector of the multi-channel graph convolutional neural network is input into the policy network, the policy network outputs the priority vector of the action, which is a scalar matrix specifically: Generate a mask vector m according to the scheduling tags of all processes, apply the mask to the policy network output to obtain the modified priority matrix Pr′, as follows: The softmax function is used to process the priority matrix Pr′ to obtain the probability distribution of process selection: S52: Value Network v φ Adopts a hidden layer architecture similar to the policy network, which shares a multi-channel graph neural network with the policy network; the value network The weight matrix is θ φ , with 2 hidden layers of dimension 32, using ReLU as the activation function; the value network input is the global feature G′ obtained after global mean pooling c , which is obtained by concatenating the global features of each channel: The output of the value network is the value of the current state and the state-action pair, which is specifically expressed as:
7. The fuzzy shop scheduling method according to claim 1, It is characterized in that The step S60 is specifically as follows: S61: Initialize the parameters of the Actor and Critic networks; S62: read in a set of manually generated fuzzy shop scheduling numbers and use the Actor network to make decisions to select scheduling actions; S63: According to the action taken and the data set, combined with the current environment state, a new state and reward are obtained, and the target value is calculated using the reward and the output of the value network in the new state to update the value network; S64: Use the output of the value network in the new state and the actual reward to calculate the advantage function, and use the advantage function to update the policy network; S65: Repeat the above steps until the policy network and the value network converge, and save the optimal decision network to solve the fuzzy workshop scheduling problem.
8. The fuzzy shop scheduling method according to claim 1, It is characterized in that The step S70 is specifically as follows: S71: Selecting the optimal fuzzy workshop scheduling decision model according to the current fuzzy workshop scheduling scenario; S72: Use the softmax function to output the probability of each action in the scheduling process, sample from the action probability distribution to obtain the scheduling action sequence; load the actual fuzzy scheduling production task into the simulation scheduling environment, and the fuzzy workshop scheduling decision model uses a greedy strategy to gradually select the action with the highest probability a t ,Right now: Perform actual scheduling based on the probability distribution of the model output; S73: Save and evaluate the scheduling order output by the model, and retain the scheduling time of the model.
9. A fuzzy shop floor scheduling system based on deep reinforcement learning and graph neural network according to the method of claim 1, Its special feature is that include: Fuzzy scheduling mathematical modeling module, used to establish and save the mathematical model of fuzzy scheduling problems to ensure the constraints and fuzziness of fuzzy problems; The fuzzy scheduling data generation module is used to generate artificial fuzzy shop scheduling data to provide enough data for the deep reinforcement learning algorithm to learn; Markov decision model module, used to communicate fuzzy shop scheduling problems with deep reinforcement learning algorithms; Graph neural network module, used to extract state change features during the scheduling process; Decision and value network module, used to establish decision and optimize decision network models; The proximal strategy optimization algorithm module is used to optimize the Actor-Critic network model and obtain the fuzzy shop scheduling decision model; The policy network evaluation module is used to evaluate the solution performance under actual fuzzy shop scheduling application scenarios and data.
