Hybrid flow manufacturing workshop scheduling method and system based on monte carlo tree search algorithm

Abstract

The application discloses a mixed flow manufacturing workshop scheduling method based on a Monte Carlo tree search algorithm, first collects workshop operation data, processes the workshop operation data into Markov decision chain data, constructs a multi-input channel convolutional neural network, uses a DDQN algorithm with a Monte Carlo tree search algorithm as a reward function according to predicted values and predicted probabilities output by the multi-input channel convolutional neural network, expands, selects, simulates, reversely updates and trains each node, obtains a knowledge model, calls the knowledge model, and outputs an optimal scheduling decision according to real-time data of each resource information of a manufacturing workshop. The knowledge model obtained by using the DDQN algorithm with the Monte Carlo tree search algorithm as the reward function is used to output the scheduling decision, so that the production efficiency of the workshop is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of job shop scheduling and artificial intelligence algorithm, and more particularly, to a hybrid flow shop scheduling method and system based on Monte Carlo tree search algorithm. BACKGROUND

[0002] Job shop scheduling is a demand in the production of hybrid flow shop (HFS). HFS is a large-scale production organization form that can meet customer customization needs, and is widely used in various key manufacturing industries. The production system of hybrid flow shop is composed of different processing steps, and the order of these steps is generally fixed. However, there are many uncertain dynamic events in the production process, such as equipment failure, urgent order insertion, quality accident, etc., which cause the production process to deviate from the preset flow. Therefore, in order to eliminate the influence of dynamic events on the execution of production process plan and maintain the stability of production and manufacturing process, a suitable dynamic scheduling mechanism should be adopted for hybrid flow shop. The adaptive scheduling of HFS is a non-deterministic polynomial (NP) problem. In order to optimize the HFS problem, many offline or online solutions have been proposed to achieve higher production efficiency. Online scheduling depends on real-time feedback from the workshop. With the rapid development of intelligent manufacturing, technologies such as Internet of Things, CPS, AGV, etc. are widely used, providing key conditions for online dynamic scheduling. Online scheduling solutions have become the mainstream research field due to their advantages in rapid response to dynamic events. Many scholars have conducted in-depth research on this problem, and past literature has mostly used DDQN algorithm for adaptive job shop scheduling. However, the value network of DDQN algorithm has difficulty in converging when solving the optimization scheduling problem of manufacturing system. The main reason is that the reward function is artificially designed, while the reward function is an immediate evaluation of the state transition behavior at a certain moment, which only relies on the current state information and cannot refer to the future development trend for evaluation. Therefore, the artificially designed reward function inevitably has certain "short-sightedness" and "inaccurate reward value setting" problems, which makes it difficult to guide the DDQN agent to train towards the scheduling optimization goal.

[0003] Existing technology discloses a large-scale flexible job shop scheduling method based on the DDQN algorithm. Specifically, it involves: first, formally expressing the large-scale scheduling problem, including defining state features, action sets, and a reward function; second, to ensure effective model training, decomposing the problem and proposing a method to feed back the action sets and reward function to the features; selecting a composite scheduling rule with high sensitivity to minimizing completion time as the action set, and designing a heuristic reward method to guide algorithm convergence; and finally, using the DDQN algorithm to solve the large-scale flexible job shop scheduling problem. This invention can efficiently and effectively solve the large-scale flexible job shop scheduling problem. However, it still suffers from the problem that the reward function can only rely on the current state information, exhibiting certain "shortsightedness" and "inaccurate reward value settings." Summary of the Invention

[0004] The purpose of this invention is to disclose a more efficient scheduling method and system for mixed-flow manufacturing workshops based on the Monte Carlo tree search algorithm.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] S1: Collect and record operational data from the moment an order is placed in the smart workshop until the finished product is completed;

[0007] S2: Classify and process the job data collected in step S1 into a Markov decision data chain;

[0008] S3: Construct a multi-channel input convolutional neural network A;

[0009] S4: Randomly select k data chains from the Markov decision data chain obtained in step S2, and integrate the data in the selected data chains into a multi-channel resource view state s according to resource attributes. i The multi-channel resource view status s i Input a multi-channel input convolutional neural network A to obtain the prediction value and prediction probability;

[0010] S5: Construct a convolutional neural network B. Based on the prediction value and prediction probability, use the DDQN algorithm with Monte Carlo search tree algorithm as the reward function. Through expansion, selection, simulation and reverse update and training of convolutional neural network B, obtain the knowledge model and the final result value FV.

[0011] S6: Determine whether the final result value FV has converged. If it has not converged, proceed to step S4. If it has converged, proceed to step S7.

[0012] S7: Invoke the knowledge model, match the real-time multi-channel status s of the workshop with the real-time data of various resources in the manufacturing workshop, and output the current optimal scheduling decision for the current task.

[0013] S8: According to the real-time data of the manufacturing workshop resource information, it is judged whether the current task is executed or not, if not, it returns to step S7, if the current task is completed, it ends running.

[0014] Further, the operation data in step S1 is specifically:

[0015] The various resource type data collected by various methods in the production process, including workshop material order allocation, workshop workpiece process scheduling, and workshop completion result; the operation data covers the changes of various resource information of "people, machines, materials, methods, and environment" in the workshop; including the working state data of production equipment collected by Internet of Things, information data of workpieces to be processed, workpiece scheduling transfer number, and other production system imported production order data, material data, and processing technology data.

[0016] Further, the operation data collected in step S1 is classified and processed into Markov decision data chain in step S2, specifically:

[0017] The workshop starts from the order allocation state s0 to the completion state s n n times of scheduling and production scheduling order to form a Markov decision chain data

[0018] s i is classified according to the resource categories of the workshop and a multi-channel resource view state is constructed, including multiple state views; is based on the flow action of the workpiece between the devices; r is the action performed in state s i into a new state s i+1 The corresponding reward;

[0019] A state view contains all the resource dimension state information considered, each resource dimension state information is represented by a matrix form of channel state view, the expression is as follows:

[0020] s i = <P m×c , M m×c , W m×c , F m×c , …, E m×c >, i ∈ (0, 1, 2, …, n-1)

[0021] Action is the transfer of a specific workpiece from a process device to a next process device, the directed flow Tr H between the workpiece θ K and the processing device is represented, the expression is as follows:

[0022] ​

[0023] According to the scheduling target tar and the degree of fit with the completion result T, a corresponding reward r is given.

[0024] Further, in step S3, a convolutional neural network A of the multi-channel input is constructed, specifically:

[0025] The network has two output ends and a hidden layer, wherein the evaluation network OUT-V output end is responsible for predicting the predicted value of the state view s i ; the policy network OUT-P output end is responsible for predicting the predicted probability of each action under the state view; the hidden layer is composed of convolutional layers, fully connected layers and activation layers.

[0026] Further, in step S4, the multi-channel resource attempt state is input to the INPUY_LAY end of the neural network.

[0027] Further, in step S5, expansion: starting from a certain leaf node, according to all enabled actions of the leaf node, the corresponding child nodes are expanded; at the same time, the leaf node state S is input to the convolutional neural network A, and the predicted probability is used to initialize the information of each child node.

[0028] Selection: according to the UCB formula:

[0029]

[0030] The child node with the maximum UCB value is selected to enter, and the child node state S_next is input to the convolutional neural network, and the predicted value of the child node is given by the evaluation network end OUT-V, and loaded into the information of the child node. At the same time, the information of all nodes on the selection path is updated, and the simulation income value V=S_next) and the access frequency N of each node are added by 1.

[0031] Further, in step S5, simulation: according to the predicted probability, combined with the UCB formula, starting from this leaf node, repeatedly simulate and expand the selected state node, simulate the search of the Monte Carlo tree, until the scheduling ends, and get the final result value FV; at the same time, record the historical data in the simulation execution process.

[0032] Further, in step S5, reverse update and training: the expected completion time is passed backward along the selected path to each node as the reverse update process; during training, the actual traversal probability Real_P of selecting a child node is calculated from the historical data generated in the simulation process based on the number of visits to each child node. The predicted probability given by the policy network OUT-P and the actual traversal probability Real_P of the Monte Carlo tree search are compared using cross-information entropy error. The final result value FV and the predicted value are compared using mean squared error, and the two together constitute the loss function. The convolutional neural network B is trained to optimize its weights; the loss function formula is:

[0033] Loss = (FV - V) 2 -(Job P ) T ×log(Real P )+c‖θ‖ 2

[0034] Furthermore, in step S6, convergence is achieved when the final result value FV is 130-140.

[0035] Furthermore, this invention also provides a scheduling system for mixed-flow manufacturing workshops based on the Monte Carlo tree search algorithm, characterized by comprising:

[0036] Data Acquisition Module: Collects and records operational data from the moment an order is placed in the smart workshop until the finished product is completed;

[0037] Classification and processing module: Classifies and processes the job data collected by the acquisition module into Markov decision data chains;

[0038] Building module: Constructs a multi-channel input convolutional neural network A;

[0039] Input extraction module: Randomly extract k data links from the Markov decision data chain obtained in step S2, and integrate the data in the extracted data chain into a multi-channel resource view state s according to resource attributes. i The multi-channel resource view status s i Input a multi-channel input convolutional neural network A to obtain the prediction value and prediction probability;

[0040] Training module: Convolutional neural network B is constructed. Based on the prediction value and prediction probability, the DDQN algorithm with Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate and back-update and train convolutional neural network B to obtain the knowledge model and the final result value FV.

[0041] Convergence judgment module: Determines whether the final result value FV has converged. If it has not converged, the input extraction module is executed. If it has converged, the output scheduling module is executed.

[0042] Scheduling output module: Invokes the knowledge model, matches the real-time multi-channel status s of the workshop with the real-time data of various resources in the manufacturing workshop, and outputs the current optimal scheduling decision for the current task.

[0043] Execution Judgment Module: Determines whether the current task has been completed based on real-time data of various resources in the manufacturing workshop. If it has not been completed, it returns to the execution scheduling output module; if the current task has been completed, it terminates the operation.

[0044] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0045] This invention utilizes the DDQN algorithm, with Monte Carlo Tree Search as the reward function, to expand, select, simulate, and back-update and train a convolutional neural network B to obtain a knowledge model. The Monte Carlo Tree Search algorithm is a binary tree-based algorithm that randomly expands, selects points to continue simulating according to the UCB formula, provides feedback, back-updates, and continues the iteration. The entire process is randomized, relying solely on its own self-learning for game-theoretic operations. It does not require human data for training and can evaluate future trends through its self-learning mechanism. Therefore, the knowledge model obtained using the DDQN algorithm with Monte Carlo Tree Search as the reward function can solve the problems of "shortsightedness" and "inaccurate reward value settings," thus achieving a more efficient solution to scheduling problems in mixed-flow manufacturing workshops. Attached Figure Description

[0046] Figure 1 This is a flowchart of the scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm described in Example 1;

[0047] Figure 2 This is a framework diagram of the scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm described in Example 3; Detailed Implementation

[0048] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0049] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0050] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] Example 1:

[0052] like Figure 1 As shown, a preferred embodiment of the present invention provides a scheduling method for mixed-flow manufacturing workshops based on the Monte Carlo tree search algorithm, comprising the following steps:

[0053] S1: Collect and record the job data from the start of an order in the smart workshop to the completion of finished product processing;

[0054] S2: Classify the job data collected in step S1 into Markov decision data chains;

[0055] S3: Construct a convolutional neural network A with multiple input channels;

[0056] S4: Randomly extract k Markov decision data chains from the obtained Markov decision data chains in step S2, and integrate the data in the extracted data chains into a multi-channel resource view state s according to resource attributes i , the multi-channel resource view state s i is input into the convolutional neural network A with multiple input channels to obtain a predicted value and a predicted probability;

[0057] S5: Construct a convolutional neural network B, and use the DDQN algorithm with the Monte Carlo search tree algorithm as the reward function to expand, select, simulate, and reversely update and train the convolutional neural network B according to the predicted value and the predicted probability, to obtain a knowledge model and a final result value FV;

[0058] S6: Determine whether the final result value FV converges, if not, execute step S4, and if yes, execute step S7;

[0059] S7: Call the knowledge model, match the real-time multi-channel state s of the workshop according to the real-time data of each resource information of the manufacturing workshop, and output the current optimal scheduling decision for the current task.

[0060] S8: Determine whether the current task is completed according to the real-time data of each resource information of the manufacturing workshop, if not, return to execute step S7, and if yes, end the operation.

[0061] In this embodiment, the DDQN algorithm with the Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate, and reversely update and train the convolutional neural network B, to obtain a knowledge model. The Monte Carlo search tree algorithm is an algorithm based on a binary tree structure, which randomly expands, selects a point to continue to simulate according to the UCB formula, feeds back, reversely updates, and continues the next round of iteration. The whole process is random, and only self-learning is used to perform game operations. No human data is needed for training, and the future development trend can be evaluated through the self-learning mechanism of the algorithm. Therefore, the knowledge model obtained by using the DDQN algorithm with the Monte Carlo search tree algorithm as the reward function can solve the problems of "short-sightedness" and "inaccurate reward value setting". Thus, the mixed-model manufacturing workshop scheduling problem can be solved more efficiently.

[0062] Embodiment Two:

[0063] The embodiment is based on embodiment one, and the following is further disclosed:

[0064] The operation data in step S1 is specifically:

[0065] Various resource type data collected by various methods in the production process, including workshop material order allocation, workshop workpiece process scheduling, and workshop completion results; the operation data covers various resource information changes of "people, machines, materials, methods, and environment" in the workshop; including real-time collected working state data of production equipment, information data of workpieces to be processed, workpiece scheduling transfer number, and other production system imported production order data, material data, and processing technology data.

[0066] In step S2, the operation data collected in step S1 is classified and processed into Markov decision data chain, specifically:

[0067] The workshop starts from the order allocation state s0 to the completion state s n , and forms a Markov decision chain data in the order of n times of scheduling and scheduling according to the resource categories of the workshop, including multiple state views.

[0068] Among them, s i is classified and constructed into a multi-channel resource view state according to the resource categories of the workshop, including multiple state views. is the flow action of the workpiece in the device; r is the action performed in state s i into a new state s i+1 , and the corresponding reward;

[0069] A state view contains all the resource dimension state information considered, and each resource dimension state information is represented by a matrix form of channel state view, and the expression is as follows:

[0070] s i = <P m×c , M m×c , W m×c , F m×c , …, E m×c >, i ∈ (0, 1, 2, …, n-1)

[0071] The action is the transfer of a specific workpiece from a certain process device to a next process device, and the directed flow Tr H between the workpiece θ K and the processing device is represented, and the expression is as follows:

[0072]

[0073] ​According to the scheduling target tar and the completion result T fitting degree, the corresponding reward r is given.

[0074] In step S3, the convolutional neural network A of the multi-channel input is constructed, specifically:

[0075] The network has two output ends and a hidden layer, wherein the evaluation network OUT-V output end is responsible for predicting the predicted value of the state view s i ; the policy network OUT-P output end is responsible for predicting the predicted probability of each action under the state view; the hidden layer is composed of convolutional layers, fully connected layers and activation layers.

[0076] In step S4, the multi-channel resource attempt state is input to the INPUY_LAY end of the neural network.

[0077] In step S5, expansion: starting from a certain leaf node, according to all enabled actions of the leaf node, the corresponding child nodes are expanded; at the same time, the leaf node state S is input to the convolutional neural network A, and the predicted probability is used to initialize the information of each child node.

[0078] Selection: according to the UCB formula:

[0079]

[0080] The child node with the maximum UCB value is selected to enter, and the child node state S_next is input to the convolutional neural network, and the predicted value of the child node is given by the evaluation network end OUT-V, and the information of the child node is loaded into the information of the selected path. All nodes on the selected path are updated, and the simulation income value V=S_next) and the access number N are added by 1.

[0081] In step S5, simulation: according to the predicted probability, the UCB formula is used to repeatedly simulate and expand the selected state nodes from the leaf node, simulate the search of the Monte Carlo tree, and obtain the final result value FV until the scheduling ends. At the same time, the historical data in the simulation execution process is recorded.

[0082] In step S5, backward update and training: the predicted completion time is transmitted to each node along the selected path for backward update; in the training process, the actual traversal probability Real_P of the selected child node is calculated from the historical data generated in the simulation process, the prediction probability given by the policy network OUT-P and the actual traversal probability Real_P of the Monte Carlo tree search are used to cross information entropy error, and the final result value FV and the predicted value are used to cross mean square error. The loss function is composed of the two, and the convolutional neural network B is trained to optimize its weight; the loss function formula is:

[0083] Loss = (FV - V) 2 -(Job P ) T ×log(Real P )+c‖θ‖ 2

[0084] In step S6, convergence is achieved when the final result value FV is 130-140.

[0085] This embodiment classifies and processes data into a Markov Decision Process (MDP). This process can describe sequential decision problems with uncertainty and achieve the optimal decision strategy through decision optimization methods. MDP is suitable for solving sequential decision problems, considering the impact of the current decision on future states and rewards, and achieving model generalization and problem solving through model learning and algorithmic solving. By classifying and processing data into MDP, the rich theory and applied algorithms of MDP can be used to solve complex decision problems, thereby achieving an intelligent and efficient decision-making process.

[0086] Example 3:

[0087] like Figure 2 As shown, a preferred embodiment of the present invention provides a scheduling system for a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm, comprising:

[0088] Data Acquisition Module: Collects and records operational data from the moment an order is placed in the smart workshop until the finished product is completed;

[0089] Classification and processing module: Classifies and processes the job data collected by the acquisition module into Markov decision data chains;

[0090] Building module: Constructs a multi-channel input convolutional neural network A;

[0091] Input extraction module: Randomly extract k data links from the Markov decision data chain obtained in step S2, and integrate the data in the extracted data chain into a multi-channel resource view state s according to resource attributes. i The multi-channel resource view status s i Input a multi-channel input convolutional neural network A to obtain the prediction value and prediction probability;

[0092] Training module: Convolutional neural network B is constructed. Based on the prediction value and prediction probability, the DDQN algorithm with Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate and back-update and train convolutional neural network B to obtain the knowledge model and the final result value FV.

[0093] Convergence judgment module: judge whether the final result value FV converges or not, if not, execute the input extraction module, if yes, execute the output scheduling module;

[0094] Output scheduling module: call the knowledge model, match the real-time multi-channel state s of the workshop according to the real-time data of each resource information of the manufacturing workshop, and output the current optimal scheduling decision for the current task.

[0095] Execution judgment module: judge whether the current task is executed or not according to the real-time data of each resource information of the manufacturing workshop, if not, return to the execution scheduling output module, if yes, end the running.

[0096] In this embodiment, the DDQN algorithm using the Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate and reversely update and train the convolutional neural network B to obtain the knowledge model. The Monte Carlo search tree algorithm is an algorithm based on a binary tree structure, which randomly expands, selects a point to continue to simulate according to the UCB formula, feeds back, reversely updates, and continues the next round of iteration. The whole process is random, and only the self-learning is used to perform the game operation. No human data is needed for training, and the self-learning mechanism can be used to evaluate the future development trend. Therefore, the knowledge model obtained by using the DDQN algorithm using the Monte Carlo search tree algorithm as the reward function can solve the problems of "short-sightedness" and "inaccurate reward value setting". Thus, the mixed flow manufacturing workshop scheduling problem can be solved more efficiently.

[0097] In summary, the embodiment of the present application provides a mixed flow manufacturing workshop scheduling method and system based on the Monte Carlo tree search algorithm. First, the workshop operation data is collected and processed into Markov decision chain data. A multi-input channel convolutional neural network is constructed. According to the predicted value and predicted probability of the multi-input channel convolutional neural network, the DDQN algorithm with the Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate, update and train each node, obtain a knowledge model, call the knowledge model, and output an optimal scheduling decision according to the real-time data of each resource information of the manufacturing workshop. The DDQN algorithm with the Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate, and update and train the convolutional neural network B to obtain a knowledge model. The Monte Carlo search tree algorithm is an algorithm based on a binary tree structure, which randomly expands, selects a point to continue to simulate according to the UCB formula, and then feeds back and updates reversely to continue the next round of iteration. The whole process is random, and only self-learning is used for game operation. No human data is needed for training, and the future development trend can be evaluated through its own self-learning mechanism. Therefore, the knowledge model obtained by using the DDQN algorithm with the Monte Carlo search tree algorithm as the reward function can solve the problems of "short-sightedness" and "inaccurate reward value setting". Thus, the mixed flow manufacturing workshop scheduling problem can be solved more efficiently.

[0098] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. All the embodiments do not need to be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A scheduling method for mixed-flow manufacturing workshops based on the Monte Carlo tree search algorithm, characterized in that, Includes the following steps: S1: Collect and record operational data from the moment an order is placed in the smart workshop until the finished product is completed; S2: Classify and process the job data collected in step S1 into a Markov decision data chain; S3: Construct a multi-channel input convolutional neural network A; S4: Randomly select k data chains from the Markov decision data chain obtained in step S2, and integrate the data in the selected data chains into a multi-channel resource view state s according to resource attributes. i The multi-channel resource view status s i Input a multi-channel input convolutional neural network A to obtain the prediction value and prediction probability; S5: Construct a convolutional neural network B. Based on the prediction value and prediction probability, use the DDQN algorithm with Monte Carlo search tree algorithm as the reward function. Through expansion, selection, simulation and reverse update and training of convolutional neural network B, obtain the knowledge model and the final result value FV. S6: Determine whether the final result value FV has converged. If it has not converged, proceed to step S4. If it has converged, proceed to step S7. S7: Call the knowledge model, match the real-time multi-channel status s of the workshop with the real-time data of various resources in the manufacturing workshop, and output the current optimal scheduling decision for the current task; S8: Determine whether the current task has been completed based on real-time data of various resources in the manufacturing workshop. If it has not been completed, return to step S7. If the current task has been completed, end the process.

2. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, The job data in step S1 is as follows: The production process involves collecting various types of resource data through multiple methods. The production process includes the placement of workshop material orders, the scheduling of workshop workpiece processes, and the completion results of the workshop. The operational data encompasses changes in various resource information related to "people, machines, materials, methods, and environment" within the workshop. This includes real-time data collected by the Internet of Things on the working status of production equipment, information data of workpieces to be processed, and workpiece scheduling and transfer numbers, as well as production order data, material data, and processing technology data imported into the workshop from other production systems.

3. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S2, the job data collected in step S1 is classified and processed into a Markov decision data chain, specifically as follows: The workshop progresses from order placement status s0 to completion status s n End, forming a Markov decision chain data according to the nth scheduling sequence. Where s i To categorize and construct multi-channel resource view states by workshop resource type, including multiple state views; This is based on the movement of workpieces between equipment in the workshop; r represents the movement of workpieces in state s. i Next action Entering a new state i+1 Corresponding rewards; A state view contains state information for all resource dimensions considered. Each resource dimension's state information is represented by a channel state view in matrix form, as shown below: s i =<P m×c ,M m×c ,W m×c ,F m×c ,……,E m×c >,i∈(0,1,2…,n-1) action To transfer a specific workpiece from one process equipment to the next process equipment, let workpiece θ H Directed flow between processing equipment and Tr K For representation, the expression is as follows: A corresponding reward r is assigned based on the degree of fit between the scheduling target tar and the completion result T.

4. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S3, a multi-channel input convolutional neural network A is constructed, specifically as follows: The network has two outputs and a hidden layer, where the OUT-V output of the valuation network is responsible for predicting the state view s. i The predictive value; the output of the policy network OUT-P is responsible for predicting the probability of each action under the state view; the hidden layer will be composed of convolutional layer, fully connected layer and activation layer.

5. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S4, the multi-channel resource view state is... Input the INPUBY_LAY end of the neural network.

6. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S5, the expansion is performed as follows: starting from a certain leaf node, the corresponding child nodes are expanded according to all the enabled actions of the leaf node; at the same time, the state S of the leaf node is input into the convolutional neural network A, and the information of each child node is initialized according to the predicted probability. Choice: Based on the UCB formula: The child node with the largest UCB value is selected for entry, and its state S_next is input into the convolutional neural network. The predicted value of the child node is given by the valuation network OUT-V and loaded into the information of the child node. At the same time, the information of all nodes on the selected path is updated, and the simulated profit value of each node V = V(S_next) is incremented by 1.

7. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S5, the simulation proceeds as follows: Based on the predicted probability and combined with the UCB formula, starting from this leaf node, the simulation repeatedly expands and selects sub-state nodes, simulating the search of the Monte Carlo tree until the scheduling ends, and the final result value FV is obtained; at the same time, the historical data during the simulation execution process is recorded.

8. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S5, reverse update and training: the expected completion time is passed back to each node along the selected path, which is the reverse update process; During training, the actual traversal probability Real_P of selecting a child node is calculated from the historical data generated during the simulation, based on the number of visits to each child node. Cross-information entropy error is applied to the predicted probability given by the policy network OUT-P and the actual traversal probability Real_P of the Monte Carlo tree search. The final result value FV and the predicted value are compared using mean squared error, and together they constitute the loss function. This loss function is then used to train the convolutional neural network B, optimizing its weights. The loss function formula is: Loss=(FV-V) 2 -(Job P ) T ×log(Real P )+c‖θ‖ 2 。 9. The scheduling method in a mixed-flow manufacturing workshop based on the Monte Carlo tree search algorithm according to claim 1, characterized in that, In step S6, convergence is achieved when the final result value FV is 130-140.

10. A scheduling system for mixed-flow manufacturing workshops based on the Monte Carlo tree search algorithm, characterized in that, include: Data Acquisition Module: Collects and records operational data from the moment an order is placed in the smart workshop until the finished product is completed; Classification and processing module: Classifies and processes the job data collected by the acquisition module into Markov decision data chains; Building module: Constructs a multi-channel input convolutional neural network A; Input extraction module: Randomly extract k data links from the Markov decision data chain obtained in step S2, and integrate the data in the extracted data chain into a multi-channel resource view state s according to resource attributes. i The multi-channel resource view status s i Input a multi-channel input convolutional neural network A to obtain the prediction value and prediction probability; Training module: Convolutional neural network B is constructed. Based on the prediction value and prediction probability, the DDQN algorithm with Monte Carlo search tree algorithm as the reward function is used to expand, select, simulate and back-update and train convolutional neural network B to obtain the knowledge model and the final result value FV. Convergence judgment module: Determines whether the final result value FV has converged. If it has not converged, the input extraction module is executed. If it has converged, the output scheduling module is executed. Scheduling output module: Invokes the knowledge model, matches the real-time multi-channel status s of the workshop with the real-time data of various resources in the manufacturing workshop, and outputs the current optimal scheduling decision for the current task; Execution Judgment Module: Determines whether the current task has been completed based on real-time data of various resources in the manufacturing workshop. If it has not been completed, it returns to the execution scheduling output module; if the current task has been completed, it terminates the operation.

Images