A "cloud-edge" collaborative predictive reactive scheduling method based on MCTS
By combining historical data analysis on the 'cloud' side and real-time monitoring on the 'edge' side during the production process, using MCTS to generate and switch scheduling solutions, the problem of high switching costs of scheduling solutions caused by uncertain processing time is solved, and production is achieved Stable optimization of the process and smoothness of the dynamic workshop.
Patent Information
- Application Number
- CN202211494563.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-11-25
AI Technical Summary
When the prior art deals with uncertain processing time disturbances during the production process, the switching cost of the scheduling scheme is high and difficult to control, and the process constraints and switching costs cannot be effectively considered comprehensively, resulting in unstable production process.
Using the MCTS-based 'cloud edge' collaborative prediction reactive scheduling method, the scheduling scheme is generated and switched through historical data analysis on the 'cloud' side and real-time monitoring and rapid response on the 'edge' side, combining Monte Carlo tree search and breadth Priority search, optimize production order scheduling, and form prediction basic scheduling solutions and alternative switching strategies.
It effectively reduces the switching time and cost of scheduling schemes, ensures smooth optimization of the production process, meets the dynamic workshop stability requirements under uncertain processing time, and realizes cross-layer scheduling and comprehensive optimization.
Smart Images

Figure CN115857446B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field, and in particular to a MCTS-based "cloud-edge" collaborative predictive reactive scheduling method. Background Art
[0002] Production quality and efficiency are the top priorities of the manufacturing industry. An enterprise must ensure the smooth optimization of the production process in order to gain a foothold in the fierce market competition. Most of the literature on scheduling ignores the disturbances that occur in the production process and rarely considers dynamic uncertainties. However, in the actual production process, the uncertainty of processing time may cause the original basic production scheduling plan to fail. Moreover, when the processing time is disturbed, it will have many adverse effects on the production process, making the dynamic scheduling problem more complicated than the static scheduling problem.
[0003] Although there are relevant studies on dynamic scheduling, current studies rarely consider the switching costs caused by switching scheduling schemes. In the actual production process, the switching costs caused by switching scheduling schemes are difficult to ignore, and enterprises need to reduce the switching costs caused by switching scheduling schemes as much as possible. Moreover, there is a connection between the original basic production scheduling scheme and the switching scheduling. When switching when disturbances occur, the switching cost needs to be as small as possible, so when selecting the basic scheduling scheme, the basic scheduling cost and the switching cost when switching scheduling should be comprehensively considered. Therefore, it is necessary to analyze the production and manufacturing process, and on the basis of comprehensive consideration of process constraints and uncertain processing time, optimize the scheduling of production orders to form a predicted basic scheduling scheme. When uncertain processing time disturbances occur in the production process, the production scheduling scheme is switched reactively, thereby forming a predictive-reactive scheduling method to ensure the smooth optimization of the production process. Summary of the invention
[0004] According to the technical problems mentioned in the above background technology, a "cloud-edge" collaborative predictive reactive scheduling method based on MCTS is provided.
[0005] The technical means adopted by the present invention are as follows:
[0006] A "cloud-edge" collaborative predictive reactive scheduling method based on MCTS, wherein the "cloud" side system includes: a predictive scheduling solution module and a scheduling solution pool having a historical data analysis module, a production task processing module, and a scheduling solution pool generation module;
[0007] The "side" system includes: basic scheduling acquisition module, conflict judgment and feedback module, alternative solution switching module and feasible solution rapid generation module;
[0008] The following steps are involved:
[0009] Step 1: Construct a scheduling solution pool for the uncertain processing time problem in the production process. The scheduling solution pool generation module generates the scheduling solution pool based on the result data of the historical data analysis module and the production task processing module;
[0010] Step 2: The basic scheduling acquisition module selects the optimal scheduling solution from the scheduling solution pool, and defines the optimal scheduling solution as the basic scheduling solution; at the same time, in the production scheduling process, the conflict judgment and feedback module performs real-time monitoring of equipment operation;
[0011] When the basic scheduling scheme is feasible, the basic scheduling scheme is adopted; if a conflict occurs, a request is sent to the "cloud" side to analyze whether there is a feasible alternative switching scheduling scheme in the scheduling scheme pool; if there is, step 3 is executed, and if not, step 4 is executed;
[0012] Step 3: Extract the optimal feasible alternative scheduling solution from the scheduling solution pool and send the solution to the device for execution;
[0013] Step 4: Through the feasible solution rapid generation module, a feasible scheduling solution is quickly re-generated according to the current execution status and the information to be processed, and the scheduling solution and conflict information are transmitted to the scheduling solution pool generation module on the "cloud" side, and the scheduling solution pool is updated according to the information.
[0014] Compared with the prior art, the present invention has the following advantages:
[0015] 1. The present invention utilizes the two parts of predictive scheduling and reactive scheduling included in predictive-reactive scheduling. In the predictive scheduling part, a method of generating and expanding Monte Carlo trees under the premise of considering uncertain factors is implemented, and a variety of production scheduling plans are formulated accordingly, and finally a scheduling plan pool is formed. The problem of long switching scheduling plans and difficult switching costs caused by conflicts in production scheduling due to uncertain processing time is solved, and the smooth optimization of the production process is ensured.
[0016] 2. The present invention organically combines "cloud-edge" collaboration with predictive-reactive scheduling. Based on the "cloud-edge" collaborative framework, it gives full play to the comprehensive effects of the high-performance computing resources on the "cloud" side and the rapid response and closely connected decision-making management on the "edge" side, thereby realizing cross-layer scheduling and comprehensive optimization of "cloud-edge" collaboration.
[0017] 3. The present invention utilizes the Monte Carlo tree search to combine the characteristics of breadth-first search and depth-first search and use simulation methods to quickly obtain the evaluation value of the scheme, and generalizes the method from solving chess game problems to solving the reactive scheduling problem of uncertain processing time prediction, and provides an improved Monte Carlo tree search method. On the basis of comprehensive consideration of process constraints and uncertain processing time, the production order is optimized to form a predictive basic scheduling scheme, multiple alternative scheduling schemes and switching strategies, which meets the requirements of dynamic workshops with uncertain processing time for production stability.
[0018] In summary, the technical solution of the present invention solves the problems in the prior art that it takes a long time to switch scheduling schemes and the total switching cost is difficult to control when encountering conflicts. In view of the practical problems such as the distributed deployment requirements of computing resources and the uncertainty of processing time, a "cloud-edge" collaborative predictive-reactive scheduling method based on improved Monte Carlo tree search is proposed. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0020] Figure 1 A basic architecture diagram of an MCTS-based industrial process "cloud-edge" collaborative predictive reactive scheduling method constructed for the present invention;
[0021] Figure 2 This is a schematic diagram of production task information of the present invention;
[0022] Figure 3 Generate a Monte Carlo tree algorithm flow chart for the present invention;
[0023] Figure 4 This is a diagram of the Monte Carlo tree algorithm structure of the present invention;
[0024] Figure 5 It is a diagram showing the form and state of the scheduling scheme pool of the present invention;
[0025] Figure 6 The switching algorithm flow chart of the present invention;
[0026] Figure 7 This is a schematic diagram of the switching module results of the alternative solution of the present invention.
[0027] Figure 8 It is a schematic diagram of the overall system of the present invention. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0030] like Figure 1-8 As shown, the present invention provides a "cloud-edge" collaborative predictive reactive scheduling method based on MCTS, wherein the "cloud" side system includes: a predictive scheduling solution module and a scheduling solution pool having a historical data analysis module, a production task processing module, and a scheduling solution pool generation module;
[0031] The "side" system includes: basic scheduling acquisition module, conflict judgment and feedback module, alternative solution switching module and feasible solution rapid generation module;
[0032] Step 1: Construct a scheduling solution pool for the uncertain processing time problem in the production process. The scheduling solution pool generation module generates the scheduling solution pool based on the result data of the historical data analysis module and the production task processing module.
[0033] Step 1-1: In the historical data analysis module, load the historical production data of each production process, and pre-process the historical processing time data, that is, use the mean replacement method to fill the missing values, use the sliding average method to smooth the noise data, and use the radius filter method to identify or delete outliers, so as to achieve the purpose of removing abnormal data and duplicate data;
[0034] Step 1-2: Use the least squares method to fit the historical data of each process to obtain the distribution and statistical characteristics of the data, such as binomial distribution, geometric distribution, hypergeometric distribution, multinomial distribution, negative binomial distribution, Poisson distribution and other discrete probability distributions, and obtain the probability distribution function corresponding to the data distribution;
[0035] Step 1-3: passing the discrete probability distribution of the uncertain processing time to the production task processing module and the scheduling scheme pool generation module; loading new production task data through the production task processing module; wherein the production task processing module includes: production task information and manufacturing resource information;
[0036] Step 1-4: Combine the results of the historical data analysis module to obtain the production process constraints, process processing time intervals, and process available equipment set information;
[0037] Step 1-5: storing and backing up the result data information according to the requirements of the scheduling solution pool generation module, and transmitting the result data to the scheduling solution pool generation module;
[0038] Step 1-6: In the scheduling solution pool generation module, a complete scheduling solution is generated based on the Monte Carlo tree search;
[0039] Step 1-7: Store the scheduling scheme pool in the form of a Monte Carlo tree; the scheduling scheme moves downward from the root node to form a complete scheduling scheme S0->S12->S22->S33; the node status of the Monte Carlo tree includes: process number, process start time interval and process machine allocation.
[0040] As a preferred embodiment, in the present application, steps 1-6 further include the following steps:
[0041] Step 1-6-1: Starting point: Monte Carlo tree from the root node (such as Figure 4 The search starts at node S0.
[0042] Step 1-6-2: Select: Figure 4 As shown in the selection step, starting from the root node (S0), the Upper Confidence Bounds (UCB) function is recursively used downward to select the child node with the most potential (S12), and the state of the child node is added to the scheduling plan until a non-terminal leaf node (S22) is selected.
[0043]
[0044] in is the value estimated by the node, n i is the number of times the node has been visited, while N is the total number of times its parent node has been visited, and c is an adjustable parameter.
[0045] Step 1-6-3: Extension: Figure 4 As shown in the expansion step, if the current leaf node is not a terminal leaf node, all unexpanded work order processes that can be started are selected on the leaf node (S22), and the time range in which the process can be started is determined based on the start and end time intervals of all its ancestor nodes. The process number, start time and assigned machine are set as the new state (S31, S32, S33, S34), and the node it points to is added to the search tree.
[0046] Step 1-6-4: Simulation: For the newly expanded nodes (S31 / S32 / S33 / S34), add all unselected work order processes to the current scheduling plan and sort them using heuristic methods to form a complete scheduling plan (S31~Sn1 / S32~Sn2 /
[0047] S33~Sn3 / S34~Sn4), and the scheduling cost of the scheme is obtained according to the scheduling sequence. The scheduling cost is composed of production and processing cost, switching scheme cost and non-switching scheduling scheme cost. In addition, during the simulation process, the results of the historical state table will be used to reduce the number of simulations to achieve the same simulation effect.
[0048] Step 1-6-5: Status optimization: Figure 4 As shown in the state optimization step, when nodes (S31, S32, S33, S34) are expanded, the state start time interval is reduced. The state intervals of each work order process in the same machine are compared, and the state start time cost is reduced for the state that can be reduced, so that the overall goal of coverage and cost is optimal, and the state is changed to (S31', S32, S33, S34').
[0049] Step 1-6-6: Subtree pruning: Figure 4 As shown in the subtree pruning step, during the Monte Carlo tree search, only the n subtrees with the best results are retained in the subtrees of the same parent node, and other subtrees (such as S33) are pruned to concentrate the computational effort on the best possible subtree; during the Monte Carlo tree search, only the n subtrees with the best results are retained in the subtrees of the same parent node, and other subtrees are pruned to concentrate the computational effort on the best possible subtree;
[0050] Step 1-6-7: Record historical status: In order to make full use of the result information of the previous Monte Carlo tree simulation, these historical status information are recorded in the historical status table. The historical status table records the status executed so far, its evaluation value, access times and other information, which is used to reduce the number of simulations.
[0051] Step 1-6-8: Back propagation: Figure 4 As shown in the back-propagation step, the scheduling cost simulated from the new extended node (S31', S32, S34') is back-propagated to each node (S12, S0) from the new node to the root node. The back-propagated scheduling cost will affect the selection strategy, and the node passed by the back-propagation will update the evaluation value and the number of visits of the node according to the back-propagated scheduling cost.
[0052] Step 1-6-9: Repeat steps 1-6-2 to 1-6-8 until the number of iterations is reached and one thousand complete scheduling solutions are generated. The Monte Carlo tree is recorded as a scheduling solution pool, which provides a basic scheduling solution and an alternative switching scheduling solution for the overall predictive-reactive scheduling.
[0053] Preferably, step 2: the basic scheduling acquisition module selects the optimal scheduling scheme from the scheduling scheme pool, and at the same time, defines the optimal scheduling scheme as the basic scheduling scheme; at the same time, in the production scheduling process, real-time monitoring of equipment operation is performed through the conflict judgment and feedback module; the scheduling scheme pool includes: scheduling schemes stored in the form of Monte Carlo trees, a method for issuing scheduling schemes, and a method for selecting scheduling schemes.
[0054] When the basic scheduling scheme is feasible, the basic scheduling scheme is adopted; if a conflict occurs, a request is sent to the "cloud" side to analyze whether there is a feasible alternative switching scheduling scheme in the scheduling scheme pool; if so, step 3 is executed, and if not, step 4 is executed.
[0055] The step 2 further comprises the following steps:
[0056] Step 2-1: The basic scheduling acquisition module sends a request to the "cloud" side; traverses all the final layer leaf nodes of the scheduling solution pool in the form of a Monte Carlo tree in the scheduling solution pool to find the global optimal complete schedule; and traces back from the leaf node to form a complete scheduling solution, which is defined as the basic scheduling and sent to the "edge" side system for execution by the device;
[0057] Step 2-2: In production scheduling, the conflict judgment and feedback module is used to monitor the production process in real time and determine whether the current scheduling plan can continue at this moment; if there is no conflict, that is, the current scheduling plan is feasible, then continue to execute the scheduling plan; otherwise, execute step 2-3;
[0058] Step 2-3: Determine whether there is a feasible alternative scheduling scheme. If there is a feasible alternative scheduling scheme, call the alternative scheme switching module. If there is a feasible alternative scheme when encountering a conflict, call the alternative scheme switching module; receive the conflict information from the conflict judgment and feedback module, feed back the production performance to the "cloud" side system, and send a request to the "cloud" side; otherwise, execute step 2-4;
[0059] Step 2-4: Call the feasible solution fast generation module and transmit the conflict information to the corresponding module; first receive the conflict information fed back by the conflict judgment and feedback module, and then quickly generate a scheduling plan through a heuristic algorithm based on the start time, processing time, machine allocation and work order process information of the completed work order process, and send it to the device for execution; at the same time, feed back the production performance to the scheduling plan pool generation module of the "cloud" side system, update the scheduling plan pool, and then the production scheduling is executed according to the updated scheduling plan pool.
[0060] Step 3: Extract the optimal feasible alternative scheduling solution from the scheduling solution pool and send the solution to the device for execution;
[0061] Step 4: Through the feasible solution rapid generation module, a feasible scheduling solution is quickly re-generated according to the current execution status and the information to be processed, and the scheduling solution and conflict information are transmitted to the scheduling solution pool generation module on the "cloud" side, and the scheduling solution pool is updated according to the information.
[0062] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0063] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0064] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0065] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0066] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0067] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A MCTS-based “cloud-edge” collaborative predictive reactive scheduling method, where: The "cloud" side system includes: a prediction scheduling solution module and a scheduling solution pool having a historical data analysis module, a production task processing module, and a scheduling solution pool generation module; The "side" system includes: basic scheduling acquisition module, conflict judgment and feedback module, alternative solution switching module and feasible solution rapid generation module; It is characterized by comprising the following steps: Step 1: construct a scheduling solution pool for the problem of uncertain processing time in the production process, and the scheduling solution pool generation module generates the scheduling solution pool according to the result data of the historical data analysis module and the production task processing module; the step 1 also includes the following steps: Step 1-1: In the historical data analysis module, load the historical production data of each production process, and pre-process the historical processing time data, that is, fill in missing values, smooth noise data, identify or delete outliers, so as to achieve the purpose of removing abnormal data and duplicate data; Step 1-2: Use the least squares method to fit the historical data of each process, determine the data distribution type, and obtain the probability distribution function of the corresponding distribution of the data; Step 1-3: passing the discrete probability distribution of the uncertain processing time to the production task processing module and the scheduling scheme pool generation module; loading new production task data through the production task processing module; wherein the production task processing module includes: production task information and manufacturing resource information; Step 1-4: Combine the results of the historical data analysis module to obtain the production process constraints, process processing time intervals, and process available equipment set information; Step 1-5: storing and backing up the result data information of the historical data analysis module according to the input data interface format requirements of the scheduling solution pool generation module, and transmitting the result data to the scheduling solution pool generation module; Step 1-6: In the scheduling solution pool generation module, a scheduling solution pool is generated based on the Monte Carlo tree search technology; Step 1-7: storing the scheduling solution pool in the form of a Monte Carlo tree; the scheduling solution is moved downward from the root node to form a complete scheduling solution; the node status of the Monte Carlo tree includes: process number, process start time interval and process machine allocation; Step 2: The basic scheduling acquisition module selects the optimal scheduling solution from the scheduling solution pool, and defines the optimal scheduling solution as the basic scheduling solution; at the same time, in the production scheduling process, the conflict judgment and feedback module performs real-time monitoring of equipment operation; When the basic scheduling scheme is feasible under the current production operation, the basic scheduling scheme is adopted; if a conflict occurs, a request is sent to the "cloud" side to analyze whether there is a feasible alternative switching scheduling scheme in the scheduling scheme pool; if there is, step 3 is executed, and if not, step 4 is executed; Step 3: Extract the best scheduling solution from the feasible solutions in the alternative scheduling solutions in the scheduling solution pool, and send the solution to the device for execution; Step 4: Through the feasible solution rapid generation module, a feasible scheduling solution is quickly re-generated according to the current execution status and the information to be processed, and the scheduling solution and conflict information are transmitted to the scheduling solution pool generation module on the "cloud" side, and the scheduling solution pool is updated according to the changed information.
2. According to the MCTS-based "cloud-edge" collaborative predictive reactive scheduling method according to claim 1, it is characterized in that: The scheduling scheme pool includes: scheduling schemes stored in the form of Monte Carlo trees, a method for issuing scheduling schemes, and a method for selecting scheduling schemes.
3. According to the MCTS-based “cloud-edge” collaborative predictive reactive scheduling method of claim 1, it is characterized in that: The step 2 further comprises the following steps: Step 2-1: The basic scheduling acquisition module sends a request to the "cloud" side; traverses all the final layer leaf nodes of the scheduling solution pool in the form of a Monte Carlo tree in the scheduling solution pool to find the global optimal complete schedule; and traces back from the leaf node to form a complete scheduling solution, which is defined as the basic scheduling and sent to the "edge" side system for execution by the device; Step 2-2: In production scheduling, the conflict judgment and feedback module is used to monitor the production process in real time and determine whether the current scheduling plan can continue at this moment; if there is no conflict, that is, the current scheduling plan is feasible, then continue to execute the scheduling plan; otherwise, execute step 2-3; Step 2-3: Determine whether there is a feasible alternative scheduling plan. If there is a feasible alternative plan when encountering a conflict, call the alternative plan switching module; receive the conflict information from the conflict judgment and feedback module, feed back the production performance to the "cloud" side system, and send a request to the "cloud" side; otherwise, execute step 2-4; Step 2-4: Call the feasible solution fast generation module and transmit the conflict information to the corresponding module; first receive the conflict information fed back by the conflict judgment and feedback module, and then quickly generate a scheduling plan through a heuristic algorithm based on the start time, processing time, machine allocation and work order process information of the completed work order process, and send it to the device for execution; at the same time, feed back the production performance to the scheduling plan pool generation module of the "cloud" side system to update the scheduling plan pool. After that, production scheduling is executed according to the updated scheduling plan pool.
4. According to the MCTS-based "cloud-edge" collaborative predictive reactive scheduling method according to claim 2, it is characterized in that: The steps 1-6 further include the following steps: Step 1-6-1: Start searching from the root node of the Monte Carlo tree; Step 1-6-2: Starting from the root node, recursively downward and select the most potential child node through the UCB function, and add the state of the child node to the scheduling plan until a non-terminal leaf node is selected; Step 1-6-3: If the current leaf node is not a terminal leaf node, select all unexpanded work order processes that can be started on the leaf node, and determine the time range in which the process can be started based on the start and end time intervals of all its ancestor nodes; set the process number, start time and assigned machine as the new state, and add the node it points to to the search tree; Step 1-6-4: For the newly expanded node, add all unscheduled work order processes to the current scheduling plan and sort them using a heuristic method to form a complete scheduling plan, and derive the scheduling cost of the plan based on the scheduling order; Step 1-6-5: After the node is expanded, the state start time interval is reduced; the state intervals of each work order process in the same machine are compared, and the state start time cost is reduced for the state that can be reduced, so that the overall goal of coverage and cost is optimal, and the state is updated; Step 1-6-6: During the Monte Carlo tree search, only the n subtrees with the best results are retained from the subtrees of the same parent node, and the other subtrees are pruned to concentrate the computation on the optimal possible subtree; Step 1-6-7: Record the historical status; Step 1-6-8: Perform back propagation; the scheduling cost simulated by the newly extended node is back propagated to each node from the new node to the root node. The scheduling cost of back propagation will affect the selection strategy, and the node passed by back propagation will update the evaluation value and visit count of the node according to the scheduling cost of back propagation; Step 1-6-9: Repeat steps 1-6-2 to 1-6-8 until the number of iterations is reached and one thousand complete scheduling solutions are generated.
5. According to the MCTS-based "cloud-edge" collaborative predictive reactive scheduling method of claim 4, it is characterized in that: The specific calculation formula of the UCB function is: in, Represents the estimated value of the node, n i represents the number of times the node has been visited, N represents the total number of times its parent node has been visited, and c represents an adjustable parameter.
6. According to the MCTS-based "cloud-edge" collaborative predictive reactive scheduling method of claim 4, it is characterized in that: The scheduling cost is composed of production and processing cost, switching plan cost and non-switching scheduling plan cost.
