Dynamic flexible job shop scheduling method considering multiple types of events

Through the deep reinforcement learning model DQN, optimizes flexible work workshop scheduling, identify dynamic events and generates scheduling rules, the problems of scheduling complexity and low resource utilization in the flexible manufacturing system are solved, and efficient dynamic flexible work workshop scheduling is achieved.

CN119026832APending Publication Date: 2024-11-26ZHEJIANG UNIV OF TECH
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411001969.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In flexible manufacturing systems, dynamic operation workshop scheduling faces the problems of high complexity and low resource utilization, and traditional methods are difficult to effectively deal with the problems of large-scale and continuous state space.

Method used

Deep reinforcement learning model DQN is adopted, combined with the dynamic flexible work workshop DFJSP model, and dynamic events are identified by simulating the real production environment, scheduling rules module is designed, and deep learning is used to optimize the scheduling strategy to generate scheduling rules sets of different types of dynamic events to realize dynamic events. Identify and schedule.

Benefits of technology

It improves the scheduling efficiency of the operation workshop, reduces the delay period, improves resource utilization, reduces the operation complexity, and achieves better dynamic and flexible operation space scheduling capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026832A_ABST
    Figure CN119026832A_ABST
Patent Text Reader

Abstract

The invention discloses a dynamic flexible job shop scheduling method considering multiple types of events, which realizes simulation of shop scheduling and design of rules through establishment of a model, completes identification of dynamic events, and realizes scheduling of the multiple types of events when a single dynamic event occurs by means of a deep reinforcement learning model. Recording the occurrence number of the dynamic event scheduling rules of the type, counting a scheduling rule set with the highest occurrence frequency, and recording the scheduling rule sets with the highest occurrence frequency corresponding to different dynamic events in the same manner; when a corresponding dynamic event occurs, the corresponding action space set is used for production scheduling; by recording the tardiness, images of the tardiness and the delivery period and the completion time of each iteration are recorded, and the minimum tardiness is obtained through a tardiness iteration graph. Compared with a traditional heuristic-based scheduling strategy, the scheduling capability of the dynamic flexible job space is effectively improved, the tardiness is reduced, the operation complexity is effectively reduced, and the operation speed is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a dynamic flexible job shop scheduling method considering multiple types of events. Background Art

[0002] As the core link of manufacturing production, the efficiency, flexibility and modernization of the job shop directly affect the development and competitiveness of the entire manufacturing industry. The research and solution of job shop problems not only helps to improve production efficiency, product quality and technical level, but also meets the goal of promoting the manufacturing industry to be intelligent, high-quality and sustainable. These efforts will promote the development of the manufacturing industry to a higher level and enhance its competitiveness in the global manufacturing industry.

[0003] The Flexible Jobshop Scheduling Problem (FJSP) refers to the problem of rationally arranging and optimizing the workflow in the job shop in a complex production environment with multiple products, multiple processes, and multiple equipment in a flexible manufacturing system in order to improve production efficiency, reduce costs, and maximize resource utilization. Its essence is that a certain process uses different available processing machines to complete the operation, and the time consumed is not the same, so the impact on the overall goal of the workshop cannot be generalized. Since the 1990s, researchers have invested a lot of energy in this problem, and the research directions can be roughly divided into the following three categories: exact algorithms, heuristic algorithms, and meta-heuristic algorithms. Among them, meta-heuristic algorithms have been favored by many scholars due to their advantages such as high efficiency, stability, and wide application range.

[0004] In recent years, more and more researchers have applied deep reinforcement learning (DRL) to the job shop scheduling problem, trying to use its powerful learning and optimization capabilities to solve this complex optimization problem. Traditional reinforcement learning methods have challenges in dealing with large-scale, continuous state space problems, but DRL can better handle these complex, high-dimensional problems by combining deep learning techniques. Although challenges still exist, DRL, as a cutting-edge method, has shown great potential in solving job shop scheduling problems and is becoming a research hotspot in this field. Summary of the invention

[0005] The object of the present invention is to provide a dynamic flexible job shop scheduling method considering multiple types of events.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A dynamic flexible job shop scheduling method considering multiple types of events includes the following steps:

[0008] 1) A dynamic flexible job shop DFJSP model is established by simulating the real production environment. The goal of the model is to minimize the total delay time of the work, constrain the operation process of machine processing workpieces, and construct a scheduling rule module including a series of scheduling rules. The scheduling rule module is used to complete the selection of different scheduling rules and the selection process of the job machine.

[0009] By establishing the model, the simulation and rule design of workshop scheduling can be realized, and the identification of dynamic events can be completed. The current patent can identify dynamic events as workpiece arrival and machine failure.

[0010] 2) Based on the DFJSP model established in step 1), the deep reinforcement learning model DQN is used for training, that is, the state information of the current dynamic event is extracted from the real production environment, the state information is converted into state feature values ​​and normalized to take a value in the range of [0,1] as the input layer of DQN, and the specific type and Q value of the current dynamic event are output after DQN training, and a scheduling rule is selected in the scheduling rule module constructed in step 1) according to the Q value;

[0011] According to the above process, when a single dynamic event occurs, the number of occurrences of the corresponding dispatch rules when the type of dynamic event occurs is recorded, and the dispatch rules with the highest occurrence frequency are counted and put into a set. Similarly, the dispatch rule sets with the highest occurrence frequency corresponding to different types of dynamic events are constructed, that is, different action space sets generated by the dispatch rule sets of different types of dynamic events are obtained;

[0012] 3) The DFJSP model of a normal dynamic flexible job shop is running. Different action space sets are generated by statistically analyzing the scheduling rule sets of different types of dynamic events. Whenever a corresponding dynamic event occurs, the corresponding action space set is used to perform production scheduling;

[0013] 4) By recording the delay (the completion time of the workpiece is later than the delivery time of the workpiece, that is, the delay), the delay and the image of the delivery time and completion time of each iteration are recorded, and the minimum delay is obtained through the delay iteration graph.

[0014] Furthermore, in the step 2), different dynamic events are identified and classified. The dynamic events implemented in this patent are divided into two categories: workpiece insertion and machine failure. The specific implementation logic of workpiece insertion is to assume that there are several jobs in the workshop from the beginning, and then the new job arrives after the Poisson distribution, so the arrival interval time between two consecutive new job insertions is affected by the exponential distribution; the implementation logic of machine failure is to pre-set a failure probability (simulating real scenarios, the occurrence of failures in normal factory processing is sometimes unpredictable, simulated as a probability), randomly select a machine before each processing, and then generate a random number with a value in the range of [0,1]. If the random number is greater than the failure probability, the machine enters a fault state, and the machine is repaired, and the repair time is set to a random number that takes into account the processing time (10-30min in this patent).

[0015] By running the trained model in large-scale examples, we can successfully complete the identification of dynamic events and the call of the corresponding scheduling rule space, and generate better delay results.

[0016] Taking a conventional job shop as an example, the complex system includes the following advance constraints: n: total number of jobs; m: total number of machines; J i : i-th job; n i : Belongs to Assignment J i The total number of operations; M k : kth machine; O i,j :Homework J i The jth operation of M i,j : Available operating machine set O i,j ;t i,j,k :Machine M k Operation O i,j Processing time; A i, :Homework J i Arrival time; D i :Homework J i The expiry date;

[0017] i,h: job index, i,h=1,2,…,n;

[0018] j,g: belongs to job J i and J h Operation index, j = 1, 2, ..., n i , g=1,2,…,n h ;

[0019] k: machine index, k = 1, 2, ..., m;

[0020] t i,j,k represents the processing time of the jth process of the i-th workpiece on machine k;

[0021] t h,g,k represents the processing time of the g-th process of the h-th workpiece on machine k;

[0022] The decision variables include the following categories:

[0023] C i,j : Operation O i,j completion time.

[0024]

[0025] X i,j,k Determines on which machine to allocate the operation, and Y i,j,h,g Determines the relative processing priority between any two operations.

[0026] Based on the above symbols, the dynamic flexible job shop (DFJSP) model used in this patent can be mathematically described as follows:

[0027]

[0028] The above formula (1) represents the total delay time of all jobs; formula (2) represents the constraints, where formula (a) indicates that the completion time of each operation must be non-negative, formula (b) indicates that each operation can only be assigned to one machine, formula (c) ensures that the job can only be processed after its arrival time, and formula (d) guarantees the priority constraint, that is, the workpiece processes need to be processed in sequence; formula (e) guarantees the capacity constraint, each machine can only process one workpiece under one process at a time.

[0029] Furthermore, in step 1), the selection of the workpiece processing procedure by the scheduling rule module is performed according to the following process: first, check whether there is a delay, that is, search whether there is a workpiece that cannot be completed on time. If there is a delayed workpiece, select the workpiece with the longest delay time for processing, otherwise select the workpiece processing procedure from the scheduling rule module; the DFJSP considered in this patent includes operation sequence and machine allocation sub-problems. The decision maker must determine not only the operation selection rule but also the machine allocation rule at each decision point. To this end, ten composite scheduling rules are proposed to select feasible operations and then allocate them to appropriate machines.

[0030] The selection of workpiece operations uses a simple heuristic scheduling rule. The biggest advantage of the heuristic rule algorithm is that even if the complexity is low, it can be well applied to dynamic real-time scheduling and complex large-scale scheduling problems. Different scheduling rules have different global sensitivities, so different scheduling rules produce different scheduling solutions.

[0031] The scheduling rule module of the present invention includes the following 10 heuristic scheduling rules: SPT, LPT, maximum delay rate, SRM, SRPT, LSO, EDD, LDD, mSLACK, and MSLACK.

[0032] The idea of ​​the action rule of this patent is to first search whether there are workpieces that cannot be completed on time. If there are delayed workpieces, the workpiece with the longest delay time is selected. Otherwise, the workpiece process is selected according to 10 different heuristic scheduling rules. The rule for machine selection is unified to select the machine with the earliest available time.

[0033] The 10 different heuristic scheduling rules of the present invention are as follows:

[0034] SPT (Shortest Processing Time): The shortest processing time rule selects the workpiece with the shortest processing time for priority processing.

[0035] LPT (Longest Processing Time): The longest processing time rule selects the workpiece with the longest processing time for priority processing.

[0036] Maximum Tardiness Rate: The workpiece with the largest current tardiness rate (i.e. the ratio of actual tardiness time to allowed tardiness time) is selected for processing first.

[0037] SRM (Slackper Remaining Operation): The slack time rule for the remaining operations is to select the workpiece with the smallest slack time (the difference between the remaining time of the workpiece and the processing time) divided by the number of remaining operations for processing first.

[0038] SRPT (Shortest Remaining Processing Time): The shortest remaining processing time rule selects the workpiece with the shortest remaining processing time for processing first.

[0039] LSO (Least Slackper Operation): The rule of least slack per operation selects the workpiece with the smallest slack per operation for priority processing.

[0040] EDD (Earliest Due Date): Earliest due date rule, the workpiece with the earliest due date is selected for processing first.

[0041] LDD (Latest Due Date): The latest due date rule selects the workpiece with the latest due date for processing first.

[0042] mSLACK (Minimum Slack): Minimum slack time rule, which selects the workpiece with the smallest slack time for processing first. Slack time refers to the difference between the workpiece due date and the current time minus the processing time.

[0043] MSLACK: represents the maximum slack time rule, which selects the workpiece with the largest slack time for processing first; calculates the sum of the remaining processing time of each unfinished workpiece, calculates the remaining slack time of each workpiece, and selects the workpiece with the longest remaining slack time.

[0044] In most reinforcement learning-based scheduling methods, state features are defined as some indicators of the production state, i.e., the number of machines / jobs on the shop floor, the remaining processing time of unfinished jobs, the current workload / queue length of each machine, etc. However, in the real world, the number of machines / jobs / operations is infinite and may be very large. If these indicators are directly used as state features, the input of DQN may vary in a wide range. This may deteriorate the performance and generality of DQN in untrained settings, since only specific production configurations are experienced during training. To address this issue, seven carefully designed state features are extracted, each of which takes values ​​in the range of [0,1] as the input of DQN. By restricting all state features in [0,1], DQN can be easily extended to different untrained production environments.

[0045] Furthermore, step 2) extracts the state information of the current dynamic event from the real production environment and converts it into 7 state features, each of which takes a value in the range of [0,1] as the input of DQN;

[0046] The seven status characteristics of each rescheduling time point are as follows:

[0047] (1) Average utilization of machines:

[0048]

[0049] (2) Standard deviation of machine utilization:

[0050]

[0051] (3) Average completion rate of operations:

[0052]

[0053] (4) Average completion rate of assignments:

[0054]

[0055] (5) Standard deviation of the completion rate:

[0056]

[0057] (6) Estimated delay rate

[0058] T cur is defined as the average completion time of the last operation on all machines at the current time t, then the estimated delayed job can be regarded as the estimated remaining processing time greater than the time from T cur jobs with idle time until their due date D; estimated delay rate It is defined as the number of all estimated delayed jobs divided by the number of unfinished jobs of all remaining jobs; if there is no estimated remaining processing time less than T cur The job has an idle time until its delivery date D, then is 0;

[0059] (7) Actual delay rate

[0060] Also consider T cur , actual delayed work is defined as work that is not completed before the deadline; actual delay rate It is defined as the actual number of delayed jobs divided by the number of unfinished jobs of all remaining jobs; if the actual remaining processing time is less than the time from T cur The job has a free time until its due date D, then is 0;

[0061] The symbols involved in the state characteristics are defined as follows:

[0062] CT K (t) is defined as k The completion time of the last operation assigned at the rescheduling time point t, OP i (t) is defined as job J i The current number of completed operations; then machine M k The utilization rate at time t is expressed as U K (t) means, At the same time, homework J i The completion rate at time t, CRJ i (t) means,

[0063] Since the goal is to minimize the total delay, we consider the current state s in turn. t and the next state s t+1 The values ​​of the three key state features are used to define the state-action pair (s t ,a t ) t , including the actual delay rate Tard a , estimated delay rate Tarde and the average machine utilization U ave .

[0064] The present invention utilizes the deep reinforcement learning model DQN training process, including the following steps:

[0065] Consider the current state s in turn t and the next state s t+1 The values ​​of the three key status characteristics of the actual delay rate Tard a , estimated delay rate Tard e and the average machine utilization U ave , when a single dynamic event occurs, the reward value is evaluated according to the action result selected by the model DQN algorithm according to the state, and the selected action is the scheduling rule selected by the output of the model DQN algorithm, including;

[0066] That is, first check the actual delay rate: the actual delay rate Tard of the next state a (t+1) is less than the actual delay rate Tard of the current state a (t), the scheduling result reward for the model DQN selecting the corresponding scheduling rule is +1, and the actual delay rate Tard of the next state a (t+1) is greater than the actual delay rate Tard of the current state a (t), the reward is -1, if the next state has the same actual delay rate as the current state, then check the estimated delay rate;

[0067] If the next state estimated delay rate Tard e (t+1) is less than the current state estimated delay rate Tard e (t), the reward is +1, if the next state estimated delay rate Tard e (t+1) is greater than the current state estimated delay rate Tard e (t), the reward is -1; if the estimated delay rate of the next state is the same as the current state, check the average machine utilization:

[0068] If the average machine utilization rate in the next state is U ave (t+1) is greater than the average machine utilization U of the current state ave (t), the reward is +1, if U ave (t+1) is less than U ave (t) but greater than 0.95*U ave (t), the reward is 0, otherwise, the reward is r t is -1;

[0069] According to the above process, the reward is an evaluation of the entire scheduling result of the model DQN selecting the corresponding scheduling rule. The higher the reward, the better the evaluation. When the same single dynamic event occurs, the frequency of occurrence of the corresponding scheduling rule output by the model DQN is higher.

[0070] Furthermore, in the step 2), in the example containing only a single dynamic event, the frequency of use of the scheduling rules corresponding to the dynamic event is counted, and the five scheduling rules with the highest frequency of use are selected to form an action space set of size 5 for this type of dynamic event.

[0071] Furthermore, in step 3), specific action sets for different dynamic events are put into the algorithm, and different action sets are used according to the different dynamic events identified, and the scheduling is completed, specifically:

[0072] For the dynamic event of workpiece insertion, the corresponding action space set is recorded as Ac1; for the dynamic event of machine failure, the corresponding action space set is recorded as Ac2; the remaining scheduling rules not included in Ac1 and Ac2 are uniformly put into the action space set Ac3; each time scheduling occurs, the current time is counted. If the current time is greater than or equal to the arrival time of the new workpiece, it is determined that the dynamic event of the workpiece arrival has occurred, and the message of the new workpiece arrival is printed, and the action space Ac1 is used to complete the scheduling (first determine whether a new workpiece has arrived, and if not, determine whether a machine failure has occurred); if a machine failure occurs, the message of the machine failure is printed and the machine name is specified, and the action space Ac2 is used to complete the scheduling. When the machine selects the faulty machine, it prints that the machine is in a faulty state and reselects the machine; when no new workpiece arrives and no machine fails, the action space set Ac3 other than Ac1 and Ac2 is used to complete the scheduling; after the scheduling is completed, fault detection is performed, and if there is a machine failure, it is repaired, and the repair information and time are displayed in the next scheduling.

[0073] Finally, in step 4), by comparing the completion time and delivery time graphs generated in each iteration, the effectiveness and convergence of the algorithm can be clearly seen. The cumulative reward value of each iteration can be seen in the output results, proving that the algorithm has been optimizing. Secondly, through the image curve of delay, it can be found that the algorithm can find the best result, that is, the minimum delay value.

[0074] The beneficial effects of the present invention are mainly manifested in: based on the deep reinforcement learning method (DQN), with the help of flexible job shop modeling and dynamic event simulation, the deep learning model is realized by using tensorflow and keras packages, and the trained deep reinforcement learning strategy model can be run in pycharm to realize a dynamic flexible job shop scheduling system that considers multiple types of events. The dynamic flexible job shop scheduling system that considers multiple types of events can efficiently solve the scheduling problem of the dynamic flexible job space. Compared with the traditional heuristic-based scheduling strategy, the dynamic flexible job shop scheduling system that considers multiple types of events can effectively improve the scheduling capability of the dynamic flexible job space and reduce delays. At the same time, due to the use of the screened action space, the operation complexity can be effectively reduced and the operation speed can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A flow chart of a dynamic flexible job shop scheduling system considering multiple types of events disclosed in one embodiment of the present invention.

[0076] Figure 2 A schematic diagram of a workpiece processing flow with certain flexibility in a physical workshop disclosed in one embodiment of the present invention.

[0077] Figure 3 The diagram is a schematic diagram of an algorithm structure without action screening disclosed in an embodiment of the present invention.

[0078] Figure 4 The schematic diagram of the algorithm structure proposed in one embodiment of the present invention is disclosed.

[0079] Figure 5 This is a schematic diagram of a process for abstracting a deep reinforcement learning model based on a parameterized model and generating results using actual examples disclosed in an embodiment of the present invention. Figure 5 The middle curve a is the delivery period of each workpiece, curve b is the actual completion time, the left figure is the final result, and the right figure is the initial result.

[0080] Figure 6 It is a schematic diagram comparing the algorithm results of a dynamic flexible job shop scheduling system considering multiple types of events disclosed in an embodiment of the present invention and an algorithm without dynamic event recognition.

[0081] The present invention will be further described below in conjunction with the accompanying drawings.

[0082] Reference Figure 1 to Figure 5 , a dynamic flexible job shop scheduling system considering multiple types of events, comprising the following steps:

[0083] Step S1, abstract and simplify the flexible job shop, abstract and simplify the flexible job shop, decide on the two arrangements of machines and workpieces each time, dynamic events include but are not limited to equipment failure, inserted orders (arrival of new workpieces), processing time changes, order cancellations, etc. This patent considers two types of dynamic events: equipment failure and inserted orders.

[0084] A dispatch rule module including a series of dispatch rules is constructed, and the dispatch rule module is used to complete the selection process of different dispatch rules and the selection process of the operating machine.

[0085] The scheduling rule module selects the workpiece processing procedure according to the following process: first check whether there is a delay, that is, search whether there is a workpiece that cannot be completed on time. If there is a delayed workpiece, select the workpiece with the longest delay time for processing, otherwise select the workpiece processing procedure from the scheduling rule module; the scheduling rule module includes the following 10 heuristic scheduling rules: SPT, LPT, maximum delay rate, SRM, SRPT, LSO, EDD, LDD, mSLACK, MSLACK.

[0086] refer to Figure 2 , in this embodiment, examples and abstractions of processes and machines are given.

[0087] Taking a conventional job shop as an example, the complex system includes the following advance constraints: n: total number of jobs; m: total number of machines; J i : i-th job; n i : Belongs to Assignment J i The total number of operations; M k : kth machine; O i,j :Homework J i The jth operation of M i,j : Available operating machine set O i,j ;t i,j,k :Machine M k Operation O i,j Processing time; A i, :Homework J i Arrival time; D i :Homework J i The expiry date;

[0088] i,h: job index, i,h=1,2,…,n;

[0089] j,g: belongs to job J i and J h Operation index, j = 1, 2, ..., n i , g=1,2,…,n h ;

[0090] k: machine index, k = 1, 2, ..., m;

[0091] t i,j,k represents the processing time of the jth process of the i-th workpiece on machine k;

[0092] t h,g,k It represents the processing time of the g-th process of the h-th workpiece on the machine k.

[0093] The decision variables include the following categories:

[0094] C i,j : Operation O i,j completion time.

[0095]

[0096] X i,j,k Determines on which machine to allocate the operation, and Y i,j,h,g Determines the relative processing priority between any two operations.

[0097] Based on the above symbols, the dynamic flexible job shop (DFJSP) model used in this patent can be mathematically described as follows:

[0098]

[0099] The above formula (1) represents the total delay time of all jobs; formula (2) represents the constraints, where formula (a) indicates that the completion time of each operation must be non-negative, formula (b) indicates that each operation can only be assigned to one machine, formula (c) ensures that the job can only be processed after its arrival time, and formula (d) guarantees the priority constraint, that is, the workpiece processes need to be processed in sequence; formula (e) guarantees the capacity constraint, each machine can only process one workpiece under one process at a time.

[0100] In most reinforcement learning-based scheduling methods, state features are defined as some indicators of the production state, i.e., the number of machines / jobs in the workshop, the remaining processing time of unfinished jobs, the current workload / queue length of each machine, etc. However, in the real world, the number of machines / jobs / operations is infinite and may be very large. If these indicators are directly used as state features, the input of DQN may vary in a wide range. This may deteriorate the performance and generality of DQN in untrained situations because only specific production configurations are experienced during training. To address this issue, the state information of the current dynamic events is extracted from the real production environment. Seven carefully designed state features are extracted, each of which takes a value in the range of [0,1] as the input of DQN. By restricting all state features to [0,1], DQN can be easily extended to different untrained production environments.

[0101] Step S2, based on the established model, seven feature values ​​are input, five hidden layers with 30 nodes, covering two types of dynamic events, and a deep reinforcement learning model with ten scheduling rules. The seven feature values ​​are input as the input layer of DQN, and after DQN training, the specific type and Q value of the current dynamic event are output, and the scheduling rule is selected in the scheduling rule module according to the Q value.

[0102] See Figure 3 ,The algorithm selects from the action space set of ten scheduling rules based on the output Q value.

[0103] According to the method of the present invention, when a single dynamic event occurs, the number of occurrences of the corresponding dispatching rules when the type of dynamic event occurs is recorded, and the dispatching rules with the highest frequency of occurrence are counted and placed in a set. Similarly, the dispatching rule sets with the highest frequency of occurrence corresponding to different types of dynamic events are constructed, that is, different action space sets generated by the dispatching rule sets of different types of dynamic events are obtained.

[0104] The present invention utilizes the deep reinforcement learning model DQN training to obtain the process of action space sets corresponding to different dynamic events, including the following steps:

[0105] Consider the current state s in turn t and the next state s t+1 The values ​​of the three key status characteristics of the actual delay rate Tard a , estimated delay rate Tard e and the average machine utilization U ave When a single dynamic event occurs, the reward value is evaluated according to the action result selected by the model DQN algorithm according to the state. The selected action is the scheduling rule selected by the output of the model DQN algorithm, including:

[0106] That is, first check the actual delay rate: the actual delay rate Tard of the next state a (t+1) is less than the actual delay rate Tard of the current state a (t), the scheduling result reward for the model DQN selecting the corresponding scheduling rule is +1, and the actual delay rate Tard of the next state a (t+1) is greater than the actual delay rate Tard of the current state a (t), the reward is -1, if the next state has the same actual delay rate as the current state, then check the estimated delay rate;

[0107] If the next state estimated delay rate Tard e (t+1) is less than the current state estimated delay rate Tard e (t), the reward is +1, if the next state estimated delay rate Tarde (t+1) is greater than the current state estimated delay rate Tard e (t), the reward is -1; if the estimated delay rate of the next state is the same as the current state, check the average machine utilization:

[0108] If the average machine utilization rate in the next state is U ave (t+1) is greater than the average machine utilization U of the current state ave (t), the reward is +1, if U ave (t+1) is less than U ave (t) but greater than 0.95*U ave (t), the reward is 0, otherwise, the reward rt is -1;

[0109] According to the above process, the reward is an evaluation of the entire scheduling result of the model DQN selecting the corresponding scheduling rule. The higher the reward, the better the evaluation. When the same single dynamic event occurs, the frequency of occurrence of the corresponding scheduling rule output by the model DQN is higher. The five scheduling rules with the highest frequency of occurrence are counted and put into a set.

[0110] Step S3, running the example, can correctly identify and use the corresponding action space.

[0111] See Figure 4 , the two types of dynamic events (new workpiece arrival and machine failure) of the present invention generate two action space sets, each set includes 5 scheduling rules. According to the different dynamic events identified, different action space sets are used for scheduling. Two independent action solution space sets are generated. After completing the dynamic event identification, different Q values ​​are output each time, and the corresponding scheduling rules are selected in the corresponding action space.

[0112] Figure 3-4 In the t ,a t ) refers to the algorithm according to the current state s t , selected action a t Output Q value, selected action a t This is the selected scheduling rule.

[0113] In the present invention, for the dynamic event of workpiece insertion, the corresponding action space set is recorded as Ac1; the dynamic event of machine failure, the corresponding action space set is recorded as Ac2; the remaining scheduling rules not included in Ac1 and Ac2 are uniformly put into the action space set Ac3.

[0114] Action space set Ac1 is a set of the following five scheduling rules (SPT, maximum delay rate, EDD, LDD, MSLACK), and action space set Ac2 is a set of the following five scheduling rules (maximum delay rate, LSO, EDD, LDD, MSLACK). The remaining scheduling rules not included in Ac1 and Ac2 are put into action space set Ac3, which includes LPT, SRM, SRPT, and mSLACK.

[0115] Each time scheduling occurs, when the dynamic event of workpiece insertion occurs, the action space Ac1 is used to complete the scheduling; when the dynamic event of machine failure occurs, the action space Ac2 is used to complete the scheduling. When the machine selection selects the faulty machine, the machine is printed to be in a faulty state and the machine is reselected; when no new workpieces arrive and no machine fails, the action space set Ac3 other than Ac1 and Ac2 is used to complete the scheduling; after the scheduling is completed, fault detection is performed, and if there is a machine failure, it is repaired, and the repair information and time are displayed in the next scheduling.

[0116] Therefore, the model of the present invention generates different action space sets by means of statistically different types of dynamic event scheduling rule sets. Whenever a corresponding dynamic event occurs, the corresponding action space set is used to perform production scheduling.

[0117] See Figure 5 The picture is composed of the delivery time and completion time curves of each workpiece. The larger the difference, the longer the delay. The left picture is the final result, and the right picture is the initial result. By comparison, it can be seen that the algorithm gradually converges, the processing time of the bottleneck process is reduced, and the delay is significantly reduced. The algorithm is effective and generates a better scheduling plan.

[0118] Step S4, by recording the target value (minimum delay), obtain the image of the delivery period and completion time of each iteration, and obtain the minimum delay through the delay iteration graph.

[0119] See Figure 6 The picture is an iteration curve, each run is iterated five times, the left picture is the iteration curve result of the algorithm operation after action screening, that is, the algorithm operation proposed by this patent, and the right picture is the iteration curve result of the algorithm operation without action screening. By reducing the size of the solution space, the present invention can reduce delays and achieve optimization, while effectively reducing the running time and computing power cost of the algorithm.

[0120] This embodiment discloses a dynamic flexible job shop scheduling system that takes into account multiple types of events. It uses pycharm, tensorflow and keras packages to implement the deep learning model. In pycharm, the trained deep reinforcement learning strategy model can be run to implement a dynamic flexible job shop scheduling system that takes into account multiple types of events. The main steps in the process are the parameterized simulation modeling of the flexible job shop model and the operation of the deep reinforcement learning model, which are all implemented on the pychram platform. The present invention will not go into details. The main function of the present invention is to propose a dynamic flexible job shop scheduling system that takes into account multiple types of events. The parameter simulation method is used to provide the degree of fit between the flexible job shop model and the real processing space in dynamic events. In addition, by Figure 6 The comparison of Gantt charts proves that a dynamic flexible job shop scheduling system that considers multiple types of events can effectively reduce the delay time of shop scheduling and reduce the complexity of computational space. Excellent scheduling can effectively solve the scheduling problems of the shop and be applied in practical fields.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0122] In conclusion, the above is only a preferred embodiment of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the patent of the present invention.

Claims

1. A dynamic flexible job shop scheduling method considering multiple types of events, characterized in that The following steps are involved: 1) A dynamic flexible job shop DFJSP model is established by simulating the real production environment. The goal of the model is to minimize the total delay time of the work, constrain the operation process of machine processing workpieces, and construct a scheduling rule module including a series of scheduling rules. The scheduling rule module is used to complete the selection of different scheduling rules and the selection process of the job machine; 2) Based on the DFJSP model established in step 1), the deep reinforcement learning model DQN is used for training, that is, the state information of the current dynamic event is extracted from the real production environment, the state information is converted into state feature values ​​and normalized to take a value in the range of [0,1] as the input layer of DQN, and the specific type and Q value of the current dynamic event are output after DQN training, and the scheduling rule is selected in the scheduling rule module constructed in step 1) according to the Q value; According to the above process, when a single dynamic event occurs, the number of occurrences of the corresponding dispatch rules when the type of dynamic event occurs is recorded, and the dispatch rules with the highest occurrence frequency are counted and put into a set. Similarly, the dispatch rule sets with the highest occurrence frequency corresponding to different types of dynamic events are constructed, that is, different action space sets generated by the dispatch rule sets of different types of dynamic events are obtained; 3) The DFJSP model of a normal dynamic flexible job shop is running. Different action space sets are generated by statistically analyzing the scheduling rule sets of different types of dynamic events. Whenever a corresponding dynamic event occurs, the corresponding action space set is used to perform production scheduling; 4) By recording the delays, we can obtain a graph of the delays and the delivery and completion time of each iteration, and obtain the minimum delay through the delay iteration graph.

2. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 1, characterized in that Step 1) The establishment of the dynamic flexible job shop DFJSP model includes the following contents: First, define the following symbols: n represents the total number of jobs; m represents the total number of machines; J i represents the i-th job; n i Indicates that it belongs to job J i The total number of operations; M k represents the kth machine; O i,j Indicates job J i The jth operation of M i,j Represents the set of available operating machines O i,j ;t i,j,k Indicates machine M k Operation O i,j Processing time; A i, Indicates job J i Arrival time; D i Homework J i The expiry date; i,h: job index, i,h=1,2,…,n; j,g: belongs to job J i and J h Operation index, j = 1, 2, ..., n i , g=1,2,…,n h ; k: machine index, k = 1, 2, ..., m; t i,j,k represents the processing time of the jth process of the i-th workpiece on machine k; t h,g,k represents the processing time of the g-th process of the h-th workpiece on machine k; The decision variables include the following categories: C i,j : Operation O i,j completion time; X i,j,k Determines on which machine to allocate the operation, and Y i,j,h,g Determine the relative processing priority between any two operations; The dynamic flexible job shop DFJSP model is described mathematically as follows: The above formula (1) represents the total delay time of all jobs; Formula (2) represents the constraints, where formula (a) indicates that the completion time of each operation must be non-negative, formula (b) indicates that each operation can only be assigned to one machine, formula (c) ensures that the job can only be processed after its arrival time, and formula (d) guarantees the priority constraint, that is, the workpiece processes need to be processed in sequence; formula (e) guarantees the capacity constraint, each machine can only process one workpiece at a time under one process.

3. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 1, characterized in that Step 1) The scheduling rule module selects the workpiece processing procedure according to the following process: first, check whether there is a delay, that is, search whether there is a workpiece that cannot be completed on time. If there is a delayed workpiece, select the workpiece with the longest delay time for processing, otherwise select the workpiece processing procedure from the scheduling rule module; The scheduling rule module includes the following 10 heuristic scheduling rules: SPT, LPT, maximum delay rate, SRM, SRPT, LSO, EDD, LDD, mSLACK, MSLACK; SPT stands for the shortest processing time rule, which gives priority to the workpiece with the shortest processing time; LPT stands for the longest processing time rule, which gives priority to the workpiece with the longest processing time; Maximum delay rate: The workpiece with the largest current delay rate is selected for priority processing. The current delay rate is the ratio of the actual delay time to the allowed delay time. SRM stands for the slack time rule for the remaining operations. The workpiece with the smallest slack time divided by the number of remaining operations is selected for processing first. The slack time is the difference between the remaining time and the processing time of the workpiece. SRPT stands for the shortest remaining processing time rule, which selects the workpiece with the shortest remaining processing time for processing first; LSO stands for the least slack time per operation rule, which selects the workpiece with the smallest slack time per operation for priority processing; EDD stands for the earliest due date rule, which selects the workpiece with the earliest due date for processing first; LDD stands for the latest due date rule, which gives priority to processing the workpiece with the latest due date; mSLACK stands for the minimum slack time rule, which selects the workpiece with the smallest slack time for processing first. The slack time refers to the difference between the workpiece due date and the current time minus the processing time; MSLACK represents the maximum relaxation time rule, which selects the workpiece with the largest relaxation time for processing first; The rule for machine selection is to select the machine with the earliest available time.

4. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 1, characterized in that Step 2) Extract the state information of the current dynamic event from the real production environment and convert it into 7 state features. Each state feature takes a value in the range of [0,1] as the input of DQN; The seven status characteristics of each rescheduling time point are as follows: (1) Average utilization of machines: (2) Standard deviation of machine utilization: (3) Average completion rate of operations: (4) Average completion rate of assignments: (5) Standard deviation of the completion rate: (6) Estimated delay rate T cur is defined as the average completion time of the last operation on all machines at the current time t, then the estimated delayed job can be regarded as the estimated remaining processing time greater than the time from T cur jobs with idle time until their due date D; estimated delay rate It is defined as the number of all estimated delayed jobs divided by the number of unfinished jobs of all remaining jobs; if there is no estimated remaining processing time less than T cur The job has an idle time until its delivery date D, then is 0; (7) Actual delay rate Also consider T cur , actual delayed work is defined as work that is not completed before the deadline; actual delay rate It is defined as the actual number of delayed jobs divided by the number of unfinished jobs of all remaining jobs; if the actual remaining processing time is less than the time from T cur The job has a free time until its due date D, then is 0; The symbols involved in the state characteristics are defined as follows: CT K (t) is defined as k The completion time of the last operation assigned at the rescheduling time point t, OP i (t) is defined as job J i The current number of completed operations of machine M k The utilization rate at time t is expressed as U K (t) means, At the same time, homework J i The completion rate at time t, CRJ i (t) means, 5. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 1, characterized in that The training process of the deep reinforcement learning model DQN includes the following steps: Consider the current state s in turn t and the next state s t+1 The values ​​of the three key status characteristics of the actual delay rate Tard a , estimated delay rate Tard e and the average machine utilization U ave , when a single dynamic event occurs, the reward value is evaluated according to the action result selected by the model DQN algorithm according to the state, and the selected action is the scheduling rule selected by the output of the model DQN algorithm, including; That is, first check the actual delay rate: the actual delay rate Tard of the next state a (t+1) is less than the actual delay rate Tard of the current state a (t), the scheduling result reward for the model DQN selecting the corresponding scheduling rule is +1, and the actual delay rate Tard of the next state a (t+1) is greater than the actual delay rate Tard of the current state a (t), the reward is -1, if the next state has the same actual delay rate as the current state, then the estimated delay rate is checked; If the next state estimated delay rate Tard e (t+1) is less than the current state estimated delay rate Tard e (t), the reward is +1, if the next state estimated delay rate Tard e (t+1) is greater than the current state estimated delay rate Tard e (t), the reward is -1; if the estimated delay rate of the next state is the same as the current state, check the average machine utilization: If the average machine utilization rate in the next state is U ave (t+1) is greater than the average machine utilization U of the current state ave (t), the reward is +1, if U ave (t+1) is less than U ave (t) but greater than 0.95*U ave (t), the reward is 0, otherwise, the reward is r t is -1; According to the above process, the reward is an evaluation of the entire scheduling result of the model DQN selecting the corresponding scheduling rule. The higher the reward, the better the evaluation. When the same single dynamic event occurs, the frequency of occurrence of the corresponding scheduling rule output by the model DQN is higher.

6. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 1, characterized in that The dynamic events described in step 2) are workpiece insertion and machine failure. The specific implementation logic of workpiece insertion is to assume that there are several jobs in the workshop from the beginning, and then new jobs arrive after Poisson distribution. Therefore, the arrival interval time between two consecutive new job insertions is affected by exponential distribution; the implementation logic of machine failure is to pre-set a failure probability, randomly select a machine before starting each processing, and then generate a random number in the range of [0,1]. If the random number is greater than the failure probability, the machine enters a failure state and is overhauled for 10-30 minutes.

7. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 1, characterized in that In the step 2), in the example containing only a single dynamic event, the usage frequency of the dispatching rules corresponding to the dynamic event is counted, and the five dispatching rules with the highest usage frequency are selected to form an action space set of size 5 for this type of dynamic event.

8. A dynamic flexible job shop scheduling method considering multiple types of events as claimed in claim 6, characterized in that In step 3), specific action sets for different dynamic events are put into the algorithm, and different action sets are used according to the different dynamic events identified, and the scheduling is completed, specifically: For the dynamic event of workpiece insertion, the corresponding action space set is recorded as Ac1; for the dynamic event of machine failure, the corresponding action space set is recorded as Ac2; the remaining scheduling rules not included in Ac1 and Ac2 are put into the action space set Ac3; Each time scheduling occurs, the current time is counted. If the current time is greater than or equal to the arrival time of the new workpiece, it is determined that a dynamic event of the workpiece arrival has occurred, and the message of the new workpiece arrival is printed, and the action space Ac1 is used to complete the scheduling. If a machine failure occurs, the message of the machine failure is printed and the machine name is specified, and the action space Ac2 is used to complete the scheduling. When the machine selection selects the faulty machine, it is printed that the machine is in a faulty state and the machine is reselected. When no new workpiece arrives and no machine fails, the action space set Ac3 other than Ac1 and Ac2 is used to complete the scheduling. After the scheduling is completed, fault detection is performed. If there is a machine failure, it is repaired, and the repair information and time are displayed in the next scheduling.

Citation Information

Cited By

  • Dynamic flexible job shop predictive scheduling method and system

    CN119717735A

  • Flexible workshop production resource scheduling method based on multi-strategy deep reinforcement learning

    CN120355202A

  • A Flexible Workshop Production Resource Scheduling Method Based on Multi-Strategy Deep Reinforcement Learning

    CN120355202B