Dynamic flexible job shop predictive scheduling method and system

Abstract

The invention provides a predictive scheduling method and system for a dynamic flexible job shop, and the method comprises the steps: collecting the operation state of a machine in real time, and building a mathematical model of a shop environment; the fault probability of each machine is predicted by taking the machine as an object and combining historical data and data collected in real time; establishing a scheduling model composed of a workshop environment and a scheduling agent adopting a reinforcement learning algorithm, in the scheduling model, taking the machine fault probability and the machine operation state obtained in the workshop environment as input data of the reinforcement learning algorithm, and outputting a corresponding optimal scheduling scheme conforming to a scheduling rule; and executing and monitoring the optimal scheduling scheme, and rescheduling when a triggering condition is met. According to the method, the problems of machine faults and new operation insertion in the dynamic production environment can be effectively solved, the scheduling performance is optimized, workpiece delay is reduced, and the overall production efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of dynamic scheduling technology in industrial manufacturing, and in particular relates to a method and system for predictive scheduling of a dynamic flexible job shop. Background Art

[0002] As the complexity and uncertainty of the manufacturing industry increase, traditional workshop scheduling methods have been unable to meet production needs in a dynamic environment, especially when dealing with events such as machine failures and job insertions, which often show slow responses, leading to production interruptions and reduced efficiency. Existing solutions usually lack predictions of machine status and cannot take preventive measures before failures occur.

[0003] The main problems of dynamic flexible workshop scheduling are as follows: 1) Existing scheduling methods usually only take remedial measures after a machine failure occurs, lack real-time monitoring and prediction of the machine's health status, and cannot identify potential failure risks in advance. This passive response method will lead to production interruptions and increased maintenance costs. 2) Existing scheduling systems are slow to respond to emergencies such as job insertion, order changes, and machine failures in a dynamic environment, and rely on manual intervention to adjust production plans. The scheduling system responds slowly and lacks real-time and intelligence. 3) Most existing scheduling systems rely on regular maintenance strategies and cannot be flexibly adjusted according to actual operating conditions. Regular maintenance may lead to excessive or insufficient maintenance, which not only increases resource waste, but also fails to respond to the risk of machine failure in a timely manner, thereby affecting the stability of the production line and the service life of the machine. Therefore, there is an urgent need for an optimization method that can predict machine failures and dynamically adjust scheduling. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a dynamic flexible job shop predictive scheduling method and system, which can effectively deal with machine failures and new job insertion problems in a dynamic production environment, optimize scheduling performance, reduce workpiece delays, and improve overall production efficiency.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a dynamic flexible job shop predictive scheduling method, comprising:

[0007] S1: Collect machine operation status in real time and establish a mathematical model of the workshop environment;

[0008] S2: Taking machines as the object, combining historical data and real-time collected data, predict the failure probability of each machine;

[0009] S3: Establish a scheduling model consisting of a workshop environment and a scheduling agent using a reinforcement learning algorithm. In the scheduling model, the machine failure probability and machine operation status obtained in the workshop environment are used as input data of the reinforcement learning algorithm, and the corresponding optimal scheduling plan that meets the scheduling rules is output;

[0010] S4: Execute the optimal scheduling plan and monitor it, and reschedule when the trigger conditions are met.

[0011] Optionally, in step S1, the machine operation status includes the current load, idle time and health status of the machine.

[0012] Optionally, in step S1, the mathematical model of the workshop environment satisfies the following constraints:

[0013]

[0014] C i，j，k >S i，j，k

[0015]

[0016]

[0017]

[0018]

[0019] In the formula, i, j, and k represent the workpiece index, process index, and machine index respectively, and the i-th workpiece is denoted as J i , O i，j represents the jth process of the i-th workpiece; C i，0 Indicates workpiece J i The start time of C i，j Indicates process O i，j Completion time; C i，j，k and S i，j，k Respectively represent process O i，j The completion time and start time on machine k; M i，j Indicates compatible process O i，j The collection of machines, ζ i，j，k Indicates workpiece J i Whether the processing is completed on machine k, if yes, it is 1, otherwise it is 0; t i，1，k Indicates process O i，1 The processing time on machine k, A i Indicates workpiece J i The arrival time of i，j_i’j’ Indicates process O i，j Is it in process O? i’，j′Before, it was 1 if yes, 0 otherwise.

[0020] Optionally, in step S2, predicting the failure probability of each machine includes:

[0021] S2.1: Extract characteristic data of the health status of the machine based on the historical operation status data of the machine and the real-time collected operation status data;

[0022] S2.2: Use principal component analysis to reduce the dimension of the extracted feature data;

[0023] S2.3: The extracted feature data is used to train the random forest model to learn the relationship between the feature data and the fault, and then predict the failure probability of the machine;

[0024] S2.4: Input the current feature data of the machine in real time, and calculate the failure probability of the machine through the random forest model as follows:

[0025] P fault (t) = f(X) = f(φ L ,φ W ,φ I )

[0026] In the formula, φ L is the machine load, φ W is the health status of the machine, φ I is the idle time; the function f(·) is learned by the random forest model and is used to calculate the failure probability of the machine in a given state.

[0027] Optionally, in step S3, the scheduling goal of the scheduling model is to minimize the delay time of all tasks, which is expressed as follows:

[0028]

[0029] In the formula, C i represents the actual delivery time of the current workpiece, D i It represents the original delivery date of the current workpiece, and n represents the number of workpieces that arrive in succession.

[0030] Optionally, in step S3, the reinforcement learning algorithm adopts DDQN, which consists of 1 input layer, multiple hidden layers and 1 output layer; the input layer inputs a state vector, and the output layer outputs 5 actions that comply with scheduling rules and are used to minimize the delay time. The number of nodes in the input layer is equal to the number of state features in the state vector, and the number of nodes in the output layer is equal to the number of actions; wherein the state vector is an input vector generated by vectorizing the state data of all machines and workpieces; the hidden layer uses a fully connected network, each layer contains multiple neurons, and uses a tanh activation function.

[0031] Optionally, in step S3, the scheduling rules include 5 rules for minimizing the delay time, which are as follows:

[0032] Rule 1: Define reference time T as the current decision point time, and the average completion time of the last process assigned to each machine is t; if a certain workpiece J i Delivery time D i Earlier than the current time point T, it is put into the deferred artifact set middle, Select the process with the minimum slack time from the set of delayed workpieces Expressed as t i Represents a certain workpiece J i The average completion time of the last process; the process is assigned to the earliest available machine based on the principle of the shortest completion time. The machine selection is expressed as T a is the earliest available time of the current machine; if several machines have the same earliest available time, they are randomly assigned to one of the machines; if the set of deferred workpieces is empty, the remaining processes of each workpiece are sorted according to the time to the delivery date, and the process with the most urgent time is selected as the next processing process; before assigning to the earliest available machine, check the health status of the machine first. If the machine has a fault warning, give priority to other machines without fault warnings; if all machines have a fault probability, select the machine with the smallest fault probability;

[0033] Rule 2: Based on the remaining time of the delivery period and the remaining processing time t of the workpiece r The ratio is calculated as follows: t r (O i，j ) represents the workpiece process O i，j The remaining processing time is 100%; the workpiece process with the minimum ratio is selected to allocate the machine, and the machine failure probability is used as a weighting factor during allocation, which is expressed as P fault (M k ) indicates machine M k The probability of failure;

[0034] Rule 3: Select the next unfinished operation in the workpiece with the greatest risk of postponement and assign it to the machine with the lowest utilization rate; the maximum risk of postponement is defined as: J max = argmax{D i -(T+t r (J i ))}, the machine with the lowest utilization is represented by Where U(M k ) is the machine M k Before allocating, the failure probability of the machine is evaluated. If the machine with the lowest utilization is also the machine with the highest failure risk, the machine selection is expressed as ω1 and ω2 are weighted coefficients of utilization and failure probability;

[0035] Rule 4: Randomly select an unfinished process, prioritize machines with high failure probability, and assign it to the earliest available machine, expressed as And P fault (M k )≤P t , P t is the maximum threshold of failure probability;

[0036] Rule 5: From the estimated unfinished workpiece operation set The process with the shortest remaining processing time is selected and assigned to the earliest available machine for processing. The machine selection is expressed as

[0037] Optionally, in step S4, the triggering condition includes: when a new task arrives at the workshop and it is predicted that a machine failure will occur at a certain time point.

[0038] Optionally, in step S4, during the rescheduling, the priorities of all workpieces are recalculated, and a maintenance window time is reserved for the faulty machine; at the same time, the Q value of the possible action is recalculated based on the current operating status to select the optimal scheduling solution.

[0039] In a second aspect, the present invention provides a dynamic flexible job shop predictive scheduling system, comprising:

[0040] The acquisition module is used to collect the machine operation status in real time and establish a mathematical model of the workshop environment;

[0041] The prediction module is used to predict the failure probability of each machine by combining historical data and real-time collected data;

[0042] An optimization module is used to establish a scheduling model consisting of a workshop environment and a scheduling agent using a reinforcement learning algorithm. In the scheduling model, the machine failure probability and machine operation status obtained in the workshop environment are used as input data of the reinforcement learning algorithm to output the corresponding optimal scheduling plan that meets the scheduling rules;

[0043] The execution module is used to execute the optimal scheduling plan and monitor it, and reschedule when the trigger conditions are met.

[0044] The beneficial effects of the present invention are:

[0045] (1) The present invention integrates machine health status monitoring and fault prediction models, can predict the risk of machine failure based on real-time data, and adjust production plans before failure occurs, effectively avoiding frequent downtime caused by machine aging and wear, thereby improving production efficiency and machine service life;

[0046] (2) The present invention adopts a deep reinforcement learning algorithm to enable the scheduling system to respond to changes in the dynamic production environment in real time, dynamically optimize the production plan based on real-time data, realize automated and intelligent scheduling, significantly improve production efficiency, shorten response time, and reduce scheduling lag;

[0047] (3) The present invention uses a predictive maintenance mechanism to determine the best maintenance time based on the real-time failure probability of the machine, thereby avoiding excessive or insufficient maintenance, reducing resource waste, ensuring that the machine is always in the best working condition, extending the service life of the machine and improving the stability of the production line;

[0048] (4) The present invention comprehensively considers factors such as workpiece priority and machine failure risk, and designs flexible scheduling rules to ensure the efficiency and continuity of the production process and reduce the risk of delays. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a dynamic flexible job shop predictive scheduling method.

[0050] Figure 2 It is the operation flow chart of the fault prediction model.

[0051] Figure 3 It is the overall framework diagram of the scheduling model.

[0052] Figure 4 It is a schematic diagram of tool life curve. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] In the entire production scheduling environment, the production equipment (i.e., machine) is denoted as M. Each machine is an independent processing unit, which contains m processing units in total, denoted as M1, M2, ..., M m ; There are n workpieces arriving successively, denoted as J1, J2, ..., J n; The number of processing steps for each workpiece is different, and each step can be completed on a different processing unit to maximize production efficiency and equipment utilization.

[0055] In one embodiment, the present invention proposes a dynamic flexible job shop predictive scheduling method, the overall implementation process is as follows: Figure 1 As shown, the specific steps include:

[0056] Step 1: In a certain production area, the operating status of each machine is monitored in real time, and the current load, idle time and tool wear of each machine are recorded. At the same time, the processing progress of all workpieces is tracked, including the completed processes and pending processes of each workpiece. The data collected by sensors and programmable logic controllers (PLCs) are transmitted to the intelligent agent of the scheduling model, used in the mathematical model of the flexible job shop, and provide input for subsequent scheduling optimization.

[0057] In this embodiment, the scheduling goal of the scheduling model is to minimize the delay time of all tasks, which is expressed as follows:

[0058]

[0059] In the formula, C i represents the actual delivery time of the current workpiece, D i Indicates the original delivery date of the current workpiece.

[0060] The constraints that the mathematical model of the workshop environment must satisfy are:

[0061]

[0062] C i，j，k >S i，j，k (3)

[0063]

[0064]

[0065]

[0066]

[0067] Where i, j, and k represent the workpiece index, process index, and machine index, respectively. The i-th workpiece is denoted by J. i , O i，j represents the jth process of the i-th workpiece. Formula (2) indicates that the completion time of each process must be a non-negative number, C i，0 Indicates workpiece J i The start time of C i，j Indicates process O i，jCompletion time; Formula (3) represents process O i，j Completion time C on machine k i，j，k At the start time S i，j，k After that; Formula (4) indicates that each process can only be allocated on one machine, M i，j Indicates compatible process O i，j The collection of machines, ζ i，j，k Indicates workpiece J i Whether the processing is completed on machine k, if yes, it is 1, otherwise it is 0; Formula (5) ensures that the workpiece can only be processed after arriving, and the completion time of the first process is greater than its start time plus the arrival time of the machine, t i，1，k Indicates process O i，1 The processing time on machine k, A i Indicates workpiece J i Arrival time; Formula (6) confirms the order constraint of the process, ε i，j _ i′，j′ Indicates process O i，j Is it in process O? i’，j′ Before, if yes, then it is 1, otherwise it is 0; Formula (7) means that only one process can be processed on each machine at a time.

[0068] Step 2: Taking the machine as the research object, combining historical data and real-time data, predict the failure probability of each machine (denoted as P fault ). The scheduling intelligence will intervene in advance and arrange relevant personnel for maintenance based on the real-time predicted probability of machine failure. The fault prediction model is based on the tool life curve and multiple state parameters, and specifically includes the following steps:

[0069] (1) In this embodiment, taking the tool as an example, the historical operation time, load level, tool wear degree of the machine, and the tool use time, cutting speed and cutting angle collected in real time are input to draw the trend of wear change over time. The curve is smoothed by the sliding window method to remove noise and extract characteristic points such as the sharp wear point of the tool life.

[0070] (2) Using feature engineering, we extract characteristic values ​​such as wear rate and life inflection point from the tool life curve, which can well reflect the current health status of the machine. We use the principal component analysis (PCA) method to reduce the dimension of the extracted features to reduce the computational complexity of the model.

[0071] (3) The extracted feature data is used to train the random forest model. The random forest model integrates multiple decision trees and uses the machine's historical operation data to learn the relationship between each feature and the fault, thereby predicting the future failure risk. The prediction performance of the model is evaluated using cross-validation to ensure its generalization ability on different data sets.

[0072] (4) During the actual operation, the current characteristic data of the machine is input in real time, and combined with the tool life curve, the failure probability of the machine is calculated. fault The calculation formula is as follows:

[0073] P fault (t) = f(X) = f(φ L ,φ W ,φ I ) (8)

[0074] Among them, φ L is the machine load, φ W is the health status of the machine, taking the tool as an example, it is the degree of tool wear, φ I is the idle time. The function f(·) is learned by the random forest model and is used to calculate the failure probability of the machine in a given state.

[0075] Specifically, the fault prediction model uses an exponential distribution model to calculate the failure probability of the machine based on the historical operation data and real-time data of the machine. When the predicted failure probability exceeds the set threshold, the maintenance plan will be automatically triggered to ensure that production is not affected by unexpected downtime. At the same time, the health score (H) of the machine is also dynamically updated for overall production risk assessment. The health score of each machine is calculated based on the failure probability and machine load:

[0076] H k =1-P fault (9)

[0077] When a machine's health score falls below a set threshold, the frequency of tasks assigned to that machine will be reduced, and a new production schedule will be generated to create a suitable maintenance time window.

[0078] To facilitate understanding of the fault prediction model of this embodiment, the complete step flow of the model is provided as follows: Figure 2 As shown in the figure, it includes steps such as data input, feature extraction, model training, real-time prediction and maintenance triggering.

[0079] Step 3: Initialize the scheduling optimization model.

[0080] This embodiment optimizes production scheduling based on the DDQN algorithm. Figure 3 As shown in Figure 1, the entire scheduling model consists of an environment and a scheduling agent. In the environment, the data obtained from the production process is processed and output as a tool life curve for fault prediction. The obtained machine failure probability and machine status are used as input data for the online network. The input data includes the workshop state vector (such as machine load L, idle time T idle , failure probability P fault, tool wear W, machine health score H, workpiece processing progress P, workpiece priority U). The output is the optimal scheduling action (denoted as A op ), and store the scheduling results in the database for subsequent scheduling adjustments.

[0081] In the initialization phase, first, construct the state vector:

[0082] NumPy is used to vectorize the status data of all machines and workpieces to generate an input vector S, which includes: tool wear, remaining process machine idle time, machine working time, system latest stop time, machine remaining processing time, workpieces that have completed all processes, completed workpiece processes, workpiece start processing time, workpiece remaining processing time, workpiece waiting processing time, remaining processing time of the current process, workpiece arrival time, workpiece delay time, and the current machine usage time.

[0083] Second, build and train DDQN:

[0084] DDQN consists of multiple fully connected layers, with 1 input layer, 1 output layer, and multiple hidden layers. The number of nodes in the input and output layers is equal to the number of state features. Each hidden layer consists of multiple nodes. The input layer is fed with the processed state vector, and the hidden layer uses a fully connected network, each layer contains multiple neurons, and uses a tanh activation function. The output layer outputs actions consisting of five scheduling rules for minimizing delays.

[0085] The DDQN training process in this embodiment is as follows: First, the Q network and the target Q network are initialized to form the behavior network and the target network respectively, and their initial parameters are set to θ and At the same time The initial value of is assigned to θ. Next, construct an experience replay pool It is used to store the experience data collected during the production process. Each piece of experience data consists of the current state s t , Action taken a t , the reward r t and the next state s t+1 In the production process, according to the current state s t Select action a through the behavior network t , after executing the action, you get a reward r t and enter a new state s t+1 , then the experience {s t , a t , r t ,s t+1} is stored in the experience replay pool. In order to optimize the behavior network, a small batch is randomly sampled from the experience replay pool. The data, i.e. the sample data, is calculated through the behavior network to calculate the Q value of the current state, and the target network is used to calculate the Q value of the next state to determine the target Q value. Specifically, the target Q value is given by the following formula:

[0086]

[0087] Among them, r t represents the immediate reward, reflecting the direct result of the action, and is usually used to encourage or punish the behavior of the agent; γ represents the discount factor, which determines the importance of the agent to future rewards, ranging from 0 to 1; a′ is the action that maximizes the Q value; s t+1 Represents the next state, which is when executing action a t The new state reached later; θ is the parameter of the online network, which is updated by gradient descent at each step; is the parameter of the target network. Finally, the parameters θ of the behavior network are updated by minimizing the following loss function through the gradient descent algorithm:

[0088]

[0089] E represents the calculation of the expected value of all experience data randomly sampled from the experience replay pool; by continuously optimizing the loss function, the parameters of the behavior network are gradually updated to improve the accuracy of action selection and the efficiency of production scheduling.

[0090] Step 4: Specific implementation of scheduling rules.

[0091] First, define the action space:

[0092] Action a represents the allocation relationship between workpieces and machines. All possible actions are included in the action set, and the optimal action is selected through the deep reinforcement learning algorithm. In order to overcome the limitations of any single rule, the action space designed in this embodiment is composed of various heuristic rules, and the appropriate scheduling rule is selected at the decision point. This embodiment designs 5 scheduling rules, which aim to allocate machines to workpieces with delay risks, taking into account the premise that the longer the machine runs, the greater the probability of failure, so as to ensure that the delay of the construction period caused by machine failure is avoided to the greatest extent.

[0093] The action space of this embodiment consists of five scheduling rules for minimizing delays:

[0094] Rule 1: First, define a reference time T as the current decision point time, and the average completion time of the last process assigned to each machine is t. i Earlier than the current time point T, it is put into the deferred artifact set In Select the process with the minimum slack time from the set of delayed workpieces Expressed as t i represents the average completion time of the last process of a certain workpiece; then the process is assigned to the machine based on the principle of the shortest completion time, and is assigned to the earliest available machine for processing, which is expressed as T a is the earliest available time of the current machine. If several machines have the same earliest available time, they are randomly assigned to one of the machines. If the set of deferred workpieces is empty, the remaining processes of each workpiece are sorted according to the time to the delivery date, and the most urgent process is selected as the next processing process. Before assigning to the earliest available machine, check the health status of the machine. If the machine has a fault warning, give priority to other machines with better health status. If all machines have a risk of failure, select the machine with the lowest risk of failure.

[0095] Rule 2: Based on the remaining time of the delivery period and the remaining processing time t of the workpiece r The ratio is calculated as follows: The machine with the smallest ratio is selected for the workpiece process allocation. The smaller the ratio, the higher the risk of delay. The allocated machine is still the earliest available machine. At the same time, the risk of machine failure is used as a weighting factor, and the ratio is multiplied by (1 + failure probability), which is expressed as This avoids choosing a machine with a high risk of failure.

[0096] Rule 3: Select the next unfinished operation in the job with the greatest risk of delay and assign it to the machine with the lowest utilization. The maximum risk of delay is defined as: J max = argmax{D i -(T+t r (J i ))}, the machine with the lowest utilization is represented by Where U(M k ) is the utilization rate of the machine. Before that, evaluate the failure probability of the machine. If the machine with the lowest utilization rate is also the machine with the highest failure risk, consider choosing a machine with a lower failure probability but a slightly higher utilization rate. The specific expression is: Where: ω1 and ω2 are weighted coefficients of utilization and failure probability, which are used to balance the impact of the two factors.

[0097] Rule 4: Randomly select an unfinished process, prioritize machines with high failure risks, ensure that the workpiece is not assigned to a machine that is about to have a problem, and then assign it to the earliest available machine, expressed as And Pfault (M k )≤P t , P t is the maximum threshold of failure probability.

[0098] Rule 5: From the estimated unfinished workpiece operation set The process with the shortest remaining processing time is selected and assigned to the earliest available machine for processing. The machine selection is expressed as

[0099] In the training phase, in order to enhance exploration, the action of each rearrangement point is randomly selected. However, when the trained DDQN is applied to the actual production process, the action with a higher Q value should have a higher probability of being selected. On the other hand, if the action with the highest Q value is always selected, reinforcement learning can easily fall into a local optimum. To this end, this embodiment adopts a dynamic ε greedy strategy to select actions. In this strategy, the initial value of ε is 0.9, and each decay The minimum value of ε is set to 0.05, and the probability of development in each step thereafter is 0.95, while the probability of exploration is 0.05. Therefore, this embodiment designs a dynamic ε-greedy strategy calculated by formula (12) to adapt to the flexible job shop scheduling problem with different solution spaces.

[0100]

[0101] Among them, n ep is the total number of training rounds; s is the training step length.

[0102] The urgency of each artifact is calculated using the following formula:

[0103]

[0104] Among them, U i Indicates the urgency of the workpiece, D i is the delivery deadline of the workpiece, The total time required for a workpiece to complete all processing steps on machine k. Different machines require different times. The more urgent the workpiece, the faster it needs to be processed.

[0105] Step 5: Real-time scheduling execution and monitoring.

[0106] (1) Execution of task assignment:

[0107] Once the scheduling decision is made, the system will issue task instructions through the central controller. Each instruction contains key information such as the workpiece number, target machine, and processing steps. After the task is issued, the human-machine interface (HMI) displays the processing status of each machine in real time. The operator can monitor and confirm the execution of the task through the HMI to ensure that the scheduling instructions are implemented correctly.

[0108] (2) Real-time monitoring and status feedback:

[0109] Each machine is equipped with a vibration sensor to monitor the machine's operating status in real time during the processing process to prevent abnormalities. The sensor data is transmitted to the central controller via the MQTT protocol to achieve real-time data feedback. At the same time, each workpiece will pass through an RFID reading point during the movement process, which can record and track the location information of the workpiece and store the data to ensure the transparency and traceability of the production process.

[0110] Step 6: Trigger rescheduling and maintenance.

[0111] The triggering of rescheduling is mainly determined by two conditions: when a new task arrives at the workshop or a machine failure is predicted at a certain point in time, rescheduling will be automatically triggered. During rescheduling, the priority of all workpieces will be recalculated, and a maintenance window will be reserved for the faulty machine to ensure the continuity of the production process as much as possible without interrupting the processing of the workpiece. The Q value of possible actions will be recalculated based on the current state to select the optimal scheduling plan. When the rescheduling decision is generated, the tasks on the high-risk machines will be migrated to the machines with lower loads, and the record of this scheduling adjustment will be stored in the scheduling log to ensure that the scheduling process is transparent and traceable, which is convenient for future analysis and optimization.

[0112] In order to more clearly demonstrate the relationship between tool wear and fault occurrence, this embodiment introduces a tool wear fault occurrence curve based on historical data (see Figure 4 ). This curve records the wear of each machine in different working time periods, revealing the law that the performance of the tool gradually decreases over time. This curve is an important reference. It helps evaluate the health status of each machine under different working conditions through accumulated wear data. In this way, the probability of machine failure can be predicted more accurately, and preventive maintenance can be performed at the right time.

[0113] During the training process, the tool wear curve data is continuously analyzed to gradually optimize its scheduling strategy. Each scheduling decision is based not only on the priority of the workpiece and the current load of the machine, but also on the wear state of the tool and the expected time of failure, so as to ensure that maintenance operations can be reasonably scheduled while ensuring efficient production. Through this curve, the system learns to schedule maintenance operations before the machine is about to reach the peak failure point, minimizing the risk of failure. This learning method based on historical data effectively reduces production interruptions caused by unexpected machine failures and improves stability.

[0114] In another embodiment, the present invention proposes a dynamic flexible job shop predictive scheduling system, which corresponds to a dynamic flexible job shop predictive scheduling method of the above embodiment, including:

[0115] The acquisition module is used to collect the machine operation status in real time and establish a mathematical model of the workshop environment;

[0116] The prediction module is used to predict the failure probability of each machine by combining historical data and real-time collected data;

[0117] An optimization module is used to establish a scheduling model consisting of a workshop environment and a scheduling agent using a reinforcement learning algorithm. In the scheduling model, the machine failure probability and machine operation status obtained in the workshop environment are used as input data of the reinforcement learning algorithm to output the corresponding optimal scheduling plan that meets the scheduling rules;

[0118] The execution module is used to execute the optimal scheduling plan and monitor it, and reschedule when the trigger conditions are met.

[0119] In this system, the functions and workflow of each module are the same as those of the dynamic flexible job shop predictive scheduling method in the aforementioned embodiment, so they will not be repeated here.

[0120] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A dynamic flexible job shop predictive scheduling method, characterized in that: include: S1: Collect machine operation status in real time and establish a mathematical model of the workshop environment; S2: Taking machines as the object, combining historical data and real-time collected data, predict the failure probability of each machine; S3: Establish a scheduling model consisting of a workshop environment and a scheduling agent using a reinforcement learning algorithm. In the scheduling model, the machine failure probability and machine operation status obtained in the workshop environment are used as input data of the reinforcement learning algorithm, and the corresponding optimal scheduling plan that meets the scheduling rules is output; S4: Execute the optimal scheduling plan and monitor it, and reschedule when the trigger conditions are met.

2. A dynamic flexible job shop predictive scheduling method as claimed in claim 1, characterized in that: In step S1, the machine operation status includes the current load, idle time and health status of the machine.

3. A dynamic flexible job shop predictive scheduling method as claimed in claim 1, characterized in that: In step S1, the mathematical model of the workshop environment satisfies the following constraints: Ci,j,k>Si,j,k In the formula, i, j, and k represent the workpiece index, process index, and machine index, respectively, and the i-th workpiece is denoted as J i , O i,j represents the jth process of the i-th workpiece; C i，0 Indicates workpiece J i The start time of C i,j Indicates process O i，j Completion time; C i，j，k and S i，j,k Respectively represent process O i，j The completion time and start time on machine k; M i，j Indicates compatible process O i，j The collection of machines, ζ i，j,k Indicates workpiece J i Whether the processing is completed on machine k, if yes, it is 1, otherwise it is 0; t i，1,k Indicates process O i,1 The processing time on machine k, A i Indicates workpiece J i Arrival time; ε i，j_i’j’ Indicates process O i,j Is it in process O? i′，j′ Before, it was 1 if yes, 0 otherwise.

4. A dynamic flexible job shop predictive scheduling method as claimed in claim 1, characterized in that: In step S2, predicting the failure probability of each machine includes: S2.1: Extract characteristic data of the health status of the machine based on the historical operation status data of the machine and the operation status data collected in real time; S2.2: Use principal component analysis to reduce the dimension of the extracted feature data; S2.3: The extracted feature data is used to train the random forest model to learn the relationship between the feature data and the fault, and then predict the failure probability of the machine; S2.4: Input the current feature data of the machine in real time, and calculate the failure probability of the machine through the random forest model as follows: P fault (t)=f(X)=f(φ L ,f W ,f I ) In the formula, φ L is the machine load, φ W is the health status of the machine, φ I is the idle time; the function f(·) is learned by the random forest model and is used to calculate the failure probability of the machine in a given state.

5. A dynamic flexible job shop predictive scheduling method as claimed in claim 1, characterized in that: In step S3, the scheduling goal of the scheduling model is to minimize the delay time of all tasks, which is expressed as follows: In the formula, C i represents the actual delivery time of the current workpiece, D i It represents the original delivery date of the current workpiece, and n represents the number of workpieces that arrive in succession.

6. A dynamic flexible job shop predictive scheduling method as claimed in claim 5, characterized in that: In step S3, the reinforcement learning algorithm adopts DDQN, which consists of 1 input layer, multiple hidden layers and 1 output layer; the input layer inputs a state vector, and the output layer outputs 5 actions that comply with the scheduling rules and are used to minimize the delay time. The number of nodes in the input layer is equal to the number of state features in the state vector, and the number of nodes in the output layer is equal to the number of actions; wherein the state vector is an input vector generated by vectorizing the state data of all machines and workpieces; the hidden layer uses a fully connected network, each layer contains multiple neurons, and uses a tanh activation function.

7. A dynamic flexible job shop predictive scheduling method as claimed in claim 6, characterized in that: In step S3, the scheduling rules include 5 rules for minimizing the delay time, which are as follows: Rule 1: Define reference time T as the current decision point time, and the average completion time of the last process assigned to each machine is t; if a certain workpiece J i Delivery time D i Earlier than the current time point T, it is put into the deferred artifact set middle, Select the process with the minimum slack time from the set of delayed workpieces Expressed as argmin{D i -(T+t i )), t i It represents the average completion time of the last process of a certain workpiece Ji; Assign machines to processes based on the principle of shortest completion time, and assign them to the earliest available machine for processing. The machine selection is expressed as T a is the earliest available time of the current machine; if several machines have the same earliest available time, they are randomly assigned to one of the machines; if the set of deferred workpieces is empty, the remaining processes of each workpiece are sorted according to the time to the delivery date, and the process with the most urgent time is selected as the next processing process; before assigning to the earliest available machine, check the health status of the machine first. If the machine has a fault warning, give priority to other machines without fault warnings; if all machines have a fault probability, select the machine with the smallest fault probability; Rule 2: Based on the remaining time of the delivery period and the remaining processing time t of the workpiece r The ratio is calculated as follows: t r (O i，j ) represents process O i，j The remaining processing time is 100%; the workpiece process with the minimum ratio is selected to allocate the machine, and the machine failure probability is used as a weighting factor during allocation, which is expressed as Indicates machine M k The probability of failure; Rule 3: Select the next unfinished operation in the workpiece with the greatest risk of postponement and assign it to the machine with the lowest utilization rate; the maximum risk of postponement is defined as: J max = argmax{D i -(T+t r (J i ))}, the machine with the lowest utilization is represented by Where U(M k ) is the machine M k Before allocating, the failure probability of the machine is evaluated. If the machine with the lowest utilization is also the machine with the highest failure risk, the machine selection is expressed as ω1 and ω2 are weighted coefficients of utilization and failure probability; Rule 4: Randomly select an unfinished process, prioritize machines with high failure probability, and assign it to the earliest available machine, expressed as And P fault (M k )≤P t , P t is the maximum threshold of failure probability; Rule 5: From the estimated unfinished workpiece operation set The process with the shortest remaining processing time is selected and assigned to the earliest available machine for processing. The machine selection is expressed as 8. A dynamic flexible job shop predictive scheduling method as claimed in claim 1, characterized in that: In step S4, the triggering conditions include: when a new task arrives at the workshop and when it is predicted that a machine failure will occur at a certain time point.

9. A dynamic flexible job shop predictive scheduling method as claimed in claim 1, characterized in that: In step S4, during the rescheduling, the priorities of all workpieces are recalculated, and a maintenance window time is reserved for the faulty machine; at the same time, the Q value of the possible action is recalculated based on the current operating status to select the optimal scheduling solution.

10. A dynamic flexible job shop predictive scheduling system, characterized in that: include: The acquisition module is used to collect the machine operation status in real time and establish a mathematical model of the workshop environment; The prediction module is used to predict the failure probability of each machine by combining historical data and real-time collected data; An optimization module is used to establish a scheduling model consisting of a workshop environment and a scheduling agent using a reinforcement learning algorithm. In the scheduling model, the machine failure probability and machine operation status obtained in the workshop environment are used as input data of the reinforcement learning algorithm to output the corresponding optimal scheduling plan that meets the scheduling rules; The execution module is used to execute the optimal scheduling plan and monitor it, and reschedule when the trigger conditions are met.

Images