Flexible job shop dynamic batch flow scheduling optimization method considering emergency order insertion and reworking based on improved DQN

Abstract

The invention relates to an improved DQN-based flexible job shop dynamic batch flow scheduling optimization method considering emergency order insertion and reworking, and belongs to the technical field of shop scheduling. The method comprises the following steps of: considering a flexible job shop dynamic batch flow scheduling multi-objective optimization mathematical model of emergency order insertion and reworking; based on a knowledge-driven batching method and a multi-agent DQN model solving method, a local optimal solution of a flexible job shop dynamic batch flow scheduling model problem considering workpiece batching is further explored. The problem that a traditional optimization method is poor in applicability and efficiency under dynamic event interference is solved, and flexible job shop dynamic batch flow optimization scheduling is achieved. According to the optimization method, the problem of flexible job shop dynamic batch flow scheduling optimization under the order insertion and workpiece reworking dynamic event can be efficiently solved, waste of resources such as energy, time and cost is effectively avoided, production interruption and maintenance cost are reduced, and a scheduling scheme with good completion time and order insertion workpiece delay time is obtained.

Description

Technical Field

[0001] This invention relates to a dynamic batch flow scheduling optimization method for flexible job shops that takes into account emergency order insertion and rework, based on an improved DQN, and belongs to the field of job shop scheduling technology. Background Technology

[0002] Discrete manufacturing is a crucial pillar of the national economy, significantly driving economic growth. Its workshop production is typically characterized by complex processes, variable batch sizes for multiple product types, and the coexistence of research and development with production, leading to frequent batch changes and difficulties in resource matching. Furthermore, numerous dynamic and uncertain factors exist in the production workshop, such as frequent emergency order insertions due to the coexistence of research and development with production; and rework of workpieces due to machine tool performance degradation and worker errors, severely impacting normal workshop production. In this complex environment, the difficulty of coordinating and optimizing workshop production resources increases dramatically. Traditional static scheduling methods are ineffective for resource allocation, and static production plans lack practical guidance. Therefore, research on dynamic batch flow scheduling methods for flexible workshops is particularly important.

[0003] In actual workshops, numerous dynamic events occur, such as emergency order insertions and machine malfunctions. The states in actual production workshops are complex and constantly changing. Flexible workshops that only consider a single event are insufficient to accurately reflect actual production. However, the dynamic scheduling problem of workshops driven by multiple events is even more complex, and solving its dynamic model takes longer. Therefore, effectively resolving conflicts among multiple dynamic events, and based on this, constructing a mathematical model for dynamic batch flow scheduling of flexible workshops with completion time and delay time of inserted workpieces as optimization objectives; and designing targeted and efficient solution methods considering the characteristics of emergency order insertions, rework, and batch scheduling are key to achieving optimization of dynamic batch flow scheduling of flexible workshops that considers emergency order insertions and rework.

[0004] Currently, scholars have conducted research on single-machine / parallel machine scheduling, flexible job shop scheduling, dynamic scheduling, and scheduling considering workpiece batching from the aspects of model construction and optimization algorithm design, and have achieved certain results. However, no relevant research has been found that comprehensively considers multiple dynamic events and flexible production batch flow scheduling. The states in actual workshops are complex and changeable; flexible job shops considering only a single event are difficult to adapt to actual production. However, the dynamic scheduling problem of job shops driven by multiple events is more complex, and solving its dynamic model takes longer. Recent research has focused on simultaneously considering multiple dynamic factors such as temporary order insertion, workpiece rework, preventative maintenance, and employee fatigue. Compared to single-event-driven scheduling, flexible job shop batch scheduling models and optimization methods considering multiple dynamic events are more able to meet actual production needs. Therefore, how to establish an energy-saving batch flow scheduling mathematical model for flexible job shops considering dynamic events and design an efficient and feasible optimization algorithm still urgently needs further exploration. Summary of the Invention

[0005] To address the aforementioned shortcomings and improvement needs of existing technologies, this invention proposes a dynamic batch flow scheduling optimization method for flexible workshops that considers emergency order insertion and rework. Its purpose is to solve the problem of insufficient robustness of traditional optimization methods caused by machine tool deterioration and flexible maintenance, and to achieve energy-saving batch flow multi-objective robust optimization scheduling for flexible workshops that considers machine tool deterioration and preventive maintenance.

[0006] To address the aforementioned technical problems, this invention proposes an energy-saving batch flow scheduling optimization method for flexible workshops that considers machine tool deterioration and preventative maintenance. The method includes the following steps: S1, constructing a multi-objective optimization mathematical model for dynamic batch flow scheduling in flexible workshops, considering emergency order insertion and rework, with completion time and delay time of inserted workpieces as optimization objectives; S2, proposing a model solution method based on knowledge-driven batching and multi-agent DQN.

[0007] According to the present invention, as a further preferred embodiment, the flexible workshop dynamic batch flow scheduling multi-objective optimization mathematical model considering emergency order insertion and rework in step S1 has the optimization objectives of minimizing the completion time and the delay time of the inserted workpieces, and the calculation formula is as follows:

[0008]

[0009] Where F is the objective function, C max DE Ti These are the workshop completion time and the delay time for inserted workpieces, respectively.

[0010] X ikjm =1

[0011] PTS ikjm =PTD ikjm +PTC ikjm +PTZ ikjm +PT ikjm

[0012]

[0013] Where i is the workpiece type number, k is the sub-batch number of workpiece i, j is the process number, M is the total number of machine tools, m is the machine tool type number, and T i For temporary insertion of single workpieces, T ij For temporary insertion of single workpiece T i The j-th process; X ikjm For decision variables, PTS is 1 if workpiece i is processed on machine m in operation j, and 0 otherwise; ikjm , These represent the total processing time of the j-th process in the k-th batch of workpiece i on machine tool m, and the total processing time of the j-th process in the insert workpiece on machine tool m, respectively; PTD ikjm PTC ikjm PTZ ikjm and PT ikjm These represent the tool setting time, workpiece disassembly time, workpiece clamping time, and machining time for the j-th process in the k-th batch of workpiece i on machine tool m, respectively. and They are respectively single workpiece T i The tool setting time, workpiece disassembly time, workpiece clamping time, and machining time for the j-th process on machine tool m.

[0014]

[0015] in, For inserting single workpiece T i The start time of the j-th process on machine tool m; D i Let i be the delivery date for workpiece i. This is the delay time for inserting a single workpiece.

[0016] In addition to meeting the basic constraints of flexible workshop scheduling, the workpiece batching constraint must also be met, as follows:

[0017]

[0018] Among them, P ik Let N be the number of sub-batches in the k-th batch of workpiece i. i Let i be the total number of workpieces i.

[0019] Process rework constraints. When a process is found to be non-conforming, rework is required for that process, as shown below:

[0020]

[0021] Among them, FT ij For the decision variable, take 1 if the j-th process of workpiece i requires rework, otherwise take 0; J Ri,j and J ij These represent the rework required for the j-th process of workpiece i and the j-th process of workpiece i, respectively.

[0022] Batch conversion constraint. When a machine tool converts its processing batch, the tool change time of the first workpiece in that batch must be taken into account, as expressed as:

[0023] PTS ikjm =PTS ikjm +PTP ikjm =PTD ikjm +PTC ikjm+PTZ ikjm +PT ikjm +PTP ikjm

[0024]

[0025] Among them, PTS ikjm 'and PTP ikjm The total machining time and tool change time on machine tool m are when the j-th pass in the k-th batch of workpiece i is changed to a different processing machine. and For inserting single workpiece T i The total machining time and tool change time on machine tool m when changing the machining machine in the j-th process.

[0026] According to the present invention, as a further preferred embodiment, step S2 includes the following sub-steps:

[0027] Design a shop floor state model based on the estimated delay rate and actual delay rate of inserted workpieces. The model is described by introducing two indicators: the estimated delay rate and the actual delay rate of inserted workpieces. The specific operation is as follows:

[0028] First, the available time of the computer bed is calculated using the following formula:

[0029]

[0030] Where AFT is the average waiting time of all available machine tools at the current time t. Indicates single workpiece T i The completion time of the j-th process on machine tool m. M represents the current moment, and M represents the number of machine tools.

[0031] Calculate the average completion time of the remaining processes for the inserted workpiece and the estimated completion time of the workpiece:

[0032]

[0033]

[0034] in, The total machining time for a single workpiece is the sum of the average completion times for machining it on different machine tools. For the remaining operations of the single workpiece; T CP (current) represents the estimated completion time of the workpiece, and AFT represents the average machine waiting time. The average completion time for the remaining processes. This refers to the current moment.

[0035] Estimated delayed jobs can be considered jobs whose remaining processing time exceeds the sum of the AFT and the current processing time. For each For unfinished insert workpieces, determine if T CP (current) > D Ti Then N ED =N ED +1, otherwise N CP =N CP +1.

[0036] in, For inserting single workpiece T i Arrival time, N ED To estimate the number of delayed workpieces, N CP This represents the number of undelayed workpieces.

[0037] The estimated delay rate for interstitial workpieces is equal to the estimated number of delayed operations divided by the total number of operations in all jobs, as shown below:

[0038]

[0039] Where Estimated_delay(t) is the estimated delay rate of the inserted workpiece, N ED To estimate the number of delayed workpieces, N CP This represents the number of undelayed workpieces.

[0040] Then, the actual delay rate of the inserted workpieces is calculated. For all inserted workpieces that have been processed, a judgment is made: if... Then N AD =N AD +1, otherwise N CP =N CP +1.

[0041] in, Let N be the completion time of single workpiece i. AD N represents the actual number of delayed workpieces. CP This represents the number of undelayed workpieces.

[0042] Based on this, the formula for calculating the actual delay rate of the inserted workpiece is:

[0043]

[0044] Where Actual_delay(t) is the actual delay rate of the inserted workpiece, N AD N represents the actual number of delayed workpieces. CP This represents the number of undelayed workpieces.

[0045] According to the present invention, as a further preferred embodiment, the efficient workshop system motion space representation method in step S2 is as follows:

[0046] Determine the estimated number of delayed workpieces N for the inserted workpiece. ED If the value is 0, then execute the static action and input the feature value into the static model for solution. The estimated number of delayed workpieces N for the single-workpiece insertion is... ED If the value is greater than 0, then two dynamic actions are executed, and the feature value is input into the dynamic DQN model for solution.

[0047] Action 1 selects workpieces based on their completion rate and delay level, and selects the machine tool with the shortest processing time for processing. Among the workpieces with estimated delays, the workpiece with the largest product of the completion rate of the inserted workpiece process multiplied by the sum of the current time and the estimated completion time and the difference between the sum and the delivery date is selected.

[0048] Then, the machine tool that can complete the processing of the current workpiece the fastest is identified. The processing time of the workpiece is divided into two segments: the first segment is the waiting time when the machine tool is still processing the previous workpiece, and the second segment is the processing time of this process on the machine tool (single workpieces are processed in batches by default). The machine tool with the smallest sum is selected for processing.

[0049]

[0050] in, Here, AFT is the estimated completion time for the currently inserted workpiece, and AFT is the machine's average waiting time. This represents the total processing time for a single workpiece.

[0051]

[0052] Among them, J i The product of the completion rate of the insert workpiece process multiplied by the sum of the current time and the estimated completion time, and the difference between the sum and the delivery date, is the product of the workpieces with the largest product. i For the delivery time of the inserted workpiece, For the current moment, T ij For inserting single workpiece T i Total number of processes, T ij(t) Inserting a single workpiece T at time t ij Number of completed processes.

[0053]

[0054] Among them, PTS Tikjm For inserting single workpiece T i The processing time of the j-th process in the k-th batch on machine tool m. Indicates single workpiece T i The moment when the j-th process is completed on machine tool m.

[0055] Action 2 selects workpieces based on remaining processing time and delivery date, i.e., selecting the workpiece with the largest difference between estimated completion time and delivery date for processing. The calculation formula is as follows:

[0056]

[0057] The workpiece is then placed on the machine tool that can process it fastest. The calculation formula for selecting the machine tool with the shortest processing time is the same as for action 1.

[0058] According to the present invention, as a further preferred embodiment, the method for representing the reward function of the workshop system in step S2 is as follows:

[0059] Design a two-stage reward function based on dynamic scheduling. The specific steps are as follows:

[0060] Step 1: After a single scheduling task is completed, compare the completion time of the current processing plan with the completion time of the optimal processing plan;

[0061] Step 2: Determine the reward value based on the difference;

[0062] Step 3: After each scheduling action, calculate the difference between the estimated delayed workpieces and the actual delayed workpieces;

[0063] Step 4: Assess machine tool utilization and provide corresponding rewards based on changes.

[0064] The beneficial effects of this invention are as follows: Firstly, with the goal of minimizing completion time and delay time for interrupted orders, a flexible job shop batch scheduling mathematical model considering emergency order interruptions and rework is constructed. Secondly, based on the problem characteristics, a multi-agent DQN method and a knowledge-driven batching method are designed to solve the model. The knowledge-driven batching method optimizes the workpiece batching sub-problem. The flexible job shop scheduling model considering workpiece batching is transformed into an MDP model to achieve collaborative optimization of multiple sub-problems, including batching, process sequencing, and machine tool allocation. Based on this, various efficient methods for representing the shop system's state space, action space, and reward function are designed. This effectively addresses the collaborative optimization problems of workpiece batching, process sequencing, and machine tool allocation in flexible job shops under dynamic events such as emergency order interruptions and workpiece rework, reducing total shop energy consumption and production costs, shortening product completion time, and providing theoretical basis and practical support for actual production of multiple varieties and variable batches under dynamic and uncertain environments. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the optimization process for the flexible job shop scheduling problem based on the improved multi-agent DQN algorithm of this invention.

[0066] Figure 2 This is the pseudocode for the reward function proposed in this invention;

[0067] Figure 3 The main effects plots for the three parameters of the improved DQN of this invention are shown below;

[0068] Figure 4 This is an example of an initial workshop scheduling scheme based on an embodiment of the present invention.

[0069] Figure 5 This invention provides an improved DQN algorithm for solving instance problems, which is a workshop rescheduling scheme. Detailed Implementation

[0070] The present invention will be further described in detail below with reference to the accompanying drawings. However, it should be understood that the embodiments are used to explain the present invention and are not intended to limit the present invention.

[0071] The present invention provides a dynamic batch flow scheduling optimization method for flexible workshops that considers emergency order insertion and rework. The steps of the method are as follows: Figure 1 As shown, it includes the following steps:

[0072] S1. Multi-objective optimization mathematical model for dynamic batch flow scheduling of flexible workshops, considering emergency order insertion and rework.

[0073] S2. A model solution method based on knowledge-driven batch processing and multi-agent DQN.

[0074] S3, Numerical Experiment.

[0075] S4, Instance Verification.

[0076] Furthermore, the multi-objective optimization mathematical model for dynamic batch flow investigation of flexible workshops, considering emergency order insertion and rework, in step S1 is as follows:

[0077] Objective function: F = min(C) max +DE Ti )

[0078] Where F is the objective function, C max DE Ti These are the workshop completion time and the delay time for inserted workpieces, respectively.

[0079] X ikjm =1

[0080] PTS ikjm =PTD ikjm +PTC ikjm +PTZ ikjm +PT ikjm

[0081]

[0082] Where i is the workpiece type number, k is the sub-batch number of workpiece i, j is the process number, M is the total number of machine tools, m is the machine tool type number, and T i For temporary insertion of single workpieces, T ij For temporary insertion of single workpiece T i The j-th process; X ikjm For decision variables, PTS is 1 if workpiece i is processed on machine m in operation j, and 0 otherwise; ikjm , These represent the total processing time of the j-th process in the k-th batch of workpiece i on machine tool m, and the total processing time of the j-th process in the insert workpiece on machine tool m, respectively; PTD ikjm PTC ikjm PTZ ikjm and PT ikjm These represent the tool setting time, workpiece disassembly time, workpiece clamping time, and machining time for the j-th process in the k-th batch of workpiece i on machine tool m, respectively. and They are respectively single workpiece T i The tool setting time, workpiece disassembly time, workpiece clamping time, and machining time for the j-th process on machine tool m.

[0083]

[0084] in, For inserting single workpiece T i The start time of the j-th process on machine tool m; D i Let i be the delivery date for workpiece i. This is the delay time for inserting a single workpiece.

[0085] In addition to meeting the basic constraints of flexible workshop scheduling, the workpiece batching constraint must also be met, as follows:

[0086]

[0087] Among them, P ik Let N be the number of sub-batches in the k-th batch of workpiece i. i Let i be the total number of workpieces i.

[0088] The process rework constraint must also be met: when a process is found to be non-conforming, the non-conforming process must be reworked, as shown below:

[0089]

[0090] Among them, FT ij For the decision variable, take 1 if the j-th process of workpiece i requires rework, otherwise take 0; J Ri,j and J ijThese represent the rework required for the j-th process of workpiece i and the j-th process of workpiece i, respectively.

[0091] Batch conversion constraint. When a machine tool converts its processing batch, the tool change time of the first workpiece in that batch must be taken into account, as expressed as:

[0092] PTS ikjm =PTS ikjm +PTP ikjm =PTD ikjm +PTC ikjm +PTZ ikjm +PT ikjm +PTP ikjm

[0093]

[0094] Among them, PTS ikjm 'and PTP ikjm The total machining time and tool change time on machine tool m are when the j-th pass in the k-th batch of workpiece i is changed to a different processing machine. and For inserting single workpiece T i The total machining time and tool change time on machine tool m when changing the machining machine in the j-th process.

[0095] According to the present invention, as a further preferred embodiment, step S2 includes the following sub-steps:

[0096] Design a shop floor state model based on the estimated delay rate and actual delay rate of inserted workpieces. The model is described by introducing two indicators: the estimated delay rate and the actual delay rate of inserted workpieces. The specific operation is as follows:

[0097] First, the available time of the computer bed is calculated using the following formula:

[0098]

[0099] Where AFT is the average waiting time of all available machine tools at the current time t. Indicates single workpiece T i The completion time of the j-th process on machine tool m. M represents the current moment, and M represents the number of machine tools.

[0100] The formula for calculating the average completion time of the remaining processes for the inserted workpiece and the estimated completion time of the workpiece is as follows:

[0101]

[0102]

[0103] in, The total machining time for a single workpiece is the sum of the average completion times for machining it on different machine tools. For the remaining operations of the single workpiece; T CP (current) represents the estimated completion time of the workpiece, and AFT represents the average machine waiting time. The average completion time for the remaining processes. This refers to the current moment.

[0104] Estimated delayed jobs can be considered jobs whose remaining processing time exceeds the sum of the AFT and the current processing time. For each For unfinished insert workpieces, determine if T CP (current) > D Ti Then N ED =N ED +1, otherwise N CP =N CP +1.

[0105] in, For inserting single workpiece T i Arrival time, N ED To estimate the number of delayed workpieces, N CP This represents the number of undelayed workpieces.

[0106] The estimated delay rate for interstitial workpieces is equal to the estimated number of delayed operations divided by the total number of operations in all jobs. The formula is as follows:

[0107]

[0108] Where Estimated_delay(t) is the estimated delay rate of the inserted workpiece, N ED To estimate the number of delayed workpieces, N CP This represents the number of undelayed workpieces.

[0109] Then, the actual delay rate of the inserted workpieces is calculated. For all inserted workpieces that have been processed, a judgment is made: if... Then N AD =N AD +1, otherwise N CP =N CP +1.

[0110] in, Let N be the completion time of single workpiece i. AD N represents the actual number of delayed workpieces. CP This represents the number of undelayed workpieces.

[0111] Based on this, the formula for calculating the actual delay rate of the inserted workpiece is:

[0112]

[0113] Where Actual_delay(t) is the actual delay rate of the inserted workpiece, N AD N represents the actual number of delayed workpieces. CP This represents the number of undelayed workpieces.

[0114] Based on this, a motion space representation method for predicting delay workpieces is designed, mainly including:

[0115] Determine the estimated number of delayed workpieces N for the inserted workpiece. ED If the value is 0, then execute the static action and input the feature value into the static model for solution. The estimated number of delayed workpieces N for the single-workpiece insertion is... ED If the value is greater than 0, then two dynamic actions are executed, and the feature value is input into the dynamic DQN model for solution.

[0116] Action 1 selects workpieces based on their completion rate and delay level, and selects the machine tool with the shortest processing time for processing. Among the workpieces with estimated delays, the workpiece with the largest product of the completion rate of the inserted workpiece process multiplied by the sum of the current time and the estimated completion time and the difference between the sum and the delivery date is selected.

[0117] Then, the machine tool that can complete the processing of the current workpiece the fastest is identified. The processing time of the workpiece is divided into two segments: the first segment is the waiting time when the machine tool is still processing the previous workpiece, and the second segment is the processing time of this process on the machine tool (single workpieces are processed in batches by default). The machine tool with the smallest sum is selected for processing.

[0118] The formula for calculating the estimated completion time of a single workpiece is as follows:

[0119]

[0120] in, Here, AFT is the estimated completion time for the currently inserted workpiece, and AFT is the machine's average waiting time. This represents the total processing time for a single workpiece.

[0121]

[0122] Among them, J i The product of the completion rate of the insert workpiece process multiplied by the sum of the current time and the estimated completion time, and the difference between the sum and the delivery date, is the product of the workpieces with the largest product. i For the delivery time of the inserted workpiece, For the current moment, T ij For inserting single workpiece T i Total number of processes, T ij(t) Inserting a single workpiece T at time t ij Number of completed processes.

[0123]

[0124] Among them, PTS Tikjm For inserting single workpiece T i The processing time of the j-th process in the k-th batch on machine tool m. Indicates single workpiece T i The moment when the j-th process is completed on machine tool m.

[0125] Action 2 selects workpieces based on remaining processing time and delivery date, i.e., selecting the workpiece with the largest difference between estimated completion time and delivery date for processing. The calculation formula is as follows:

[0126]

[0127] The workpiece is then placed on the machine tool that can process it fastest. The calculation formula for selecting the machine tool with the shortest processing time is the same as for action 1.

[0128] Furthermore, the pseudocode for the reward function of the optimization method for the dynamic scheduling problem of shop floor scheduling based on improved DQN in step S2 is as follows: Figure 2 As shown, the specific implementation steps are as follows:

[0129] Step 1: After a single scheduling task is completed, compare the completion time of the current processing plan with the completion time of the optimal processing plan;

[0130] Step 2: Determine the reward value based on the difference;

[0131] Step 3: After each scheduling action, calculate the difference between the estimated delayed workpieces and the actual delayed workpieces;

[0132] Step 4: Assess machine tool utilization and provide corresponding rewards based on changes.

[0133] Furthermore, step S3 includes the following sub-steps:

[0134] To improve the training effect of the proposed optimization method for dynamic batch flow scheduling in flexible workshops that considers emergency order insertion and rework, an orthogonal experimental method was adopted to calibrate the effectiveness of the model's three main parameters: empirical buffer size, discount factor, and learning rate. Table 1 provides a set of three-factor, three-level L9(44) orthogonal matrices. To evaluate the effectiveness of the proposed method, 12 test cases of different problem sizes were randomly generated. A 10×15 scheduling case of medium size was selected from all the cases for experimentation and trained using the DQN model, with completion time as the performance index. The results of each experiment are recorded as shown in Table 2. The main effect diagrams of each parameter of the algorithm are shown in Table 2. Figure 3As shown in the figure. The neural network in the algorithm adopts a 10-layer structure, which includes a 7-node input layer, a 16-node output layer, and 8 hidden layers, each containing 256 nodes.

[0135] In the dynamic flexible job shop scheduling problem, rescheduling strategies are crucial for adjusting production plans in response to changes in the shop environment, significantly impacting the performance of the scheduling scheme. This invention validates the effectiveness of three rescheduling mechanisms—partial rescheduling, full rescheduling, and right-shift rescheduling—based on multiple test cases. The results are shown in Table 3. As can be seen from Table 3, when dealing with the large-scale LFJSP problem, the full rescheduling strategy significantly outperforms the partial and right-shift rescheduling strategies in terms of completion time.

[0136] To further verify the effectiveness of the proposed algorithm, a full rescheduling strategy was adopted, and the performance of the proposed DQN algorithm, heuristic rules, and genetic algorithm was compared under different problem sizes, as shown in Tables 4 and 5, respectively. The sum of completion time and single-job delay time was used as the performance index, with the first part of the completion time representing the overall completion time and the second part representing the single-job delay time. Table 4 shows that all rules are significantly inferior to the DQN algorithm. Table 5 shows that the improved DQN algorithm generally outperforms the GA algorithm in optimizing the objective value.

[0137] Table 1. Parameter Selection for the Three-Factor, Three-Level Orthogonal Experiment

[0138] Table 2 Comparison of orthogonal experiment results

[0139] Table 3 Performance Comparison of Three Rescheduling Mechanisms under Different Problem Sizes

[0140] Table 4 Comparison of Single Heuristic Action and Improved DQN Algorithm under Five Problem Sizes

[0141] Table 5 compares the performance of GA and the improved DQN algorithm in solving dynamic scheduling problems at different problem sizes.

[0142] Furthermore, the example verification in step S5 comes from a production workshop of an aerospace company. In engineering applications, thin-walled flat plate-type complex aerospace components are single-piece workpieces. Single-piece workpieces differ from ordinary workpieces only in size, and their process flow is the same as that of ordinary workpieces. Two processes in channel-type complex components are randomly selected for rework. Based on the workshop scheduling scheme under the optimal batching scheme, considering the interference of two dynamic events, namely workpiece rework and temporary single-piece insertion, a scheduling scheme for three rescheduling mechanisms under dynamic events is obtained. The arrival time of the single-piece workpiece is set to 20, the delivery period of the single-piece workpiece is 330, and the number of rework processes is 2. The workshop scheduling scheme before implementing the full rescheduling strategy is as follows: Figure 4 As shown. The proposed optimization method for dynamic batch flow scheduling of flexible workshops, based on improved DQN and considering emergency order insertion and rework, is used to solve the problem. The initial scheduling scheme and the optimized rescheduling scheme are shown below. Figure 5 As shown, the feasibility and applicability of the dynamic scheduling optimization method designed in this invention in a real workshop are demonstrated.

[0143] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it. They should not be used to limit the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the essence of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A flexible job shop dynamic batch flow scheduling optimization method considering emergency insertion and rework based on improved DQN, characterized in that, The method comprises the following steps: S1, considering the flexible job shop dynamic batch flow scheduling multi-objective optimization mathematical model of emergency order insertion and rework; S2, based on the knowledge-driven batch method and the model solving method of multi-agent DQN.

2. The improved DQN-based dynamic job shop scheduling optimization method considering urgent insertion and rework of flexible batch flow, according to claim 1, wherein, The flexible job shop dynamic batch flow scheduling multi-objective optimization mathematical model considering emergency order insertion and rework mainly comprises: The optimization objective of the model is to minimize the completion time and the delay time of the inserted workpiece, and the calculation formula is: Wherein, F is the objective function, C max , DE Ti are the workshop completion time and the delay time of inserted workpieces, respectively, X ikjm = 1, PTS ikjm = PTD ikjm + PTC ikjm + PTZ ikjm + PT ikjm , where i is the workpiece type serial number, k is the sub-lot batch number of workpiece i, j is the process serial number, M is the total number of machine tools, m is the machine tool type serial number, T i is the temporary insertion single workpiece, T ij is the temporary insertion single workpiece T i is the jth process; X ikjm is the decision variable, 1 if the jth process of workpiece i is processed on machine m, otherwise 0; PTS ikjm , are the total processing time of the jth process in the kth batch of workpiece i on machine m and the total processing time of the jth process in the temporary insertion single workpiece on machine m, respectively; PTD ikjm , PTC ikjm , PTZ ikjm and PT ikjm are the tool setting time, workpiece disassembly time, workpiece clamping time and processing time of the jth process in the kth batch of workpiece i on machine m, respectively; and are the tool setting time, workpiece disassembly time, workpiece clamping time and processing time of the jth process in the temporary insertion single workpiece T i on machine m, respectively, else wherein, is the insertion of the single piece T i is the start time of the jth operation on the machine m; D i is the delivery time of the piece i, is the delay time of the insertion of the single piece, In addition to meeting the basic constraints of flexible job shop scheduling, the workpiece batching constraints also need to be met, which are represented as: where P ik is the number of sub-lots in the kth batch of workpieces i, N i is the total number of workpieces i, Rework constraints, when an unqualified process is found, the rework operation needs to be performed on the unqualified process, which is represented as: where FT ij is the decision variable, taking value 1 if the jth operation of the workpiece i needs rework, otherwise taking value 0; J Ri,j and J ij is the jth operation of the workpiece i that needs rework and the jth operation of the workpiece i, respectively, When the machine tool processing batch is converted, the first workpiece in the batch needs to consider the machine tool tool changing time, which is represented as: PTS ikjm ' = PTS ikjm + PTP ikjm = PTD ikjm + PTC ikjm + PTZ ikjm + PT ikjm + PTP ikjm wherein PTS ikjm and PTP ikjm is the total machining time and tool change time on machine m when changing machining machine for the jth process in the kth batch of workpieces i; and is the total machining time and tool change time on machine m when changing machining machine for the jth process in the kth batch of workpieces i; i is the total machining time and tool change time on machine m when changing machining machine for the jth process in the kth batch of workpieces i; 3. The improved DQN-based dynamic job shop scheduling optimization method considering urgent insertion and rework of flexible batch flow, according to claim 1 or 2, characterized in that, The workshop state design based on the delay rate of inserted workpieces and the actual delay rate of inserted workpieces mainly comprises: The workshop state model is described by introducing two indexes of the estimated delay rate of inserted workpieces and the actual delay rate of inserted workpieces, and the specific operation is as follows: First, calculate the available time of the machine tool: where AFT is the average waiting time at all available machines at the current time t, denotes a single insertion workpiece T i the end time of the jth process on machine m, denotes the current time, and M denotes the number of machines, Calculate the average completion time of the remaining processes of the inserted workpiece and the estimated completion time of the workpiece: wherein, Ttotal is the total processing time of the single-insert workpiece, which is the sum of the average completion times of processing on different machine tools, T is the remaining process of the single-insert workpiece, CP AFT is the average waiting time of the machine, T is the average completion time of the remaining process, t is the current time, The estimated delay job can be considered as the job whose remaining processing time exceeds the sum of AFT and current processing time. For each job, the following steps are performed: If the unfinished insert job is judged, and if T CP (current) > D Ti , then N ED = N ED + 1, otherwise N CP = N CP + 1, wherein, for inserting single pieces T i time of arrival, N ED for estimated delayed pieces, N CP for non-delayed pieces, The estimated delay rate of the inserted workpiece is equal to the number of estimated delay operations divided by the number of processes of all jobs, which is represented as: Wherein, Estimated_delay(t) is the estimated delay rate of inserted single workpiece, N ED is the number of estimated delay workpieces, N CP is the number of non-delay workpieces, Then the actual delay rate of the inserted workpiece is calculated, and all the completed inserted workpieces are judged. If then N AD = N AD + 1, otherwise N CP = N CP + 1, wherein, is the completion time of the inserted workpiece i, N AD is the actual number of delayed workpieces, and N CP is the number of non-delayed workpieces, The calculation formula of the actual delay rate of the inserted workpiece is: Wherein, Actual_delay(t) is the actual delay rate of the inserted single workpiece, N AD is the actual delayed workpiece number, N CP is the non-delayed workpiece number.

4. The improved DQN-based dynamic job shop scheduling optimization method considering urgent insertion and rework of flexible batch flow, according to claim 3, wherein, Intelligent agent based on dynamic scheduling mainly comprises: Two DQN networks are introduced to correspond to two groups of actions in the dynamic scheduling problem. Before performing each action, it is first determined whether the estimated number of delayed jobs N ED of the inserted job is 0. If it is 0, the static action is performed and the characteristic value is input into the static model for solving. If the estimated number of delayed jobs N ED of the inserted job is greater than 0, the two dynamic actions are performed and the characteristic value is input into the dynamic DQN model for solving. The optimization objective function is to minimize the completion time and the delivery delay time of the inserted job. Action 1 is to select the workpiece based on the completion rate and delay degree, and select the machine tool with the shortest completion processing time for processing. In the estimated delay, select the workpiece with the maximum product of the completion rate of the inserted workpiece process, the current time, the sum of the estimated completion time and the difference between the delivery period, and then find the machine tool that can complete the processing of the workpiece the fastest. The processing time of the workpiece is divided into two parts, the first part is the waiting time of the machine tool processing the previous workpiece, and the second part is the processing time of the process on the machine tool (the inserted workpiece is defaulted to be processed in a batch), and the machine tool with the smallest sum is selected for processing. The specific formula is as follows: wherein, AFT is the average waiting time of the machine, is the total machining time of the single-insert workpiece, wherein J i is the product of the maximum product of the workpiece for which the product of the completion rate of the single insertion work process times the sum of the current time plus the estimated completion time minus the delivery period is the largest, D i is the delivery period of the single insertion workpiece, is the current time, T ij is the total number of processes of the single insertion workpiece T i , T ij(t) is the number of completed processes of the single insertion workpiece T ij at time t, wherein PTS Tikjm is the processing time of the jth operation on the kth batch of single-insert workpieces T i on machine m, denotes the time at which the jth operation on the single-insert workpiece T i is completed on machine m, Action 2 is to select the workpiece based on the remaining processing time and the delivery period, that is, to select the workpiece with the maximum difference between the estimated completion time and the delivery period for processing, and the calculation formula is as follows: Then put the workpiece on the machine tool that can complete the processing of the workpiece the fastest, select the machine tool with the shortest processing time, and the calculation formula is the same as action 1.

5. The improved DQN-based dynamic job shop scheduling optimization method considering urgent insertion and rework of flexible batch flow, according to claim 4, wherein, Two-stage reward function based on dynamic scheduling mainly comprises: Step 1: After completing a single scheduling task, compare the completion time of the current processing scheme with the completion time of the optimal processing scheme; Step 2: Give a reward value according to the difference; Step 3: Calculate the difference between the estimated delay workpiece and the actual delay workpiece after each scheduling action is executed; Step 4: Evaluate the machine tool utilization rate and give corresponding rewards according to the changes.

Images