A two-stage rolling production scheduling method for flow shop

Abstract

The present invention discloses a two-stage flow shop rolling production scheduling method, including: obtaining the processing information of the workshop; calculating the judgment conditions of the continuous production of the two-stage flow shop according to the processing information, establishing a linear programming model by means of 0-1 decision variables, the linear programming model including an objective function with the goal of minimizing the completion time and the constraint conditions of the objective function, the constraint conditions of the objective function are set according to the judgment conditions of the continuous production of the two-stage flow shop; solving the linear programming model, obtaining the initial scheduling scheme of the processing sequence of different batches of workpieces in the same machine to be determined, as the initial parameters of the Q learning algorithm; updating the reward and punishment table of the Q learning algorithm according to the semi-finished product inventory generated by different processing sequences of the workpiece, and using the Q learning algorithm to determine the scheduling scheme with the highest reward and punishment value, so as to determine the processing sequence of the workpiece in each machine. The present invention can ensure that the work can be uninterrupted when the debugging equipment is frequently switched due to a wide variety of products, thereby reducing the manufacturing cost.

Description

Technical Field

[0001] The invention relates to a workshop scheduling technology, and in particular to a two-stage flow workshop rolling production scheduling method. Background Art

[0002] With the vigorous promotion of the "dual carbon" goal, new energy vehicles are developing rapidly and are shaking and replacing the position of traditional fuel vehicles. From 2019 to 2023, global sales of new energy vehicles continued to rise. As the world's largest automobile market, China's demand for new energy vehicles continues to grow. This has brought new development opportunities to automobile supporting companies. As an important accessory, automotive glass faces greater challenges in production and scheduling. Glass manufacturing companies urgently need to shift from large batches and few varieties to small batches of multiple varieties. In addition, in order to meet customer requirements for high quality and short delivery time, a large amount of semi-finished product inventory needs to be set up in the production process. If there is no load between key processes, it will affect product quality and production continuity. How to improve the manufacturing efficiency and quality of automotive glass and realize intelligent production is an urgent problem for supporting companies.

[0003] Automotive glass products need to go through five steps in sequence for processing: pretreatment, bending, lamination, high pressure and final inspection. In the first step, the machine switches from producing one product to another and needs to be shut down for debugging. Therefore, this step requires debugging time. In the second step, bending, when the machine changes the mold to process another product, the machine can work continuously without interruption, and the debugging time can be ignored. Other processes do not involve debugging time, and steps 2 to 5 can be integrated as the second stage. The system is regarded as a two-stage assembly line. In addition, when different glass products are bent, the product should be heated to a given temperature. If the machine in step 2 cannot run continuously, the product will have quality problems.

[0004] Many scholars have made great efforts in scheduling two-stage flow shop operation systems. With the goal of minimizing the completion time of a two-stage flow shop, Reference 1 (Chen, Z., Zheng, X., Zhou, SC, Liu, C., & Chen, HP (2019). Quantum-inspired ant colony optimization algorithm for a two-stage permutation flow shop with batch processing machines. International Journal of Production Research, 58, 5945-5963.) discloses a quantum-inspired ant colony optimization algorithm. For a two-stage blocking arrangement production line, Reference 2 (Zheng, X., Zhou, SC, Xu, R., & Chen, HP (2019). Energy-efficient scheduling for multi-objective two-stage flow shop using a hybrid ant colony optimization algorithm. International Journal of Production Research, 58, 4103-4120.) discloses a hybrid ant colony algorithm with total energy cost and completion time as optimization objectives. Since there is no waiting requirement between the two stages, document 3 (Zhong, W., & Shi, Y. (2018). Two-stage no-wait hybrid flowshop scheduling with inter-stage flexibility. Journal of Combinatorial Optimization, 35, 108-125.) discloses an efficient scheduling algorithm to minimize the completion time of a two-stage hybrid production line, in which each stage is equipped with a parallel machine. However, the above document does not involve continuous production constraints in the scheduling process. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a two-stage flow shop rolling production scheduling method.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A two-stage flow shop rolling production scheduling method, comprising:

[0008] Obtaining processing information of the workshop; the processing information includes the number of workpieces to be processed, the processing time of each process, the number of parallel machines included in each stage of the machine, and the processing power of the machine;

[0009] Calculating the judgment conditions for continuous production of the two-stage flow shop according to the processing information, and establishing a linear programming model by means of 0-1 decision variables, wherein the linear programming model includes an objective function with the goal of minimizing the completion time and constraints of the objective function, and the constraints of the objective function are set according to the judgment conditions for continuous production of the two-stage flow shop;

[0010] Solve the linear programming model to obtain the initial scheduling scheme for the processing order of different batches of workpieces on the same machine, which is used as the initial parameters of the Q learning algorithm;

[0011] The reward and penalty table of the Q learning algorithm is updated according to the semi-finished product inventory generated by different processing sequences of the workpiece. The Q learning algorithm is used to determine the scheduling plan with the highest reward and penalty value as the final scheduling plan to determine the processing order of the workpiece on each machine.

[0012] To optimize the above technical solutions, the specific measures taken also include:

[0013] Furthermore, the determination conditions for continuous production in a two-stage flow shop are calculated based on the processing information as follows:

[0014] Calculate the completion time T of the jth batch of workpieces processed by the i-th machine in the first stage ij And the time H when the jth batch of workpieces processed by the i-th machine in the first stage can be rolled out after processing ij , the calculation formula is as follows:

[0015]

[0016]

[0017] In the formula, θ i Indicates the initial state of the system, the i-th machine M in the first stage 1i The number of workpieces processed on the machine, α represents the number of workpieces processed by the i-th machine M in the first stage when the system is in the initial state. 1i The time required to process a single workpiece, b ij Indicates the i-th machine M in the first stage 1i The number of the jth batch of workpieces processed on the machine, σ represents the preparation time for machine commissioning in the first stage, and m(i) represents the number of the i-th machine M in the first stage. 1i The processing batches allocated on represents a positive integer from 1 to m, where m represents the number of machines set up in the first stage. represents a positive integer from 1 to m(i);

[0018] Calculate the time ψ required to complete a single workpiece in the second stage, the formula is as follows:

[0019] ψ=μ / n

[0020] Where μ represents the jth machine M in the second stage 2j The time required to process a single workpiece in the first stage, n represents the number of machines used to process the workpiece in the second stage;

[0021] Traverse and assign to the i-th machine M in the first stage 1i The processing completion time of all batches of workpieces is obtained to obtain the minimum processing completion time T min , Where, T im(i) Indicates the allocation to the i-th machine M in the first stage 1i The processing completion time of the m(i)th batch of workpieces;

[0022] The conditions for setting up two-stage continuous production are as follows:

[0023] At time [0,T min ], the workpiece productivity in the first stage is greater than or equal to the workpiece productivity in the second stage, which can be expressed as The two-stage workshop maintains continuous processing, and at any time t∈(0,T min ];

[0024] v it Indicates that in the first stage, the i-th machine M 1i Average productivity of machined workpieces;

[0025] Calculate the minimum product batch Δp to ensure two-stage continuous processing, the formula is as follows:

[0026] Δp=σ / (m*ψ-α)

[0027] In the formula, σ represents the preparation time for machine debugging in the first stage, m represents the number of machines used to process the workpiece in the first stage, ψ represents the time required to complete the processing of a single workpiece in the second stage, and α represents the number of machines M of the i-th machine in the first stage when the system is in the initial state. 1i The time required to process a single workpiece;

[0028] According to the minimum product batch Δp to ensure continuous processing in two stages, the i-th machine in the first stage can produce rolling time H im(i) The conditions for two-stage continuous production are used to deduce the conditions for continuous production in a two-stage flow shop:

[0029] The first judgment condition: If any batch number b ij ≥Δp, but Established, any t∈(0,T min ];

[0030] The second judgment condition: If the batch number of any workpiece group but Established, any t∈(0,T min ].

[0031] Furthermore, the linear programming model is established by using 0-1 decision variables:

[0032] Establish an objective function with the goal of minimizing completion time;

[0033] Objective function: minC max

[0034] In the formula, C max It indicates the time required to complete all batches of workpieces in the first stage;

[0035] The constraints of the objective function are set according to the judgment conditions of continuous production in the two-stage flow shop, specifically:

[0036] The constraints of the objective function are as follows:

[0037] Constraint 1:

[0038] In the formula, x ij represents a 0-1 decision variable. If the j-th batch of workpieces is on the i-th machine M in the first stage 1i If processing, it is 1, otherwise it is 0; x iu is a 0-1 decision variable. If the quantity is b iu The uth batch of workpieces is on machine M 1i Processing, b iu <Δp,x iu If yes, it is 1, otherwise it is 0;

[0039] Constraint 1 means that each workpiece group can only be assigned to a certain machine in the first stage. Upper processing;

[0040] Constraint 2:

[0041] In the formula, δ i Indicates the i-th machine M in the first stage 1i The number of additional workpieces, r 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of workpieces in stock; b iu Indicates the i-th machine M in the first stage 1iThe number of the u-th batch of workpieces processed;

[0042] Constraint 2 means that the total processing time of all batches of workpieces is not less than C max ;

[0043] Constraint three:

[0044] Constraint three means that in the time period (0, T min ], meeting the two-stage continuous production conditions The two-stage workshop is capable of rolling production;

[0045] Constraint 4:

[0046] Constraint 4 indicates that the product processing batch does not meet the two-stage continuous production conditions. The system allocates some additional batches to the first stage of processing and merges them into the first batch of workpieces to ensure two-stage continuous processing;

[0047] Constraint 5:

[0048] In the formula, r 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces, r represents the total inventory in the initial state of the system;

[0049] Constraint five indicates that the total inventory in the initial state of the system is r;

[0050] Constraint 6: x ij =0 or

[0051] Constraint 7: x iu =0 or

[0052] Constraints six and seven represent the value range of the 0-1 decision variable.

[0053] Furthermore, the reward and penalty table of the Q learning algorithm is updated according to the semi-finished product inventory generated by different processing sequences of the workpieces. The Q learning algorithm is used to determine the scheduling plan with the highest reward and penalty value as the final scheduling plan to determine the processing sequence of the workpieces on each machine. The specific process is as follows:

[0054] Step 1: Initialize the parameters of the Q learning algorithm, including the maximum number of iterations ite max , learning rate φ, discount rate γ, greedy selection probability ε, initial scheduling plan B ite and Q table, iteration number ite=1; scheduling scheme B ite The row number i=1

[0055] Step 2: Update the greedy selection probability ε according to the greedy selection probability strategy. The formula is as follows:

[0056]

[0057] In the formula, ite represents the number of iterations, ite max represents the maximum number of iterations of the algorithm, Represents a random number;

[0058] Step 3: Select an activity in the current state s according to the Q value The calculation formula is as follows:

[0059]

[0060] In the formula, randomA means randomly selecting an action from set A, and rand means a random number;

[0061]

[0062] a * Represents the element q in the Q table ij The action corresponding to the maximum value, arg max is a function that describes the maximum value returned from the function;

[0063] Step 4. Take Action And observe the output next state s ite ′, used to update the Q table;

[0064] Step 5: Select scheduling plan B ite The i-th row in B ：，i ; Update the reward and penalty table R according to the minimum batch size Δp of two-stage continuous processing ite ;

[0065] Step 6, i=i+1, if i=m, the inner termination condition of the algorithm is met, and the process goes to step 7, otherwise it returns to step 5;

[0066] Step 7: Update the Q table according to the following calculation formula;

[0067]

[0068] Where: s ite Represents the state of the algorithm's iteth iteration, Indicates the action performed at the iteth iteration. Indicates state s ite Execute an action The updated Q value when s ite ′ represents the state of time step ite+1, Indicates the action performed at the ite+1th iteration; R ite represents the reward value of the ite-th iteration; φ∈[0,1] represents the learning rate; γ represents the discount rate;

[0069] Step 8: Get the new scheduling plan B based on the updated Q table ite ′, check the scheduling plan B according to the judgment conditions of continuous production in the two-stage flow shop ite 'Whether the two-stage continuous production is met, if the continuous production conditions are met, update the scheduling plan B ite =B ite ′, calculate the scheduling plan B ite The corresponding semi-finished product inventory is adjusted according to the semi-finished product inventory. ite The reward and punishment value in R ite =d kj (m(i)×m(i)), d kj =d kj +|d kj |;d kj Indicates machine M 1i The jth batch of workpieces is processed in the kth place; if the continuous production conditions are not met, the scheduling plan is not updated, and the reward and punishment table R is corrected. ite , d kj =min{d kj ,0};

[0070] Step 9, ite=ite+1, if the number of iterations ite reaches the maximum number of iterations ite of the algorithm max ,, output the scheduling plan B with the highest reward and penalty value ite , otherwise, return to step 2.

[0071] Furthermore, step 5 specifically includes:

[0072] Step 5.1, b ij =b i(j+1) =Δp,d kj = + M;

[0073] Step 5.2, b ij ≥Δp,b ij +b i(j+1) <2*Δp,d kj =-M;

[0074] Step 5.3, b ij ≥Δp,b ij +b i(j+1) >2*Δp,d kj =min{2*Δp / (b ij +b i(j+1) ), Δp};

[0075] Where: B ite =b ij (m×J), b ij Indicates machine M 1i The number of the jth batch of workpieces, b i(j+1) Indicates machine M 1i The number of the j+1th batch of workpieces, m represents the number of machines in the first stage, represents the number of batches of workpieces assigned to the first stage, R ite =d kj (m(i)×m(i)), d kj Indicates machine M 1i The jth batch of workpieces is processed at the kth position, and +M and -M represent the set positive and negative numbers respectively.

[0076] Furthermore, in step 8, the calculation scheduling scheme B ite The corresponding semi-finished product inventory is as follows:

[0077] Use the following piecewise function to calculate the semi-finished product inventory:

[0078] WIP(i, t) = t / α - t / ψ + r 1i +max{(δ i -r 1i ),0},0 <t≤θ i α

[0079] WIP(i, t) = θ i -t / ψ+r 1i +max{(δ i -r 1i ), 0}, θ i α <t≤θ i α+σ

[0080] WIP(i, t) = t / α - t / ψ + θ i +r 1i +max{(δ i -r 1i ), 0}, θ i α+σ <t≤T i1

[0081] WIP(i, t) = b i1 -t / ψ+θ i +r 1i +max{(δ i -r 1i ), 0}, T i1 <t≤H i1

[0082]

[0083]

[0084] Where, WIP(i, t) represents the first stage of the system at time t, the i-th machine M 1i The semi-finished product inventory at 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces; T i1 Indicates machine M 1i The time when the first batch of workpieces is processed; H i1 Indicates machine M 1i The first batch of workpieces is put into production, and the second batch of workpieces is also completed and debugged. i(j+1) Indicates machine M 1i The rolling production moment of the j+1th batch of workpieces.

[0085] The beneficial effects of the present invention are:

[0086] The problem of ensuring uninterrupted production in the two-stage assembly line when switching between multiple varieties of automotive glass manufacturing is solved, and the problem of product quality problems caused by the inability to produce continuously when changing product processing batches is effectively reduced. In addition, with the goal of minimizing semi-finished product inventory, the Q-learning algorithm designed further optimizes the scheduling plan, thereby reducing manufacturing costs. Since the manufacturing of automotive glass workshops has changed from traditional small-batch production of a few varieties to small-batch production of many varieties, the system needs to switch products frequently. The automotive glass manufacturing process needs to ensure continuous production in the two-stage workshop. According to the two-stage rolling production scheduling method of the present invention, it can ensure uninterrupted work when frequent switching and debugging equipment is caused by a wide variety of varieties, while reducing manufacturing costs. Moreover, with the semi-finished product inventory as the goal, the proposed Q-learning algorithm can further optimize the scheduling plan. The proposed scheduling method has been verified to be feasible and effective through experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is a flow chart of the rolling production scheduling method for a two-stage flow shop;

[0088] Figure 2 Flowchart of the Q-learning algorithm. DETAILED DESCRIPTION

[0089] The present invention will now be described in further detail with reference to the accompanying drawings.

[0090] Classic flow shop scheduling problems generally use multiple workstations to process multiple products. Many of the production processes can be viewed as two-stage flow shop scheduling problems (TFSP), such as the automotive glass manufacturing process. Automotive glass manufacturers produce various types of glass, such as car front windshields, car rear windshields, and car side window glass. Glass for different purposes has different production processes, and glass for the same purpose has different processes due to different products, which brings difficulties and challenges to the production process. Usually, automotive glass manufacturing mainly goes through five processes, such as pretreatment, bending, polyvinyl butyral (PVB) stretching and lamination, high pressure, and final inspection and packaging. Among them, the first process is pretreatment. When transferring from processing one batch to processing another batch, the machine needs to be stopped for debugging, that is, the preparation time needs to be set. For the second process, bending, different types of automotive glass have different bending angles. By reasonably arranging the production plan, the debugging time of the bending process of different products can be ignored to ensure continuous processing of products, that is, there is no preparation time between the 2nd to 5th process. Therefore, the first process is usually regarded as the first stage, and the 2nd to 5th processes are completed in sequence in the second stage. In addition, it should be noted that the bending of the second process must be completed at a certain temperature, and the machine must ensure uninterrupted production in the second stage while meeting the needs of rolling production.

[0091] The present invention studies the scheduling problem of a two-stage flow shop with multiple products and small batches. When the system switches from processing one product type to another, setup is required during this transition. Based on actual industrial needs, this work assumes that setup time is required in the first stage, while setup time can be ignored in the second stage. In addition, it also requires that the two-stage flow shop runs uninterruptedly. In order to solve the scheduling problem of this system, the present invention derives the conditions for checking whether the two-stage flow shop can continue to produce products without interruption at any time.

[0092] (2) On this basis, a linear programming model was established to find the optimal solution with the goal of minimizing the completion time.

[0093] (3) Based on (1) and (2), a Q-learning algorithm is designed to further optimize the scheduling scheme with the goal of minimizing the inventory of semi-finished products.

[0094] In one embodiment, the present invention proposes a two-stage flow shop rolling production scheduling method, the flow chart of which is as follows: Figure 1 As shown, including:

[0095] Obtaining processing information of the workshop; the processing information includes the number of workpieces to be processed, the processing time of each process, the number of parallel machines included in each stage of the machine, and the processing power of the machine;

[0096] Calculating the judgment conditions for continuous production of the two-stage flow shop according to the processing information, and establishing a linear programming model by means of 0-1 decision variables, wherein the linear programming model includes an objective function with the goal of minimizing the completion time and constraints of the objective function, and the constraints of the objective function are set according to the judgment conditions for continuous production of the two-stage flow shop;

[0097] The determination conditions for calculating the continuous production of the two-stage flow shop according to the processing information are specifically:

[0098] Calculate the completion time T of the jth batch of workpieces processed by the i-th machine in the first stage ij And the time H when the jth batch of workpieces processed by the i-th machine in the first stage can be rolled out after processing ij , the calculation formula is as follows:

[0099]

[0100]

[0101] In the formula, θ i Indicates the initial state of the system, the i-th machine M in the first stage 1i The number of workpieces processed on the machine, α represents the number of workpieces processed by the i-th machine M in the first stage when the system is in the initial state. 1i The time required to process a single workpiece, b ij Indicates the i-th machine M in the first stage 1i The number of the jth batch of workpieces processed on the machine, σ represents the preparation time for machine commissioning in the first stage, and m(i) represents the number of the i-th machine M in the first stage. 1i The processing batches allocated on represents a positive integer from 1 to m, where m represents the number of machines set up in the first stage. represents a positive integer from 1 to m(i);

[0102] Calculate the time ψ required to complete a single workpiece in the second stage, the formula is as follows:

[0103] ψ=μ / n

[0104] Where μ represents the jth machine M in the second stage 2j The time required to process a single workpiece in the first stage, n represents the number of machines used to process the workpiece in the second stage;

[0105] Traverse and assign to the i-th machine M in the first stage 1i The processing completion time of all batches of workpieces is obtained to obtain the minimum processing completion time T min , Where, T im(i)Indicates the allocation to the i-th machine M in the first stage 1i The processing completion time of the m(i)th batch of workpieces;

[0106] The conditions for setting up two-stage continuous production are as follows:

[0107] At time [0, T min ], the workpiece productivity in the first stage is greater than or equal to the workpiece productivity in the second stage, which can be expressed as The two-stage workshop maintains continuous processing, and at any time t∈(v, T min ];

[0108] v it Indicates that in the first stage, the i-th machine M 1i Average productivity of machined workpieces;

[0109] Calculate the minimum product batch Δp to ensure two-stage continuous processing, the formula is as follows:

[0110] Δp=σ / (m*ψ-α)

[0111] In the formula, σ represents the preparation time for machine debugging in the first stage, m represents the number of machines used to process the workpiece in the first stage, ψ represents the time required to complete the processing of a single workpiece in the second stage, and α represents the number of machines M of the i-th machine in the first stage when the system is in the initial state. 1i The time required to process a single workpiece;

[0112] According to the minimum product batch Δp to ensure continuous processing in two stages, the i-th machine in the first stage can produce rolling time H im(i) The conditions for two-stage continuous production are used to deduce the conditions for continuous production in a two-stage flow shop:

[0113] The first judgment condition: If any batch number of workpieces but Established, any t∈(0, T min ];

[0114] The second judgment condition: If the batch number of any workpiece group but Established, any t∈(0, T min ].

[0115] Meeting one of the above conditions means that the two-stage flow shop meets the rolling production requirements.

[0116] The linear programming model established by using 0-1 decision variables is specifically as follows:

[0117] Establish an objective function with the goal of minimizing completion time;

[0118] Objective function: min C max

[0119] In the formula, C max It indicates the time required to complete all batches of workpieces in the first stage;

[0120] The constraints of the objective function are set according to the judgment conditions of continuous production in the two-stage flow shop, specifically:

[0121] The constraints of the objective function are as follows:

[0122] Constraint 1:

[0123] In the formula, x ij represents a 0-1 decision variable. If the j-th batch of workpieces is on the i-th machine M in the first stage 1i If processing, it is 1, otherwise it is 0; x iu is a 0-1 decision variable. If the quantity is b iu The uth batch of workpieces is on machine M 1i Processing, b iu <Δp,x iu If yes, it is 1, otherwise it is 0;

[0124] Constraint 1 means that each workpiece group can only be assigned to a certain machine in the first stage. Upper processing;

[0125] Constraint 2:

[0126] In the formula, δ i Indicates the i-th machine M in the first stage 1i The number of additional workpieces, r 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of workpieces in stock; b iu Indicates the i-th machine M in the first stage 1i The number of the u-th batch of workpieces processed;

[0127] Constraint 2 means that the total processing time of all batches of workpieces is not less than C max ;

[0128] Constraint three:

[0129] Constraint three means that in the time period (0, T min ], meeting the two-stage continuous production conditions The two-stage workshop is capable of rolling production;

[0130] Constraint 4:

[0131] Constraint 4 indicates that the product processing batch does not meet the two-stage continuous production conditions. The system allocates some additional batches to the first stage of processing and merges them into the first batch of workpieces to ensure two-stage continuous processing;

[0132] Constraint 5:

[0133] In the formula, r 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces, r represents the total inventory in the initial state of the system;

[0134] Constraint five indicates that the total inventory in the initial state of the system is r;

[0135] Constraint 6: x ij =0 or

[0136] Constraint 7: x iu =0 or

[0137] Constraints six and seven represent the value range of the 0-1 decision variable.

[0138] Solve the linear programming model and obtain the initial scheduling scheme for different batches of workpieces on the same machine, which is used as the initial parameter of the Q learning algorithm. The workpieces assigned to each machine in the initial scheduling scheme are not sequential. The difference in processing sequence will directly affect the semi-finished product inventory in the production process.

[0139] The reward and penalty table of the Q learning algorithm is updated according to the semi-finished product inventory generated by different processing sequences of the workpieces, and the scheduling plan with the highest reward and penalty value is determined by the Q learning algorithm as the final scheduling plan to determine the processing order of the workpieces on each machine. The specific process is:

[0140] Step 1: Initialize the parameters of the Q learning algorithm, including the maximum running time t max , learning rate φ, discount rate γ, greedy selection probability ε, initial scheduling plan B ite and Q table, iteration number ite=1; scheduling scheme B ite The row number i=1

[0141] Step 2: Update the greedy selection probability ε according to the greedy selection probability strategy. The formula is as follows:

[0142]

[0143] In the formula, ite represents the number of iterations, ite maxrepresents the maximum running time of the algorithm, Represents a random number, the value range is The design of the calculation formula for the greedy selection probability ε is derived from the ε-greedy algorithm with inverse proportional decay, which has better learning effect than the ε-greedy algorithm with a fixed ε value.

[0144] Step 3: Select an activity in the current state s with a certain probability according to the Q value The calculation formula is as follows:

[0145]

[0146] In the formula, randomA means randomly selecting an action from set A, rand means a random number, and the value range is rand∈[0,1]; when rand>1-ε, randomA means randomly selecting an action from set A Otherwise select a * ,

[0147]

[0148] a * Represents the element q in the Q table ij The action corresponding to the maximum value, arg max is a function that describes the maximum value returned from the function;

[0149] Step 4. Take Action And observe the output next state s ite ′, used to update the Q table;

[0150] Step 5: Select scheduling plan B ite The i-th row in B ：，i ; Update the reward and penalty table R according to the minimum batch size Δp of two-stage continuous processing ite ; Specifically include:

[0151] Step 5.1, b ij =b i(j+1) =Δp,d kj = + M;

[0152] Step 5.2, b ij ≥Δp,b ij +b i(j+1) <2*Δp,d kj =-M;

[0153] Step 5.3, b ij ≥Δp,b ij +b i(j+1) >2*Δp,d kj =min{2*Δp / (b ij +bi(j+1) ), Δp};

[0154] Where: B ite =b ij (m×J), b ij Indicates machine M 1i The number of the jth batch of workpieces, b i(j+1) Indicates machine M 1i The number of the j+1th batch of workpieces, m represents the number of machines in the first stage, represents the number of batches of workpieces assigned to the first stage, R ite =d kj (m(i)×m(i)), d kj Indicates machine M 1i The jth batch of workpieces is processed at the kth position, and +M and -M represent the set positive and negative numbers respectively.

[0155] Step 6, i=i+1, if i=m, the inner termination condition of the algorithm is met, and the process goes to step 7, otherwise it returns to step 5;

[0156] Step 7: Update the Q table according to the following calculation formula;

[0157]

[0158] Where: s ite Represents the state of the algorithm's iteth iteration, Indicates the action performed at the iteth iteration. Indicates state s ite Execute an action The updated Q value when s ite ′ represents the state of time step ite+1, Indicates the action performed at the ite+1th iteration; R ite represents the reward value of the ite-th iteration; φ∈[0,1] represents the learning rate; γ represents the discount rate; if γ approaches 0, it means that γ considers short-term rewards more, and if γ approaches 1, it means that γ considers long-term cumulative rewards more. It should be emphasized that the update of Q-learning does not necessarily use The sampled data.

[0159] Step 8: Get the new scheduling plan B based on the updated Q table ite ′, check the scheduling plan B according to the judgment conditions of continuous production in the two-stage flow shop ite 'Whether the two-stage continuous production is met, if the continuous production conditions are met, update the scheduling plan B ite =B ite ′;

[0160] Computational Scheduling Scheme B iteThe corresponding semi-finished product inventory is as follows:

[0161] Use the following piecewise function to calculate the semi-finished product inventory:

[0162] WIP(i, t) = t / α - t / ψ + r 1i +max{(δ i -r 1i ),0},0 <t≤θ i α (2.11)

[0163] WIP(i, t) = θ i -t / ψ+r 1i +max{(δ i -r 1i ), 0}, θ i α <t≤θ i α+σ (2.12)

[0164] WIP(i, t) = θ i +t / α-t / ψ+r 1i +max{(δ i -r 1i ), 0}, θ i α+σ <t≤T i1 (2.13)

[0165] WIP(i, t) = b i1 -t / ψ+θ i +r 1i +max{(δ i -r 1i ), 0}, T i1 <t≤H i1 (2.14)

[0166]

[0167]

[0168] Where, WIP(i, t) represents the first stage of the system at time t, the i-th machine M 1i The semi-finished product inventory at 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces; T i1 Indicates machine M 1i The time when the first batch of workpieces is processed; H i1 Indicates machine M 1i The first batch of workpieces is put into production, and the second batch of workpieces is also completed and debugged. i(j+1) Indicates machine M 1iThe rolling production moment of the j+1th batch of workpieces.

[0169] Formula (2.11) represents time 0 <t≤θ i Within the range of α, the calculation formula for the inventory of semi-finished products of each machine in the initial state of the system; Formula (2.12) represents the time θ i α <t≤θ i Within the range of α+σ, the formula for calculating the inventory of semi-finished products of each machine before the first batch of workpieces are processed is: Formula (2.13) Time θ i α+σ <t≤T i1 Within the range, the calculation formula for the inventory of semi-finished products of each machine before the first batch of workpieces is completed; Formula (2.14) Time T i1 <t≤H i1 The formula for calculating the inventory of semi-finished products of each machine from the completion of the first batch of workpieces to the start of processing of the second batch of workpieces is as follows: Formula (2.15) represents the time H ij <t≤H i(j+1) -σ, the calculation formula for the inventory of semi-finished products of each machine before the jth batch of workpieces is completed; and formula (2.16) represents the time H i(j+1) -σ <t≤H i(j+1) The formula for calculating the inventory of semi-finished products of each machine from the completion of the jth batch of workpieces to the start of processing of the j+1th batch of workpieces is within the range.

[0170] Modify the reward and punishment table R according to the inventory of semi-finished products ite The reward and punishment value in R ite =d kj (m(i)×m(i)), d kj =d kj +|d kj |;d kj Indicates machine M 1i The jth batch of workpieces is processed in the kth place; if the continuous production conditions are not met, the scheduling plan is not updated, and the reward and punishment table R is corrected. ite ,d kj =min{d kj ,0};

[0171] Step 9, ite=ite+1, if the number of iterations ite reaches the maximum number of iterations ite of the algorithm max , output the scheduling plan B with the highest reward and penalty value ite , otherwise, return to step 2.

[0172] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A two-stage flow shop rolling production scheduling method, characterized in that: include: Obtaining processing information of the workshop; the processing information includes the number of workpieces to be processed, the processing time of each process, the number of parallel machines included in each stage of the machine, and the processing power of the machine; The determination conditions for continuous production of a two-stage flow shop are calculated according to the processing information, and a linear programming model is established by adopting a 0-1 decision variable method. The linear programming model includes an objective function with the goal of minimizing the completion time and constraints of the objective function. The constraints of the objective function are set according to the determination conditions for continuous production of a two-stage flow shop. The determination conditions for continuous production of a two-stage flow shop calculated according to the processing information are specifically: Calculate the completion time T of the jth batch of workpieces processed by the i-th machine in the first stage ij And the time H when the jth batch of workpieces processed by the i-th machine in the first stage can be rolled out after processing ij , the calculation formula is as follows: ； In the formula, Indicates the initial state of the system, the i-th machine M in the first stage 1i The number of workpieces processed on the machine, α represents the number of workpieces processed by the i-th machine M in the first stage when the system is in the initial state. 1i The time required to process a single workpiece, b ij Indicates the i-th machine M in the first stage 1i The number of workpieces in the jth batch processed on represents the preparation time for machine debugging in the first stage, and m(i) represents the i-th machine M in the first stage 1i The processing batches allocated on represents a positive integer from 1 to m, where m represents the number of machines set up in the first stage. = {1,…, m(i)}, represents a positive integer from 1 to m(i); Calculate the time required to complete a single workpiece in the second stage of processing , the formula is as follows: ； Where μ represents the jth machine M in the second stage 2j The time required to process a single workpiece in the first stage, n represents the number of machines used to process the workpiece in the second stage; Traverse and assign to the i-th machine M in the first stage 1i The processing completion time of all batches of workpieces is obtained to obtain the minimum processing completion time T min , T min = min(T im(i) , i ∈ );where T im(i) Indicates the allocation to the i-th machine M in the first stage 1i The processing completion time of the m(i)th batch of workpieces; The conditions for setting up two-stage continuous production are as follows: At time [0, T min ], the workpiece productivity in the first stage is greater than or equal to the workpiece productivity in the second stage, which can be expressed as , then the two-stage workshop maintains continuous processing, and at any time t ∈ (0, T min ]; Indicates that in the first stage, the i-th machine M 1i Average productivity of machined workpieces; Calculate the minimum processing batch Δp to ensure two-stage continuous processing, the formula is as follows: ； In the formula, represents the preparation time for machine debugging in the first stage, m represents the number of machines used to process the workpiece in the first stage, Indicates the time required to complete a single workpiece in the second stage of processing. Indicates the initial state of the system, the i-th machine M in the first stage 1i The time required to process a single workpiece; According to the minimum processing batch Δp to ensure continuous processing in two stages, the i-th machine in the first stage can produce rolling production time H im(i) The conditions for continuous production in two stages are used to deduce the conditions for continuous production in two stages of flow workshops: The first judgment condition: If any batch number of workpieces is b ij ≥Δp,i∈ ,j∈ ,but holds, any t ∈ (0, T min ]; The second judgment condition: If the batch number of any workpiece group ≥m(i)Δp, i∈ , j∈ ,but Established, any t∈(0,T min ]; Solve the linear programming model to obtain the initial scheduling scheme for the processing order of different batches of workpieces on the same machine, which is used as the initial parameters of the Q learning algorithm; The reward and penalty table of the Q learning algorithm is updated according to the semi-finished product inventory generated by different processing sequences of the workpiece. The Q learning algorithm is used to determine the scheduling plan with the highest reward and penalty value as the final scheduling plan to determine the processing order of the workpiece on each machine.

2. The two-stage flow shop rolling production scheduling method according to claim 1 is characterized in that: The linear programming model established by using 0-1 decision variables is specifically as follows: Establish an objective function with the goal of minimizing completion time; Objective function: min C max In the formula, C max It indicates the time required to complete all batches of workpieces in the first stage; The constraints of the objective function are set according to the judgment conditions of continuous production in the two-stage flow shop, specifically: The constraints of the objective function are as follows: Constraint 1: ; In the formula, represents a 0-1 decision variable. If the j-th batch of workpieces is on the i-th machine M in the first stage 1i If processing, it is 1, otherwise it is 0; x iu is a 0-1 decision variable. If the quantity is b iu The uth batch of workpieces is on machine M 1i Processing, b iu < Δp,x iu If yes, it is 1, otherwise it is 0; Constraint 1 means that each workpiece group can only be assigned to a certain machine M in the first stage. 1i , i ∈ Upper processing; Constraint 2: + + α×max{(δ i –r 1i ),0}≤ C max ,i∈ ,j ≠ u In the formula, δ i Indicates the i-th machine M in the first stage 1i The number of additional workpieces, r 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces; b iu Indicates the i-th machine M in the first stage 1i The number of the u-th batch of workpieces processed; Constraint 2 means that the total processing time of all batches of workpieces is not less than C max ; Constraint three: ; Constraint three means that in the time period (0, T min ], meeting the two-stage continuous production conditions ≥ m(i)*Δp, i ∈ , j∈ , the two-stage workshop is capable of rolling production; Constraint 4: ; Constraint 4 indicates that the product processing batch does not meet the two-stage continuous production conditions. The system allocates some additional batches to the first stage of processing and merges them into the first batch of workpieces to ensure two-stage continuous processing; Constraint 5: ; In the formula, r 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces, r represents the total inventory in the initial state of the system; Constraint five indicates that the total inventory in the initial state of the system is r; Constraint 6: x ij = 0 or 1, i ∈ , j ∈ ; Constraint 7: x iu = 0 or 1, i ∈ , u∈ ; Constraints six and seven represent the value range of the 0-1 decision variable.

3. The two-stage flow shop rolling production scheduling method according to claim 1 is characterized in that: The process of updating the reward and penalty table of the Q learning algorithm according to the semi-finished product inventory generated by different processing sequences of the workpieces, and using the Q learning algorithm to determine the scheduling plan with the highest reward and penalty value as the final scheduling plan to determine the processing sequence of the workpieces in each machine is as follows: Step 1: Initialize the parameters of the Q learning algorithm, including the maximum number of iterations ite max , learning rate φ, discount rate , Greedy selection probability ε, initial scheduling plan B ite and Q table, iteration number ite = 1; scheduling scheme B ite The row number i = 1 Step 2: Update the greedy selection probability ε according to the greedy selection probability strategy. The formula is as follows: ； In the formula, ite represents the number of iterations, ite max represents the maximum number of iterations of the algorithm, Represents a random number; Step 3: Select an activity ȃ in the current state s according to the Q value ite , the calculation formula is as follows: ； In the formula, randomA means randomly selecting an action from set A, and rand means a random number; ； Represents the element q in the Q table ij The action corresponding to the maximum value. arg max is a function that describes the maximum value returned from the function. Step 4. Take action. ite And observe the output next state , used to update the Q table; Step 5: Select scheduling plan B ite The i-th row in B :,i ; Update the reward and penalty table R according to the minimum batch size Δp of two-stage continuous processing ite ; Step 6, i = i + 1, if i = m, the inner termination condition of the algorithm is met, and go to step 7, otherwise return to step 5; Step 7: Update the Q table according to the following calculation formula; ； Where: s ite represents the state of the algorithm at the iteth iteration, ȃ ite represents the action performed at the iteth iteration, Q(s ite ,ȃ ite ) indicates state s ite Perform an action ite The updated Q value when represents the state of time step ite+1, Indicates the action performed at the ite+1th iteration; R ite represents the reward value of the iteth iteration; φ ∈ [0, 1] represents the learning rate; represents the discount rate; Step 8: Get a new scheduling plan based on the updated Q table , check the scheduling plan according to the judgment conditions of continuous production in the two-stage flow shop Whether the two-stage continuous production is met, if the continuous production conditions are met, update the scheduling plan B ite = , calculate the scheduling plan B ite The corresponding semi-finished product inventory is adjusted according to the semi-finished product inventory. ite The reward and punishment value in R ite = d kj (m(i)×m(i)), d kj = d kj + |d kj |;d kj Indicates machine M 1i The jth batch of workpieces is processed in the kth place; if the continuous production conditions are not met, the scheduling plan is not updated, and the reward and punishment table R is corrected. ite , d kj = min{d kj , 0}; Step 9, ite = ite + 1, if the number of iterations ite reaches the maximum number of iterations ite of the algorithm max , output the scheduling plan B with the highest reward and penalty value ite , otherwise, return to step 2.

4. The two-stage flow shop rolling production scheduling method according to claim 3 is characterized in that: Step 5 specifically includes: Step 5.1, b ij = b i(j+1) = Δp, d kj = +M; Step 5.2, b ij ≥ Δp, b ij + b i(j+1) < 2*Δp, d kj = -M; Step 5.3, b ij ≥ Δp, b ij + b i(j+1) > 2*Δp, d kj = min{2*Δp / (b ij + b i(j+1) ), Δp}; Where: B ite = b ij (m×J), b ij Indicates machine M 1i The number of the jth batch of workpieces, b i(j+1) Indicates machine M 1i The number of workpieces in the j+1th batch, m represents the number of machines in the first stage, J = max{m(i)| i ∈ } represents the number of batches of workpieces assigned to the first stage, R ite = d kj (m(i)×m(i)), d kj Indicates machine M 1i The jth batch of workpieces is processed at the kth position, and +M and -M represent the set positive and negative numbers respectively.

5. The two-stage flow shop rolling production scheduling method according to claim 3 is characterized in that: In step 8, the calculation scheduling scheme B ite The corresponding semi-finished product inventory is as follows: Use the following piecewise function to calculate the semi-finished product inventory: ； ； , ； Where, WIP(i, t) represents the i-th machine M in the first stage of the system at time t. 1i The semi-finished product inventory at 1i Indicates the allocation to the i-th machine M in the first stage 1i The number of inventory workpieces; T i1 Indicates machine M 1i The time when the first batch of workpieces is processed; H i1 Indicates machine M 1i The first batch of workpieces is put into production, and the second batch of workpieces is also completed and debugged. i(j+1) Indicates machine M 1i The rolling production moment of the j+1th batch of workpieces.

Images