Multi-target energy-saving distributed assembly zero-waiting flow shop scheduling optimization system
By combining differential evolution algorithm and reinforcement learning, a multi-objective energy-saving distributed assembly zero-wait flow workshop scheduling optimization system is designed, which solves the high-dimensional and nonlinear feature problems of traditional methods in multi-objective distributed assembly zero-wait flow workshop scheduling, and realizes the high-dimensional and nonlinear feature problems of traditional methods in multi-objective distributed assembly zero-wait flow workshop scheduling. Effective optimization of energy consumption and time, and improve the flexibility and adaptability of the system.
Patent Information
- Application Number
- CN202410876245.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional optimization methods are difficult to effectively deal with the high-dimensional, nonlinear and multimodal features in the scheduling problem of multi-objective distributed assembly zero-wait flow workshops, especially lacking flexibility and adaptability under dynamic changes and uncertainty conditions, making it difficult to minimize at the same time. Total energy consumption and maximum assembly completion time.
Using a multi-objective energy-saving distributed assembly zero-wait flow workshop scheduling optimization system, combined with differential evolution algorithm and reinforcement learning, design workpiece sequence initialization, accelerated operation and energy-saving operation modules, and reinforcement learning-driven decision-making modules, and guide differentials through Q-learning training results. Evolutionary algorithm optimization, select the appropriate operation operator and damage size to achieve adaptive optimization of the system.
The assembly scheduling optimization under energy consumption constraints is realized, which effectively reduces the total energy consumption and assembly completion time, improves the flexibility and adaptability of the system, and can find high-quality feasible solutions in complex optimization problems.
Smart Images

Figure CN119065325A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of distributed green production scheduling, and specifically is a multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system. Background Art
[0002] In the context of economic globalization, enterprises are no longer limited to production and supply in a single country. Instead, they disperse production links around the world by establishing a transnational supply chain network to obtain lower costs and higher efficiency. Distributed production scheduling is an important scheduling mode in the field of intelligent manufacturing to connect different production units and optimize the entire production process. It is a development trend of the manufacturing industry in the context of globalization. Distributed production scheduling aims to efficiently allocate and utilize multiple resources, such as factories, machines, raw materials and manpower, and how to coordinate these resources to meet production needs, reduce production cycles and reduce production costs. Through distributed production scheduling, enterprises can achieve cross-regional production activities, optimize supply chain management, and enhance production flexibility and market responsiveness. Multi-objective distributed production scheduling problems involve optimizing multiple objectives simultaneously in a distributed manufacturing system, such as minimizing production costs, shortening delivery time, improving resource utilization and reducing environmental impact. This type of scheduling problem not only requires coordinating production units in different geographical locations, but also finding the best balance between various competing objectives, such as how to achieve energy savings and emission reduction while maintaining high production efficiency. Green scheduling focuses on how to achieve a balance between environmental and economic goals in the production process. The application of multi-objective distributed production in green scheduling can not only help enterprises reduce production costs and improve market competitiveness, but also promote the green image and social responsibility of enterprises, which is of great significance for promoting sustainable development. Therefore, the study of distributed green production scheduling has important practical significance and academic value.
[0003] In the field of intelligent manufacturing, solving complex optimization problems is the core of achieving efficient production processes, optimal resource allocation, and intelligent production environments. Complex optimization problems, such as production scheduling, resource allocation, and logistics management, usually have characteristics such as high dimensionality, nonlinearity, multimodality, uncertainty, and multiple constraints. The energy-saving distributed assembly zero-wait flow shop scheduling problem is an important problem in distributed manufacturing and green manufacturing. The core lies in taking appropriate optimization methods to obtain effective solutions. Traditional optimization methods, including linear programming, nonlinear programming, integer programming, and dynamic programming, are often difficult to effectively deal with the characteristics of high dimensionality, nonlinearity, and multimodality when facing such problems, and are prone to fall into local optimal solutions. In addition, the adaptability to problems under dynamic changes and uncertainty is poor, which is not conducive to the optimization requirements under multiple constraints. This leads to the fact that traditional optimization methods are not flexible and adaptable enough to meet the needs of solving complex optimization problems in intelligent manufacturing. In contrast, metaheuristic algorithms have attracted widespread attention from researchers in recent years due to their excellent global search capabilities, high adaptability, and good scalability. Differential Evolution (DE) algorithm, as a kind of meta-heuristic algorithm, is widely used to solve various optimization problems due to its simple structure, fewer control parameters and high adaptability. Reinforcement learning is a powerful decision optimization method. Its ability to autonomously mine problem features and accumulate problem information for decision optimization helps the algorithm to quickly find effective feasible solutions in the solution space. Using the training results of reinforcement learning to guide the differential evolution algorithm to obtain high-quality feasible solutions is a reasonable solution. Summary of the invention
[0004] The purpose of the present invention is to address the problems existing in the prior art and provide a multi-objective energy-saving distributed assembly zero-wait flow-shop scheduling optimization system to solve an energy-efficient distributed assembly no-wait flow-shop scheduling problem (EEDANWFSP) under energy consumption constraints, with the goal of minimizing total energy consumption (TEC) and maximum assembly completion time (C_max).
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system, characterized in that: the system includes a workpiece sequence initialization module, an optimization module based on a differential evolution algorithm, an acceleration operation and energy-saving operation module, and a decision module driven by reinforcement learning;
[0007] In the workpiece sequence initialization module, an optimal allocation rule based on random sequences is designed to generate better initial solutions;
[0008] In the optimization module based on differential evolution algorithm, a framework based on differential evolution algorithm is designed to optimize the performance of the solution;
[0009] In the acceleration operation and energy-saving operation module, according to the characteristics of distributed assembly zero-wait flow shop scheduling under energy consumption constraints, an acceleration operation and a deceleration operation are designed to reduce the assembly completion time of the workpiece while reducing the processing energy consumption;
[0010] In the decision-making module driven by reinforcement learning, the results of Q-learning training are used for the selection of destruction size d and operation operator in the differential evolution algorithm optimization module.
[0011] Preferably, in the workpiece sequence initialization module, a certain number of random sequences are first generated based on the optimal allocation rule of random sequences, and each individual corresponds to a random sequence; secondly, the workpieces to be processed are tentatively inserted into different positions in the factory in sequence, and the final position is the position with the shortest completion time; finally, the solution corresponding to each individual is constructed into a concurrent population.
[0012] Specifically, we first use the FRB2 heuristic strategy to sort the workpieces in each product. For each product, the processing time of all its workpieces is summarized into a list PV h , and sort the list in descending order. For product h, set an initial workpiece sequence Then, according to the list PV h In the processing time sequence, select the corresponding workpiece J. If workpiece J has been added to the sequence before In Next, explore artifact J in the sequence Among all possible insertion positions, a position that minimizes the completion time is selected to insert workpiece J. This process is repeated until all workpiece sequences of all products are successfully initialized.
[0013] According to the LPT rule, the factory sequence initialization method is proposed to allocate all products to the factory. After completing the initialization of all workpiece sequences, the start assembly time τ is calculated. s. Then, using the LPT rule, sort the products in descending order according to the start assembly time to form a list Λ. Then, initialize the factory sequence Π and assign the first F products in the list Λ to the F factories in order. For the remaining products, the algorithm tests different insertion positions in the factory sequence Π and selects a position that minimizes the assembly completion time for insertion. Repeat the above process until all products are assigned to the factory sequence.
[0014] Preferably, in the optimization module based on the differential evolution algorithm, the traditional differential evolution algorithm is improved by selecting individuals from the parent generation for evolutionary operations, and instead two individuals are selected from the parent generation and one individual is selected from an external archive set, which helps to achieve a good balance between global search and local search. In addition, a HBOI crossover operator is designed by combining the binomial crossover operator in the DE algorithm and the order insertion operator commonly used to solve scheduling problems.
[0015] Specifically, a mutation operator is designed to operate the working sequence of the selected individuals, Q represents the population size, X i (t) represents the i-th target individual (i=1.2,...,Q) in the t-th generation. The first mutation operation is used to operate the job sequences on machine 1 and machine 2 respectively. An individual is randomly selected from the external archive EAS(t-1) and X e (t-1) means: two individuals X are randomly selected from the t-1 generation population. a (t-1) and X b (t-1); The job sequences of these three individuals on each machine are extracted as S e (t-1), S a (t-1) and S b (t-1); variant individual V i (t) operation sequence VS i (t) is based on S a (t-1) and S b The difference information between (t-1) and the operation sequence S e (t-1) is generated after transformation; the mutation operation is shown in the following formula:
[0016]
[0017] Where a and b are randomly selected from 1.2, ..., Q, satisfying a≠b≠i, F is the variation scaling factor used to adjust the size of the difference vector between two individuals, F∈[0,1]; Formula (1) includes three operators. Operator represents the position difference between two workpiece sequences, The result of the operation is the same as S a (t) and S b(t) have the same dimensions; for example, S a (t) = {3, 2, 4, 1, 6, 5} and S b (t) = {2, 1, 4, 5, 3, 6}, for S a The first job in (t) is b The corresponding position in (t) is 5, that is, the operation result is 5; the operations of other jobs are similar, and the final result is
[0018] In order to ensure that LDE can be effectively iterated within the feasible domain, a HBOI crossover operator is designed by combining the binomial crossover operator in the DE algorithm and the order insertion operator commonly used to solve scheduling problems; as shown in formula (2):
[0019]
[0020] Where US, S and VS are the test individual, target individual and variant individual respectively, r is a random number between [0, 1], p c is the crossover probability; if r≤p c , then take the target individual S as the basic individual and the variant individual VS as the reference individual, perform sequential insertion operation to generate US; otherwise, exchange VS and S and perform the same operation.
[0021] Preferably, in the acceleration operation and energy-saving operation module, in order to reduce the total assembly time, this module implements acceleration operations on the processes located on the critical path of the key factory. The initial step is to identify the key factory in the current solution, and traverse the workpiece sequence of each product in the key factory in turn. For those workpieces on the critical path, try to improve their speed level on the critical path. After each acceleration, it is necessary to recalculate the assembly completion time and energy consumption, and update the Pareto optimal solution set and external archives. This process continues until all workpieces of all products in the factory are traversed.
[0022] Preferably, in the decision-making module driven by reinforcement learning, the module constructs a method for adaptively selecting the destruction size d based on reinforcement learning Q-learning. In order to select a suitable operation operator, a method for adaptively selecting 6 operation operators based on the reinforcement learning Q-learning mechanism is also constructed, thereby avoiding blind search in the iterative process.
[0023] Specifically, the reinforcement learning driven decision module proposes a method for adaptively selecting the destruction size d based on reinforcement learning Q-learning. A Q table with 2 rows and s columns is established, denoted as q_table_d 2,s, the two states S1 without a new non-dominated solution and S2 with a new non-dominated solution are used as q_table_d 2,s If the selected destruction size d is used for iteration and no new non-dominated solution is obtained, the state is S1, otherwise it is S2; 1 to s selectable destruction sizes d are used as q_table_d 2,s s columns, destruction size d is taken as the action, the destruction size d is selected through the Q table using the ε-greedy selection method, and the Q table is updated using formula (27), and the reward value r t+1 According to whether there is a new non-dominated solution in the current iteration, if there is a new non-dominated solution, then r t+1 =1, otherwise r t+1 = -1;
[0024] Q(s t , a t )←Q(s t , a t )+α(r t+1 +γmax Q(s t+1 , a t+1 )-Q(s t , a t )) (25)
[0025] According to the characteristics of EEDANWFSP, six operation operators based on exchange and insertion are proposed. The six operation operators are:
[0026] Workpiece exchange: randomly select a workpiece of a product to exchange its position; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, continue to select other workpieces;
[0027] Workpiece insertion: randomly extract workpieces from the product's workpiece sequence and explore all possible positions for insertion; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive, and Q table; otherwise, return to the original position and continue to select the next workpiece;
[0028] Product insertion in key factories: randomly select a product from the key factory and try to insert it into all possible positions of the factory; if a new non-dominated solution is generated, update the Pareto optimal solution and external archive; otherwise, the product returns to its original position and continues to extract the next product;
[0029] Product insertion between factories: Extract a product from a key factory and explore all possible insertion locations in other non-key factories; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, put the product back to its original position and select the next product in the key factory;
[0030] Product exchange within key factories: Randomly select products for exchange within key factories; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, continue to select other products in the key factory;
[0031] Product exchange between factories: select products in key factories and products in non-key factories for exchange; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, continue to select the next product in the key factory;
[0032] The key factory is the factory with the longest assembly completion time;
[0033] In order to select the appropriate operator, the reinforcement learning Q-learning mechanism is introduced to adaptively select 6 operators, thereby avoiding blind search in the iterative process. A Q table with 2 rows and 6 columns is established, denoted as q_table_operator 2,6 , taking the two states of not obtaining a new non-dominated solution S1 and obtaining a new non-dominated solution S2 as q_table_operator 2,6 The two rows indicate that when the selected operator is used to iterate without obtaining a new non-dominated solution, the state is S1, otherwise it is S2. The columns indicate actions, and each operator is an action. To initialize the Q-table q_table_operator, a random local search method is used in the local search phase before the iteration. First, a sequence of operators is randomly generated without repetition, and the operators in this sequence are executed in sequence. During the iteration, the ε-greedy selection method is used to select operators through the Q-table.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] (1) The present invention analyzes the characteristics of the multi-objective energy-saving distributed assembly zero-wait flow shop scheduling problem, constructs a hybrid linear integer programming model for this type of problem, and proposes an evaluation criterion that simultaneously minimizes the maximum assembly completion time and total energy consumption.
[0036] (2) The present invention proposes a reinforcement learning driven differential evolution algorithm to solve the problem, designs six operation operators, and uses reinforcement learning method to self-learn and adjust the selection probability of the operation operators. At the same time, reinforcement learning method is used to adaptively select the destruction size d.
[0037] (3) According to the characteristics of this problem, the present invention designs a knowledge-driven energy-saving strategy, which can optimize the total delay and total energy consumption simultaneously in one step.
[0038] (4) The logic of the present invention is simple, easy to implement and expand. The optimization system can be extended to most scheduling problems in manufacturing production systems, such as site selection problems, vehicle path planning problems, etc. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 A schematic diagram of the structure of an energy-saving distributed assembly zero-wait flow shop scheduling optimization system provided by an example of the present invention;
[0040] Figure 2 This is a schematic diagram of the framework of the discrete differential evolution algorithm driven by reinforcement learning in the present invention;
[0041] Figure 3 Schematic diagram of crossover operation of differential evolution algorithm in the present invention;
[0042] Figure 4 It is a schematic diagram of the deceleration operation in the present invention;
[0043] Figure 5 The experimental comparison results of this system and other systems for solving the problem are shown in Figure 1. (a) is the interval diagram of the HV index of the four algorithms under the 95% confidence interval; (b) is the interval diagram of the C metric of the four algorithms under the 95% confidence interval; (c) is the result diagram of the Bonferroni-Dunn experiment; (d) is the curve formed by the Pareto frontier of the four algorithms. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0045] Example 1
[0046] Each factory in EEDANWFSP consists of two stages: zero-wait flow shop processing stage and product assembly stage. Figure 1As shown in the Gantt chart of EEDANWFSP, there are F factories, each of which has a processing factory and an assembly factory. There are s products that need to be produced, each of which is assembled from several workpieces. The s products are distributed to F factories, and the workpieces required to compose the products are processed in the processing factories in the factories, and then these workpieces are assembled into products in the assembly factories. All processing factories are the same zero-wait flow workshops, and each processing factory has m machines. All workpieces are processed sequentially on m machines, and the processing sequence cannot be changed. In order to meet the zero-wait constraint, there is no waiting time between the consecutive processes of the same workpiece during the workpiece processing process. All workpieces can only be processed on one machine in a processing factory at the same time, and all machines can only process one workpiece at the same time. All workpieces are independent of each other, and each workpiece can start processing at time 0, ignoring the machine setup time and transmission time, and there is an infinite buffer between these two stages. In the assembly factory, the product can only be assembled after all the workpieces of the product have been processed in the processing factory. The assembly of the latter product can only start after the assembly of the previous product is completed. In order to meet the distributed constraints, once all products are assigned to a specific factory, all artifacts belonging to the product can only be processed in the specific factory, the product can only be assembled into products in the specific factory, and the artifacts of the product cannot be mixed with the artifacts of other products and cannot be separated from the product.
[0047] MILP model description of EEDANWFSP:
[0048] Min{C max ,TEC} (3)
[0049]
[0050] C h ≥(C j,m ·G j,h )+t h (13)
[0051]
[0052] In the MILP model of the energy-saving distributed zero-wait flow shop scheduling problem, the goal is to minimize the maximum completion time (C_max) and total energy consumption (TEC) of the product, as shown in formula (3). To achieve this goal, the model includes the following constraints: Constraint (4) ensures that each workpiece is processed in only one designated factory to avoid duplicate processing or position errors of workpieces. Constraint (5) prevents workpieces from being transferred to another factory during processing to ensure the continuity of processing. Constraint (6) requires that operations be performed at a single speed level, which helps to standardize the calculation of processing time and energy consumption. Constraint (7) determines the actual processing time based on the selected speed level. Constraints (8-10) are the core requirements of zero-wait flow shop scheduling to ensure seamless connection of the production process. Constraint (11) requires that all workpieces of the same product must be processed in the same factory. Constraint (12) prevents products from being repeatedly arranged in the processing queue. Constraints (13) and (14) respectively ensure that product assembly only starts after all related workpieces are processed and in a specific order. The model calculates the completion time of product assembly through formula (15), while formulas (16) and (17) define the energy consumption of the machine in processing and idle states respectively. Formula (18) is used to calculate the total energy consumption. Formulas (19-22) define the decision variables.
[0053] Example 2
[0054] 1. Workpiece sequence initialization module
[0055] First, the FRB2 heuristic strategy is used to sort the workpieces in each product. For each product, the processing time of all its workpieces is summarized into a list PV h , and sort the list in descending order. For product h, set an initial workpiece sequence Then, according to the list PV h In the processing time sequence, select the corresponding workpiece J. If workpiece J has been added to the sequence before In Next, explore artifact J in the sequence Among all possible insertion positions, a position that minimizes the completion time is selected to insert workpiece J. This process is repeated until all workpiece sequences of all products are successfully initialized.
[0056] According to the LPT rule, the factory sequence initialization method is proposed to allocate all products to the factory. After completing the initialization of all workpiece sequences, the start assembly time τ is calculated. s. Then, using the LPT rule, sort the products in descending order according to the start assembly time to form a list Λ. Then, initialize the factory sequence Π and assign the first F products in the list Λ to the F factories in order. For the remaining products, the algorithm tests different insertion positions in the factory sequence Π and selects a position that minimizes the assembly completion time for insertion. Repeat the above process until all products are assigned to the factory sequence.
[0057] 2. Discrete differential evolution algorithm crossover and mutation module
[0058] like Figure 3 As shown, we design a mutation operator to operate the working sequence of the selected individuals. Q represents the population size, X i (t) represents the i-th target individual (i=1.2,...,Q) in the t-th generation, using the first mutation operation to operate the job sequence on machine 1 and machine 2 respectively. An individual is randomly selected from the external archive EAS(t-1) and represented by X e (t-1) represents. Randomly select two individuals X from the t-1 generation population a (t-1) and X b (t-1). The job sequences of these three individuals on each machine are extracted as S e (t-1), S a (t-1) and S b (t-1). Mutated individual V i (t) operation sequence VS i (t) is based on S a (t-1) and S b The difference information between (t-1) and the operation sequence S e (t-1) is transformed. The mutation operation is shown in the following formula:
[0059]
[0060] Where a and b are randomly selected from 1.2, ..., Q, satisfying a≠b≠i, and F is the variation scaling factor used to adjust the size of the difference vector between two individuals, F∈[0,1]. Formula (5.23) includes three operators. Operator represents the position difference between two workpiece sequences, The result of the operation is the same as S a (t) and S b (t) are of the same dimension. For example, S a (t) = {3, 2, 4, 1, 6, 5} and S b (t) = {2, 1, 4, 5, 3, 6}, for S a The first job in (t) is bThe corresponding position in (t) is 5, that is, the operation result is 5. The operations of other jobs are similar, and the final result is
[0061] To ensure that LDE can be effectively iterated within the feasible domain, we design an HBOI crossover operator by combining the binomial crossover operator in the DE algorithm and the order insertion operator commonly used to solve scheduling problems. As shown in formula (22):
[0062]
[0063] Where US, S and VS are the test individual, target individual and variant individual respectively, r is a random number between [0, 1], p c is the crossover probability. If r≤p c , then take the target individual S as the basic individual and the variant individual VS as the reference individual, perform sequential insertion operations to generate US; otherwise, swap VS and S and perform the same operation. For clarity, Figure 3 The crossover operation of the HBOI operator is shown.
[0064] 3. Acceleration operation and energy-saving operation module
[0065] like Figure 4 As shown in Figure 1, the processes on the critical path of the critical factories are accelerated. The initial step is to identify the critical factories Π in the current solution x. c , and for Π c For those workpieces J on the critical path, try to improve their speed level V on the critical path. i,J After each acceleration, the assembly completion time and energy consumption need to be recalculated, and the Pareto optimal solution set and external archive are updated. This process continues until all workpieces of all products in the factory are traversed.
[0066] A greedy deceleration method is proposed to perform deceleration operations on the workpieces of each product located in all factories. After each deceleration operation, the assembly completion time and energy consumption are recalculated, and the Pareto optimal solution set and external archive are updated. This process continues until all workpieces have been decelerated.
[0067] 4. Reinforcement Learning Technology Module
[0068] A method based on reinforcement learning Q-learning to adaptively select the destruction size d is proposed. A Q table with 2 rows and s columns is established, denoted as q_table_d 2,s , the two states S1 without a new non-dominated solution and S2 with a new non-dominated solution are used as q_table_d 2,sIf the selected destruction size d is used for iteration and no new non-dominated solution is obtained, the state is S1, otherwise it is S2. 1 to s selectable destruction sizes d are used as q_table_d 2,s s columns, and destroy size d as the action. Use the ε-greedy selection method to select the destruction size d through the Q table, and use formula (27) to update the Q table. The reward value r t+1 According to whether there is a new non-dominated solution in the current iteration, if there is a new non-dominated solution, then r t+1 =1, otherwise r t+1 =-1.
[0069] Q(s t , a t )←Q(s t , a t )+α(r t+1 +γmax Q(s t+1 , a t+1 )-Q(s t , a t )) (25)
[0070] According to the characteristics of EEDANWFSP, six operation operators based on exchange and insertion are proposed. The specific operations of the six operation operators are shown in Table 1. Among them, the key factory refers to the factory with the longest assembly completion time.
[0071] Table 1 Description of six operation operators
[0072]
[0073]
[0074] In order to select the appropriate operator, the reinforcement learning Q-learning mechanism is introduced to adaptively select 6 operators, thereby avoiding blind search in the iterative process. A Q table with 2 rows and 6 columns is established, denoted as q_table_operator 2,6 , taking the two states of not obtaining a new non-dominated solution S1 and obtaining a new non-dominated solution S2 as q_table_operator 2,6 The two rows of , when the selected operator is used to iterate without obtaining a new non-dominated solution, the state is S1, otherwise it is S2. The columns represent actions, and each operator is an action. To initialize the Q-table q_table_operator, a random local search method is used in the local search phase before the iteration. First, a sequence of operators is randomly generated without repetition, and the operators in this sequence are executed in sequence. In the iteration process, the ε-greedy selection method is used to select operators through the Q-table.
[0075] Example 3
[0076] System flow: The basic framework of LDE is as follows: Figure 2 As shown. First, use the initialization method proposed in the present invention to initialize the solution x. Update the Pareto optimal solution set. Perform a random local search on the solution x and update the q_table_operator. In the iteration stage, first perform mutation and crossover operations, then randomly select a solution x′ from the Pareto optimal solution set or external archive, and then randomly replace the plant sequence or speed sequence in the current solution x with the plant sequence or speed matrix of the solution x′, use the ε-greedy selection method to select a destruction size d in q_table_d, and use d for destruction and reorganization. Then use the ε-greedy selection method to select an operation operator o in q_table_operator, and execute the operation operator o on the solution x. Update q_table_d and q_table_operator. Perform acceleration operations and energy-saving operations on the solution x, and repeat the above process until the termination condition is met. The algorithm pseudocode is shown in Algorithm 1.
[0077]
[0078]
[0079] Example 4
[0080] The performance of the system in processing the EEDANWFSP problem is evaluated using large-scale examples from the standard test set proposed by Hatami et al. Specifically, the large-scale examples involve the number of workpieces n as {100, 200, 500}, the number of machines m as {5, 10, 20}, the number of factories f as {4, 6, 8}, and the number of products s as {30, 40, 50}, with 81 different configurations, each with 10 independent instances, for a total of 810 instances. All these instances can be accessed and downloaded through the website http: / / soa.iti.es / . In terms of evaluating the effectiveness of the solution set of multi-objective optimization problems, this section uses the C metric and the HV indicator to evaluate the Pareto solution set generated by the algorithm from the perspectives of quantity and comprehensiveness.
[0081] The interval diagram of HV index of four algorithms under 95% confidence interval is as follows: Figure 5 As shown in (a), LDE outperforms the other three comparison algorithms at all time levels c and has significant differences. The interval diagram of the C metric of the four algorithms under the 95% confidence interval is shown in Figure 5As shown in (b), LDE has significant differences compared with NSGA-II and RLBSO at all time levels. When c = 20, there is no significant difference between LDE and DE. As the CPU time increases, LDE and DE begin to have significant differences. The Bonferroni-Dunn test is a post hoc test to evaluate the significance level of all algorithms. The Bonferroni-Dunn test is shown in Figure 5 As shown in (c), the average rank value of LDE is higher than the test thresholds of 95% and 90%, while the average rank value of the other three comparison algorithms is lower than the test thresholds of 95% and 90%. From this, it can be seen that LDE has significant differences from the other comparison algorithms. Figure 5 (d) is the Pareto frontier diagram of some examples of the four algorithms, where the blue dots are LDE, the red stars are DE, the green hexagons are NSGA-II, and the purple diamonds are RLBSO. The closer the curve of the Pareto frontier formed by the algorithm is to the lower left corner, the better the quality of the Pareto frontier of the algorithm. Figure 5 (d) It can be seen that the curve formed by the LDE non-dominated solution set is closer to the origin than the other three comparison algorithms and the number of non-dominated solutions is significantly larger than the other three comparison algorithms.
[0082] In summary, LDE is superior to the other three comparison algorithms in both counting indicators and comprehensive indicators. From the Pareto frontier diagram, it can be seen that the quality of the Pareto frontier obtained by LDE is better than the other three comparison algorithms, which also proves that LDE is an effective algorithm for solving EEDANWFSP.
[0083] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. Multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system, characterized by: The system includes a workpiece sequence initialization module, an optimization module based on differential evolution algorithm, an acceleration operation and energy-saving operation module, and a decision-making module driven by reinforcement learning; In the workpiece sequence initialization module, a certain number of random sequences are first generated based on the optimal allocation rule of random sequences, and each individual corresponds to a random sequence; secondly, the workpieces to be processed are tentatively inserted into different positions in the factory in the order of the sequence, and the final position is the position with the smallest completion time; finally, the solution corresponding to each individual is constructed into a concurrent population; In the optimization module based on differential evolution algorithm, a framework based on differential evolution algorithm is designed to optimize the performance of the solution; In the acceleration operation and energy-saving operation module, according to the characteristics of distributed assembly zero-wait flow shop scheduling under energy consumption constraints, an acceleration operation and a deceleration operation are designed to reduce the assembly completion time of the workpiece while reducing the processing energy consumption; In the decision-making module driven by reinforcement learning, the results of Q-learning training are used for the selection of destruction size d and operation operator in the differential evolution algorithm optimization module.
2. The multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system according to claim 1 is characterized by: In the workpiece sequence initialization module, the FRB2 heuristic strategy is first used to sort the workpieces in each product. For each product, the processing time of all its workpieces is summarized into a list PV h , and sort the list in descending order; for product h, set an initial workpiece sequence Then, according to the list PV h In the processing time sequence, select the corresponding workpiece J. If workpiece J has been added to the sequence before In Move out; Next, explore artifact J in the sequence Among all possible insertion positions, select a position that minimizes the completion time to insert workpiece J; this process is repeated until the workpiece sequences of all products are successfully initialized; According to the LPT rule, a factory sequence initialization method is proposed to allocate all products to the factory. After completing the initialization of all workpiece sequences, the start assembly time τ is calculated. s ; Then, using the LPT rule, sort the products in descending order according to the start assembly time to form a list Λ; then, initialize the factory sequence ∏, and assign the first F products in the list Λ to F factories in sequence; for the remaining products, the algorithm tests different insertion positions in the factory sequence ∏ and selects a position that minimizes the assembly completion time for insertion; repeat the above process until all products are assigned to the factory sequence.
3. The multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system according to claim 1 is characterized by: In the optimization module based on differential evolution algorithm, a mutation operator is designed to operate the working sequence of the selected individuals. Q represents the population size, X represents the i (t) represents the i-th target individual in the t-th generation (i = 1.2, ..., Q). The first mutation operation is used to operate the job sequences on machine 1 and machine 2 respectively. An individual is randomly selected from the external archive EAS(t-1) and X e (t-1) means: two individuals X are randomly selected from the t-1 generation population. a (t-1) and X b (t-1); The job sequences of these three individuals on each machine are extracted as S e (t-1), S a (t-1) and S b (t-1); variant individual V i (t) operation sequence VS i (t) is based on S a (t-1) and S b The difference information between (t-1) and the operation sequence S e (t-1) is generated after transformation; the mutation operation is shown in the following formula: Where a and b are randomly selected from 1.2,…,Q, satisfying a≠b≠i, F is the variation scaling factor used to adjust the size of the difference vector between two individuals, F∈[0,1]; operator represents the position difference between two workpiece sequences, The result of the operation is the same as S a (t) and S b (t) have the same dimensions; for example, S a (t) = {3, 2, 4, 1, 6, 5} and S b (t) = {2, 1, 4, 5, 3, 6}, for S a The first job in (t) is b The corresponding position in (t) is 5, that is, the operation result is 5; the operations of other jobs are similar, and the final result is In order to ensure that LDE can be effectively iterated within the feasible domain, a HBOI crossover operator is designed by combining the binomial crossover operator in the DE algorithm and the order insertion operator commonly used to solve scheduling problems; as shown in formula (2): Where US, S and VS are the test individual, target individual and variant individual respectively, r is a random number between [0,1], p c is the crossover probability; if r≤p c , then take the target individual S as the basic individual and the variant individual VS as the reference individual, perform sequential insertion operation to generate US; otherwise, exchange VS and S and perform the same operation.
4. The multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system according to claim 1 is characterized by: In the acceleration operation and energy-saving operation module, in order to reduce the total assembly time, this module implements acceleration operations on the processes located on the critical path of the key factory. The initial step is to identify the key factory in the current solution and traverse the workpiece sequence of each product in the key factory in turn. For those workpieces on the critical path, try to improve their speed level on the critical path. After each acceleration, the assembly completion time and energy consumption need to be recalculated, and the Pareto optimal solution set and external archive are updated. This process continues until all workpieces of all products in the factory are traversed.
5. The multi-objective energy-saving distributed assembly zero-wait flow shop scheduling optimization system according to claim 1 is characterized by: In the decision-making module driven by reinforcement learning, a method based on reinforcement learning Q-learning to adaptively select the destruction size d is proposed. A Q table with 2 rows and s columns is established, denoted as q_table_d 2,s , the two states S1 without a new non-dominated solution and S2 with a new non-dominated solution are used as q_table_d 2,s If the selected destruction size d is used for iteration and no new non-dominated solution is obtained, the state is S1, otherwise it is S2; 1 to s selectable destruction sizes d are used as q_table_d 2,s s columns, destruction size d is taken as the action, the destruction size d is selected through the Q table using the ε-greedy selection method, and the Q table is updated using formula (27), and the reward value r t+1 According to whether there is a new non-dominated solution in the current iteration, if there is a new non-dominated solution, then r t+1 =1, otherwise r t+1 = -1; Q(s t ,a t )←Q(s t ,a t )+α(r t+1 +γmax Q(s t+1 ,a t+1 )-Q(s t ,a t )) (25) According to the characteristics of EEDANWFSP, six operation operators based on exchange and insertion are proposed. The six operation operators are: Workpiece exchange: randomly select a workpiece of a product to exchange its position; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, continue to select other workpieces; Workpiece insertion: randomly extract workpieces from the product's workpiece sequence and explore all possible positions for insertion; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive, and Q table; otherwise, return to the original position and continue to select the next workpiece; Product insertion in key factories: randomly select a product from the key factory and try to insert it into all possible positions of the factory; if a new non-dominated solution is generated, update the Pareto optimal solution and external archive; otherwise, the product returns to its original position and continues to extract the next product; Product insertion between factories: Extract a product from a key factory and explore all possible insertion locations in other non-key factories; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, put the product back to its original position and select the next product in the key factory; Product exchange within key factories: Randomly select products for exchange within key factories; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, continue to select other products in the key factory; Product exchange between factories: select products in key factories and products in non-key factories for exchange; if a new non-dominated solution is generated, update the Pareto optimal solution, external archive and Q table; otherwise, continue to select the next product in the key factory; The key factory is the factory with the longest assembly completion time; In order to select the appropriate operator, the reinforcement learning Q-learning mechanism is introduced to adaptively select 6 operators, thereby avoiding blind search in the iterative process; a Q table with 2 rows and 6 columns is established, denoted as q_table_operator 2,6 , taking the two states of not obtaining a new non-dominated solution S1 and obtaining a new non-dominated solution S2 as q_table_operator 2,6 The two rows indicate that when the selected operator is used to iterate without obtaining a new non-dominated solution, the state is S1, otherwise it is S2. The columns indicate actions, and each operator is an action. To initialize the Q-table q_table_operator, a random local search method is used in the local search phase before the iteration. First, a sequence of operators is randomly generated without repetition, and the operators in this sequence are executed in sequence. During the iteration, the ε-greedy selection method is used to select operators through the Q-table.
Citation Information
Cited By
Multi-modal multi-target reentrant hybrid flow shop scheduling method
CN121981506A
A multi-modal multi-objective re-entrant hybrid flow shop scheduling method
CN121981506B
