Multi-stage and multi-level assembly workshop energy-saving scheduling multi-objective optimization method

Abstract

The invention aims to provide a multi-stage and multi-level assembly workshop energy-saving scheduling multi-objective optimization method, relates to the technical field of assembly workshop scheduling, establishes a mathematical model of a problem in stages, and designs a problem-driven energy-saving strategy triggering mechanism according to an individual state to improve the quality of a solution and the search efficiency; secondly, two heuristic rules and a random generation mode are adopted to cooperatively construct an initialized population considering high quality and diversity, a dynamic self-adaptive adjustment strategy for assimilation operator parameters is realized through Q-learning, and the convergence speed is increased while population diversity is ensured, so that the exploration and mining capabilities of the algorithm are better balanced; a revolution operation guided by hyper-heuristic variable neighborhood search is designed, and found high-quality colonies are searched carefully; and competition is replaced by joint empire aggressive operation, so that coevolution and information interaction and sharing of multiple empire are realized, and non-dominated solutions which are distributed more uniformly are found.

Description

Technical Field

[0001] The present invention relates to the technical field related to assembly workshop scheduling, and in particular to a multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop. Background Art

[0002] The multi-stage assembly job shop scheduling problem (MsAJSP) involves multiple stages from processing to assembly. Its model is more realistic and has received widespread attention. Current studies simplify products into single-level assembly structures or ignore product structure differences, which are only applicable to single-level or fixed-structure products. However, in actual production, products are usually complex assemblies composed of multiple parts, with a tree-like multi-level structure composed of components and sub-assemblies. In addition, as the number of product structure levels and the number of manufacturing stages increase, the constraints become more complex, resulting in machine idle time between and within stages, which in turn causes a large amount of idle energy consumption. Therefore, the energy-saving scheduling problem for multi-stage and multi-level assembly job shops (EEMsAJSP_ML) for multi-level complex products needs further study.

[0003] At present, it is a relatively effective method to deal with complex scheduling problems with multiple processes and stages. There are two main operation modes: one is to execute the block shifting strategy only for the solutions in the non-dominated solution set to further reduce the consumption, but this operation will lose some potential solutions, resulting in a local rather than a global optimal solution; the other is to embed the block shifting strategy into the individual decoding process to reduce consumption in order not to lose the global optimal solution, but it will cause a waste of computing resources and seriously drag down the algorithm search efficiency. Therefore, there is an urgent need to design an efficient block shifting strategy to improve the quality of the solution and the search efficiency without losing the global solution.

[0004] However, as an effective evolutionary algorithm framework, most existing studies of the Imperialist Competitive Algorithm (ICA) use the Uniform Crossover (UX) operator to implement the assimilation process of the global search of the algorithm. Due to the high complexity of the EEMsAJSP_ML problem and the huge solution space, traditional static parameters may be difficult to guarantee the search efficiency and quality. Therefore, it is necessary to design a reasonable parameter dynamic adaptive adjustment mechanism to balance the exploration and mining of the algorithm. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention provides the following technical solutions:

[0006] A multi-objective optimization method for energy-saving scheduling in a multi-stage and multi-level assembly workshop is proposed. The mathematical model of the EEMsAJSP_ML problem is established in stages, and a decoding strategy with an energy-saving trigger mechanism is designed based on the nature of the problem. An improved imperial competition algorithm IICA-QL based on Q-learning is designed to solve the problem model. The specific optimization method includes the following steps:

[0007] S1: Establish a mathematical model of the problem. Establish a mathematical model of the EEMsAJSP_ML problem in the processing stage, transportation stage, and assembly stage.

[0008] Total energy consumption during processing Release energy consumption for the machine during the processing stage, Set the energy consumption of the machine for the processing phase, is the energy consumption of the machine during the processing stage, is the idle energy consumption of the machine during the processing phase;

[0009] Total energy consumption during transportation

[0010] Total energy consumption during assembly is the energy consumption of the machine during the assembly phase, is the idle energy consumption of the machine during the assembly phase;

[0011] The maximum completion time of the system is the time it takes to complete the assembly of all products.

[0012] In all three phases, the sorted set Π(π=[π st1 ,π st2 ,π st3 ]∈Π) to find an optimal sorting π * , so that the completion time C of the three stages is minimized at the same time max and total energy consumption TEC;

[0013]

[0014] TEC(π)=EC st1 +EC st2 +EC st3 +EC aux

[0015]

[0016] Construction of production system completion time C max and the total energy consumption TEC, while minimizing the three-stage completion time C max and total energy consumption TEC;

[0017] S2: Encoding and decoding design, according to the coupling properties between the various stages of EEMsAJSP_ML, the processing procedures in the processing stage are encoded, the machine allocation in the processing stage and the heuristic decoding rules used in the subsequent transportation and assembly stages are decoded, and the energy-saving strategy decoding trigger mechanism is designed. The specific operations of the energy-saving strategy decoding trigger mechanism are as follows:

[0018] S2.1: The result of the scheduling scheme obtained by using the strategy-free decoding strategy and the main scheduling for the individual is C max (π) and TEC(π), and record the idle time;

[0019] S2.2: Calculate the upper limit ES of energy consumption saved by adopting the energy-saving strategy of the scheduling scheme UB And energy consumption lower limit E LB ;

[0020]

[0021] E LB =TEC(π)-ES UB ;

[0022] S2.3: Determine the solution (C max (π),E LB ) and the contemporary non-dominated solution set. If it is not dominated by the contemporary non-dominated solution set or they are not mutually dominated, go to S2.5, otherwise go to S2.4;

[0023] S2.4: Output the result C of the scheduling scheme using non-strategy decoding max (π) and TEC(π);

[0024] S2.5: According to the product assembly sequence of the scheduling plan, each product is assembled in reverse order from back to front, and Determine whether the block transfer condition is met;

[0025] If satisfied, then according to

[0026]

[0027] Update the completion time of the process, otherwise, determine the previous process of the process;

[0028] S2.6: If all processes are judged, then use Determine the idle interval that meets the profit and loss time and shut down and output the result after the scheduling scheme executes the energy-saving strategy C max (π) and TEC(π)', update the non-dominated solution set, otherwise, return to S2.5;

[0029] in, Indicates the process on machine m After that, in your free time, Indicates that on machine k The end time of the insertable idle interval, express The next process; T off-on Indicates the time required for the machine to turn on and off, EC off-on Indicates the energy consumption of the switch. Indicates the energy consumption per unit time of the machine;

[0030] S3: Initialize the empire, use heuristic rules to generate a certain number of individuals, and randomly generate the remaining individuals. Select N im Colonial countries, building the initial empire;

[0031] A variety of heuristic methods are used to mix and initialize the population. The product aggregation PA and the parts aggregation rule JA generate α×Ps national individuals respectively, and the remaining national individuals are randomly generated.

[0032]

[0033] N col =PS-N im

[0034]

[0035] NC k =round(Pow k ×N col )

[0036] Calculate the cost of each country in the initial population in turn right After sorting in non-ascending order, select the N with the largest cost im Countries as N im The empire's colonial countries, other countries in the population are colonies, calculate the number of colonies N col , calculate the colonial country Imp k The normalized power of each colonial country is used to determine the number of colonies NC k , and finally randomly select a corresponding number of colonies to be divided into the corresponding empire population, and delete the selected colonies from the original colony population;

[0037] IICA-QL National Pi i represents a solution to the problem, where i = 1, 2…, Ps, Ps is the size of the population, and all countries in the g-th generation constitute the g-th generation population Pop(g), that is, Pop(g) = {π g,1 ,π g,2 ,…,π g,Ps};

[0038] S4: Construct an external archive set Ω, and update the external archive set according to the non-dominated solution, and dynamically update it during the iteration process;

[0039] S5: Assimilation based on adaptive adjustment of Q-learning operator parameters. Select UX operator parameters according to the Q-table, then perform multi-directional collaborative assimilation in each empire and update the Q-table. At the same time, retain colonies according to the elite retention strategy. The specific steps are as follows:

[0040] S5.1: Co-evolution operation using two different uniform crossover operators UX and two-point crossover operator TPX;

[0041] S5.2: Status, the status of the entire population is determined by judging the convergence index ΔGD of the non-dominated solution set and the reference set and the diversity index ΔRatio of the coverage solution space ratio.

[0042]

[0043] ΔRatio=ψ t+1 |-ψ t |

[0044] Where ΔGD represents the non-dominated solution and the Pareto frontier PF obtained in the t+1th generation compared with the tth generation * The difference in generation distance, PF * is the preset reference point, ΔRatio represents the ratio of the sub-region occupied by the contemporary non-dominated solution in the target space, and the sub-region represents dividing the target space into N non-overlapping regions through N weight vectors; where |PF t |,ψ t |represents the number of non-dominated solutions of the tth generation and the proportion of the target space occupied;

[0045] S5.3: Action, the action set A consists of 10 actions. By judging the current state of the population, the action with the largest Q value in the Q table is selected according to the ε-greedy strategy;

[0046] S5.4: ε-greedy, using the sigmoid-based ε-greedy strategy for the agent to learn. The mathematical expression of ε-greedy is as follows:

[0047]

[0048] S5.5: Rewards. By calculating the ΔGD and ΔRatio after executing an action, the agent will receive the corresponding rewards, and according to

[0049]

[0050] Update the Q value of the action in the corresponding state of the Q table, where α and γ are the learning rate and discount factor respectively, and A is the set of optional actions;

[0051] S5.6: Colonial renewal strategy, using the elite retention strategy to select colonies that need to be retained after the assimilation process; k Individuals generated during the assimilation process are placed into the colony offspring population k After the assimilation operation is completed within the empire, the original colony population and the colony offspring population are merged into a new population SubPop k , all SubPop k SubPop after individual merging k The individuals in the non-dominated sorting and crowding calculation are performed, and NC is selected k The number of colonies is used as the next generation of colonies. At the same time, the greedy strategy is used to select the assimilation results of the colonial countries, that is, for the new solution generated by the assimilation of the colonial countries, if it is not dominated by the old solution, the new solution will replace the original colonial country, otherwise no replacement will be made;

[0052] S6: Based on the revolution of hyper-heuristic variable neighborhood search, the high-quality colonies within the empire execute the high-dimensional strategy constructed by the two-dimensional probability model to select the low-level neighborhood search operation sequence, and update the probability model according to the feedback information given by the environment after executing the selected neighborhood group;

[0053] Define NS = {NS G,k (1),NS G,k (2),…,NS G,k (l)} is the neighborhood operation sequence performed by the kth individual in the Gth generation, where l represents the length of the variable neighborhood search using the two-dimensional probability model P G (x,y) serves as a high-level strategy to guide the selection of low-level variable neighborhood search operator sequences;

[0054]

[0055] The initialization of the two-dimensional probability model and the update of the Gth generation are shown in the following equations respectively;

[0056] P 0 (x,y)=1 / l,x,y=1,2,…,l

[0057] P G (x,y)=(1+reward)×P G-1 (x,y),x=1,2,…,l,y=NS(1),NS(2),…,NS(l)

[0058] After completing the revolution operation, calculate the cost of each country in the empire, select the country with the largest cost as the new colonial country, and the original colonial country will become a colony.

[0059]

[0060] P G (x,y) is updated;

[0061] S7: Implement joint empire competition, achieve co-evolution of empires and strengthen information interaction and sharing among empires;

[0062] S7.1: Calculate and rank the total cost of each imperial group

[0063]

[0064] Among them, Emp k For the Empire Imp k The set of colonies owned, ξ is the coefficient of the proportion of colonies to the total cost of the empire;

[0065] S7.2: The top N im -1 Empire's colonies are crossed with random solutions using POX to generate a new solution that replaces the corresponding number of the worst colonies in the worst empire.

[0066] S8: If the iteration termination condition is met, stop searching; otherwise, return to S5.

[0067] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0068] This paper proposes an improved imperial competition algorithm IICA-QL based on Q-learning to minimize the maximum completion time and total energy consumption. Firstly, the mathematical model of the problem is established in stages, and a problem-driven energy-saving strategy trigger mechanism is designed according to the individual state to improve the solution quality and search efficiency; secondly, IICA-QL is proposed to solve EEMsAJSP_ML. In IICA-QL, two heuristic rules and random generation methods are used to collaboratively construct an initialization population that takes into account both high quality and diversity; a dynamic adaptive adjustment strategy for the assimilation operator parameters is implemented through Q-learning to improve the convergence speed while ensuring population diversity, so as to better balance the algorithm's exploration and mining capabilities; a revolutionary operation guided by a hyper-heuristic variable neighborhood search is designed to conduct a detailed search for discovered high-quality colonies; the joint empire invasion operation is used to replace competition to achieve the co-evolution of multiple empires and information interaction and sharing, so as to find a more evenly distributed non-dominated solution; compared with other algorithms, specifically, IICA-QL's performance in the GD indicator shows that it can find solutions in the solution space that are very close to the Pareto optimal frontier, indicating that IICA-QL has obvious advantages in convergence, and from the HV indicator, it can be found that the value obtained by IICA-QL is much larger than that of the other three algorithms, indicating that IICA-QL can not only find high-quality solutions, but also maintain the diversity of solutions and cover a wider Pareto frontier area. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 EEMsAJSP_ML schematic diagram;

[0070] Figure 2 Coding diagram;

[0071] Figure 3 Schematic diagram of energy-saving strategy decoding trigger mechanism;

[0072] Figure 4 Q table update schematic diagram;

[0073] Figure 5 Schematic diagram of neighborhood structure;

[0074] Figure 6 Algorithm flow chart;

[0075] Figure 7 Algorithm comparison box plots of various indicators;

[0076] Figure 8 Algorithm running speed comparison chart;

[0077] Fig. 9 Algorithm comparison box plots of various indicators;

[0078] Fig.10Box plot of the comparison results between IICA-QL and BPBMO / PSO-GA / KBOA. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] Embodiment 1:

[0081] This embodiment is the establishment of the mathematical model of the EEMsAJSP_ML problem:

[0082] Table 1 Notes on mathematical model

[0083]

[0084]

[0085] The symbols used to define the EEMsAJSP_ML mathematical model are shown in Table 1. The entire production process of the product is divided into three stages: 1) Processing stage: There are N processing workshops in total M The parts complete all the processing steps on the optional processing machines according to the processing process route. 2) Transportation stage: The parts that have completed all the processing steps are transported by N T The same type of vehicles with a maximum load of Gmax are transported to the assembly workshop in batches. 3) Assembly stage: There are NK assembly machines in the assembly workshop. After all the parts of the product are complete, they are assembled in N K The entire production process of the product is completed by assembling it on an assembly machine. Figure 1 As shown. The problem makes the following assumptions:

[0086] 1) Any machine in the processing and assembly stages can only handle one process at the same time; once the processing process, transportation process and assembly process start, they cannot be interrupted; 2) All machines and vehicles are available at time zero; 3) Parts (products) can only enter the next process after the previous processing (assembly) process is completed; 4) The transportation time between machines for two consecutive processing (assembly) processes is not considered; 5) The same parts can exist between different products; 6) The same processing (assembly) process can be used for the same part (product) at most once; 7) The vehicle maintains a constant speed during transportation, and the load does not exceed the maximum load of the vehicle; 8) The first setup time of the machine is included in the workpiece release time.

[0087] The construction of the three-stage mathematical model of EEMsAJSP_ML with two optimization objectives is as follows:

[0088] The maximum completion time of the system is the time it takes to complete the assembly of all products.

[0089]

[0090] The total energy consumption of the entire production system consists of four parts: 1) total energy consumption in the processing stage; 2) total energy consumption in the transportation stage; 3) total energy consumption in the assembly stage. The total energy consumption of the three stages is calculated below:

[0091] Processing stage: The total energy consumption in this stage is composed of the energy consumption of the processing machine. The energy consumption generated by the processing machine is composed of four states of the machine, namely: release state, setting state, working state and idle state; the energy consumption of the processing machine in each state is constructed as follows:

[0092] Release energy consumption: refers to the energy consumption of each machine before the first arrival of the workpiece on the machine. The calculation is as follows:

[0093]

[0094] Energy consumption during setup: refers to the energy consumption of changing fixtures for different workpieces before the machine is in working state. The calculation of is as follows:

[0095]

[0096] Working energy consumption: refers to the energy consumption generated by the processing machine in working state. The calculation is as follows:

[0097]

[0098] Idle energy consumption: refers to the energy consumption of the machine during the non-set time between two consecutive processing steps. The calculation is as follows:

[0099]

[0100] Total energy consumption in the processing phase EC st1 The calculation is as follows:

[0101]

[0102] 2) Transportation stage: The total energy consumption in this stage is composed of vehicle energy consumption. The energy consumption generated by the vehicle is composed of two parts: loaded energy consumption and unloaded energy consumption. The total energy consumption in the transportation stage is EC st2 The calculation is as follows:

[0103]

[0104] 3) Assembly phase: The total energy consumption in this phase is composed of the energy consumption of the assembly machine. The energy consumption generated by the assembly machine is composed of two states of the machine: working state and idle state. The energy consumption of the assembly machine in each state is constructed as follows:

[0105] Working energy consumption: refers to the energy consumption generated when the assembly machine is in working state. Working energy consumption of the machine during the assembly stage The calculation is as follows:

[0106]

[0107] Idle energy consumption: refers to the energy consumption of the assembly machine when it is waiting for the next assembly process. The calculation is as follows:

[0108]

[0109] Total energy consumption in assembly phase EC st3 The calculation is as follows:

[0110]

[0111] Optimization goal: In all three stages, the sorted set Π(π=[π st1 ,π st2 ,π st3 ]∈Π) to find an optimal sorting π * , so that the completion time C of the three stages is minimized at the same time max And the total energy consumption TEC:

[0112]

[0113] TEC(π)=EC st1 +EC st2 +EC st3 +EC aux (13)

[0114]

[0115] Embodiment 2:

[0116] This embodiment is a codec design;

[0117] According to the coupling properties between the various stages of EEMsAJSP_ML, this embodiment only encodes the processing procedures in the processing stage, and uses heuristic rules to decode the machine allocation in the processing stage and the subsequent stages.

[0118] This paper uses a process-based coding method to characterize the solution to the problem. The process of the workpiece is represented by the number of the workpiece, and the number of times the workpiece number appears is the process number of the workpiece. Since the product design order demand considered in this paper is large and different products may have the same parts, it is necessary to read the product information once to determine the workpiece required for the product and assign it a unique number for identification. In order to better demonstrate the coding process, take the problem that the demand for product 1 is 2 and the demand for product 2 is 1 as an example, where product 1 is composed of components 1 and 2, product 2 is composed of components 3 and 4, component 1 is composed of part 1, component 2 is composed of parts 2 and 3, component 3 is composed of parts 2 and 4, and component 4 is composed of parts 5, 6, and 7. Each part is processed by a corresponding process. The specific coding method of this problem is as follows: Figure 2 shown.

[0119] In order to narrow the solution space and find the near-optimal solution to the problem as much as possible, the heuristic decoding rules used for machine allocation in the processing stage and subsequent transportation and assembly stages are as follows:

[0120] 1) Machine allocation in the processing stage: The earliest completion time rule is used to allocate the process to the machine that can complete the process the earliest. If multiple machines have the same time to complete the process, the machine with the lowest energy consumption is selected according to the minimum energy consumption priority rule;

[0121] 2) Rules for the transportation phase: Under the premise of meeting the vehicle load constraints, the first-completed-first-transported rule is adopted to organize the transportation of workpieces in batches, that is, the workpieces are loaded into available vehicles in the order of completion time, and the workpieces transported by a single vehicle must not be overloaded. Subsequent workpieces are transported by subsequent vehicles according to the same rule;

[0122] 3) Assembly stage rules: When the parts transported to the assembly workshop can meet the requirements of product assembly, they are assembled according to the product assembly process. If the assembly conditions of multiple products are met at the same time, the product with the least parts is prioritized to determine the assembly order of the products. When assembling the products, the machine allocation of the assembly process adopts the same rule as the machine allocation in the processing stage to determine the assembly machine for the process;

[0123] This embodiment proposes an adaptive triggering mechanism for an energy-saving strategy, which adaptively determines the effect of executing the energy-saving strategy to ensure that potential high-quality solutions are not lost while improving the algorithm search efficiency. The specific operations are:

[0124] Step 1: Use the strategy-free decoding strategy and the main scheduling strategy to obtain the result C of the scheduling solution. max (π) and TEC(π), and record the idle time;

[0125] Step 2: According to formula (15) and formula (16), the energy consumption limit ES saved by the scheduling scheme after adopting the energy-saving strategy is obtained:UB And energy consumption lower limit E LB ;

[0126] Step 3: Determine the solution (C max (π),E LB ) and the contemporary non-dominated solution set. If it is not dominated by the contemporary non-dominated solution set or they are not mutually dominated, go to Step 5, otherwise go to Step 4;

[0127] Step 4: Output the result C of the scheduling scheme using non-strategy decoding max (π) and TEC(π);

[0128] Step 5: According to the product assembly sequence of the scheduling plan, each product is reversed from the back to the front in the process order and whether the block transfer condition is met according to formula (17); if it is met, the completion time of the process is updated according to formula (18); otherwise, the previous process of the process is determined;

[0129] Step 6: If all processes are judged, then use formula (19) to determine the idle interval shutdown that meets the profit and loss time and output the result C after the scheduling scheme executes the energy-saving strategy max (π) and TEC(π)', update the non-dominated solution set, otherwise, return to Step 5;

[0130]

[0131] E LB =TEC(π)-ES UB (16)

[0132]

[0133] in, Indicates the process on machine m After that, in your free time, Indicates that on machine k The end time of the insertable idle interval, express The next process. off-on Indicates the time required for the machine to turn on and off, EC off-on Indicates the energy consumption of the switch. Represents the energy consumption per unit time of the machine; the process of energy-saving strategy decoding trigger mechanism is as follows Figure 3 shown.

[0134] Embodiment 3:

[0135] This example uses IICA-QL to solve EEMsAJSP_ML;

[0136] 3.1 Initializing the Empire

[0137] In IICA-QL, country π i represents a solution to the problem, where i = 1, 2…, Ps, Ps is the size of the population. All countries in the gth generation constitute the gth generation population Pop(g), that is, Pop(g) = {π g,1 ,π g,2 ,…,π g,Ps In order to obtain a high-quality initial population while taking into account the diversity of the population, a variety of heuristic methods are used to mix and initialize the population. α×Ps national individuals are generated by product aggregation and part aggregation rules respectively, and the remaining national individuals are randomly generated, so that the algorithm has a higher search starting point while taking into account the search width.

[0138] Then, the cost of each country in the initial population is calculated in turn according to formula (20): right After sorting in non-ascending order, select the N with the largest cost im Countries as N im The number of colonies N is calculated according to formula (21): col Then, according to formula (22), the Imp of the colonial country is calculated. k The normalized power of each colonial country is used to determine the number of colonies NC k , and finally randomly select a corresponding number of colonies to be divided into the corresponding empire population, and delete the selected colonies from the original colonial population.

[0139]

[0140] N col =PS-N im (twenty one)

[0141]

[0142] NC k =round(Pow k ×N col ) (twenty three)

[0143] Before the iteration begins, an empty external archive set Ω is established to store the non-dominated set of the initial population and is dynamically updated during the iteration process.

[0144] 3.2 Assimilation based on adaptive adjustment of Q-learning operator parameters

[0145] During the assimilation process, the Uniform Crossover (UX) operator and the Two Point Crossover (TPX) operator are used, which are respectively applied to the process of colonies moving to colonial countries and external archives. Among them, the UX operator guides the evolution of the population as the main evolutionary operator, and can also be used as a local search, which plays an important role in the quality of population evolution. The size of the parameter η of the UX operator determines the number of high-quality genes passed from high-quality parents to offspring. In the early stages of evolution, a larger η helps the algorithm converge. However, in the later stages of evolution, a larger η will cause poor population diversity, causing the algorithm to fall into a local optimum. Therefore, this embodiment designs a Q-learning-based UX operator parameter adaptive adjustment strategy to dynamically adjust the evolution of the population, so that the algorithm can converge to the Pareto frontier more efficiently.

[0146] 1) Status: The population is in different states of convergence and diversity during the evolution process. The status of the entire population is determined by judging the convergence index ΔGD of the non-dominated solution set and the reference set and the diversity index ΔRatio of the coverage solution space ratio, where ΔGD represents the difference between the non-dominated solution and the Pareto frontier PF obtained in the t+1 generation compared with the t generation. * The difference in generation distance, PF * is the preset reference point, ΔRatio represents the proportion of the sub-region occupied by the contemporary non-dominated solution in the target space, and the sub-region represents the division of the target space into N non-overlapping regions by N weight vectors. ΔGD represents the degree of convergence between the t+1th generation and the tth generation. The smaller the ΔGD, the better the convergence. ΔRatio represents the degree of change in population diversity. ΔRatio < 0 indicates that the population diversity decreases, and ΔRatio > 0 indicates that the population diversity increases. The calculation methods of ΔGD and ΔRatio are respectively formula (24) and formula (25), where |PF t |、|ψ t |represents the number of non-dominated solutions in the tth generation and the proportion of the target space occupied. In the evolution process, there are four states: 1) ΔGD>=0, ΔRatio>0; 2) ΔGD>=0, ΔRatio<=0; 3) ΔGD<0, ΔRatio>0; 4) ΔGD<0, ΔRatio<=0;

[0147]

[0148] ΔRatio=|ψ t+1 |-|ψ t | (25)

[0149] 2) Action: The action set A consists of 10 actions, i.e., 10 level values ​​of the UX operator parameters: Α={0.30,0.35,0.40,0.45,0.5,0.55,0.6,0.65,0.7,0.75}. By judging the current state of the population, the action with the largest Q value in the Q table is selected according to the ε-greedy strategy.

[0150] 3) ε-greedy: Since the agent is prone to fall into the local optimum when learning without prior knowledge of Q value in the initial iteration process, the ε-greedy strategy based on sigmoid is adopted for the agent to learn. The mathematical expression of ε-greedy is as follows:

[0151]

[0152] 4) Reward: By calculating ΔGD and ΔRatio after executing an action, the agent will receive the corresponding reward and update the Q value of the action in the corresponding state of the Q table according to formula (27). If ΔGD>=0, ΔRatio>0,; if ΔGD>=0, ΔRatio<=0, reward=-2; if ΔGD<, ΔRatio>0, reward=4; if ΔGD<0, ΔRatio<=0, reward=2.

[0153]

[0154] Among them, α and γ are learning rate and discount factor respectively, and A is the optional action set.

[0155] Figure 4 An example is given. In the tth generation, the population is in state S3 and the action selected is A2. After executing this action, the population changes to state S3 in the t+1th generation. The action selected in this state is A 10 , the state of the t+2 generation population is converted to S4.

[0156] In order to achieve rapid population convergence, a retention strategy with elites is used to select colonies that need to be retained after the assimilation process. k Individuals generated during the assimilation process are placed into the colony offspring population k After the assimilation operation is completed within the empire, the original colony population and the colony offspring population are merged into a new population SubPop k , all SubPop k SubPop after individual merging k The individuals in the non-dominated sorting and crowding calculation are performed, and NC is selected kThe number of colonies is selected as the next generation of colonies. At the same time, the greedy strategy is used to select the assimilation results of the colonial countries, that is, for the new solution generated by the assimilation of the colonial countries, if it is not dominated by the old solution, the new solution will replace the original colonial country, otherwise no replacement will be made.

[0157] 3.3 Revolution of Hyper-heuristic Variable Neighborhood Search

[0158] Colonial revolution is the process of further searching and mining the neighborhood of the better colonies after the assimilation operation is completed. This embodiment designs multiple neighborhood search operations, and uses high-level information constructed by a two-dimensional probability matrix to guide the selection of the corresponding low-level variable neighborhood search operation group. The high-level strategy dynamically adjusts the selection of variable neighborhood operation operators through the feedback information given by the environment after each use of the low-level variable neighborhood search operation group, thereby improving the search performance of the algorithm.

[0159] In order to implement an effective perturbation strategy for individuals, this embodiment designs two neighborhood operations based on single processes and five neighborhood operations based on process blocks as neighborhood search operators.

[0160] NS1: Individual code single process insertion operation. Randomly select a process from the individual code and insert the process into other positions of the individual code. NS2: Individual code single process exchange operation. Randomly select two different processes from the individual code and exchange their positions. NS3: Individual code reversal operation. Randomly select two positions from the individual code and reverse the process fragment between the two positions. NS4: Individual code fragment exchange operation. Select two process fragments from the individual code and exchange their positions. NS5: Individual code fragment insertion operation. Randomly select two process fragments from the individual code and insert the fragment into other positions of the individual code. NS6: Individual code fragment shuffle operation. Randomly select a process fragment in the individual code and randomly shuffle the processes in the fragment. NS7: Individual code fragment reread gene dispersion operation. Divide the individual code into several fragments and disperse the positions of the consecutively repeated genes in each fragment.

[0161] Define NS = {NS G,k (1),NS G,k (2),…,NS G,k (l)} is the neighborhood operation sequence performed by the kth individual in the Gth generation, where l represents the length of the variable neighborhood search. The two-dimensional probability model P shown in formula (28) is adopted. G (x,y) serves as a high-level strategy to guide the selection of low-level variable neighborhood search operator sequences.

[0162]

[0163] The initialization of the two-dimensional probability model and the update of the Gth generation are shown in Equation (29) and Equation (30) respectively.

[0164] P 0 (x,y)=1 / l,x,y=1,2,…,l (29)

[0165] P G (x,y)=(1+reward)×P G-1 (x,y),x=1,2,…,l,y=NS(1),NS(2),…,NS(l)(30)

[0166]

[0167] After completing the revolution operation, calculate the cost of each country in the empire, select the country with the largest cost as the new colonial country, and the original colonial country will become a colony. According to formula (31), P G (x,y) is updated;

[0168] 3.4 United Empire Invasion

[0169] This embodiment designs a joint empire competition mechanism to replace the standard empire competition operation, treating each empire as a different region of the population collaborative search solution space, and further strengthening the information sharing and interaction between empires on the basis of maintaining the collaborative evolutionary search of each empire, so as to improve diversity and algorithm search efficiency.

[0170] The specific operation of the united empire competition is as follows: the total cost of each empire group is calculated and ranked according to formula (32), where Emp k For the Empire Imp k The set of colonies owned by the empire, ξ is the ratio of the colonies to the total cost of the empire, set to 0.1. im -1 empire's colonies and random solutions are executed to generate new solutions by POX crossover to replace the worst empire's corresponding number of worst colonies. By uniting empires to compete, on the one hand, the co-evolution of empires can be maintained, and on the other hand, information sharing and interaction between empires can be strengthened and diversity can be improved, which is conducive to improving the search efficiency of the algorithm.

[0171]

[0172] Embodiment 4:

[0173] This example is the simulation experiment results and analysis of IICA-QL;

[0174] The IICA-QL programming mentioned in this example is implemented using MATLAB R2021a. Since no scholar has yet studied EEMsAJSP_ML, the test data is generated randomly. The problem scale is represented by O / J / A_M / T / K, where O / J / A represents the number of processing steps, the number of parts, and the number of assembly steps, respectively, and M / T / K represents the total number of processing machines, the total number of transport vehicles, and the total number of assembly machines, respectively. The experimental time of each scale is T (i) Determined according to the size of the specific problem, T (i) =O×M+J×T+A×K.

[0175] In order to comprehensively evaluate the convergence and diversity of the Pareto solution set obtained by the algorithm, the following four indicators are used to evaluate the algorithm performance.

[0176] 1) C-metric solution set coverage: A and B are the non-dominated solution sets obtained by algorithm A and algorithm B. C(A,B) represents the ratio of the number of solutions in solution set B that are dominated by at least one solution in solution set A to all solutions in solution set B. The larger the C(A,B) value is, the better the performance of A is.

[0177] 2) GD (generational distance): evaluates the approximation between the Pareto frontier PF obtained by the algorithm and the optimal Pareto frontier PF*, where PF* is the true non-dominated solution set among all non-dominated solution sets obtained by the algorithm. For each solution x belonging to PF, find the solution y in the closest PF* and calculate its Euclidean distance. GD is the average shortest Euclidean distance. The smaller its value, the better the convergence, and the closer the found PF is to the optimal Pareto frontier.

[0178] 4) HV (Hypervolume): The HV index is a unary index that represents the volume of the target space enclosed by the non-dominated solution set obtained by the algorithm and the reference point. The reference point is set to (1.01, 1.01). The hypervolume index is a comprehensive index for evaluating the convergence and diversity of the non-dominated solution set obtained by the algorithm. The larger its value, the better the overall performance of the algorithm.

[0179] In order to determine the optimal values ​​of the parameters in IICA-QL, the experimental design method was used to set the parameters. There are 7 parameters in IICA-QL, each parameter has three levels to choose from, and the orthogonal array L27 (37) is selected. The meaning of each parameter and its optional parameter level are shown in Table 2. A medium experimental scale (82 / 42 / 71_3 / 3 / 3) was selected for the experiment, and each parameter combination was run 10 times for a running time of T. (i)=O×M+J×T+A×K. The hypervolume index (HV) of each parameter combination after each run was calculated, and the average of 10 HVs was calculated as the response value (RV). In addition, the average RV and influence of each parameter level are shown in Table 3.

[0180] Table 2 Parameter level values

[0181]

[0182] Table 3 Average RVs and ranks of each parameter

[0183]

[0184]

[0185] In order to verify the effectiveness of the energy-saving strategy decoding trigger mechanism, IICA-QL (denoted as A) is compared with IICA-QL that executes the energy-saving strategy for Pareto solutions (IICA-QL-PFE, denoted as B) and IICA-QL that uses energy-saving strategy decoding for all individuals (IICA-QL-AE, denoted as C). With the same calculation time as the termination condition, the average values ​​of C-metric, GD and HV obtained by running each algorithm independently 10 times on 18 examples are shown in Table 4, and the bold indicates the optimal value of each indicator. The running time of each algorithm at each scale is recorded in Table 5.

[0186] From Table 4 and Figure 8 From the C-metric results, it can be seen that in the 18 test cases, C(A,B) is greater than C(B,A), and there are 17 cases where C(A,C) is greater than C(C,A), which means that most of the non-dominated solutions obtained by IICA-QL can dominate the non-dominated solutions obtained by IICA-QL-PFE and IICA-QL-AE; from the GD index and HV index, it can be seen that in most cases, the value of IICA-QL is better than that of IICA-QL-PFE and IICA-QL-AE, and the value of IICA-QL-AE is better than that of IICA-QL-PFE, indicating that IICA-QL is the best, followed by IICA-QL-AE, and IICA-QL-PFE is the worst, indicating that the energy-saving strategy decoding trigger mechanism improves the quality of algorithm search. In addition, from Table 5 and Fig. 9 It can be seen that the running speed of IICA-QL is comparable to that of IICA-QL-PFE, and is significantly better than that of IICA-QL-AE, indicating that the energy-saving strategy decoding trigger mechanism improves the efficiency of algorithm search. Therefore, the energy-saving strategy decoding trigger mechanism can effectively improve the performance of the algorithm.

[0187] Table 4 Comparison results between IICA-QL and IICA-QL-PFE / IICA-QL-AE algorithms

[0188]

[0189]

[0190] Table 5 Running speed results of IICA-QL and IICA-QL-PFE / IICA-QL-AE algorithms

[0191]

[0192]

[0193] In order to verify the effectiveness of dynamic adaptive adjustment of operator parameters based on Q-learning, IICA-QL (denoted as A) is compared with IICA-QL (IICA-nQL, denoted as B) without using this strategy, where the UX parameter is set to 0.55. With the same calculation time as the termination condition, each algorithm is independently run 10 times on 18 examples. The average values ​​of C-metric, GD and HV are shown in Table 6, and the bold indicates the optimal value of each indicator.

[0194] From Table 6 and Fig.10 From the C-metric results, it can be seen that in 18 test cases, C(A, B) is greater than C(B, A) in 17 cases, and C(A, B) shows an increasing trend, which means that most of the non-dominated solutions obtained by IICA-QL can dominate the non-dominated solutions obtained by IICA-nQL, and the advantage becomes more obvious as the scale increases. From the GD and HV indicators, it can be seen that the values ​​of IICA-QL are better than those of IICA-nQL in all cases, indicating that IICA-QL has better convergence and diversity than IICA-nQL. Therefore, the dynamic adaptive adjustment strategy of operator parameters based on Q-learning can effectively improve the performance of the algorithm.

[0195] Table 6 Comparison results between IICA-QL and IICA-nQL algorithms

[0196]

[0197]

[0198] Since there is no research on solving EEMsAJSP_ML, in order to verify the effectiveness of the algorithm, IICA-QL is compared with BPBMO, PSO-GA and KBOA, which are similar to EEMsAJSP_ML. The parameters of BPBMO, PSO-GA and KBOA are all set in the literature. The running time T of the four algorithms is determined according to the problem scale (i) Each algorithm was run independently 10 times, and the average value of each evaluation index was calculated. The C index, GD value, and HV of each algorithm under 18 problem scales are shown in Tables 7 and 8. The bold font indicates the optimal value of each index.

[0199] As can be seen from Table 7 and Figure 11, the GD and HV values ​​obtained by IICA-QL in all examples are better than those of BPBMO, PSO-GA, and KBOA. Specifically, the performance of IICA-QL in the GD index shows that it can find a solution very close to the Pareto optimal frontier in the solution space, indicating that IICA-QL has obvious advantages in convergence. From the HV index, it can be found that the value obtained by IICA-QL is much larger than that of the other three algorithms, indicating that IICA-QL can not only find high-quality solutions, but also maintain the diversity of solutions and cover a wider Pareto frontier area. Therefore, the effectiveness of IICA-QL in solving MsEEFAJSP_ML is verified.

[0200] Table 7 Comparison of GD and HV results between IICA-QL and BPBMO / PSO-GAKBOA algorithms

[0201]

[0202]

[0203] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A multi-objective optimization method for energy-saving scheduling in a multi-stage and multi-level assembly workshop, establishing a problem model of EEMsAJSP_ML, and solving the problem model through IICA-QL, characterized in that: The mathematical model of the EEMsAJSP_ML problem is established in stages, and a decoding strategy with an energy-saving trigger mechanism is designed based on the nature of the problem. The improved imperial competition algorithm IICA-QL based on Q-learning is designed to solve the problem model. The specific optimization method includes the following steps: S1: Establish a mathematical model of the problem, and establish a mathematical model of the EEMsAJSP_ML problem in the processing stage, transportation stage, and assembly stage, and construct the completion time C of the production system max and the total energy consumption TEC, while minimizing the three-stage completion time C max and total energy consumption TEC; S2: Codec design, according to the coupling nature between each stage of EEMsAJSP_ML, the processing procedures in the processing stage are encoded, the machine allocation in the processing stage and the heuristic decoding rules used in the subsequent transportation and assembly stages are decoded, and the energy-saving strategy decoding trigger mechanism is designed; S3: Initialize the empire, use heuristic rules to generate a certain number of individuals, and randomly generate the remaining individuals. Select N im Colonial countries, building the initial empire; S4: Construct an external archive set Ω and update the external archive set according to the non-dominated solution; S5: Assimilation based on adaptive adjustment of Q-learning operator parameters, selecting UX operator parameters according to the Q-table, then performing multi-directional collaborative assimilation in each empire and updating the Q-table, while retaining colonies according to the retention strategy with elites; S6: Based on the revolution of hyper-heuristic variable neighborhood search, the high-quality colonies within the empire execute the high-dimensional strategy constructed by the two-dimensional probability model to select the low-level neighborhood search operation sequence, and update the probability model according to the feedback information given by the environment after executing the selected neighborhood group; S7: Implement joint empire competition, achieve co-evolution of empires and strengthen information interaction and sharing among empires; S8: If the iteration termination condition is met, stop searching; otherwise, return to S5.

2. The multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop according to claim 1 is characterized in that: In S1, the maximum completion time of the system is the time it takes to complete the assembly of all products. Total energy consumption during processing Total energy consumption during transportation Total energy consumption during assembly In all three phases, the sorted set π(π=[π st1 ,π st2 ,π st3 ]∈π) to find an optimal sorting π * , so that the completion time C of the three stages is minimized at the same time max and the total energy consumption TEC.

3. The multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop according to claim 1 is characterized in that: In S2, the specific operations of designing the energy-saving strategy decoding trigger mechanism are: S2.1: The result of the scheduling scheme obtained by using the strategy-free decoding strategy and the main scheduling for the individual is C max (π) and TEC(π), and record the idle time; S2.2: Calculate the upper limit ES of energy consumption saved by adopting the energy-saving strategy of the scheduling scheme UB And energy consumption lower limit E LB ; AND LB =TEC(π)-ES UB ; S2.3: Determine the solution (C max (π),E LB ) and the contemporary non-dominated solution set. If it is not dominated by the contemporary non-dominated solution set or they are not mutually dominated, go to S2.5, otherwise go to S2.4; S2.4: Output the result C of the scheduling scheme using non-strategy decoding max (π) and TEC(π); S2.5: According to the product assembly sequence of the scheduling plan, each product is assembled in reverse order from back to front, and Determine whether the block transfer condition is met; If satisfied, then according to Update the completion time of the process, otherwise, determine the previous process of the process; S2.6: If all processes are judged, then use Determine the idle interval that meets the profit and loss time and shut down and output the result after the scheduling scheme executes the energy-saving strategy C max (π) and TEC(π)', update the non-dominated solution set, otherwise, return to S2.5; in, Indicates the process on machine m The idle time after that, l=a,a+1,…, Indicates that on machine k The end time of the insertable idle interval, express The next process; T off-on Indicates the time required for the machine to turn on and off, EC off-on Indicates the energy consumption of the switch. Indicates the energy consumption per unit time of the machine.

4. The multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop according to claim 1 is characterized in that: In S3, a variety of heuristic methods are used to mix and initialize the population. The product aggregation PA and the parts aggregation rule JA generate α×Ps national individuals respectively, and the remaining national individuals are randomly generated; according to N col =PS-N im NC k =round(Pow k ×N col ) Calculate the cost of each country in the initial population in turn i=1,2…,Ps,right After sorting in non-ascending order, select the N with the largest cost im Countries as N im The empire's colonial countries, other countries in the population are colonies, calculate the number of colonies N col , calculate the colonial country Imp k The normalized power of each colonial country is used to determine the number of colonies NC k , and finally randomly select a corresponding number of colonies to be divided into the corresponding empire population, and delete the selected colonies from the original colony population; IICA-QL National Pi i represents a solution to the problem, where i = 1, 2…, Ps, Ps is the size of the population, and all countries in the g-th generation constitute the g-th generation population Pop(g), that is, 5. The multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop according to claim 1 is characterized in that: In S5, the specific steps include: S5.1: Co-evolution operation using two different uniform crossover operators UX and two-point crossover operator TPX; S5.2: Status, the status of the entire population is determined by judging the convergence index ΔGD of the non-dominated solution set and the reference set and the diversity index ΔRatio of the coverage solution space ratio. ΔRatio=ψ t+1 -ψ t Where ΔGD represents the non-dominated solution and the Pareto frontier PF obtained in the t+1th generation compared with the tth generation * The difference in generation distance, PF * is the preset reference point, ΔRatio represents the ratio of the sub-region occupied by the contemporary non-dominated solution in the target space, and the sub-region represents dividing the target space into N non-overlapping regions through N weight vectors; where PF t , t They represent the number of non-dominated solutions of the tth generation and the proportion of the target space they occupy; S5.3: Action, the action set A consists of 10 actions. By judging the current state of the population, the action with the largest Q value in the Q table is selected according to the ε-greedy strategy; S5.4: ε-greedy, using the sigmoid-based ε-greedy strategy for the agent to learn. The mathematical expression of ε-greedy is as follows: S5.5: Rewards. By calculating the ΔGD and ΔRatio after executing an action, the agent will receive the corresponding rewards, and according to Update the Q value of the action in the corresponding state of the Q table, where α and γ are the learning rate and discount factor respectively, and A is the set of optional actions; S5.6: Colony renewal strategy, using a retention strategy with elites to select colonies that need to be retained after the assimilation process.

6. The multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop according to claim 1 is characterized in that: In S6, two single-process-based neighborhood operations and five process-block-based neighborhood operations are designed as neighborhood search operators. High-level information constructed by a two-dimensional probability matrix is ​​used to guide the selection of the corresponding low-level variable neighborhood search operation group. The high-level strategy dynamically adjusts the selection of the variable neighborhood operation operator through the feedback information given by the environment after each application of the low-level variable neighborhood search operation group.

7. The multi-objective optimization method for energy-saving scheduling of a multi-stage and multi-level assembly workshop according to claim 1 is characterized in that: In S7, the specific operations of the United Empire competition are: S7.1: Calculate and rank the total cost of each imperial group Among them, Emp k For the Empire Imp k The set of colonies owned, ξ is the coefficient of the proportion of colonies to the total cost of the empire; S7.2: The top N im -1 Empire's colonies are crossed with random solutions using POX to generate a new solution that replaces the corresponding number of the worst colonies in the worst empire.

Images