Multi-objective optimization method for energy-saving scheduling of multi-stage multi-level assembly job shop

Abstract

The application aims at providing a multi-stage multi-level assembly job shop energy-saving scheduling multi-objective optimization method, relates to the technical field of assembly shop scheduling, and establishes a mathematical model of the problem in stages, and designs a problem-driven energy-saving strategy triggering mechanism according to the individual state, so that the quality and search efficiency of the solution are improved. Secondly, two heuristic rules and a random generation method are used to construct an initialization population that takes into account high quality and diversity, and a dynamic self-adaptive adjustment strategy for the assimilation operator parameters is realized through Q-learning, so that the convergence speed is improved while the population diversity is ensured, so that the exploration and mining ability of the algorithm is better balanced. A revolutionary operation guided by a hyper-heuristic variable neighborhood search is designed, and a high-quality colony found is searched in detail. The joint empire invasion operation is used to replace the competition, realizes the cooperative evolution and information interaction sharing of multiple empires, and thus finds non-dominated solutions with more uniform distribution.

Description

TECHNICAL FIELD

[0001] The present application relates to the related technical field of assembly shop scheduling, and in particular to a multi-stage multi-level assembly job shop energy-saving scheduling multi-objective optimization method. BACKGROUND

[0002] Multi-stage assembly job shop scheduling problem (MsAJSP) involves multiple stages from machining to assembly, and its model is more in line with reality, so it has attracted widespread attention. Current research simplifies products into single-level assembly structure or ignores product structure differences, which is only applicable to single-level or fixed-structure products. However, in actual production, products are usually complex assemblies composed of multiple components, with tree-shaped multi-level structure composed of components and sub-assemblies. In addition, as the number of product structure levels and manufacturing stages increases, the constraints become more complex, resulting in idle time between stages and within stages, resulting in a large amount of idle energy consumption. Therefore, the multi-stage multi-level assembly job shop energy-saving scheduling problem (EEMsAJSP_ML) for multi-level complex products needs further research.

[0003] The current method for handling multi-process multi-stage complex scheduling problems is relatively effective. There are two main operation methods: one is to only execute the block shifting strategy for the solutions in the non-dominated solution set to further reduce consumption, but this operation will lose a part of potential solutions, resulting in a local but not global optimal solution; the second is to embed the block shifting strategy into the individual decoding process to reduce consumption, but this will cause waste of computing resources and seriously slow down the search efficiency of the algorithm. Therefore, there is an urgent need to design an efficient block shifting strategy to improve the quality of the solution and search efficiency without losing the global solution.

[0004] However, the current Imperialist Competitive Algorithm (ICA) as an effective evolutionary algorithm framework, for the assimilation process of global search of the algorithm, the existing research mostly uses the uniform crossover operator (Uniform crossover, UX) to realize, due to the high complexity and large solution space of the EEMsAJSP_ML problem, the traditional static parameters may be difficult to guarantee the search efficiency and quality. Therefore, a reasonable parameter dynamic self-adaptive adjustment mechanism needs to be designed to balance the exploration and mining of the algorithm. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides the following technical solutions:

[0006] A multi-stage multi-level assembly job shop energy-saving scheduling multi-objective optimization method, the mathematical model of EEMsAJSP_ML problem is established in stages, and a decoding strategy with energy-saving trigger mechanism is designed based on the problem properties, an improved imperial competitive algorithm IICA-QL based on Q-learning is designed to solve the problem model, the specific optimization method includes the following steps:

[0007] S1: establish the mathematical model of the problem, establish the mathematical model of EEMsAJSP_ML problem in processing stage, transportation stage and assembly stage,

[0008] Total energy consumption of processing stage Release energy consumption for processing stage machines, Set energy consumption for processing stage machines, Working energy consumption for processing stage machines, Idle energy consumption for processing stage machines;

[0009] Total energy consumption of transportation stage

[0010] Total energy consumption of assembly stage Working energy consumption for assembly stage machines, Idle energy consumption for assembly stage machines;

[0011] The maximum completion time of the system is the assembly completion time of all products

[0012] Find an optimal order π * Among all three-stage sorting sets Π(π=[π st1 ,π st2 ,π st3 ]∈Π) so as to minimize the three-stage completion time C max And total energy consumption TEC;

[0013]

[0014] TEC(π)=EC st1 +EC st2 +EC st3 +EC aux

[0015]

[0016] Build production system completion time C max And total energy consumption TEC, minimize the three-stage completion time C max And total energy consumption TEC;

[0017] S2: Encoding and decoding design, according to the coupling properties between each stage of EEMs AJSP_ML, the processing stage is encoded for the processing stage, the decoding rule of the heuristic decoding rule adopted by the machine allocation of the processing stage and the subsequent transportation and assembly stage is decoded, and the decoding trigger mechanism of the energy saving strategy is designed, and the specific operation of the decoding trigger mechanism of the energy saving strategy is:

[0018] S2.1: The individual adopts a strategy-free decoding strategy and the master scheduling to obtain the result C of the scheduling scheme max (π) and TEC(π), and record the idle time;

[0019] S2.2: Calculate the upper limit of energy saving ES of the scheduling scheme after adopting the energy saving strategy UB and the lower limit of energy consumption E LB ;

[0020]

[0021] E LB = TEC(π)-ES UB ;

[0022] S2.3: Determine the relationship between the solution (C max (π), E LB ) and the current non-dominated solution set, if it is not dominated or mutually dominated by the current non-dominated solution set, go to S2.5, otherwise go to S2.4;

[0023] S2.4: Output the result C max (π) and TEC(π) of the scheduling scheme adopting the strategy-free decoding;

[0024] S2.5: According to the product assembly sequence of the scheduling scheme, from back to front, each product is processed in reverse order, and according to determine whether the block moving condition is met;

[0025] If it is met, update the completion time of the process according to

[0026]

[0027] the previous process of the process, otherwise, determine the previous process of the process;

[0028] S2.6: If all processes are determined, use to determine whether the idle interval shutdown of the profit and loss time is met and output the result C max (π) and TEC(π)' of the scheduling scheme after the energy saving strategy is executed, update the non-dominated solution set, otherwise, return to S2.5;

[0029] wherein, indicates the process In the free time afterwards, Indicates machine k End time of insertable idle intervals, express The immediate follow-up process; T off-on EC indicates the time required for the machine to power on and off. off-on Indicates the power consumption for starting and stopping the device. This indicates the machine's energy consumption per unit time.

[0030] S3: Initialize the empire. Generate a certain number of individuals using heuristic rules, and generate the remaining individuals randomly. Select N. im A colonial state, building the initial empire;

[0031] The population is initialized using a hybrid approach employing multiple heuristics. Individuals representing α×Ps of different countries are generated by the product aggregation rule (PA) and the part aggregation rule (JA), while the remaining individuals representing different countries are generated randomly.

[0032]

[0033] N col =PS-N im

[0034]

[0035] NC k =round(Pow k ×N col )

[0036] Calculate the cost for each country in the initial population sequentially. right After sorting in non-ascending order, select N with the highest cost. im N countries as N im Calculate the number of colonies N, where each empire is a colony of another population. col Calculate the Imp of the colonial state k The unified forces, and the number of colonies determined based on the unified forces of each colonial nation (NC) k Finally, a certain number of colonies are randomly selected and assigned to the corresponding imperial population, and the selected colonies are removed from the original colonial population.

[0037] IICA-QL National π i Let i represent a solution to the problem, where i = 1, 2, ..., Ps, and Ps is the population size. All countries in generation g constitute the population Pop(g) of generation g, i.e., Pop(g) = {π...} g,1 ,π g,2 ,…,π g,Ps};

[0038] S4: Constructing external archive set Ω and updating external archive set according to non-dominated solutions and dynamically updating in the iteration process;

[0039] S5: Based on assimilation of adaptive adjustment of Q-learning operator parameters, selecting UX operator parameters according to Q table, then performing multi-directional collaborative assimilation on each empire and updating Q-table, while preserving the colony according to the reservation strategy with elite, the specific steps are as follows:

[0040] S5.1: Adopting collaborative evolution operation of two different uniform crossover operators UX and two-point crossover operator TPX;

[0041] S5.2: State, judging the state of the whole population by judging the convergence index ΔGD of non-dominated solution set and reference set and the diversity index ΔRatio of coverage solution space ratio,

[0042]

[0043] ΔRatio=ψ t+1 |-ψ t |

[0044] Where ΔGD represents the difference of the generation distance of non-dominated solutions obtained in the t+1 generation compared with the t generation and the pareto front PF * , PF * is the preset reference point, ΔRatio represents the proportion of the non-dominated solutions obtained in the current generation in the sub-region of the target space, and the sub-region represents the N non-overlapping regions divided by N weight vectors; Where |PF t |, ψ t | represents the number of non-dominated solutions in the t generation and the proportion of the target space occupied, respectively.

[0045] S5.3: Action, the action set A consists of 10 actions, by judging the state of the current population, the action with the maximum Q value in the Q table is selected according to the ε-greedy strategy;

[0046] S5.4: ε-greedy, adopting sigmod-based ε-greedy strategy for agent learning, the mathematical expression of ε-greedy is as follows:

[0047]

[0048] S5.5: Reward, by calculating ΔGD and ΔRatio after executing an action, the agent will obtain the corresponding reward, and according to,

[0049]

[0050] updating the Q value of the action in the corresponding state of the Q table, where a and g are learning rate and discount factor respectively, and A is the set of available actions;

[0051] S5.6: colony update strategy, using elitist reservation strategy to select the colonies that need to be reserved after the assimilation process; merging the original colony population and the colony offspring population into a new population SubPop k The individuals generated in the assimilation process are put into the colony offspring population Offspring k After the assimilation operation is completed within the empire, the original colony population and the colony offspring population are merged into a new population SubPop k After merging all the SubPop k The individuals in SubPop k are non-dominantly sorted and crowdedness calculated, and NC k number of colonies are selected as the next generation of colonies. At the same time, the assimilation results of the colonies are selected using a greedy strategy, that is, if the new solution generated by the assimilation of the colonies is not dominated by the old solution, the new solution replaces the original colony, otherwise no replacement is made;

[0052] S6: revolution based on hyper-heuristic variable neighborhood search, high-dimensional strategy based on two-dimensional probability model is used to select low-level neighborhood search operation sequence in the high-quality colonies within the empire, and the probability model is updated according to the feedback information given by the environment after the selected neighborhood group is executed;

[0053] Define NS = {NS G,k (1), NS G,k (2), …, NS G,k (l)} as the neighborhood operation sequence executed by the kth individual in the Gth generation, where l represents the length of variable neighborhood search, and two-dimensional probability model P G (x, y) is used as a high-level strategy to guide the selection of low-level variable neighborhood search operator sequence;

[0054]

[0055] The initialization of the two-dimensional probability model and the update of the Gth generation are as follows:

[0056] P 0 (x, y) = 1 / l, x, y = 1, 2, …, l

[0057] P G (x, y) = (1 + reward) x P G-1 (x, y), x = 1, 2, …, l, y = NS(1), NS(2), …, NS(l)

[0058] After the revolution operation is completed, the cost of each country within the empire is calculated, the country with the highest cost is selected as the new colonial country, and the original colonial country becomes a colony, according to

[0059]

[0060] P G (x,y) is updated;

[0061] S7: Perform joint empire competition to realize empire co-evolution while strengthening information interaction and sharing between empires;

[0062] S7.1: Calculate the total cost of each empire group and rank

[0063]

[0064] Where Emp k is the colony set owned by the empire Imp k , and ξ is the proportion coefficient of the total cost of the empire occupied by the colony.

[0065] S7.2: Replace the worst colonies of the worst empire with the new solutions generated by POX crossover of the colonial countries of the top N im -1 empires and the random solution.

[0066] S8: If the iteration termination condition is met, stop the search, otherwise return to S5.

[0067] Compared with the prior art, the technical scheme of the present application has the following beneficial effects:

[0068] This invention proposes an improved Empire Competition Algorithm IICA-QL based on Q-learning to minimize the maximum completion time and total energy consumption. First, a mathematical model of the problem is established in stages, and a problem-driven energy-saving strategy triggering mechanism is designed based on individual states to improve the quality of solutions and search efficiency. Second, an IICA-QL for solving EEMsAJSP_ML is proposed. In IICA-QL, two heuristic rules and a random generation method are used to collaboratively construct an initial population that balances high quality and diversity. Q-learning is used to dynamically and adaptively adjust the parameters of the assimilation operator, improving convergence speed while maintaining population diversity, thus better balancing the algorithm's exploration and mining capabilities. A hyperheuristic variable neighborhood search-guided revolution operation is designed to conduct a detailed search of discovered high-quality colonies. A joint empire invasion operation replaces competition, enabling co-evolution and information sharing among multiple empires, thereby finding more evenly distributed non-dominated solutions. Specifically, compared with other algorithms, IICA-QL's performance on the GD metric shows that it can find solutions very close to the Pareto optimal front in the solution space, indicating a significant advantage in convergence. Furthermore, the HV metric shows that IICA-QL achieves a value much higher than the other three algorithms, indicating that IICA-QL not only finds high-quality solutions but also maintains solution diversity, covering a wider Pareto front region. Attached Figure Description

[0069] Figure 1 EEMsAJSP_ML diagram;

[0070] Figure 2 Encoding diagram;

[0071] Figure 3 Schematic diagram of the energy-saving strategy decoding trigger mechanism;

[0072] Figure 4 Q table update diagram;

[0073] Figure 5 Neighborhood structure diagram;

[0074] Figure 6 Algorithm flowchart;

[0075] Figure 7 Algorithm comparison of box plots for various indicators;

[0076] Figure 8 Algorithm running speed comparison chart;

[0077] Figure 9 Algorithm comparison of box plots for various indicators;

[0078] Figure 10IICA-QL vs. BPBMO / PSO-GA / KBOA comparative results box plot. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0080] Embodiment 1

[0081] This embodiment is the establishment of a mathematical model for EEMsAJSP_ML problem:

[0082] Table 1 Mathematical model annotation

[0083]

[0084]

[0085] The symbols used for the EEMsAJSP_ML mathematical model are shown in Table 1. The entire production process of the product is divided into three stages, which are: 1) processing stage: the processing workshop has N M machines, and the parts complete all processing procedures on the optional processing machines according to the processing route. 2) transportation stage: the parts that have completed all processing procedures are transported to the assembly workshop in batches by N T vehicles with a maximum load of Gmax. 3) assembly stage: the assembly workshop has NK assembly machines. The product is assembled on the N K assembly machines according to the assembly process after the parts belonging to the product are complete, thereby completing the entire production process of the product. The EEMsAJSP_ML model is shown in Figure 1 Fig. 1. The problem is assumed as follows:

[0086] 1) Any machine in the processing stage and the assembly stage can only process one procedure at the same time; once the processing procedure, the transportation process and the assembly procedure start, they cannot be interrupted; 2) all machines and vehicles are available at zero time; 3) the parts (products) can only enter the next procedure after the previous processing (assembly) procedure is completed; 4) the transportation time between two consecutive processing (assembly) procedures in the machines is not considered; 5) there can be the same parts between different products; 6) the same processing (assembly) procedure in the same part (product) is at most 1 time; 7) the vehicle maintains a uniform speed during the transportation process, and the load does not exceed the maximum load of the vehicle; 8) the machine first setup time is included in the workpiece release time.

[0087] The mathematical model for EEMsAJSP_ML with two optimization objectives and three stages is constructed as follows:

[0088] The maximum system completion time is the time required for all products to be assembled.

[0089]

[0090] The total energy consumption within the entire production system consists of four parts: 1) total energy consumption in the processing stage; 2) total energy consumption in the transportation stage; 3) total energy consumption in the assembly stage. The total energy consumption for each of these three stages is calculated below:

[0091] Processing Stage: The total energy consumption in this stage consists of the energy consumption of the processing machine. The energy consumption generated by the processing machine is composed of the machine's four states: released state, set-up state, working state, and idle state; the energy consumption of the processing machine in each state is constructed as follows:

[0092] Released energy consumption: refers to the machine's energy consumption before the first arrival of a workpiece on each machine. Machine released energy consumption during the processing phase. The calculation is as follows:

[0093]

[0094] Setting energy consumption: refers to the energy consumed when changing tooling fixtures for different workpieces before the machine enters working condition. Machine setting energy consumption during the processing stage. The calculation is as follows:

[0095]

[0096] Working energy consumption: refers to the energy consumed by a processing machine while it is in operation. Working energy consumption of the machine during the processing phase. The calculation is as follows:

[0097]

[0098] Idle energy consumption: refers to the energy consumed by a machine during the non-set time period between two consecutive processing steps. Idle energy consumption of a machine during the processing phase. The calculation is as follows:

[0099]

[0100] Total energy consumption during the processing stage (EC) st1 The calculation is as follows:

[0101]

[0102] 2) Transportation Phase: The total energy consumption in this phase consists of vehicle energy consumption. Vehicle energy consumption comprises two parts: loaded energy consumption and unloaded energy consumption. Total energy consumption EC in the transportation phase. st2 The calculation is as follows:

[0103]

[0104] 3) Assembly Stage: The total energy consumption in this stage consists of the energy consumption of the assembly machine. The energy consumption generated by the assembly machine is composed of two states: working state and idle state. The energy consumption of the assembly machine in each state is constructed as follows:

[0105] Operating energy consumption: refers to the energy consumption generated when the assembly machine is in operation. Operating energy consumption of the machine during the assembly phase. The calculation is as follows:

[0106]

[0107] Idle energy consumption: refers to the energy consumed by an assembly machine while waiting for the next assembly step. Idle energy consumption of machines during the assembly phase. The calculation is as follows:

[0108]

[0109] Total energy consumption during assembly phase (EC) st3 The calculation is as follows:

[0110]

[0111] Optimization objective: To sort the set Π(π=[π) in all three stages. st1 ,π st2 ,π st3 Find an optimal sorting π in ∈ π. * This minimizes the three-stage completion time C. max And total energy consumption (TEC):

[0112]

[0113] TEC(π)=EC st1 +EC st2 +EC st3 +EC aux (13)

[0114]

[0115] Example 2:

[0116] This embodiment is a codec design;

[0117] Based on the coupling nature between the stages of EEMsAJSP_ML, this embodiment only encodes the processing steps in the processing stage, and uses heuristic rules for decoding the machine allocation in the processing stage and subsequent stages.

[0118] In this paper, the encoding method based on process is used to represent the solution of the problem, and the process of the workpiece is represented by the number of the workpiece. The number of the workpiece is the number of the process. Since the product design order demand and the different products may have the same parts, it is necessary to read the product information once to determine the required workpiece and give it a unique number for identification. In order to better show the encoding process, take the problem of product 1 demand 2 and product 2 demand 1 as an example, where product 1 is composed of parts 1 and 2, product 2 is composed of parts 3 and 4, part 1 is composed of part 1, part 2 is composed of part 2 and part 3, part 3 is composed of part 2 and part 4, and part 4 is composed of part 5, part 6 and part 7. Each part is processed by corresponding process. The specific encoding method of this problem is shown in Figure 2 .

[0119] In order to reduce the solution space and find the near optimal solution of the problem, the heuristic decoding rules for machine allocation in processing stage and subsequent transportation and assembly stage are as follows:

[0120] 1) Machine allocation in processing stage: adopt the earliest completion time rule to allocate the process to the machine that can complete the process earliest, if there are multiple machines to complete the process at the same time, select the machine with the smallest energy consumption according to the minimum energy consumption priority rule;

[0121] 2) Transportation stage rule: under the premise of meeting the vehicle load constraint, adopt the rule of first completing work and then transporting to organize batch transportation of workpieces, that is, workpieces are loaded into available vehicles in the order of completion time, and the workpieces transported by a single vehicle cannot be overloaded, and subsequent workpieces are transported by subsequent vehicles according to the same rule;

[0122] 3) Assembly stage rule: when the parts transported to the assembly workshop can meet the product assembly, assemble according to the product assembly process, if the assembly conditions of multiple products are met at the same time, adopt the rule of least part product priority to determine the assembly sequence of the product, and when the product is assembled, the machine allocation of the assembly process adopts the same rule as the machine allocation in the processing stage to determine the assembly machine for the process;

[0123] The embodiment proposes an adaptive triggering mechanism of energy saving strategy, which adaptively judges the effect of executing energy saving strategy to ensure not to lose potential high quality solution while improving the search efficiency of algorithm; the specific operation is:

[0124] Step 1: obtain the result C of the scheduling scheme by using the non-strategy decoding strategy and the master scheduling for the individual max (π) and TEC(π), and record the idle time;

[0125] Step 2: obtain the upper limit of energy saving ES of the scheduling scheme after adopting the energy saving strategy according to formula (15) and formula (16) respectivelyUB and lower limit of energy consumption E LB ;

[0126] Step 3: Determine the solution (C) max (π),E LB If the solution is not dominated by the contemporary non-dominated solution set or they are mutually non-dominated, proceed to Step 5; otherwise, proceed to Step 4.

[0127] Step 4: Output the result C of the scheduling scheme using strategy-free decoding. max (π) and TEC(π);

[0128] Step 5: According to the product assembly sequence of the scheduling scheme, process each product in reverse order from back to front and check whether the block moving condition is met according to formula (17); if it is met, update the completion time of the process according to formula (18); otherwise, determine the previous process of the process.

[0129] Step 6: If all processes are judged, then use equation (19) to determine the idle interval shutdown that meets the profit and loss time and output the result C after the scheduling scheme has completed the energy-saving strategy. max If (π) and TEC(π)' are found, update the non-dominated solution set; otherwise, return to Step 5.

[0130]

[0131] E LB =TEC(π)-ES UB (16)

[0132]

[0133] in, Indicates the process on machine m In the free time afterwards, Indicates machine k End time of insertable idle intervals, express The immediate following process. T off-on Indicates the time required for the machine to power on and off, EC off-on Indicates the power consumption for starting and stopping the device. This represents the machine's energy consumption per unit time; the process of decoding and triggering the energy-saving strategy is as follows: Figure 3 As shown.

[0134] Example 3:

[0135] This example demonstrates IICA-QL for solving EEMsAJSP_ML;

[0136] 3.1 Initializing the Empire

[0137] In IICA-QL, the country π i Let i represent a solution to the problem, where i = 1, 2, ..., Ps, and Ps is the population size. All countries in generation g constitute the population Pop(g) of generation g, i.e., Pop(g) = {π} g,1 ,π g,2 ,…,π g,Ps To obtain a high-quality initial population while also considering population diversity, a combination of heuristics is used to initialize the population. Individuals representing α×Ps countries are generated using product aggregation and part aggregation rules, while individuals representing other countries are generated randomly. This approach ensures the algorithm has a high starting point for the search while also considering the breadth of the search.

[0138] Subsequently, the cost of each country in the initial population is calculated sequentially according to equation (20). right After sorting in non-ascending order, select N with the highest cost. im N countries as N im The number of colonies N is calculated according to formula (21), where each empire is a colony of another nation. col Then, based on equation (22), the Imp of the colonial state is calculated. k The unified forces, and the number of colonies determined based on the unified forces of each colonial nation (NC) k Finally, a certain number of colonies are randomly selected and assigned to the corresponding imperial population, and the selected colonies are removed from the original colonial population.

[0139]

[0140] N col =PS-N im (twenty one)

[0141]

[0142] NC k =round(Pow k ×N col ) (twenty three)

[0143] Before the iteration begins, an empty external archive set Ω is created to store the non-dominated set of the initial population, and is dynamically updated during the iteration process.

[0144] 3.2 Assimilation Based on Adaptive Adjustment of Q-learning Operator Parameters

[0145] In the assimilation process, the uniform crossover (UX) operator and the two-point crossover (TPX) operator are executed, which are applied to the process of moving the colony to the colonial state and the external archive set, respectively. The UX operator is used as the main evolution operator to guide the evolution process of the population, and can also be used as a local search, which plays an important role in the quality of population evolution. The size of the parameter η of the UX operator determines the number of high-quality genes passed from the parent to the offspring. In the early stage of evolution, a larger η helps the algorithm converge, but in the later stage of evolution, a larger η will cause poor population diversity, leading the algorithm to fall into a local optimum. Therefore, an adaptive adjustment strategy for the UX operator parameter based on Q-learning is designed to dynamically adjust the evolution process of the population, so that the algorithm can converge to the Pareto front more efficiently.

[0146] 1) State: The population is in different convergence and diversity states in the evolution process. The state of the whole population is determined by judging the convergence index ΔGD of the non-dominated solution set and the reference set and the diversity index ΔRatio of the coverage solution space ratio, where ΔGD represents the difference between the generation distance of the non-dominated solution obtained in the t+1 generation and the t generation and the Pareto front PF * , PF * is a preset reference point, and ΔRatio represents the proportion of the non-dominated solution obtained in the current generation in the sub-region of the target space. The sub-region represents that the target space is divided into N non-overlapping regions by N weight vectors. ΔGD represents the convergence degree of the t+1 generation compared with the t generation. The smaller ΔGD is, the better the convergence is. ΔRatio represents the change degree of population diversity. ΔRatio<0 indicates that the population diversity decreases, and ΔRatio>0 indicates that the population diversity increases. The calculation methods of ΔGD and ΔRatio are formula (24) and formula (25), respectively, where |PF t | and |ψ t | represent the number of non-dominated solutions in the t generation and the proportion of the target space occupied, respectively. In the evolution process, there are four states: 1) ΔGD>=0, ΔRatio>0; 2) ΔGD>=0, ΔRatio<=0; 3) ΔGD<0, ΔRatio>0; 4) ΔGD<0, ΔRatio<=0;

[0147]

[0148] ΔRatio=|ψ t+1 |-|ψ t | (25)

[0149] 2) Action: The action set A consists of 10 actions, i.e. 10 levels of UX operator parameter: A = {0.30, 0.35, 0.40, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75}, the action with the maximum Q value in the Q table is selected according to the ε-greedy strategy by judging the state of the current population.

[0150] 3) ε-greedy: Since the Q value is usually without prior knowledge in the initial iteration process, the agent is prone to fall into local optimum during learning, therefore the ε-greedy strategy based on sigmod is adopted for agent learning, the mathematical expression of ε-greedy is as follows:

[0151]

[0152] 4) Reward: The agent will obtain the corresponding reward by calculating the ΔGD and ΔRatio after executing an action, and updating the Q value of the action in the corresponding state of the Q table according to formula (27). If ΔGD >= 0, ΔRatio > 0, reward = 4; if ΔGD >= 0, ΔRatio <= 0, reward = -2; if ΔGD < 0, ΔRatio > 0, reward = 4; if ΔGD < 0, ΔRatio <= 0, reward = 2.

[0153]

[0154] Wherein, α and γ are learning rate and discount factor respectively, and A is the set of available actions.

[0155] Figure 4 An example is given, the population is in state S3 at the tth generation, the selected action is A2, after executing the action, the state of the population at the t+1th generation becomes S3, the selected action in this state is A 10 , the state of the population at the t+2th generation is converted to S4.

[0156] In order to realize the rapid convergence of population, the elitist reservation strategy is adopted to select the colonies that need to be reserved after the assimilation process. The specific operation is as follows: the individuals generated in the assimilation process of the empire Imp k are put into the colony offspring population Offspring k , after the assimilation operation is completed in the empire, the original colony population and the colony offspring population are merged into a new population SubPop k , all SubPop k individuals are merged, and the individuals in SubPop k are sorted and crowdedness calculated, and NC kThe number of colonies as the next generation of colonies. At the same time, the assimilation results of the colonizing country are selected using a greedy strategy, that is, if the new solution generated by the assimilation of the colonizing country is not dominated by the old solution, the new solution replaces the original colonizing country, otherwise no replacement is made.

[0157] 3.3 Revolution based on hyper-heuristic variable neighborhood search

[0158] The colony revolution is a process of further searching and mining the neighborhood of the better colony after the assimilation operation is completed. This embodiment designs various neighborhood search operations, uses high-level information constructed by a two-dimensional probability matrix to guide the selection of the corresponding low-level variable neighborhood search operation group, and dynamically adjusts the selection of the variable neighborhood operation operator through the feedback information given by the environment after each use of the low-level variable neighborhood search operation group, thereby improving the search performance of the algorithm.

[0159] In order to realize the effective perturbation strategy of the individual, this embodiment designs two single-process-based neighborhood operations and five process block-based neighborhood operations as neighborhood search operators.

[0160] NS1: Single-process insertion operation of individual code. Randomly select a process from the individual code, and insert the process into other positions of the individual code. NS2: Single-process exchange operation of individual code. Randomly select two different processes from the individual code to exchange positions. NS3: Reverse operation of individual code. Randomly select two positions from the individual code, and reverse the process fragments between the two positions. NS4: Fragment exchange operation of individual code. Exchange the positions of two process fragments in the individual code. NS5: Fragment insertion operation of individual code. Randomly select two process fragments from the individual code and insert the fragment into other positions of the individual code. NS6: Fragment disorder operation of individual code. Randomly select a process fragment in the individual code, and randomly disturb the processes in the fragment. NS7: Fragment re-reading gene dispersion operation of individual code. Divide the individual code into several fragments, and disperse the continuously repeated genes in each fragment.

[0161] Define NS = {NS G,k (1), NS G,k (2), …, NS G,k (l) as the neighborhood operation sequence performed by the kth individual of the Gth generation, where l represents the length of the variable neighborhood search. A two-dimensional probability model P G (x, y) shown in equation (28) is used as a high-level strategy to guide the selection of the low-level variable neighborhood search operator sequence.

[0162]

[0163] The initialization of the two-dimensional probability model and the update of the Gth generation are shown in equations (29) and (30), respectively.

[0164] P 0 (x,y)=1 / l,x,y=1,2,…,l (29)

[0165] P G (x,y)=(1+reward)×P G-1 (x,y),x=1,2,…,l,y=NS(1),NS(2),…,NS(l)(30)

[0166]

[0167] After completing the revolution operation, the cost of each country within the empire is calculated, the country with the highest cost is selected as the new colonial country, and the original colonial country becomes a colony. The P G (x,y) is updated according to equation (31);

[0168] 3.4 Joint Empire Invasion

[0169] This embodiment designs a joint empire competition mechanism to replace the standard empire competition operation. Each empire is regarded as a population searching different regions of the solution space. On the basis of maintaining the cooperative evolution search of each empire, the information sharing and interaction between empires are further strengthened to improve the diversity and search efficiency of the algorithm.

[0170] The specific operation of joint empire competition is as follows: the total cost of each empire group is calculated according to equation (32) and ranked, where Emp k is the colony set owned by the empire Imp k , ξ is the proportion coefficient of the total cost of the colony occupied by the empire, and is set to 0.1. The colonial countries of the top N im -1 empires are replaced by the new solutions generated by POX crossover with random solutions, and the worst colonies of the corresponding number of worst empires are replaced. Through joint empire competition, on the one hand, the cooperative evolution of each empire can be maintained, and on the other hand, the information sharing and interaction between empires can be strengthened and the diversity can be improved, which is beneficial to improve the search efficiency of the algorithm.

[0171]

[0172] Embodiment 4:

[0173] This embodiment is the simulation experiment results and analysis of IICA-QL;

[0174] The IICA-QL programming in this embodiment is implemented by MATLAB R2021a. Since no scholars have studied EEMsAJSP_ML at present, the test data is generated in a random manner. The problem size is represented by O / J / A_M / T / K, where O / J / A represent the number of processing procedures, the number of parts, and the number of assembly procedures, respectively, and M / T / K represent the total number of processing machines, the total number of transport vehicles, and the total number of assembly machines, respectively. The experimental time T (i) According to the specific problem size, T (i) = O x M + J x T + A x K.

[0175] In order to comprehensively evaluate the convergence and diversity of the Pareto solution set obtained by the algorithm, the following four indexes are used to evaluate the performance of the algorithm.

[0176] 1) C-metric solution set coverage rate: A and B are the non-dominated solution sets obtained by algorithm A and algorithm B, and C(A, B) represents the proportion of solutions in solution set B that are dominated by at least one solution in solution set A. The larger the value of C(A, B) is, the better the performance of A is.

[0177] 2) GD (generation distance): It evaluates the approximation degree of the pareto frontier PF obtained by the algorithm and the optimal pareto frontier PF*. PF* is the true non-dominated solution set in all non-dominated solution sets obtained by all algorithms. For each solution x belonging to PF, find the solution y in PF* closest to it and calculate its Euclidean distance. GD is the average shortest Euclidean distance. The smaller the value is, the better the convergence is, and the closer the PF found is to the optimal pareto frontier.

[0178] 4) HV (Hypervolume, hyper volume): HV index is a unary index, which represents the volume of the region surrounded by the non-dominated solution set obtained by the algorithm and the reference point. The reference point is set to (1.01, 1.01). The hyper volume index is a comprehensive index for evaluating the convergence and diversity of the non-dominated solution set obtained by the algorithm. The larger the value is, the better the comprehensive performance of the algorithm is.

[0179] In order to determine the optimal value of the parameters in IICA-QL, the experimental design method is used for parameter setting. There are 7 parameters in IICA-QL, and each parameter has three levels to be selected. The orthogonal array L27(37) is selected, and the meanings of the parameters and the levels of the parameters are shown in Table 2. The medium experimental size (82 / 42 / 71_3 / 3 / 3) is selected for testing, each parameter combination is run for 10 times, and the running time is T (i)= O x M + J x T + A x K. The hyper-volume index (HV) of each parameter combination after each run was calculated, and the average of 10 HVs was taken as the response value (RV). In addition, the average RVs and influence of each level of parameter are shown in Table 3.

[0180] Table 2 Parameter level values

[0181]

[0182] Table 3 Average RVs and ranks of each parameter

[0183]

[0184]

[0185] To verify the effectiveness of the energy-saving strategy decoding trigger mechanism, IICA-QL (denoted as A) was compared with IICA-QL with energy-saving strategy decoding for Pareto solutions (IICA-QL-PFE, denoted as B) and IICA-QL with energy-saving strategy decoding for all individuals (IICA-QL-AE, denoted as C). The average of C-metric, GD and HV obtained by each algorithm in 18 test cases independently run 10 times with the same calculation time as the termination condition is shown in Table 4, and the bold indicates the optimal value of each indicator. The running time of each algorithm in each scale is recorded in Table 5.

[0186] From the C-metric results of Table 4 and Figure 8 It can be seen that in 18 test cases, C(A, B) is greater than C(B, A), and C(A, C) is greater than C(C, A) in 17 cases, which indicates that the non-dominated solutions obtained by IICA-QL can dominate the non-dominated solutions obtained by IICA-QL-PFE and IICA-QL-AE in most cases. From the GD index and HV index, it can be seen that the values of IICA-QL are better than those of IICA-QL-PFE and IICA-QL-AE in most cases, and the values of IICA-QL-AE are better than those of IICA-QL-PFE, which indicates that IICA-QL is the best, followed by IICA-QL-AE, and IICA-QL-PFE is the worst, indicating that the energy-saving strategy decoding trigger mechanism improves the search quality of the algorithm. In addition, from Table 5 and Figure 9 It can be seen that the running speed of IICA-QL is comparable to that of IICA-QL-PFE, and significantly better than that of IICA-QL-AE, indicating that the energy-saving strategy decoding trigger mechanism improves the efficiency of the algorithm search. Therefore, the energy-saving strategy decoding trigger mechanism can effectively improve the performance of the algorithm.

[0187] Table 4 Comparison results of IICA-QL and IICA-QL-PFE / IICA-QL-AE algorithms

[0188]

[0189]

[0190] Table 5 Running speed results of IICA-QL and IICA-QL-PFE / IICA-QL-AE algorithms

[0191]

[0192]

[0193] To verify the effectiveness of the dynamic self-adaptive adjustment of operator parameters based on Q-learning, IICA-QL (denoted as A) is compared with IICA-QL without using this strategy (IICA-nQL, denoted as B), where UX parameter is set to 0.55. The average values of C-metric, GD and HV obtained by each algorithm in 18 test cases independently running 10 times are shown in Table 6, and the optimal value of each index is in bold.

[0194] From the C-metric results of Table 6 and Figure 10 It can be seen that in 17 of the 18 test cases, C(A, B) is greater than C(B, A), and C(A, B) shows a growing trend, which means that the non-dominated solutions obtained by IICA-QL can dominate the non-dominated solutions obtained by IICA-nQL in most cases, and the advantage is more obvious as the size increases. From the GD index and HV index, it can be seen that the values of IICA-QL are better than those of IICA-nQL on all cases, which shows that IICA-QL has better convergence and diversity than IICA-nQL. Therefore, the dynamic self-adaptive adjustment strategy of operator parameters based on Q-learning can effectively improve the performance of the algorithm.

[0195] Table 6 Comparison results of IICA-QL and IICA-nQL algorithms

[0196]

[0197]

[0198] Since there is no research on solving EEMsAJSP_ML, in order to verify the effectiveness of the algorithm, IICA-QL is compared with the algorithms BPBMO, PSO-GA and KBOA which are similar to EEMsAJSP_ML. The parameters of BPBMO, PSO-GA and KBOA are set as the parameters in the literature. The running time T of the four algorithms is determined according to the problem size (i) Each algorithm is independently run 10 times, and the average value of each evaluation index is calculated. The C index, GD value and HV of each algorithm under 18 problem sizes are shown in Tables 7 and 8, and the optimal value of each index is represented in bold.

[0199] As shown in Tables 7 and Figure 10 It can be seen that the GD value and HV value obtained by IICA-QL are better than BPBMO, PSO-GA and KBOA on all examples. Specifically, the performance of IICA-QL on the GD index shows that it can find solutions very close to the Pareto optimal front in the solution space, indicating that IICA-QL has obvious advantages in convergence. From the HV index, it can be found that the value obtained by IICA-QL is much larger than that of the other three algorithms, indicating that IICA-QL can not only find high-quality solutions, but also maintain the diversity of solutions and cover a wider Pareto front area. Therefore, the effectiveness of IICA-QL in solving MsEEFAJSP_ML is verified.

[0200] Table 7 Comparison of GD and HV results of IICA-QL with BPBMO / PSO-GA / KBOA algorithms

[0201]

[0202]

[0203] The preferred embodiments of the above disclosed application are only used to help explain the application. The preferred embodiments do not describe all the details, nor limit the application to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the specification. The embodiments are selected and described in detail in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and utilize the application. The application is limited by the claims and their entire scope and equivalents.

Claims

1. A multi-stage, multi-level assembly workshop energy-saving scheduling multi-objective optimization method, which establishes an EEMsAJSP_ML problem model and solves the problem model using IICA-QL, characterized in that... The mathematical model of the EEMsAJSP_ML problem is established in stages, and a decoding strategy with an energy-saving triggering mechanism is designed based on the problem properties. An improved Empire Competition Algorithm IICA-QL based on Q-learning is designed to solve the problem model. The specific optimization method includes the following steps: S1: Establish a mathematical model for the problem, dividing it into processing, transportation, and assembly stages. Develop a mathematical model for the EEMsAJSP_ML problem and construct the production system completion time. and total energy consumption Simultaneously minimize the three-stage completion time. and total energy consumption ; S2: Encoding and decoding design. Based on the coupling nature between the stages of EEMsAJSP_ML, the processing procedures in the processing stage are encoded, the machine allocation in the processing stage and the heuristic decoding rules used in the subsequent transportation and assembly stages are decoded, and an energy-saving strategy decoding trigger mechanism is designed. S3: Initialize the empire, using heuristic rules to generate a certain number of individuals, and randomly generate the remaining individuals. A colonial state, building the initial empire; S4: Construct an external archive set And update the external archive set based on the non-dominated solution; S5: Assimilation based on adaptive adjustment of Q-learning operator parameters. UX operator parameters are selected according to the Q-table. Multi-directional collaborative assimilation is then performed in each empire and the Q-table is updated. Meanwhile, colonies are retained according to the retention strategy with elites. S6: A revolution based on hyperheuristic variable neighborhood search. High-quality colonies within the empire execute a high-dimensional strategy constructed from a two-dimensional probability model to select a sequence of low-level neighborhood search operations, and update the probability model based on feedback information from the environment after executing the selected neighborhood group. S7: To execute joint imperial competition, achieve coordinated imperial evolution, and strengthen information exchange and sharing between empires; S8: If the iteration termination condition is met, stop the search; otherwise, return to S5. In step S2, the specific operation of designing the energy-saving strategy decoding trigger mechanism is as follows: S2.1: The result of the scheduling scheme is obtained by applying a strategy-free decoding strategy and a master scheduling method to the individual. and And record the free time; S2.2: Calculate the upper limit of energy savings after adopting the energy-saving strategy in this scheduling scheme. and lower limit of energy consumption ; ； ； S2.3: Determine the solution The relationship with the contemporary non-dominated solution set is as follows: if it is not dominated by the contemporary non-dominated solution set or they are mutually non-dominated, then proceed to S2.5; otherwise, proceed to S2.

4. S2.4: Output the result of the scheduling scheme using policy-free decoding. and ; S2.5: According to the product assembly sequence of this scheduling scheme, process each product in reverse order from back to front, and according to... Determine if the block shifting condition is met; If satisfied, then according to ； Update the completion time of the process; otherwise, check the previous process. S2.6: If all processes are judged, then adopt... Determine the idle interval that meets the profit / loss time requirement for shutdown and output the result after the energy-saving strategy of the scheduling scheme has been executed. and Update the non-dominated solution set; otherwise, return to S2.

5. in, Indicates the process on machine m In the free time afterwards, ; Indicates machine k End time of insertable idle intervals, express The immediate following process; This indicates the time required for the machine to turn on and off. Indicates the power consumption for starting and stopping the device. This indicates the energy consumption of the machine per unit of time.

2. The multi-stage, multi-level assembly workshop energy-saving scheduling multi-objective optimization method according to claim 1, characterized in that, In S1, the maximum system completion time is the time required for all products to be assembled. ; Total energy consumption during the processing stage ; Total energy consumption during the transportation phase ; Total energy consumption during assembly stage ; The sorting set in all three stages ( Find an optimal sorting This minimizes the completion time of all three stages simultaneously. and total energy consumption .

3. The multi-stage, multi-level assembly workshop energy-saving scheduling multi-objective optimization method according to claim 1, characterized in that, In S3, a hybrid approach using multiple heuristics is employed to initialize the population, generated separately by product aggregation rules (PA) and part aggregation rules (JA). Individuals from certain countries are generated randomly, while individuals from other countries are generated randomly; according to... ； ； ； ； Calculate the cost for each country in the initial population sequentially. , ,right After sorting in non-ascending order, select the one with the highest cost. National countries as Count the number of colonies of an empire whose colonies include other countries within the same ethnic group. Calculating colonial countries The unification forces, and the number of colonies to be divided based on the unification forces of each colonial power. Finally, a certain number of colonies are randomly selected and assigned to the corresponding imperial population, and the selected colonies are removed from the original colonial population. IICA-QL countries Represents a solution to the problem, where , For the size of the population, the first All countries constitute the first Generation population ,Right now .

4. The multi-stage, multi-level assembly workshop energy-saving scheduling multi-objective optimization method according to claim 1, characterized in that, In S5, the specific steps include: S5.1: Co-evolutionary operation using two different uniform crossover operators UX and two-point crossover operators TPX; S5.2: State, determined by the convergence index of the non-dominated solution set and the reference set. and the diversity index of the ratio of covered solution space To determine the state of the entire population, ； ； in Indicates the first generation and first The non-dominated solutions obtained by comparing the generation with the Pareto front The difference in generational distance, As a preset reference point, This represents the proportion of the sub-regions in the target space occupied by the non-dominated solutions obtained in the present time, where a sub-region is defined as N non-overlapping regions divided into the target space by N weight vectors; where , They represent the first The number of non-dominated solutions and the proportion of the target space they occupy; S5.3: Actions, Action Sets It consists of 10 actions, which determine the current state of the population and, based on... The greedy strategy selects the action with the largest Q value in the Q-table; S5.4: -greedy, using sigmoid-based... The -greedy strategy is used for agent learning. The mathematical expression for -greedy is as follows: ； S5.5: Reward, calculated after completing an action. and The intelligent agent will receive corresponding rewards, and based on... ； Update the Q-value of the action in the corresponding state of the Q-table, where, and These are the learning rate and the discount factor, respectively. For optional action sets; S5.6: Colony renewal strategy, which uses an elite retention strategy to select colonies that need to be retained after the assimilation process.

5. The multi-stage, multi-level assembly workshop energy-saving scheduling multi-objective optimization method according to claim 1, characterized in that, In S6, two neighborhood operations based on single processes and five neighborhood operations based on process blocks are designed as neighborhood search operators. High-level information constructed by a two-dimensional probability matrix is ​​used to guide the selection of the corresponding low-level variable neighborhood search operation group. The high-level strategy dynamically adjusts the selection of variable neighborhood operation operators based on the feedback information given by the environment after each application of the low-level variable neighborhood search operation group.

6. The multi-stage, multi-level assembly workshop energy-saving scheduling multi-objective optimization method according to claim 1, characterized in that, In S7, the specific operation of the Allied Empire Competition is as follows: S7.1: Calculate the total cost for each imperial group and rank them. ； in For the Empire The collection of colonies it possesses. The percentage of the total cost of the empire accounted for by the colonies; S7.2: Rank the top Each empire's colonies are cross-generated with random solutions using POX to replace the worst empire with the corresponding number of its worst colonies.

Images