Intelligent flexible workshop multi-target AGVs integrated scheduling optimization method and system

Abstract

The invention discloses an intelligent flexible workshop multi-target AGVs integrated scheduling optimization method and system, and belongs to the technical field of intelligent manufacturing workshop scheduling optimization. The invention discloses an intelligent flexible workshop multi-target AGVs integrated scheduling optimization method. The method comprises the steps of S1, deconstructing a multi-target AGVs integrated scheduling feasibility into a population, and initializing the population; s2, sorting the initial population based on Pareto layering and the degree of congestion; s3, dividing the population into two equal populations Pup and Pdown according to the sequence; s4, optimizing the population Pup based on NSGAII to obtain a new population # imgabs0 # S5, optimizing the population Pdown based on Q-Learning to obtain a new population # imgabs1 # S6, screening elite individuals based on the Pup, the Pdown, the # imgabs2 # and the # imgabs3 #, and constructing a new population Pnext; s7, judging whether the improvement of the new population Pnext is finished or not; if yes, outputting an optimal solution set; and otherwise, turning the new population Pnext to S3.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent manufacturing workshop scheduling optimization, and specifically relates to a multi-objective AGVs integrated scheduling optimization method and system for an intelligent flexible workshop. Background Art

[0002] In intelligent manufacturing, the integrated scheduling of flexible operations and automated guided vehicles (AGVs) is a key component in improving production efficiency, reducing costs, and enhancing enterprise competitiveness. Flexible operations require production lines to quickly adapt to changes in product types and processes. AGVs, as automated logistics equipment, can achieve real-time collaboration with flexible operations through integrated scheduling, dynamically planning routes based on production needs and ensuring accurate and timely material delivery.

[0003] To achieve efficient, cost-effective, and sustainable development in a green and sustainable manner, smart workshops must simultaneously address multiple objectives, including total completion time, energy consumption, production costs, and product quality. Manufacturing companies must strike a balance between improving production efficiency and reducing operating costs. Therefore, multi-objective flexible workshop AGV integrated scheduling aligns with global green manufacturing trends and can lay the foundation for companies to gain a competitive advantage in future markets. Existing research has largely focused on total completion time as a single optimization objective, excessively focusing on production efficiency while ignoring the energy waste associated with frequent AGV scheduling and equipment idleness. Energy consumption has become a crucial metric that cannot be ignored in manufacturing system optimization. In actual production, machines and AGVs not only consume significant energy during operation but also generate significant static energy consumption when idle. Therefore, effectively reducing overall energy consumption while ensuring production efficiency is a key issue that intelligent scheduling systems urgently need to address. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a multi-objective AGVs integrated scheduling optimization method and system for an intelligent flexible workshop, which solves the problems in the prior art.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop includes the following steps:

[0007] S1, construct the feasible solution of multi-objective AGVs integrated scheduling into a population and initialize the population;

[0008] S2, sorting the initial population based on Pareto stratification and crowding;

[0009] S3, divide the population into two equal populations P according to the sortingup and P dwown ;

[0010] S4. Optimize the population P based on NSGAII up to obtain a new population

[0011] S5. Optimize the population P based on Q-Learning dwon to obtain a new population

[0012] S6. Screen elite individuals based on P up and P down , and to construct a new population P next ;

[0013] S7. Judge whether the improvement of the new population P next is completed; if so, output the optimal solution set; otherwise, transfer the new population P next to S3.

[0014] Furthermore, the population is divided into three types of subgroups: subgroup subP1 with random characteristics, heuristic subgroup subP2 with short completion time, and heuristic subgroup subP3 with low energy consumption;

[0015] The initialization rules for the three types of subgroups are as follows:

[0016] 1) Initialization of subgroup subP1: Randomly generate an operation processing sequence, and randomly select a feasible processing machine and a feasible AGV cart for each operation in the sequence;

[0017] 2) Initialization of subgroup subP2: Randomly generate an operation processing sequence, and select the machine with the shortest processing time for each operation in the sequence; Calculate the transportation time required to transport the operation from the current position to the assigned processing machine for each AGV cart, and assign the AGV with the shortest transportation time to the operation;

[0018] 3) Initialization of subgroup subP3: Randomly generate an operation processing sequence, and randomly select a feasible processing machine and a feasible AGV cart for each operation in the sequence; Randomly select two operations from the set of operations assigned to each processing machine, determine the positions i and j of the two operations in the processing sequence, and i < j, and exchange the operation at position j in the processing sequence with the operation at position i + 1.

[0019] Furthermore, the steps to optimize the population P based on NSGAII up to obtain a new population include:

[0020] S41, based on the binary bidding game from the population P up Select individual x i and x j ;

[0021] S42, set the crossover probability p c , the POX operator is used to cross the individual operation processing sequence, and the MPX operator is used to cross the individual machine selection sequence and the AGV selection sequence;

[0022] S43, set the mutation probability p m , the SIM operator is used to mutate the individual operation processing sequence, and the MRM operator is used to mutate the individual machine selection sequence and AGV selection sequence;

[0023] S44, adding new individuals to the new population

[0024] S45, judge the new population Is the improvement finished? If so, output the new population. Otherwise, go to S41.

[0025] Furthermore, based on Q-Learning, we optimize the population P down , get a new population The steps include:

[0026] S51, build a Q-Learning state-aware decision framework, including state space, action space, reward function and strategy;

[0027] S52, initialize the Q-table, the number of rows is the number of states in the state space, the number of columns is the number of actions in the action space, and all elements in the Q-table are set to 0;

[0028] S53, calculate population P down State_P down =(S 1t ,S 2t ,S 3t )

[0029] S54, using the selection operator from the population P down Select two individuals x from i and x j ;

[0030] S55, select action a in the action space based on the ε greedy strategy t ;

[0031] S56, for x i and x j Execute action a t Get the new individual x'i and x' j , join the new population

[0032] S57, calculate the population state State_P down =(S 1(t+1) ,S 2(t+1) ,S 3(t+1) );

[0033] S58, calculating reward R;

[0034] S59, Update

[0035] S510, determine the new population Is the improvement finished? If so, output the new population. Otherwise, go to S52.

[0036] Furthermore, the state space includes three dimensions: population completion time optimization degree Population energy consumption optimization and population dispersion Calculation formula for three-dimensional indicators:

[0037]

[0038] std_f(x i )=(std_f1(x i ),std_f2(x i ))

[0039]

[0040] Among them, f q (x i ) is individual x i The qth objective function value, maxf q and minf q They represent the qth target maximum function value and minimum function value of all individuals in the population, std_f q (x i ) represents individual x i The result after normalization of the qth objective function value; represents the g-th generation population P g The sum of the qth objective function values of all individuals, represents the mean individual in the population, represents the g-th generation population P g The average value of the qth objective function of all individuals in ; represents the g-th generation population P g The sum of all discrete distances between individuals; Indicates the degree of improvement of the g-th generation population compared to the initial population in the q-th objective function value, Indicates the degree of change in the distribution breadth of the g-th generation population compared to the initial population in the target space.

[0041] Furthermore, the action space is constructed from four perspectives: global search, local mining, energy consumption optimization, and total completion time optimization, including:

[0042] Action 1: Set the crossover probability p c and mutation probability p m , select two individuals to perform crossover mutation;

[0043] Action 2: Crossover probability p c Unchanged, increase the mutation probability p m ≥0.8, crossover mutation was performed on two individuals;

[0044] Action 3: Crossover probability p c Unchanged, set the mutation probability p m =0, only perform crossover operation on two individuals;

[0045] Action 4: Randomly select an operation in the individual operation processing sequence and replace its current processing machine with another available machine;

[0046] Action 5: Randomly select an operation in the individual operation processing sequence and replace its current AGV with another available AGV;

[0047] Action 6: Calculate the current load of each processing machine and each AGV; determine the processing machine with the smallest load, denoted as M minload ; Determine the AGV with the smallest load, denoted as V minload ; For the operation processing sequence assigned to M minload Each operation of M minload Is it an optional processing machine for the operation? If so, move the operation from the current machine to M minload ; For the operation processing sequence assigned to V minload The operation set randomly selects an operation and updates the AGV car to V minload ;

[0048] Action 7: Determine the critical path on the individual processing machines, identify the operations with the longest processing time on the critical path, and move them to machines with shorter processing time than the current ones;

[0049] Action 8: Determine the critical path on the individual processing machine, randomly select two adjacent operations on the critical path, and swap the processing order of the operations;

[0050] Action 9: Determine the critical path on the individual processing machine, calculate the transport waiting time of all operations on the critical path, and replace the AGV car assigned to the operation with the longest transport waiting time with the AGV car with the least current load.

[0051] Furthermore, the reward function is:

[0052]

[0053] Where R is the reward function value, f q (x i ) is the parent individual x i The qth objective function value, f q (x' i ) is the new individual x' i The qth objective function value.

[0054] Furthermore, the Q-Learning state-aware decision framework adopts the ε-greedy strategy as the action selection mechanism:

[0055]

[0056] Where t and T represent the current iteration number and the total number of iterative optimizations, respectively, and ε t is the exploration rate of the tth iteration, ε max and ε min are the upper and lower bounds of the exploration rate ε, respectively.

[0057] An intelligent flexible workshop multi-objective AGVs integrated scheduling optimization system, including:

[0058] Initialization module: constructs the feasible solution of multi-objective AGVs integrated scheduling into a population and initializes the population;

[0059] Population sorting module: sorts the initial population based on Pareto stratification and crowding;

[0060] Population division module: divide the population into two equal populations P according to the sorting up and P down ;

[0061] The first population optimization module: Optimize the population P based on NSGAII up , get a new population

[0062] The second population optimization module: Optimize the population P based on Q-Learning down , get a new population

[0063] Population screening module: Based on Pup 、P down 、 and To screen elite individuals and construct a new population P next ;

[0064] And, judgment module: judge the new population P next Is the improvement finished? If so, output the optimal solution set; otherwise, the new population P next Go to the Population Segmentation module.

[0065] A computer storage medium stores a readable program, which, when run by a processor, can execute the above-mentioned intelligent flexible workshop multi-objective AGVs integrated scheduling optimization method.

[0066] An electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0067] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned intelligent flexible workshop multi-objective AGVs integrated scheduling optimization method.

[0068] Beneficial effects of the present invention:

[0069] 1. This paper adopts NSGAII to build a global optimization architecture. Through non-dominated sorting and congestion calculation, it effectively balances the conflict between the two objectives of total completion time and energy consumption, obtains a uniformly distributed Pareto optimal solution set, and provides diversified scheduling solutions for decision makers of intelligent manufacturing systems.

[0070] 2. The present invention utilizes the dynamic adaptive learning capability of Q-Learning to integrate important factors affecting the efficiency of the scheduling scheme during the iterative optimization process to construct a state-aware decision-making framework, incrementally update the Q-table in real time, and improve the stability and robustness of the intelligent manufacturing system.

[0071] 3. The present invention combines multi-objective optimization with reinforcement learning, which can achieve flexible switching between global and local in the process of searching complex solution spaces, promote the optimization of resource allocation while reducing system energy consumption, and help the intelligent manufacturing system develop in the direction of intelligence and greenness. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0073] Figure 1 Example diagram of the multi-objective flexible workshop AGVs integrated scheduling problem of the present invention

[0074] Figure 2 Flowchart of the multi-objective AGVs integrated scheduling optimization method of the present invention;

[0075] Figure 3 Q-Learning state space diagram of the present invention;

[0076] Figure 4 The C-index result graph of the present invention on the FJSP standard dataset;

[0077] Figure 5 The Pareto frontier of the present invention on the FJSP standard data set. DETAILED DESCRIPTION

[0078] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0079] Example 1

[0080] like Figure 2 As shown, a multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop includes the following steps:

[0081] S1, construct the feasible solution of multi-objective AGVs integrated scheduling into a population and initialize the population;

[0082] The population is divided into three subgroups: 1) subgroup subP1 with random characteristics, which is used to improve the dispersion of the search; 2) heuristic subgroup subP2 with short completion time, which is used to explore the solution set that minimizes the completion time; 3) heuristic subgroup subP3 with low energy consumption, which is used to explore the solution set that minimizes the energy consumption.

[0083] The number of individuals in each subgroup is distributed according to the ratio of X:Y:Z and initialized based on different rules, specifically:

[0084] 1) Subgroup subP1 initialization: randomly generate an operation processing sequence, and randomly select a feasible processing machine and a feasible AGVs for each operation in the sequence;

[0085] 2) Initialization of subgroup subP2: Randomly generate the operation processing sequence, and select the machine with the shortest processing time for each operation in the sequence; Calculate the transportation time required for each AGV to transport the operation from the current position to the assigned processing machine, and assign the AGV with the shortest transportation time to the operation;

[0086] 3) Initialization of subgroup subP3: Randomly generate the operation processing sequence, and randomly select a feasible processing machine and feasible AGVs for each operation in the sequence; In the set of operations assigned to each processing machine, randomly select two operations, determine the positions i and j of the two operations in the processing sequence, and i < j, and exchange the operation at position j in the processing sequence with the operation at position i + 1.

[0087] S2. Sort the initial population based on Pareto stratification and crowding degree;

[0088] The specific steps for sorting the initial population are as follows:

[0089] S21. Perform Pareto non-dominated sorting on the initial population to obtain the non-dominated layer where each individual is located, where font[0] is the optimal layer, font[1] is the second best, and so on;

[0090] S22. For each non-dominated layer font[i], determine the maximum value i and the minimum value of the n individuals included in the layer on the q-th objective function

[0091] S23. Sort the individuals in ascending order based on the q-th objective function value;

[0092] S24. Set the crowding degree of the first and last individuals after sorting on the q-th objective to ∞, that is

[0093] S25. Calculate the crowding degree of the remaining individuals on the q-th objective where and are the q-th objective function values of the (i + 1)-th and (i - 1)-th individuals after sorting respectively;

[0094] S26. For each individual i in the population, the total crowding degree is

[0095] S27. Sort each individual in ascending order based on the total crowding degree.

[0096] S3. Divide the initial population into two equal populations P up and P down ;

[0097] Among them, the initial population contains N individuals, the population P up and P down Contains The specific steps of division are:

[0098] S31, starting from the non-dominated optimal layer, sequentially add all individuals of font[i] to P up , after joining, determine the population P up Is the number of individuals in greater than If yes, execute S32; otherwise, continue to add the next non-dominated layer font[i+1] individuals;

[0099] S32, calculate the population P up The number of individuals num that still need to be added is

[0100] S33, select the first num individuals from the crowdedness sorted list of font[i] and add them to P up ;

[0101] S34, add the remaining individuals of the initial population to P down .

[0102] S4, optimize the population P based on NSGAII up , get a new population

[0103] Optimize population P based on NSGAII up , get a new population The steps include:

[0104] S41, based on the binary bidding game from the population P up Select individual x i and x j ;

[0105] Among them, from the population P up Two individuals x are randomly selected from i and x′ i , compare the non-dominated layers where the two individuals are located, if x i The non-dominated layer is better than x′ i The non-dominated layer of i , otherwise select individual x′ i If x i and x′ i In the same non-dominated layer, we further compare x i and x′ i The congestion degree, if x i The congestion degree is better than x′ iIf the congestion degree is such, then select individual x i , otherwise select individual x' i . Subsequently, select another individual x in the same way j .

[0106] S42. Set the crossover probability p c , and use the POX operator to perform crossover on the individual operation processing sequence, and use the MPX operator to perform crossover on the individual machine selection sequence and AGV selection sequence;

[0107] Among them, if the randomly generated probability is less than the crossover probability p c , then perform the crossover operation; the POX operator randomly extracts several operations from individual x i , and retains their relative order in the sequence; then insert the unextracted operations into the remaining positions in the x j individual; the MPX operator randomly selects multiple pairs of position points in the x i individual and the x j [[ID=​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​S45, judge the new population Is the improvement finished? If so, output the new population. Otherwise, go to S41;

[0113] Among them, the new population The number of individuals in the population will continue to increase during the iteration process. The number of individuals and the previous generation population P up When the number of individuals is the same, it means that a sufficient number of new individuals have been generated and the improvement can be ended.

[0114] S5, based on Q-Learning to optimize the population P down , get a new population

[0115] Optimize population P based on Q-Learning down , get a new population The steps include:

[0116] S51, build a Q-Learning state-aware decision framework, including state space, action space, reward function and strategy;

[0117] 1) State Space

[0118] The state space contains three dimensions, and the three dimensional indicators are used to measure the g-generation population P g Different states, namely the degree of optimization of population completion time Population energy consumption optimization Population dispersion The three-dimensional indicators are calculated as follows:

[0119]

[0120] std_f(x i )=(std_f1(x i ),std_f2(x i )) (3)

[0121]

[0122] Among them, f q (x i ) is individual x i The qth objective function value, maxf q and minf q They represent the qth target maximum function value and minimum function value of all individuals in the population, std_f q (x i ) represents individual x i The result after normalization of the qth objective function value. represents the g-th generation population P g The sum of the qth objective function values of all individuals, represents the mean individual in the population, represents the g-th generation population P g The average value of the qth objective function of all individuals in . represents the g-th generation population P g The sum of all discrete distances between individuals. Indicates the degree of improvement of the g-th generation population compared to the initial population in the q-th objective function value, Indicates the degree of change in the distribution breadth of the g-th generation population compared to the initial population in the target space.

[0123] Formula (1) is used to normalize the optimization objectives of two different scales. After all N individuals are normalized, the g-th generation population P is calculated using formula (2): g The sum of the two objective function values of total completion time and energy consumption is used to calculate the g-generation population P by formula (3). g Based on the above statistical information, the three-dimensional indicators of the state space are calculated by formula (4) and formula (5) respectively. and It is used to reflect the overall improvement of the current population compared to the initial population in terms of the two objectives, and is used to guide Q-Learning to identify the optimization direction of the population. It is used to characterize the distribution breadth of the population in the target space and to assist in determining whether the population has fallen into a local optimum or has lost diversity.

[0124] To facilitate Q-Learning in classifying and identifying population states, two partitioning parameters are set within the value ranges of the three dimensional indicators in the state space. The value range of each dimensional indicator is divided into three level areas: low, middle, and high. The division method is as follows:

[0125]

[0126] In formula (6), k1 and k2 are set as custom partition parameters, and in formula (7), m1 and m2 are set as custom partition parameters. Finally, the three-dimensional state space is divided into 27 finite state sets, each of which is represented by State_P g =(S1, S2, S3). The state here is for the population P g Abstract representation in three-dimensional state space. State_P in S53 down =(S 1t ,S 2t ,S 3t ) represents the population P downThe specific state obtained after the tth iteration.

[0127] 2) Action Space

[0128] The action space is constructed from four perspectives: global search, local mining, energy consumption optimization, and total completion time optimization. Specifically, it includes:

[0129] Action 1: Set the crossover probability p c and mutation probability p m , select two individuals to perform crossover mutation;

[0130] Action 2: Crossover probability p c Unchanged, increase the mutation probability p m ≥0.8, cross-mutation is performed on two individuals to enhance population diversity;

[0131] Action 3: Crossover probability p c Unchanged, set the mutation probability p m =0, only perform crossover operation on two individuals to avoid destroying high-quality genes;

[0132] The above actions (Action 1-Action 3) promote individual participation in recombination to produce a large number of new individuals by reasonably setting the crossover probability, exploring the unknown areas of the solution space. Further, by adjusting the mutation probability, we can maintain the excellent genes of existing individuals while trying to expand to more distant areas.

[0133] Action 4: Randomly select an operation in the individual operation processing sequence and replace its current processing machine with another available machine;

[0134] Action 5: Randomly select an operation in the individual operation processing sequence and replace its current AGV with another available AGV;

[0135] The above actions (action 4 and action 5) complete the detailed mining of the local environment of excellent individuals by replacing the processing machine and transport cart of the current individual.

[0136] Action 6: Calculate the current load of each processing machine and each AGV; determine the processing machine with the smallest load, denoted as M minload ; Determine the AGV with the smallest load, denoted as V minload ; For the operation processing sequence assigned to M minload Each operation of M minload Is it an optional processing machine for the operation? If so, move the operation from the current machine to M minload ; For the operation processing sequence assigned to V minload The operation set randomly selects an operation and updates the AGV car to V minload .

[0137] The above action (action 6) balances the energy consumption between machines and small workshops by transferring operations to machines and AGVs with the smallest loads, thereby reducing the total energy consumption of each unit.

[0138] Action 7: Determine the critical path on the individual processing machines, identify the operations with the longest processing time on the critical path, and move them to machines with shorter processing time than the current ones;

[0139] Action 8: Determine the critical path on the individual processing machine, randomly select two adjacent operations on the critical path, and swap the processing order of the operations;

[0140] Action 9: Determine the critical path on the individual processing machine, calculate the transport waiting time of all operations on the critical path, and replace the AGV car assigned to the operation with the longest transport waiting time with the AGV car with the least current load.

[0141] The above actions (Action 7-Action 9) attempt to reduce the total completion time of each individual task by exchanging the order of operations, adjusting the machine, and adjusting the AGV.

[0142] 3) Reward Function

[0143] The reward function in the Q-Learning state perception decision framework is shown in formula (8):

[0144]

[0145] Where R is the reward function value, f q (x i ) is the parent individual x i The qth objective function value, f q (x' i ) is the new individual x' i The qth objective function value.

[0146] Formula (8) sets rewards for the effectiveness of actions at four levels. If the new individual x' i Compared to the parent individual x i If both objective function values are improved at the same time, the maximum reward is set to 1; if there is improvement on one objective function value, the reward is 0.5; if the new individual does not improve on both objective values but explores a new position in the solution space, the reward is 0.05; if none of the above three effects are achieved, the reward is negative.

[0147] 4) Strategy

[0148] The Q-Learning state-aware decision framework adopts the ε-greedy strategy as the action selection mechanism, and sets the ε thresholds to ε max and ε min, then:

[0149]

[0150] Where t and T represent the current iteration number and the total number of iterative optimizations, respectively, and ε t is the exploration rate of the tth iteration, ε max and ε min They are the upper and lower bounds of the exploration rate ε respectively.

[0151] Formula (9) is the dynamic update mechanism for ε, where t and T represent the current iteration number and the total number of iterative optimizations, respectively; the maximum and minimum values are set. By controlling the gradual decay of ε during the learning process, the algorithm maintains a reasonable balance between global search and local mining. In the early stages of the algorithm, more random actions can be selected to train the Q-table. As the number of iterations increases, there is a greater probability of selecting the most appropriate action.

[0152] S52, initialize the Q-table, the number of rows is the number of states in the state space, the number of columns is the number of actions in the action space, and all elements in the Q-table are set to 0;

[0153] The Q-table is a table with 27 rows and 9 columns. Each row corresponds to a state in the state space, and each column corresponds to an action in the action space. The element in the i-th row and j-th column represents the reward that can be obtained after performing the j-th action in the i-th state.

[0154] S53, calculate population P down State_P down =(S 1t ,S 2t ,S 3t )

[0155] Among them, State_P down =(S 1t ,S 2t ,S 3t ) represents the population P down The specific state obtained after the tth iteration, S 1t ,S 2t Calculated by formula (6), S 3t Calculated by formula (7).

[0156] S54, using the selection operator from the population P down Select two individuals x from i and x j ;

[0157] Among them, from the population P down The process of selecting individuals is the same as that of S41.

[0158] S55, select action a in the action space based on the ε greedy strategy t ;

[0159] Among them, find the row corresponding to the current state of the population calculated in S53 in the Q table, select the action with the largest Q value in the row with a probability of 1-ε; and select other actions in the row with a probability of ε.

[0160] S56, for x i and x j Execute action a t Get the new individual x' i and x' j , join the new population

[0161] Among them, action a t is the action selected by S55 based on the ε-greedy strategy in the action space, x i and x j are the two individuals selected by S54 for x i and x j Execute action a t Then we will get two new individuals x' i and x′ j , add the new individuals obtained into the new population

[0162] S57, calculate the population state State_P down =(S 1(t+1) ,S 2(t+1) ,S 3(t+1) );

[0163] Among them, the population changes after the new individual S56 joins, and the population state needs to be recalculated according to formula (6) and formula (7).

[0164] S58, calculating reward R;

[0165] Among them, it can be seen from formula (8) that if the new individual x' i Compared to the parent individual x i If both objective function values are improved at the same time, the reward is 1; if one objective function value is improved, the reward is 0.5; if the new individual does not improve on both objective values compared to the parent individual but explores a new position in the solution space, the reward is 0.05; if none of the above three effects are achieved, the reward is negative.

[0166] S59, Update

[0167] Among them, Q(s t ,a t ) indicates that in the current state st Next select action a t The corresponding Q value; α represents the learning rate, with a value range of 0<α≤1, which controls the influence of the current feedback on the original Q value. The larger the value, the more dependent on the current experience; R represents the state s t Next, perform action a t The reward obtained later; γ represents the discount factor, which ranges from 0≤γ<1 and measures the importance of future rewards. The larger the value, the more emphasis is placed on long-term returns. Indicates that in the next state s t+1 In the , all optional actions correspond to the maximum Q value; the update formula combines the current reward and the future optimal return to make a weighted correction to the current Q value.

[0168] S510, determine the new population Is the improvement finished? If so, output the new population. Otherwise, go to S52;

[0169] Among them, the population The number of individuals will continue to increase during the iteration process. The number of individuals and the previous generation population P down When the number of individuals is the same, it means that a sufficient number of new individuals have been generated and the improvement can be ended.

[0170] S6, based on P up 、P down 、 and To screen elite individuals;

[0171] The screening process includes:

[0172] S61, based on P up 、P down 、 and All individuals in construct the population P al l ;

[0173] S62, for P all The individuals in the group are sorted by non-dominated layers, with font[0] being the best layer, font[1] being the second best, and so on;

[0174] S63, starting from the optimal non-dominated layer, sequentially add all individuals in the i-th non-dominated layer font[i] to P next , judge P next Is the number of individuals in font[i+1] greater than the population size N? If yes, execute S64; if no, continue to add the individuals in font[i+1] to P next ;

[0175] S64, calculate population P next The number of individuals num that needs to be added is num=(NP next Current number of individuals);

[0176] S65, calculate the crowding degree of each individual in font[i], sort them from large to small, select the first num individuals to join P next .

[0177] S7, determine the population P next Is the improvement finished? If so, output the optimal solution set; otherwise, next Go to S3;

[0178] In order to control the algorithm calculation cost and prevent invalid improvement, the maximum number of iterations T is set. After each population iterative improvement is completed, it is judged whether the current number of iterations t satisfies t>T. If yes, it means the improvement is completed and output P next As the final optimal solution set; otherwise, proceed to the next improvement. This ensures that the algorithm has good controllability and stable convergence.

[0179] Based on similar inventive concepts, an embodiment of the present invention also provides a computer storage medium storing a readable program, which, when executed by a processor, can execute the above-mentioned intelligent flexible workshop multi-objective AGVs integrated scheduling optimization method.

[0180] Based on similar inventive concepts, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0181] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned intelligent flexible workshop multi-objective AGVs integrated scheduling optimization method.

[0182] Based on similar inventive concepts, an embodiment of the present invention also provides a computer program product, including computer instructions, which instruct a computing device to perform operations corresponding to the above-mentioned intelligent flexible workshop multi-objective AGVs integrated scheduling optimization method.

[0183] Example 2

[0184] In this embodiment, the technical solution of the present invention is described through specific examples.

[0185] In this embodiment, the multi-objective flexible workshop AGVs integrated scheduling problem is as follows: Figure 1As shown in the figure, with the optimization goal of minimizing both total completion time and energy consumption, the pre-order relationship between different operations within the same job and the maximum load of the AGVs are considered to rationally allocate processing machines and transport vehicles to each operation. Due to the vast size of the problem solution space, it is impossible to find an optimal solution within an acceptable timeframe. Furthermore, various uncertainties exist in the actual operational environment, posing significant challenges to the solution. As shown in the figure, Scheduling Schemes A and B employ different methods to complete the transportation and processing of the operations, resulting in significant differences in total completion time and energy consumption between Solutions A and B.

[0186] In response to the above issues, Figure 2 A flow chart of the hybrid scheduling optimization method according to an embodiment of the present invention is given. The main idea is to combine the multi-objective optimization capability of NSGAII with the adaptive learning capability of Q-Learning to efficiently locate high-quality solutions in a complex and huge solution space. This method uses a population with diverse characteristics to initialize the configuration of individuals with different search capabilities. After non-dominated sorting and congestion calculation, the first half of the individuals are handed over to NSGA-II for global search, and the second half of the individuals are handed over to the Q-Learning module for adaptive learning. Subsequently, the parent population is merged with the new population optimized by NSGAII and Q-Learning to screen elite individuals, and continue to be iteratively improved as the next generation population. This method not only ensures the diversity of solutions and the continuous improvement of the Pareto frontier, but also can adaptively select adaptive actions based on the diverse state of the population, thereby improving the search efficiency and convergence quality of the algorithm in complex scheduling problems.

[0187] Figure 3 The state space diagram of Q-Learning in this embodiment is shown, and the three coordinate axes correspond to Three indicators. Each dimension is divided into three levels based on two custom boundary parameters. The combination of the three dimensions forms a state space containing 27 discrete state points, which assists Q-Learning in finely perceiving and classifying the population's progress on the two optimization objectives and the diversity of the solution set, thereby adaptively and dynamically deciding the most appropriate action. The points of different colors in the figure represent different state regions of the population, guiding Q-Learning to identify the population's evolutionary trends and the current optimization quality of the population. Specifically, when a state point is in the high-optimization, low-dispersion region (such as the points circled in red in the figure), it indicates that the algorithm is stuck in a local optimum and the population is converging too quickly, and diversity should be enhanced or local perturbations should be applied. On the other hand, when it is in the low-optimization, high-dispersion region (such as the points circled in blue in the figure), it indicates that the current evolution quality is poor and a strong convergence strategy can be used to accelerate progress. This state space model provides a clear and stable learning foundation for Q-Learning, enabling it to have stronger adaptability and policy control capabilities in complex scheduling problems.

[0188] In order to comprehensively evaluate the performance of the NSGA-II and Q-Learning hybrid optimization method (denoted as NSGA-II-Q) proposed in the present invention, this embodiment tests the algorithm on the FJSP standard data set, and selects the intermediate distance GD and C indicators to compare and analyze the results. GD measures the average distance between the obtained Pareto frontier and the true Pareto frontier. The smaller the value, the closer the overall result set is to the optimal frontier. The C indicator is used to measure the dominance relationship between the solution sets obtained by the two algorithms. The closer the value is to 1, the more advantageous the current solution set is in terms of non-domination. When calculating the GD indicator, the non-dominated solution set obtained by the NSGA-II-Q algorithm proposed in the present invention is used as the optimal reference Pareto frontier of GD, and the advantages and disadvantages of each algorithm are evaluated while ensuring relative fairness.

[0189] The GD test results of the three multi-objective optimization algorithms NSGA-II-Q, NSGA-II, SPEA2, and MOEA / D on the FJSP dataset are shown in Table 1 below;

[0190] Table 1 GD test results of four multi-objective algorithms on the FJSP dataset

[0191]

[0192]

[0193] The experimental results in Table 1 show that the GD values of the three algorithms on both datasets are not only consistently greater than 0, but also significantly larger in several test instances, causing the overall solution set to deviate significantly from the optimal frontier. NSGA-II-Q's advantage over the other three algorithms stems from the Q-Learning module's adaptive dynamic adjustment of the evolutionary strategy during the optimization process. This enables the algorithm to more effectively guide the solution set toward the optimal direction in complex scheduling environments, thereby improving the overall quality of the final solution set.

[0194] Figure 4The C-index comparison results on the FJSP dataset are further presented, comparing NSGA-II-Q with three multi-objective optimization algorithms: NSGA-II, SPEA2, and MOEA / D. Darker colors in the figure represent higher coverage of the solution set obtained by the NSGA-II-Q algorithm compared to the solution sets obtained by the three comparison algorithms. Values closer to 1 indicate a stronger dominance of the solution set obtained by the NSGA-II-Q algorithm compared to the solution sets obtained by the three comparison algorithms. As can be seen from the results, NSGA-II-Q achieved a coverage of 1 or close to 1 for all three comparison algorithms in all 10 groups of examples, particularly achieving 100% coverage compared to MOEA / D. This further demonstrates that the solution set obtained by NSGA-II-Q according to the present invention has significant advantages in both distribution quality and dominance.

[0195] Figure 5 The Pareto frontiers of the three multi-objective optimization algorithms NSGA-II-Q, NSGA-II, SPEA2, and MOEA / D on the three data instances FJSP1, FJSP5, and FJSP10 in the FJSP dataset are shown. Figure 5 (a) in the figure corresponds to the Pareto frontier distribution of the FJSP1 dataset. NSGA-II-Q (blue curve) outperforms the other compared algorithms in most solutions, particularly in the low makespan region. Its corresponding energy consumption is also lower, forming a more optimal Pareto frontier, demonstrating its excellent convergence and solution quality advantages for small-scale problems. Figure 5 Panel (b) shows the optimization results for the FJSP5 dataset, achieving denser non-dominated solutions close to the ideal region in both makespan and energy consumption. In contrast, the solution set of the MOEA / D algorithm is sparsely distributed and generally shifted to the upper right, indicating that it exhibits poor Pareto approximation performance on this problem. Figure 5 Panel (c) shows the performance of each algorithm on the FJSP10 dataset. NSGA-II-Q's solution set not only falls outside the frontier of the compared algorithms but also achieves lower makespan and energy consumption combinations in most cases, demonstrating its strong multi-objective collaborative optimization capabilities. In particular, when the energy consumption range is approximately 2600 to 2700, the solutions provided by NSGA-II-Q have a clear advantage in the makespan dimension, further demonstrating its robustness and superiority in solving large-scale, complex scheduling problems.

[0196] As can be seen from the figure, the Pareto solution set obtained by the NSGA-II-Q algorithm can achieve a good balance between the two objectives of completion time and energy consumption, and is superior to the three comparison algorithms in terms of solution quality, solution quantity, and solution diversity.

[0197] Example 3

[0198] Based on the multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop in Example 1, this embodiment proposes a multi-objective AGVs integrated scheduling optimization system for an intelligent flexible workshop, specifically including:

[0199] Initialization module: constructs the feasible solution of multi-objective AGVs integrated scheduling into a population and initializes the population;

[0200] Population sorting module: sorts the initial population based on Pareto stratification and crowding;

[0201] Population division module: divide the population into two equal populations P according to the sorting up and P down ;

[0202] The first population optimization module: Optimize the population P based on NSGAII up , get a new population

[0203] The second population optimization module: Optimize the population P based on Q-Learning down , get a new population

[0204] Population screening module: Based on P up 、P down 、 and To screen elite individuals and construct a new population P next ;

[0205] And, judgment module: judge the population P next Is the improvement finished? If so, output the optimal solution set; otherwise, next Go to the Population Segmentation module.

[0206] The method of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CDROM, RAM, floppy disk, hard disk or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It will be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0207] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention as claimed.

Claims

1. A multi-objective AGVs integrated scheduling optimization method for intelligent flexible workshops, characterized by: It includes the following steps: S1. Decompose the multi-objective AGVs integrated scheduling feasibility into a population and initialize the population; S2. Sort the initial population based on Pareto stratification and crowding degree; S3, divide the population into two equal populations P according to the sorting up and P down ; S4, optimize the population P based on NSGAII up , get a new population S5, based on Q-Learning to optimize the population P down , get a new population S6, based on P up 、P down 、 and To screen elite individuals and construct a new population P next ; S7, determine the new population P next Is the improvement finished? If so, output the optimal solution set; Otherwise, the new population P next Go to S3.

2. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 1 is characterized in that: The population is divided into three types of subgroups: subgroup subP1 with random characteristics, heuristic subgroup subP2 with short completion time, and heuristic subgroup subP3 with low energy consumption; The initialization rules for the three types of subgroups are as follows: 1) Initialization of subgroup subP1: Randomly generate an operation processing sequence, and randomly select a feasible processing machine and a feasible AGV cart for each operation in the sequence; 2) Initialization of subgroup subP2: Randomly generate an operation processing sequence, and select the machine with the shortest processing time for each operation in the sequence; Calculate the transportation time required to transport the operation from the current position to the assigned processing machine for each AGV cart, and assign the AGV with the shortest transportation time to the operation; 3) Initialization of subgroup subP3: Randomly generate an operation processing sequence, and randomly select a feasible processing machine and a feasible AGV cart for each operation in the sequence; In the operation set assigned to each processing machine, randomly select two operations, determine the positions i and j of the two operations in the processing sequence, and i < j, and exchange the operation at position j in the processing sequence with the operation at position i + 1.

3. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 1 is characterized in that: Optimize population P based on NSGAII up , get a new population The steps include: S41, based on the binary bidding game from the population P up Select individual x i and x j ; S42, set the crossover probability p c , the POX operator is used to cross the individual operation processing sequence, and the MPX operator is used to cross the individual machine selection sequence and the AGV selection sequence; S43, set the mutation probability p m , the SIM operator is used to mutate the individual operation processing sequence, and the MRM operator is used to mutate the individual machine selection sequence and AGV selection sequence; S44, adding new individuals to the new population S45, judge the new population Is the improvement finished? If so, output the new population. Otherwise, go to S41.

4. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 1 is characterized in that: Optimize population P based on Q-Learning down , get a new population The steps include: S51. Construct a Q-Learning state-aware decision framework, including a state space, an action space, a reward function, and a policy; S52. Initialize the Q-table, where the number of rows is the number of states in the state space and the number of columns is the number of actions in the action space, and set all elements in the Q-table to 0; S53, calculate population P down State_P down =(S 1t ,S 2t ,S 3t ) S54, using the selection operator from the population P down Select two individuals x from i and x j ; S55, select action a in the action space based on the ε greedy strategy t ; S56, for x i and x j Execute action a t Get the new individual x′ i and x j ' , join the new population S57, calculate the population state State_P down =(S 1(t+1) ,S 2(t+1) ,S 3(t+1) ); S58. Calculate the reward R; S59, Update S510, determine the new population Is the improvement finished? If so, output the new population. Otherwise, go to S52.

5. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 4 is characterized in that: The state space contains three dimensions: population completion time optimization degree Population energy consumption optimization and population dispersion Calculation formula for three-dimensional indicators: Among them, f q (x i ) is individual x i The qth objective function value, maxf q and minf q They represent the qth target maximum function value and minimum function value of all individuals in the population, std_f q (x i ) represents individual x i The result after normalization of the qth objective function value; represents the g-th generation population P g The sum of the qth objective function values of all individuals, represents the mean individual in the population, represents the g-th generation population P g The average value of the qth objective function of all individuals in ; represents the g-th generation population P g The sum of all discrete distances between individuals; Indicates the degree of improvement of the g-th generation population compared to the initial population in the q-th objective function value, Indicates the degree of change in the distribution breadth of the g-th generation population compared to the initial population in the target space.

6. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 4 is characterized in that: The action space is constructed from four perspectives: global search, local mining, energy consumption optimization, and total completion time optimization, including: Action 1: Set the crossover probability p c and mutation probability p m , select two individuals to perform crossover mutation; Action 2: Crossover probability p c Unchanged, increase the mutation probability p m ≥0.8, crossover mutation was performed on two individuals; Action 3: Crossover probability p c Unchanged, set the mutation probability p m =0, only perform crossover operation on two individuals; Action 4: Randomly select an operation in the individual operation processing sequence and replace its current processing machine with other available machines; Action 5: Randomly select an operation in the individual operation processing sequence and replace its current AGV cart with other available AGV carts; Action 6: Calculate the current load of each processing machine and each AGV; determine the processing machine with the smallest load, denoted as M minload ; Determine the AGV with the smallest load, denoted as V minload ; For the operation processing sequence assigned to M minload Each operation of M minload Is it an optional processing machine for the operation? If so, move the operation from the current machine to M minload ; For the operation processing sequence assigned to V minload The operation set randomly selects an operation and updates the AGV car to V minload ; Action 7: Determine the critical path on the individual processing machine, identify the operation with the longest processing time on the critical path, and move it to a machine with a shorter current processing time; Action 8: Determine the critical path on the individual processing machine, randomly select two adjacent operations on the critical path, and exchange the processing order of the operations; Action 9: Determine the critical path on the individual processing machine, calculate the transportation waiting time of all operations on the critical path, and replace the AGV cart assigned to the operation with the longest transportation waiting time with the AGV cart with the least current load.

7. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 5 is characterized in that: The reward function is: Where R is the reward function value, f q (x i ) is the parent individual x i The qth objective function value, f q (x′ i ) is a new individual x′ i The qth objective function value.

8. The multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop according to claim 4 is characterized in that: The Q-Learning state-aware decision framework adopts an ε-greedy strategy as the action selection mechanism: Where t and T represent the current iteration number and the total number of iterative optimizations, respectively, and ε t is the exploration rate of the tth iteration, ε max and ε min are the upper and lower bounds of the exploration rate ε, respectively.

9. An intelligent flexible workshop multi-objective AGVs integrated scheduling optimization system, characterized by: It includes: Initialization module: Decompose the multi-objective AGVs integrated scheduling feasibility into a population and initialize the population; Population sorting module: Sort the initial population based on Pareto stratification and crowding degree; Population division module: divide the population into two equal populations P according to the sorting up and P down ; The first population optimization module: Optimize the population P based on NSGAII up , get a new population The second population optimization module: Optimize the population P based on Q-Learning down , get a new population Population screening module: Based on P up 、P down 、 and To screen elite individuals and construct a new population P next ; And, judgment module: judge the new population P next Is the improvement finished? If so, output the optimal solution set; Otherwise, the new population P next Go to the Population Segmentation module.

10. A computer storage medium storing a readable program, characterized in that: When the program is run by the processor, it can execute the intelligent flexible workshop multi-objective AGVs integrated scheduling optimization method described in any one of claims 1-8.

11. An electronic device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the multi-objective AGVs integrated scheduling optimization method for an intelligent flexible workshop as described in any one of claims 1-8.

Images