A distributed heterogeneous hybrid flow shop scheduling method considering limited AGVs

Abstract

The application discloses a kind of distributed heterogeneous mixed flow shop scheduling method considering limited AGV, this method is first for the distributed heterogeneous mixed flow shop scheduling model considering limited AGV design fusion heuristic rule and random mechanism's hybrid initial population generation strategy;Then design global evolution mechanism based on NSGA-Ⅱ, realize global search to scheduling scheme;Finally design local search mechanism based on double-layer PPO cooperation, realize the fine development of high-quality scheduling scheme.The application is aimed at the problems such as difficult machine and AGV collaborative scheduling in distributed heterogeneous mixed flow shop, large-scale problem easy to premature convergence, carbon emission and completion time difficult to collaborative optimization, through the organic combination of hybrid initialization, global evolution and double-layer reinforcement learning local search, efficient multi-objective optimization is realized, and the collaborative scheduling efficiency of machine and AGV in distributed workshop and green production level in large-scale scene are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and production scheduling technology, and in particular to a distributed heterogeneous hybrid flow shop scheduling method that considers a limited number of AGVs. Background Technology

[0002] Driven by the iterative updates of automation technology and the deepening of global trade, manufacturing enterprises are facing immense pressure to deliver orders due to a surge in volume. To effectively address this challenge, the traditional centralized production model of a single factory is gradually transforming into a distributed manufacturing model with multi-factory collaboration. This distributed architecture can not only significantly improve the overall efficiency of the production line but also greatly expand capacity while balancing equipment load. However, in real-world industrial scenarios, each branch factory often exhibits significant heterogeneity in terms of equipment specifications, number of machines, and production environment. This constitutes the Distributed Heterogeneous Hybrid Flowshop Scheduling Problem (DHHFSP).

[0003] In traditional manufacturing workshops, when using manual labor or forklifts to move workpieces and materials, the handling time cannot be precisely controlled. Therefore, traditional workshop scheduling problems often ignore the handling time of workpieces and materials, or consider handling time within the processing time. With the continuous deepening of the intelligent manufacturing concept, multi-variety, small-batch, short-cycle, and personalized production models have become the norm. AGVs, with their advantages of high efficiency, reliability, energy saving, and strong environmental adaptability, have achieved automated distribution in workshop logistics, reducing production costs and improving production efficiency. More and more manufacturing workshops are using AGVs for material transportation. AGVs (Automated Guided Vehicles) enable more precise control of handling time, and in many application scenarios, transportation time accounts for a significant proportion and cannot be ignored. Therefore, it is necessary to simultaneously consider the machine resources for processing workpieces and the AGV resources for transporting workpieces, integrating and scheduling both to construct a highly flexible and efficient intelligent production system. Therefore, the rational allocation of transportation and handling tasks, the planning of AGV travel paths, and the collaborative cooperation between machines and AGVs—that is, the joint scheduling of AGVs and machines—become key aspects of the DHHFSP problem.

[0004] Hybrid flow shop scheduling is widely used in various fields such as automobile manufacturing, textiles, steel, and semiconductors. However, when solving complex combinatorial optimization problems with exact algorithms, the search space of hybrid flow shop scheduling problems typically has high dimensionality, especially as the number of workpieces increases, leading to exponential growth. This can cause the algorithm to encounter problems such as excessively long computation time and low search efficiency during global search. Therefore, approximate methods are used to solve the flow shop scheduling problem.

[0005] For large-scale problems, existing heuristic, metaheuristic, and reinforcement learning algorithms are widely used. Heuristic algorithms, relying on simple and intuitive rules, can generate scheduling schemes in a short time, based on specific priority principles, such as prioritizing tasks with shorter processing times. This approach can quickly obtain feasible solutions, but due to its relatively singular decision-making basis and lack of comprehensive exploration of the overall solution space, the quality of the generated scheduling schemes varies, making it difficult to guarantee approximation of the optimal solution. Metaheuristic algorithms, on the other hand, use unique search strategies to deeply explore different regions of the solution space within an acceptable timeframe, thereby obtaining high-quality near-optimal solutions. These algorithms can better balance solution quality and computation time when dealing with large-scale scheduling problems. In recent years, deep reinforcement learning has also been increasingly applied to solving scheduling problems. Deep reinforcement learning involves agents interacting with the environment, continuously trying and learning optimal strategies. Compared with traditional algorithms, deep reinforcement learning can dynamically adjust scheduling strategies according to the real-time state of the workshop, making it particularly suitable for dynamically changing production environments. However, deep reinforcement learning faces challenges such as long training times and the difficulty of handling large-scale state and action spaces. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a distributed heterogeneous hybrid flow shop scheduling method considering a limited number of AGVs. This method solves the problems of inconsistent scheduling scheme quality generated by existing algorithms when dealing with large-scale problems, difficulty in guaranteeing approximation of the optimal solution, long training time, and difficulty in handling large-scale state and action spaces.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] A distributed heterogeneous hybrid flow shop scheduling method considering a limited number of AGVs includes the following steps:

[0009] Step 1: Read the workshop status information and establish a distributed heterogeneous hybrid flow workshop scheduling model that considers a limited number of AGVs, with the optimization goal of minimizing the maximum completion time and carbon dioxide emissions. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model.

[0010] Step 2: Design a global evolution mechanism based on NSGA-II. Select outstanding individuals from the initial population as parent individuals through a binary tournament. Then, perform crossover and mutation operations on the parent individuals to generate offspring individuals. Subsequently, adopt an elite retention strategy to merge the parent and offspring into a double-sized population. Obtain Pareto front individuals through non-dominated sorting. Prioritize retaining individuals with low non-dominated levels to the rated population size. If the non-dominated levels are the same, prioritize retaining individuals with large crowding distances.

[0011] Step 3: Design a local search mechanism based on two-layer PPO collaboration. The main agent and sub-agents help individuals at the Pareto front select the most suitable neighborhood search operator at different levels, thereby realizing the refined development of elite solutions and effectively improving the convergence speed of the algorithm.

[0012] Furthermore, the workshop status information mentioned in step 1 includes: the number of workpieces, the number of stages, the number of factories, the number of AGVs, the number of machines in each stage of each factory, the machine load and idle energy consumption in each stage, the rotation speed of each machine, the standard processing time of each workpiece in each factory, the raw material area, the finished product area in each factory, and the location of each machine.

[0013] Furthermore, the constraints of the distributed heterogeneous hybrid flow shop scheduling model described in step 1 include:

[0014] (1) The machine and AGV can only serve one workpiece at a time;

[0015] (2) The processing and transportation of the workpiece must not be interrupted;

[0016] (3) The AGV transportation time depends on the physical distance between the machines;

[0017] (4) Before the workpiece is completed in the current stage and transported to the next stage by the AGV, it needs to wait in the current machine.

[0018] Furthermore, the initial population described in step 1 is generated in three parts:

[0019] First, the first batch of individuals is generated using the minimum time heuristic rule, greedily selecting the machine with the earliest completion time and the AGV with the shortest transportation time;

[0020] Secondly, the second part of the individuals is generated using the minimum carbon emission heuristic rule, prioritizing the combination of machines and AGVs with low processing energy consumption and short idle paths;

[0021] Finally, the third group of individuals is generated using a completely random method;

[0022] This hybrid strategy provides both a high-quality initial solution for the algorithm and ensures population diversity.

[0023] Each individual uses a four-layer coding structure: workpiece sorting (JS layer), factory allocation (FA layer), machine allocation (MA layer), and AGV allocation (AA layer).

[0024] Furthermore, a two-way decoding mechanism is implemented on individuals in the initial mixed population from step 1 to calculate their fitness values.

[0025] Furthermore, in step 2, the selection of excellent individuals is to perform non-dominated sorting on the individuals participating in the binary tournament, select individuals with low non-dominated levels as parent individuals, and implement a two-way decoding mechanism on the offspring individuals generated by the crossover and mutation operation on the parent individuals in step 2 to calculate their fitness values.

[0026] Furthermore, in step 3, in order to enhance the ability to discover elite solutions, a local search mechanism based on two-layer PPO collaboration is designed. This mechanism is achieved by agents working collaboratively at two levels: the global search layer and the factory-level search layer.

[0027] The global search layer consists of a master agent, which is mainly responsible for the local search of workpiece sequences and factory assignments. There are three search strategies, which correspond to the action set of the master agent.

[0028] The factory-level search layer deploys a sub-agent for each factory. Each sub-agent is mainly responsible for the local search of workpiece sequences, machine allocation, and AGV allocation within that factory. There are four search strategies, which correspond to the action set of each sub-agent.

[0029] Each level of the agent normalizes the current individual's old solution as the state input and dynamically outputs the action probability distribution to select the optimal strategy;

[0030] After generating a new solution through the aforementioned dynamic output actions, a bidirectional decoding mechanism is implemented for the new solution to calculate its fitness value. The reward value is calculated based on the dominance relationship between the new and old solutions and the improvement margin of the target value. Subsequently, the state transition quadruple is stored in the experience pool. The state transition quadruple consists of the input state (normalized from the old solution), the action performed, the reward value, and the resulting new state (normalized from the new solution). The agent is trained by randomly sampling batches of data to continuously optimize its decision-making capabilities.

[0031] The bidirectional decoding mechanism is as follows: based on the scheduling information determined by forward decoding, the completion time of all workpieces is determined, and reverse derivation is performed from the last processing stage to the first stage. If idle time is observed after the workpiece, a right shift strategy is executed without affecting other processing operations. The completion time of all workpieces cannot be later than the smaller of the time when the AGV starts transporting the workpiece and the start time of the next workpiece being processed by the current machine.

[0032] Another aspect of the present invention provides a computer-readable access medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0033] Step 1: Read the workshop status information and establish a distributed heterogeneous hybrid flow workshop scheduling model that considers a limited number of AGVs, with the optimization goal of minimizing the maximum completion time and carbon dioxide emissions. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model.

[0034] Step 2: Design a global evolution mechanism based on NSGA-II. Select outstanding individuals from the initial population as parent individuals through a binary tournament. Then, perform crossover and mutation operations on the parent individuals to generate offspring individuals. Subsequently, adopt an elite retention strategy to merge the parent and offspring into a double-sized population. Obtain Pareto front individuals through non-dominated sorting. Prioritize retaining individuals with low non-dominated levels to the rated population size. If the non-dominated levels are the same, prioritize retaining individuals with large crowding distances.

[0035] Step 3: Design a local search mechanism based on two-layer PPO collaboration. The main agent and sub-agents help individuals at the Pareto front select the most suitable neighborhood search operator at different levels, thereby realizing the refined development of elite solutions and effectively improving the convergence speed of the algorithm.

[0036] Another aspect of the present invention provides a processing apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the following steps:

[0037] Step 1: Read the workshop status information and establish a distributed heterogeneous hybrid flow workshop scheduling model that considers a limited number of AGVs, with the optimization goal of minimizing the maximum completion time and carbon dioxide emissions. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model.

[0038] Step 2: Design a global evolution mechanism based on NSGA-II. Select outstanding individuals from the initial population as parent individuals through a binary tournament. Then, perform crossover and mutation operations on the parent individuals to generate offspring individuals. Subsequently, adopt an elite retention strategy to merge the parent and offspring into a double-sized population. Obtain Pareto front individuals through non-dominated sorting. Prioritize retaining individuals with low non-dominated levels to the rated population size. If the non-dominated levels are the same, prioritize retaining individuals with large crowding distances.

[0039] Step 3: Design a local search mechanism based on two-layer PPO collaboration. The main agent and sub-agents help individuals at the Pareto front select the most suitable neighborhood search operator at different levels, thereby realizing the refined development of elite solutions and effectively improving the convergence speed of the algorithm.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] (1) This invention abandons the single random initialization method and adopts a hybrid strategy that combines minimum completion time priority, minimum carbon emission priority and random generation. This design not only provides the population with high-quality solutions on both objectives in the early stage of the algorithm, but also ensures the diversity of the population through random individuals, effectively preventing the algorithm from getting stuck in local optima in the early stage of evolution and accelerating the convergence speed.

[0042] (2) This invention decomposes the complex distributed scheduling problem into two levels: global coordination between factories and local optimization within factories. It successfully reduces the dimensionality of the massive joint action space, which would otherwise expand exponentially with the problem size, into two low-dimensional, decoupled independent action subspaces. Simultaneously, normalized state inputs ensure that the structure and computational load of the neural network are no longer affected by the expansion of production scale. This effectively avoids the curse of dimensionality that traditional deep reinforcement learning is prone to encounter when dealing with large-scale scheduling problems, significantly reducing the trial-and-error costs and computational overhead of the agent. It fundamentally solves the technical bottlenecks of excessively long training times and difficulty in handling large-scale states and joint action spaces in existing technologies, enabling the algorithm to maintain extremely high solution efficiency even in large-scale industrial scenarios.

[0043] (3) By introducing a bidirectional decoding mechanism, the present invention reduces machine idle time through reverse decoding, thereby significantly reducing the total carbon emissions of the workshop while ensuring production capacity. Attached Figure Description

[0044] Figure 1 A Gantt chart is used to consider the scheduling problem of a distributed heterogeneous hybrid flow shop with a limited number of AGVs;

[0045] Figure 2 Here is an example diagram of a four-layer coding structure;

[0046] Figure 3 Here is an example diagram of reverse decoding;

[0047] Figure 4 This is a schematic diagram of the main agent's action 1 in Example 1;

[0048] Figure 5 This is a schematic diagram of the main agent's action 2 in Example 1;

[0049] Figure 6This is a schematic diagram of the main agent's action 3 in Example 1, where Q1-Q5 represent 1-5 child agents respectively;

[0050] Figure 7 This is a schematic diagram of sub-agent action 1 in Example 1;

[0051] Figure 8 This is a schematic diagram of sub-agent action 2 in Example 1;

[0052] Figure 9 This is a schematic diagram of sub-agent action 3 in Example 1;

[0053] Figure 10 This is a schematic diagram of sub-agent action 4 in Example 1;

[0054] Figure 11 This is a flowchart of the local search process;

[0055] Figure 12 Here is the overall algorithm flowchart;

[0056] Figure 13 The results show the Pareto front comparison between the algorithm of the present invention in Example 1 and five advanced algorithms in a 10-workpiece scale example.

[0057] Figure 14 The results show the Pareto front comparison between the algorithm of the present invention in Example 1 and five advanced algorithms in a 20-workpiece scale example.

[0058] Figure 15 The results show the Pareto front comparison between the algorithm of the present invention in Example 1 and five advanced algorithms in a 40-workpiece scale example.

[0059] Figure 16 The results show the Pareto front comparison between the algorithm of the present invention in Example 1 and five advanced algorithms in a scalar example with 60 workpieces;

[0060] Figure 17 The results show the Pareto front comparison between the algorithm of the present invention and five advanced algorithms in Example 1 on a scale of 80 workpieces. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0062] Example 1

[0063] A distributed heterogeneous hybrid flow shop scheduling method considering a limited number of AGVs includes the following steps:

[0064] Step 1: Read the workshop status information, specifically including: number of workpieces, number of stages, number of factories, number of AGVs, number of machines in each stage of each factory, machine load and idle energy consumption in each stage, speed of each machine, standard processing time of each workpiece in each factory, raw material area, finished product area in each factory, and location of each machine. With minimizing the maximum completion time and carbon dioxide emissions as the optimization objective, establish a distributed heterogeneous hybrid flow workshop scheduling model considering a limited number of AGVs. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model.

[0065] In this embodiment, 30 test instances of different sizes were constructed, with the number of workpieces ranging from [number missing]. The range of the number of stages is... The range of the number of factories is... These three parameters together constitute Examples of different sizes. The number of AGVs in each factory depends on the factory size, using the formula... The calculation shows that, among which This represents the total number of machines in the factory. The standard processing time for each workpiece in each stage is uniformly generated from the interval [10, 90]. The processing speed factor for each machine ranges from... The processing speed conversion factor range is... The machine's operating power is uniformly generated from the interval [5,15], and its idle power is uniformly generated from the interval [1,3], both in kW. The AGV's load power and idle power are set to 5kW and kW respectively. The carbon emission factor is set at... The transport time of the AGV depends on the speed of the AGV and the distance between the machines. In this invention, the speed of the AGV is set to 1 m / s. The horizontal coordinates of each machine position are uniformly generated from the interval [5, 100], and the vertical coordinates are uniformly generated from the range [5, 80]. The distance between the machines is the Euclidean distance between these two coordinates. The following key constraints are considered: the machine and AGV can only serve one workpiece at a time; the workpiece processing and transport process cannot be interrupted; the AGV transport time depends on the physical distance between the machines; the workpiece must wait on the current machine before completing the current stage of processing and being transported to the next stage by the AGV.

[0066] The initial population was generated in three parts;

[0067] First, the first batch of individuals is generated using the minimum time heuristic rule, greedily selecting the machine with the earliest completion time and the AGV with the shortest transportation time;

[0068] Secondly, the second part of the individuals is generated using the minimum carbon emission heuristic rule, prioritizing the combination of machines and AGVs with low processing energy consumption and short idle paths;

[0069] Finally, the third group of individuals is generated using a completely random method;

[0070] This hybrid strategy provides both a high-quality initial solution for the algorithm and ensures population diversity.

[0071] A reasonable initialization can balance the quality of the solution and the diversity of the population. Specifically, this invention divides the initial population into three parts, employing the following strategies: minimum time priority strategy, minimum carbon emission priority strategy, and random generation strategy. The pseudocode for the initialization is shown in Algorithm 1:

[0072]

[0073] Each individual uses a four-layer coding structure: workpiece sorting (JS) layer, factory allocation (FA) layer, machine allocation (MA) layer, and AGV allocation (AA) layer, such as... Figure 2 As shown, the JS layer is [9, 6, 5, 2, 1, 8, 3, 4, 7], which means the processing order of the first stage is 9→6→5→2→1→8→3→4→7. The FA, MA, and AA layers represent the information of workpieces 1 to 9 from left to right, and this order remains unchanged in subsequent evolution processes. For example, workpiece 3 is assigned to factory 2, and the first stage is processed on machine 3. Workpiece 3 is transported from the raw material area to the machine via AGV1 for the first stage of processing. During encoding, the JS, MA, and AA layers correspond to the information of the first processing stage; the information for subsequent processing stages is determined during decoding.

[0074] A bidirectional decoding mechanism is implemented for individuals in the initial mixed population to calculate their fitness values. Specifically, the bidirectional decoding mechanism involves: based on the scheduling information determined by forward decoding, determining the completion time of all workpieces; proceeding in reverse order from the last processing stage to the first stage; if idle time is observed after a workpiece, a right-shift strategy is executed without affecting other processing operations. The completion time of all workpieces cannot be later than the smaller of the time the AGV begins transporting the workpiece and the start time of the next workpiece being processed by the current machine. Figure 3 As shown, there is idle time after workpiece 1 is processed on machine 1 in stage 2. Without affecting other workpieces, workpiece 1 is moved to the right, reducing the machine's idle time and thus reducing carbon dioxide emissions caused by idle time. The specific steps are shown in pseudocode in Algorithm 2:

[0075]

[0076]

[0077] Step 2: Design a global evolution mechanism based on NSGA-II. A binary tournament is used to select superior individuals from the initial population as parent individuals. The selection method involves ranking the participants in the binary tournament using non-dominance ranking, choosing individuals with lower non-dominance levels as parent individuals, and then performing crossover and mutation operations on the parent individuals to generate offspring individuals. Specifically:

[0078] Crossover operations: The process priority crossover (POX) operator is used for the JS layer to preserve the optimal process structure; the uniform crossover (UX) operator is used for the FA, MA, and AA layers.

[0079] Mutation operation:

[0080] A hybrid mutation strategy (random swapping or reverse mutation) is used for the JS layer.

[0081] Single-point mutations were applied to layers FA, MA, and AA.

[0082] A bidirectional decoding mechanism is implemented for offspring individuals generated by crossover mutation operations in parent individuals to calculate their fitness values. Then, an elite retention strategy is adopted to merge the parent and offspring into a double-sized population. Pareto front individuals are obtained through non-dominated sorting. Individuals with low non-dominated levels are retained up to the nominal population size. If the non-dominated levels are the same, individuals with large crowding distances are retained.

[0083] Step 3: Design a local search mechanism based on two-layer PPO collaboration. The main agent and sub-agents help the Pareto frontier individuals select the most suitable neighborhood search operators at different levels to achieve refined development of elite solutions and effectively improve the convergence speed of the algorithm. In order to enhance the ability to mine elite solutions, a local search mechanism based on two-layer PPO collaboration is designed. This mechanism is made up of agents working together at two levels: the global search layer and the factory-level search layer.

[0084] The global search layer consists of a master agent, which is mainly responsible for the local search of workpiece sequences and factory assignments. There are three search strategies, which correspond to the action set of the master agent.

[0085] Main agent action 1: Find the factory with the longest completion time, randomly select a job on the critical path, and replace it with the factory with the longest completion time, such as... Figure 4 As shown, factory 2 is the factory with the longest completion time, and workpiece 3 is the workpiece on its critical path. Therefore, workpiece 3 is replaced with factory 1, which has the shortest longest completion time.

[0086] Main agent action 2: Randomly select a job on the critical path of the factory with the longest completion time and swap it with a job on the non-critical path of the factory with the shortest completion time, such as... Figure 5 As shown, factory 2 is the factory with the longest completion time, factory 1 is the factory with the shortest completion time, workpiece 3 is the workpiece on the critical path of factory 2, and workpiece 5 is the workpiece on the non-critical path of factory 1. Then, change the processing factory of workpiece 3 and workpiece 5.

[0087] Action 3 of the main agent: Randomly select three adjacent genes at any position, execute all possible permutations, and finally select the best permutation, such as... Figure 6 As shown, three adjacent genes, namely workpiece 4, workpiece 1 and workpiece 3, are selected. There are 6 possible random permutations and combinations. The one that maximizes the overall performance of completion time and carbon dioxide emissions is selected from the 6 combinations.

[0088] The factory-level search layer deploys a sub-agent for each factory. Each sub-agent is primarily responsible for the local search of workpiece sequences, machine allocation, and AGV allocation within its respective factory. There are four search strategies, corresponding to the action set of each sub-agent, specifically:

[0089] Sub-agent action 1: Randomly select a workpiece on the critical path, replace its AGV with the earliest available AGV, and replace it with the machine that will complete it earliest, such as... Figure 7 As shown, workpiece 4 is a workpiece on the critical path. The machine for it is replaced with the earliest available machine 1, and the AGV for it is replaced with the earliest available AGV2.

[0090] Sub-agent action 2: Randomly select a job on a non-critical path and replace it with the machine that emits the least CO2, such as... Figure 8 As shown, workpiece 4 is a workpiece on a non-critical path, and its replacement machine is machine 1, which has the lowest CO2 emissions.

[0091] Sub-agent action 3: Select the workpiece with the latest completion time on the critical path, search backwards from the current workpiece, exchange its processing order with the workpiece with the shortest completion time, and assign the exchanged workpiece to the earliest available AGV, such as... Figure 9 As shown, workpiece 3 is the workpiece with the latest completion time on the critical path. Searching backward from workpiece 3, it is found that workpiece 4 has the shortest completion time. Therefore, the processing order of workpiece 3 and workpiece 4 is changed, and the AGV for workpiece 3 is replaced with the earliest available AGV2.

[0092] Sub-agent action 4: Select the workpiece with the latest completion time on the critical path, search backwards from the current workpiece, insert it before the workpiece with the shortest completion time, and assign the swapped workpiece to the earliest available AGV, such as... Figure 10 As shown, workpiece 3 is the workpiece with the latest completion time on the critical path. Searching backward from workpiece 3, it is found that workpiece 4 has the shortest completion time. Therefore, workpiece 3 is inserted before workpiece 4 for processing, and the AGV for workpiece 3 is replaced with the earliest available AGV2.

[0093] Each level of the agent normalizes the current individual's old solution as the state input and dynamically outputs the action probability distribution to select the optimal strategy;

[0094] To enable neural networks to handle scheduling problems of varying scales, a normalization process is used to construct the state vector S. t State vector S t It includes four dimensions: workpiece sequence, factory allocation, machine allocation, and AGV allocation.

[0095]

[0096] Where OS is the workpiece sequence, FA is the factory allocation sequence, MA is the machine allocation sequence, and AGV is the AGV allocation sequence; N, F, M, and K are the total number of workpieces, the total number of factories, the number of machines in the first stage of the current factory, and the total number of AGVs, respectively.

[0097] After generating a new solution by performing the above dynamic output actions, a bidirectional decoding mechanism is implemented on the new solution to calculate its fitness value. The reward value is calculated based on the dominance relationship between the new solution and the old solution and the improvement of the target value. Then, the state transition quadruple is stored in the experience pool. The state transition quadruple consists of the input state (normalized old solution), the action performed, the reward value, and the new state obtained (normalized new solution). The agent is trained by randomly sampling batch data to continuously optimize the agent's decision-making ability.

[0098] The reward value is calculated based on the dominance relationship between the new and old solutions and the improvement margin of the target value. Let f1 be the maximum completion time and f2 be the carbon dioxide emissions. The improvement rate is... , Divided into and ,

[0099] in, , , representing the improvement rates on the two objective functions respectively:

[0100]

[0101] This reward mechanism encourages agents not only to find non-dominated solutions but also to optimize the target value as much as possible. Through this design, the main agent eliminates global bottlenecks by migrating critical tasks; the sub-agents, from a microscopic perspective, perform refined repairs on specific factories, fully utilizing the idle time of equipment and AGVs. This hierarchical architecture decouples the originally huge joint action space into two low-dimensional spaces, effectively avoiding the curse of dimensionality in large-scale scheduling problems, allowing the algorithm to maintain efficient convergence in later stages. The pseudocode for the local search based on the two-layer PPO is shown in Algorithm 3, and the local search flowchart is as follows. Figure 11 As shown:

[0102]

[0103]

[0104] System Flow:

[0105] The algorithm mainly consists of three core stages. First, the initialization stage employs a hybrid strategy combining minimum time and minimum energy greediness with random generation to improve the quality of initial solutions while ensuring population diversity. Second, the global search relies on the NSGA-II framework, combining forward and reverse bidirectional decoding to reduce machine idle energy consumption, and uses multi-layer crossover operators and non-dominated sorting mechanisms to select high-quality offspring. Finally, the local search stage utilizes a two-layer PPO architecture, working collaboratively between the master agent's cross-factory load balancing and the sub-agents' in-factory resource optimization to deeply mine elite solutions. The algorithm pseudocode is shown in Algorithm 4, and the algorithm flowchart is as follows. Figure 12 As shown:

[0106]

[0107]

[0108] like Figure 13-17 The figure shows a comparison of the Pareto front results between the method of this invention and five existing advanced algorithms (DQNMMA, ILMPE, MOMPCEA, MONSII, and NSGA2MDDQN) under different workpiece sizes. The horizontal axis in the figure represents the maximum completion time, and the vertical axis represents the total carbon emissions.

[0109] pass Figure 13-17 The comparison curves of each subplot can be clearly observed:

[0110] 1. Significant Convergence Advantage: In all test cases, the method of this invention consistently ranked to the lower left of the curves of the other five comparative algorithms. This means that, under the same completion time, the method of this invention can achieve lower carbon emissions; or, under the same carbon emission constraints, it can achieve a faster completion time. This overall dominance over the solution sets of other algorithms fully demonstrates that the proposed two-layer PPO collaborative mechanism has extremely high accuracy and search capability in approximating the true Pareto front, solving the problem that existing technologies struggle to balance production efficiency and green indicators.

[0111] 2. Diversity of Distribution: In terms of curve coverage, the Pareto front generated by the method of this invention has a more uniform and widespread distribution. In contrast, the solution sets generated by other algorithms often cluster in certain local areas, or the curves exhibit discontinuities. The method of this invention can provide a series of continuous and smooth trade-off solutions, from extremely high-speed, high-energy-consumption to low-speed, low-energy-consumption, offering a richer and more flexible selection space for practical production decisions.

[0112] Example 2

[0113] A computer-readable access medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0114] Step 1: Read the workshop status information and establish a distributed heterogeneous hybrid flow workshop scheduling model that considers a limited number of AGVs, with the optimization goal of minimizing the maximum completion time and carbon dioxide emissions. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model.

[0115] Step 2: Design a global evolution mechanism based on NSGA-II. Select outstanding individuals from the initial population as parent individuals through a binary tournament. Then, perform crossover and mutation operations on the parent individuals to generate offspring individuals. Subsequently, adopt an elite retention strategy to merge the parent and offspring into a double-sized population. Obtain Pareto front individuals through non-dominated sorting. Prioritize retaining individuals with low non-dominated levels to the rated population size. If the non-dominated levels are the same, prioritize retaining individuals with large crowding distances.

[0116] Step 3: Design a local search mechanism based on two-layer PPO collaboration. The main agent and sub-agents help individuals at the Pareto front select the most suitable neighborhood search operator at different levels, thereby realizing the refined development of elite solutions and effectively improving the convergence speed of the algorithm.

[0117] Example 3

[0118] A processing apparatus includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the following steps:

[0119] Step 1: Read the workshop status information and establish a distributed heterogeneous hybrid flow workshop scheduling model that considers a limited number of AGVs, with the optimization goal of minimizing the maximum completion time and carbon dioxide emissions. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model.

[0120] Step 2: Design a global evolution mechanism based on NSGA-II. Select outstanding individuals from the initial population as parent individuals through a binary tournament. Then, perform crossover and mutation operations on the parent individuals to generate offspring individuals. Subsequently, adopt an elite retention strategy to merge the parent and offspring into a double-sized population. Obtain Pareto front individuals through non-dominated sorting. Prioritize retaining individuals with low non-dominated levels to the rated population size. If the non-dominated levels are the same, prioritize retaining individuals with large crowding distances.

[0121] Step 3: Design a local search mechanism based on two-layer PPO collaboration. The main agent and sub-agents help individuals at the Pareto front select the most suitable neighborhood search operator at different levels, thereby realizing the refined development of elite solutions and effectively improving the convergence speed of the algorithm.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A distributed heterogeneous hybrid flow shop scheduling method considering a limited number of AGVs, characterized in that, Includes the following steps: Step 1: Read the workshop status setting information, and establish a distributed heterogeneous hybrid flow workshop scheduling model that considers a limited number of AGVs with the optimization goal of minimizing the maximum completion time and carbon dioxide emissions. Design a hybrid initial population generation strategy that integrates heuristic rules and random mechanisms for the distributed heterogeneous hybrid flow workshop scheduling model. Step 2: Design a global evolution mechanism based on NSGA-II. Select outstanding individuals from the initial population as parent individuals through a binary tournament. Then, perform crossover and mutation operations on the parent individuals to generate offspring individuals. Subsequently, adopt an elite retention strategy to merge the parent and offspring into a double-sized population. Obtain Pareto front individuals through non-dominated sorting. Prioritize retaining individuals with low non-dominated levels to the rated population size. If the non-dominated levels are the same, prioritize retaining individuals with large crowding distances. Step 3: Design a local search mechanism based on two-layer PPO collaboration. This mechanism utilizes a master agent and sub-agents to help individuals at the Pareto front select the most suitable neighborhood search operator at different levels, enabling refined development of elite solutions. This mechanism consists of agents working collaboratively at two levels: a global search layer and a factory-level search layer. The global search layer comprises a master agent primarily responsible for local searches of workpiece sequences and factory allocations, employing three search strategies corresponding to the master agent's action set. The factory-level search layer deploys a sub-agent for each factory, with each sub-agent primarily responsible for local searches of workpiece sequences, machine allocations, and AGV allocations within that factory, employing four search strategies corresponding to each sub-agent's action set. Each level of the agent normalizes the current individual's old solution as the state input and dynamically outputs the action probability distribution to select the optimal strategy. After generating a new solution by executing the above dynamic output action, a two-way decoding mechanism is implemented on the new solution to calculate its fitness value. The reward value is calculated based on the dominance relationship between the new solution and the old solution and the improvement range of the target value. Then, the state transition quadruple is stored in the experience pool. The agent is trained by randomly sampling batch data to continuously optimize the agent's decision-making ability. The two-way decoding mechanism is as follows: based on the scheduling information determined by forward decoding, the completion time of all workpieces is determined. The process is reversed from the last processing stage to the first stage. If idle time is observed after a workpiece, a right shift strategy is executed without affecting other processing operations. The completion time of all workpieces cannot be later than the smaller of the time when the AGV starts transporting and the start time of the next workpiece processed by the current machine. The reward value is calculated based on the dominance relationship between the new solution New and the old solution Old and the improvement range of the target value. Let f1 be the maximum completion time, f2 be the carbon dioxide emission, and the improvement rate be... , Divided into and ,in, ， These represent the improvement rates on the two objective functions: ， Furthermore, the main agent's action 1: find the factory with the longest completion time, randomly select a workpiece on the critical path, and replace it with the factory with the longest completion time; the main agent's action 2: randomly select a workpiece on the critical path of the factory with the longest completion time and exchange it with a workpiece on the non-critical path of the factory with the shortest completion time; the main agent's action 3: randomly select three adjacent genes at any position, execute all possible permutations, and finally select the best permutation.

2. The method for scheduling a distributed heterogeneous hybrid assembly line considering a limited number of AGVs according to claim 1, characterized in that, The workshop status setting information mentioned in step 1 includes: number of workpieces, number of stages, number of factories, number of AGVs, number of machines in each stage of each factory, machine load and idle energy consumption in each stage, speed of each machine, standard processing time of each workpiece in each factory, raw material area, finished product area in each factory, and location of each machine.

3. The method for scheduling a distributed heterogeneous hybrid assembly line considering a limited number of AGVs according to claim 1, characterized in that, The constraints of the distributed heterogeneous hybrid flow shop scheduling model described in step 1 include: (1) The machine and AGV can only serve one workpiece at a time; (2) The processing and transportation of the workpiece must not be interrupted; (3) The AGV transportation time depends on the physical distance between the machines; (4) Before the workpiece is completed in the current stage and transported to the next stage by the AGV, it needs to wait in the current machine.

4. The method for scheduling a distributed heterogeneous hybrid assembly line considering a limited number of AGVs according to claim 1, characterized in that, The initial population described in step 1 is generated in three parts; First, the first batch of individuals is generated using the minimum time heuristic rule, greedily selecting the machine with the earliest completion time and the AGV with the shortest transportation time; Secondly, the second part of the individuals is generated using the minimum carbon emission heuristic rule, prioritizing the combination of machines and AGVs with low processing energy consumption and short idle paths; Finally, the third group of individuals is generated using a completely random method; Each individual uses a four-layer coding structure: workpiece sorting (JS layer), factory allocation (FA layer), machine allocation (MA layer), and AGV allocation (AA layer).

5. A distributed heterogeneous hybrid flow shop scheduling method considering a limited number of AGVs according to claim 1, characterized in that, A two-way decoding mechanism is applied to individuals in the initial mixed population from step 1 to calculate their fitness values.

6. A distributed heterogeneous hybrid flow shop scheduling method considering a limited number of AGVs according to claim 5, characterized in that, In step 2, the selection of superior individuals is carried out by non-dominant sorting of the individuals participating in the binary tournament, selecting individuals with low non-dominant levels as parent individuals, and implementing a two-way decoding mechanism for the offspring individuals generated by the crossover and mutation operation of the parent individuals in step 2 to calculate their fitness values.

7. A computer-readable access medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the distributed heterogeneous hybrid flow shop scheduling method considering finite AGVs as described in any one of claims 1-6.

8. A processing apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the distributed heterogeneous hybrid flow shop scheduling method considering finite AGVs as described in any one of claims 1-6.

Images