A Hybrid Flow Shop Scheduling Method Based on a Two-Agent Neighborhood Search Algorithm

Abstract

This invention belongs to the field of production scheduling technology and discloses a hybrid assembly line workshop scheduling method based on a dual-agent neighborhood search algorithm. The specific steps are as follows: Step 1: Study the real-world production scenario of the hybrid assembly line workshop, identify constraints, and analyze the main scheduling problems affecting the production cycle. This invention integrates the advantages of Deep Q-Network (DQN) and metaheuristic algorithms, and proposes a dual-agent neighborhood search algorithm. This algorithm can quickly respond to changes in the workshop environment and dynamically adjust the scheduling strategy, aiming to provide a new and efficient solution for hybrid assembly line workshop scheduling and its derivative problems. This invention constructs two agents that can interact with the workshop environment in real time: the first agent can determine the processing sequence of workpieces at each stage and quickly generate promising initial solutions; the second agent can select the optimal neighborhood search strategy and dynamically adjust the search direction to accelerate the convergence of the algorithm.

Description

Technical Field

[0001] This invention belongs to the field of production scheduling technology, specifically a hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm. Background Technology

[0002] Hybrid flow shop scheduling (HFSP) and its variants are common and complex combinatorial optimization problems in industrial production systems, involving multiple jobs, multiple stages, and multiple machines. These problems combine the characteristics of traditional flow shop scheduling and parallel machine scheduling, exhibiting significant NP-hard characteristics. HFSP is widely found in various industries such as chemical production, electronic assembly, biopharmaceuticals, and circuit board printing. By optimizing the scheduling of production tasks and resources, the production efficiency and operational effectiveness of related manufacturing enterprises can be significantly improved.

[0003] Currently, most traditional scheduling algorithms rely on fixed rules or strategies to determine job order and machine allocation. These methods typically ignore the dynamic changes in the shop floor environment as production tasks progress. As the production environment continuously changes, existing scheduling rules cannot respond to these changes in real time, leading to low production efficiency. Therefore, how to adjust scheduling strategies in real time to respond to environmental changes in a dynamic production environment is a key challenge in solving High-Frequency Calculations (HFSP) and its derivatives. Based on research on HFSP by scholars both domestically and internationally, its solution methods can be categorized into exact algorithms, heuristic algorithms, metaheuristic algorithms, and algorithms based on deep or reinforcement learning. Exact algorithms, such as branch and bound, dynamic programming, and integer linear programming (ILP), solve HFSP by applying all... Possible scheduling schemes are calculated, and the optimal solution is found using mathematical models and constraints. However, as the problem size increases, the computational complexity of these exact algorithms grows exponentially, making them ineffective in solving large-scale problems. In this case, heuristic and metaheuristic algorithms are widely used to balance the quality of the solution with the computation time. Heuristic algorithms can quickly generate feasible solutions, but they usually cannot guarantee global optimality. Metaheuristic algorithms, through more complex search mechanisms and perturbation strategies, can find solutions close to global optimality in a shorter time. However, most existing metaheuristic algorithms are based on fixed rules, lack flexibility, and are difficult to respond to dynamic changes in the workshop environment in a timely manner, thus negatively impacting production efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a hybrid flow shop scheduling method based on a dual-agent neighborhood search algorithm to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a hybrid flow shop scheduling method based on a dual-agent neighborhood search algorithm, the specific steps of which are as follows:

[0006] Step 1: Study the real-world production scenario of a hybrid assembly line workshop, identify constraints, and analyze the main scheduling issues affecting the production cycle:

[0007] In the HFSP workshop, there are J workpieces that need to go through K processing stages in sequence, and each stage has M processes. k ≥1 selectable machine, and at least one stage satisfies M k >1, k∈K, in each stage, workpiece j can and can only choose one machine for processing. After completing the processing of the current stage, the task immediately enters the subsequent processing stage. j∈J, all workpieces and machines are ready at time 0. A machine can only process one workpiece at a time and the processing process is not allowed to be interrupted. The processing time of all workpieces in each stage is known, and the transportation time of workpieces between the two stages and the preparation time of machines are not considered. The optimization objective is to minimize the production cycle, that is, to obtain the minimum maximum completion time. The main scheduling task is to determine the workpiece sequence and machine allocation for each stage.

[0008] Step 2: Based on the characteristics of the problem, design appropriate encoding and decoding strategies to symbolically represent the solution:

[0009] Encoding is a method of representing a solution. The main task of hybrid flow shop scheduling is to determine the sequence of workpieces and machine allocation for each stage, using a two-dimensional matrix Z with K rows and J columns. K×J The solution is represented by K, where K is the number of stages and J is the number of workpieces. For example, a workshop has 3 stages and 5 workpieces, and its solution is represented as follows:

[0010]

[0011] Decoding transforms the solution into a practical scheduling scheme, where, for each stage of the workpiece sequence, workpiece j is in matrix Z. K×J The position of the k-th row indicates its processing order in the k-th stage. For example, according to matrix Z... 3×5 The workpiece sequence in the first stage is {3,2,1,5,4}, that is, the processing order of the workpieces in the first stage is workpiece 3, workpiece 2, workpiece 1, workpiece 5 and workpiece 4. For machine allocation, the "first idle" rule is adopted, that is, the workpiece is assigned to the machine that is first idle for processing. If there are multiple idle machines at the same time, it is randomly assigned to one of them for processing.

[0012] Step 3: Construct an agent based on Markov decision processes, and design the state space, action space, state transition, reward function, and neural network architecture:

[0013] Two agents, Agent1 and Agent2, were constructed based on Markov decision process. Agent1 determines the workpiece sequence at each stage according to the characteristics of the workpiece to be processed at each decision point, thereby quickly generating an initial solution. Agent2 selects the optimal neighborhood perturbation strategy according to the workshop environment under the current solution, driving the solution to evolve in a promising direction.

[0014] Step 4: Introduce an adaptive exploration mechanism and a dynamic acceptance criterion to balance the algorithm's development and exploration capabilities:

[0015] During the training of Agent1 and Agent2, in order to prevent the agents from always choosing the action with the highest Q value and over-relying on specific state-action combinations, thereby limiting their exploration of other strategies and affecting their ability to identify potential optimal solutions, an adaptive exploration mechanism based on the ε-greedy policy is introduced to balance the action selection probability of the agents.

[0016] At each decision point, a random number τ∈[0,1] is generated. If τ is greater than the threshold ε, the agent will choose the action with the highest Q value in the current state. Conversely, if τ is less than or equal to ε, the agent will randomly select an action from the available actions. The threshold ε changes linearly during training. Where E represents the total number of training iterations and e represents the current training iteration; as training progresses, ε gradually decreases, which encourages the algorithm to make full use of the knowledge it has learned while still retaining a certain exploratory ability. This adaptive exploration mechanism ensures the balance of the learning process, which not only promotes the identification of the optimal strategy, but also encourages the exploration of new possibilities.

[0017] Similar to the adaptive exploration mechanism described above, in order to balance the local search capability and global search capability of the algorithm, a dynamic acceptance criterion is constructed. At the end of each iteration of the algorithm, the current solution is compared with the optimal solution. If the current solution is better than the optimal solution, the optimal solution is updated with the current solution and used as the initial solution for the next iteration. Otherwise, a random number δ∈[0,1] is generated.

[0018] Step 5: Train the agent and construct a two-agent neighborhood search algorithm:

[0019] Agents Agent1 and Agent2 are trained with different parameter settings, but the training steps are the same. First, the experience replay buffer buffer_m and reward buffer_r are initialized, and the evaluation network Q and the target network Q are constructed. * Q = Q * Next, the training cycle begins. In each training round, the agent must first reset the environment and initialize the cumulative reward r. c At each decision point t, the agent bases its decision on the current state s.t Select action a t Execute action a t To make the agent enter the next state s t+1 Calculate the instant reward r t and the state transition (s t ,a t ,r t ,s t+1 The data is stored in the experience replay buffer, and a batch of data (s) is randomly sampled from it once buffer_m reaches a predefined threshold. t ,a t ,r t ,s t+1 Using the evaluation network Q and the target network Q), * Calculate the Q-value and loss function Δloss, then minimize Δloss using the Adam optimizer and update the evaluation network Q. At the end of each training iteration, calculate the average reward r over the most recent 10 epochs. ave and compared it with the best recorded average reward r best Compare, if r ave >r best Then save the current target network Q. * ;

[0020] Step Six: Generate an efficient scheduling scheme to achieve real-time scheduling and dynamic adjustment of production, and compare the proposed method with commonly used methods to verify the efficiency of the proposed method:

[0021] During the workshop production process, sensors monitor workshop equipment, workpieces, and environmental data in real time and feed this data back to the workshop scheduling system. The scheduling system analyzes the current real-time data and provides a dual-agent neighborhood search algorithm to obtain the optimal operation under the current workshop environment, thereby generating the optimal scheduling decision scheme. Subsequently, the workshop scheduling system executes the decision scheme to achieve efficient scheduling planning of workshop production and shorten the production cycle of workpieces.

[0022] To further verify the efficiency of the scheduling method based on the dual-agent neighborhood search algorithm in solving the scheduling of hybrid flow shops, it was compared with the genetic algorithm (GA) and iterative greedy algorithm (IG), which are commonly used to solve such problems, on 100 simulation instances.

[0023] As a preferred embodiment of the present invention, the state space design method for Agent1 and Agent2 in step three is as follows: In HFSP, there are K stages and J workpieces, with a total of K×J decision points. At each decision point, Agent1 can only schedule workpieces that have reached the current stage, according to... The processing time, arrival time, and waiting time of the workpiece are used to design 12 states for Agent1; Agent2 selects the best neighborhood search strategy based on the workshop environment under the current solution and designs 6 states for it based on the workpiece and machine state characteristics.

[0024] As a preferred technical solution of the present invention, the action space design method of Agent1 and Agent2 in step three is as follows: the action of Agent1 is from the set The strategy for selecting workpieces is based on the set. Nine job selection rules (JSRs) are constructed based on the processing time, arrival time, and waiting time of the workpieces. Agent2 mainly selects the optimal neighborhood search strategy to generate new solutions based on the current workshop environment. These neighborhood search strategies constitute the action space of Agent1. Based on the swap and insert operators, four neighborhood search strategies (NS) are designed for it.

[0025] As a preferred embodiment of the present invention, the reward functions of Agent1 and Agent2 in step three are designed as follows: the reward function is set to minimize the difference in system completion time between two consecutive decisions, and the reward r at time t is... t The negative value of the difference between these two completion times, i.e. in, This represents the completion time at decision time t. This represents the completion time of decision time t-1.

[0026] As a preferred technical solution of the present invention, the state transition design method of Agent1 and Agent2 in step three is as follows: at each decision point t, the agent determines the state transition based on the current state s. t ,Execute action a t and obtain rewards from the environment. t In response, this interaction causes the system to transition to a new state. t+1 In Agent1 and Agent2, a quadruple (s) is used. t ,a t ,r t ,s t+1 The state transition is represented by a ) and stored in the experience pool for subsequent training and decision-making.

[0027] As a preferred technical solution of the present invention, the network structure design of Agent1 and Agent2 in step three is as follows: both Agent1 and Agent2 use a five-layer fully connected neural network. The input of the network is the state features of the agent, and the output corresponds to the action space.

[0028] As a preferred embodiment of the present invention, the dynamic acceptance criterion described in step four... Where T represents the total running time of the algorithm, and t represents the current running time.

[0029] As a preferred technical solution of the present invention, the specific comparison method in step six of the 100 simulation instances is as follows: 100 simulation instances were collected, with stage sizes of {3, 5, 8, 10} and workpiece numbers of {40, 60, 80, 100, 120}, resulting in a total of 4 × 5 = 20 combinations. Each combination generates 5 different instances, thus resulting in a total of 100 instances. In each instance, the number of machines in each stage is randomly generated within the range of [1-5], and the processing time is randomly generated within the range of [10-50]. For each instance, the method proposed in this invention, the genetic algorithm, and the iterative greedy algorithm are solved 10 times respectively. The average target value of each algorithm's 10 solutions is calculated, and the average value (Ave) of the 5 instances in each combination is calculated. At the same time, the relative percentage growth (RPI) index is introduced for measurement.

[0030] As a preferred embodiment of the present invention, the formula for measuring the relative percentage growth (RPI) index is as follows:

[0031]

[0032] Among them, C best C is the optimal objective value obtained by all algorithms. avg The average value of 10 solutions for each algorithm.

[0033] The beneficial effects of this invention are as follows:

[0034] This invention integrates the advantages of Deep Q-Networks (DQN) and metaheuristic algorithms, proposing a dual-agent neighborhood search algorithm. This algorithm can quickly respond to changes in the workshop environment and dynamically adjust the scheduling strategy, aiming to provide a new and efficient solution for hybrid flow shop scheduling and its derivative problems. This invention constructs two agents capable of interacting with the workshop environment in real time: the first agent can determine the processing sequence of workpieces at each stage and quickly generate promising initial solutions; the second agent can select the optimal neighborhood search strategy and dynamically adjust the search direction to accelerate the convergence of the algorithm. At the same time, in order to reduce the agent's dependence on specific state-actions, this invention designs an adaptive exploration mechanism based on an ε-greedy strategy to enhance the agent's exploration ability in the entire state space. In addition, a dynamic acceptance criterion is introduced at the end of the iterative evolution process to balance the local and global exploration capabilities of the algorithm. Attached Figure Description

[0035] Figure 1 This is a flowchart of the present invention;

[0036] Figure 2 This is a schematic diagram of the state space table for Agent1 in this invention;

[0037] Figure 3 This is a schematic diagram of the Agent2 state space table of the present invention;

[0038] Figure 4 This is a schematic diagram of the Agent1 action space table of the present invention;

[0039] Figure 5 This is a schematic diagram of the algorithm results table of the present invention;

[0040] Figure 6 This is a flowchart of the dual-agent neighborhood search algorithm of the present invention;

[0041] Figure 7 This represents the confidence interval for comparison using the algorithm of this invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figures 1 to 7 As shown, this embodiment of the invention provides a hybrid workflow scheduling method based on a dual-agent neighborhood search algorithm, with the following specific steps:

[0044] Step 1: Study the real-world production scenario of a hybrid assembly line workshop, identify constraints, and analyze the main scheduling issues affecting the production cycle:

[0045] In the HFSP workshop, there are J workpieces that need to go through K processing stages in sequence, and each stage has M processes. k ≥1 selectable machine, and at least one stage satisfies M k >1, k∈K, in each stage, workpiece j can and can only choose one machine for processing. After completing the processing of the current stage, the task immediately enters the subsequent processing stage. j∈J, all workpieces and machines are ready at time 0. A machine can only process one workpiece at a time and the processing process is not allowed to be interrupted. The processing time of all workpieces in each stage is known, and the transportation time of workpieces between the two stages and the preparation time of machines are not considered. The optimization objective is to minimize the production cycle, that is, to obtain the minimum maximum completion time. The main scheduling task is to determine the workpiece sequence and machine allocation for each stage.

[0046] Step 2: Based on the characteristics of the problem, design appropriate encoding and decoding strategies to symbolically represent the solution:

[0047] Encoding is a method of representing a solution. The main task of hybrid flow shop scheduling is to determine the sequence of workpieces and machine allocation for each stage, using a two-dimensional matrix Z with K rows and J columns. K×J The solution is represented by K, where K is the number of stages and J is the number of workpieces. For example, a workshop has 3 stages and 5 workpieces, and its solution is represented as follows:

[0048]

[0049] Decoding transforms the solution into a practical scheduling scheme, where, for each stage of the workpiece sequence, workpiece j is in matrix Z. K×J The position of the k-th row indicates its processing order in the k-th stage. For example, according to matrix Z... 3×5 The workpiece sequence in the first stage is {3,2,1,5,4}, that is, the processing order of the workpieces in the first stage is workpiece 3, workpiece 2, workpiece 1, workpiece 5 and workpiece 4. For machine allocation, the "first idle" rule is adopted, that is, the workpiece is assigned to the machine that is first idle for processing. If there are multiple idle machines at the same time, it is randomly assigned to one of them for processing.

[0050] Step 3: Construct an agent based on Markov decision processes, and design the state space, action space, state transition, reward function, and neural network architecture:

[0051] Two agents, Agent1 and Agent2, were constructed based on Markov decision process. Agent1 determines the workpiece sequence at each stage according to the characteristics of the workpiece to be processed at each decision point, thereby quickly generating an initial solution. Agent2 selects the optimal neighborhood perturbation strategy according to the workshop environment under the current solution, driving the solution to evolve in a promising direction.

[0052] Step 4: Introduce an adaptive exploration mechanism and a dynamic acceptance criterion to balance the algorithm's development and exploration capabilities:

[0053] During the training of Agent1 and Agent2, in order to prevent the agents from always choosing the action with the highest Q value and over-relying on specific state-action combinations, thereby limiting their exploration of other strategies and affecting their ability to identify potential optimal solutions, an adaptive exploration mechanism based on the ε-greedy policy is introduced to balance the action selection probability of the agents.

[0054] At each decision point, a random number τ∈[0,1] is generated. If τ is greater than the threshold ε, the agent will choose the action with the highest Q value in the current state. Conversely, if τ is less than or equal to ε, the agent will randomly select an action from the available actions. The threshold ε changes linearly during training. Where E represents the total number of training iterations and e represents the current training iteration; as training progresses, ε gradually decreases, which encourages the algorithm to make full use of the knowledge it has learned while still retaining a certain exploratory ability. This adaptive exploration mechanism ensures the balance of the learning process, which not only promotes the identification of the optimal strategy, but also encourages the exploration of new possibilities.

[0055] Similar to the adaptive exploration mechanism described above, in order to balance the local search capability and global search capability of the algorithm, a dynamic acceptance criterion is constructed. At the end of each iteration of the algorithm, the current solution is compared with the optimal solution. If the current solution is better than the optimal solution, the optimal solution is updated with the current solution and used as the initial solution for the next iteration. Otherwise, a random number δ∈[0,1] is generated.

[0056] Step 5: Train the agent and construct a two-agent neighborhood search algorithm:

[0057] Agents Agent1 and Agent2 are trained with different parameter settings, but the training steps are the same. First, the experience replay buffer buffer_m and reward buffer_r are initialized, and the evaluation network Q and the target network Q are constructed. * Q = Q * Next, the training cycle begins. In each training round, the agent must first reset the environment and initialize the cumulative reward r. c At each decision point t, the agent bases its decision on the current state s. t Select action a t Execute action a t To make the agent enter the next state s t+1 Calculate the instant reward r t and the state transition (s t ,a t ,r t ,s t+1 The data is stored in the experience replay buffer, and a batch of data (s) is randomly sampled from it once buffer_m reaches a predefined threshold. t ,a t ,r t ,s t+1 Using the evaluation network Q and the target network Q), * Calculate the Q-value and loss function Δloss, then minimize Δloss using the Adam optimizer and update the evaluation network Q. At the end of each training iteration, calculate the average reward r over the most recent 10 epochs.ave and compared it with the best recorded average reward r best Compare, if r ave >r best Then save the current target network Q. * ;

[0058] Step Six: Generate an efficient scheduling scheme to achieve real-time scheduling and dynamic adjustment of production, and compare the proposed method with commonly used methods to verify the efficiency of the proposed method:

[0059] During the workshop production process, sensors monitor workshop equipment, workpieces, and environmental data in real time and feed this data back to the workshop scheduling system. The scheduling system analyzes the current real-time data and provides a dual-agent neighborhood search algorithm to obtain the optimal operation under the current workshop environment, thereby generating the optimal scheduling decision scheme. Subsequently, the workshop scheduling system executes the decision scheme to achieve efficient scheduling planning of workshop production and shorten the production cycle of workpieces.

[0060] To further verify the efficiency of the scheduling method based on the dual-agent neighborhood search algorithm in solving the scheduling of hybrid flow shops, it was compared with the genetic algorithm (GA) and iterative greedy algorithm (IG), which are commonly used to solve such problems, on 100 simulation instances.

[0061] This invention proposes a dual-agent neighborhood search algorithm by combining the real-time decision-making capability of DQN with the iterative evolution process of metaheuristic algorithms, and constructs an intelligent scheduling method based on it. This method, through the real-time interaction between two agents and the production environment, can quickly generate high-quality scheduling strategies in complex dynamic environments, significantly improving the flexibility and efficiency of scheduling. It provides a new technical path for solving HFSP and its derivative problems and has important application value.

[0062] In step three, the state space design for Agent1 and Agent2 is as follows: In HFSP, there are K stages and J workpieces, with a total of K×J decision points. At each decision point, Agent1 can only schedule workpieces that have reached the current stage, according to... The processing time, arrival time, and waiting time of the workpiece are used to design 12 states for Agent1; Agent2 selects the best neighborhood search strategy based on the workshop environment under the current solution and designs 6 states for it based on the workpiece and machine state characteristics.

[0063] in, This represents the set of workpieces that have reached stage k and have not yet been processed.

[0064] In step three, the action space design method for Agent1 and Agent2 is as follows: Agent1's action is from the set The strategy for selecting workpieces is based on the set. Nine job selection rules (JSRs) are constructed based on the processing time, arrival time, and waiting time of the workpieces. Agent2 mainly selects the optimal neighborhood search strategy to generate new solutions based on the current workshop environment. These neighborhood search strategies constitute the action space of Agent1. Based on the swap and insert operators, four neighborhood search strategies (NS) are designed for it.

[0065] The four neighborhood search strategies (NS) are: NS1 (Random Swap): In the random selection phase, two jobs are randomly selected from the job sequence and their positions in the sequence are swapped; NS2 (Max-Min Swap): In the first phase, the positions of the job with the longest total completion time and the job with the shortest total completion time in the sequence are swapped; NS3 (Max Insertion): A job is randomly selected from the machine with the longest total processing time, removed from the sequence in the current phase, and then randomly inserted into other positions in the sequence; NS4 (Min Insertion): A job is randomly selected from the machine with the shortest total processing time, removed from the sequence in the current phase, and then randomly inserted into other positions in the sequence. It should be noted that when the job sequence in a certain phase changes, the job sequence in subsequent phases must also be adjusted accordingly. If the job sequence in phase k changes, the job sequence in subsequent phases will be redefined by Agent1.

[0066] In step three, the reward functions for Agent1 and Agent2 are designed as follows: the reward function is set to minimize the difference in system completion time between two consecutive decisions, and the reward r at time t is... t The negative value of the difference between these two completion times, i.e. in, This represents the completion time at decision time t. This represents the completion time of decision time t-1.

[0067] Agent1 optimizes the scheduling process by reducing the difference in completion time when the system makes consecutive decisions. Therefore, the reward function is set to minimize the difference in system completion time between two consecutive decisions.

[0068] In step three, the state transition design for Agent1 and Agent2 is as follows: at each decision point t, the agent determines the state transition based on the current state s. t ,Execute action a t and obtain rewards from the environment. t In response, this interaction causes the system to transition to a new state. t+1 In Agent1 and Agent2, a quadruple (s) is used. t ,at ,r t ,s t+1 The state transition is represented by a ) and stored in the experience pool for subsequent training and decision-making.

[0069] In reinforcement learning, state transition refers to the process by which an agent moves from one state to another through an action.

[0070] In step three, the network structure design of Agent1 and Agent2 is as follows: both Agent1 and Agent2 use a five-layer fully connected neural network. The input of the network is the state features of the agent, and the output corresponds to the action space.

[0071] To enhance the network's ability to capture complex feature interactions, all hidden layers employed the non-linear activation function ReLU, and the number of neurons in these hidden layers was set to 256, 128, 64, and 32, respectively.

[0072] Among them, the dynamic acceptance criterion in step four, Where T represents the total running time of the algorithm, and t represents the current running time.

[0073] In the dynamic acceptance criterion, if δ≥ε, the current solution is used in the next iteration; if δ<ε, a new solution is generated by the agent and used in the next iteration.

[0074] The specific comparison method in step six among 100 simulation instances is as follows: 100 simulation instances were collected, with stage sizes of {3, 5, 8, 10} and workpiece numbers of {40, 60, 80, 100, 120}, resulting in a total of 4 × 5 = 20 combinations. Each combination generates 5 different instances, thus resulting in a total of 100 instances. In each instance, the number of machines in each stage is randomly generated within the range of [1-5], and the processing time is randomly generated within the range of [10-50]. For each instance, the proposed method, genetic algorithm, and iterative greedy algorithm are solved 10 times each. The average target value of each algorithm's 10 solutions is calculated, and the average value (Ave) of the 5 instances in each combination is calculated. At the same time, the relative percentage growth (RPI) index is introduced for measurement.

[0075] Based on the algorithm comparison results and the confidence intervals of the result data, it can be seen that the dual-agent neighborhood search algorithm proposed in this invention can effectively solve the hybrid flow shop scheduling problem and is significantly better than the genetic algorithm and the iterative greedy algorithm.

[0076] The formula for the Relative Percentage Growth (RPI) indicator is as follows:

[0077]

[0078] Among them, C best C is the optimal objective value obtained by all algorithms. avg The average value of 10 solutions for each algorithm.

[0079] Relative percentage growth (RPI) is a metric used to measure how much a value has grown relative to its original or baseline value. This metric is widely used in many fields such as economics, business, education, and healthcare to assess data changes at different points in time or under different conditions.

[0080] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0081] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A hybrid flow shop scheduling method based on a dual-agent neighborhood search algorithm, characterized in that, The specific steps are as follows: Step 1: Study the real-world production scenario of a hybrid assembly line workshop, identify constraints, and analyze the scheduling issues that primarily affect the production cycle: In the HFSP workshop, there are Each workpiece needs to pass through in sequence. There are several processing stages, each with its own... The machine can be selected, and at least one stage must meet the requirements. , At each stage, the workpiece Only one machine can be selected for processing. After the current stage of processing is completed, the task will immediately proceed to the next processing stage. All workpieces and machines are ready at time 0. A machine can only process one workpiece at a time and the processing process is not allowed to be interrupted. The processing time of all workpieces at each stage is known, and the transportation time of workpieces between two stages and the preparation time of machines are not considered. The optimization goal is to minimize the production cycle, that is, to obtain the minimum maximum completion time. The scheduling task is to determine the workpiece sequence and machine allocation for each stage. Step 2: Based on the characteristics of the problem, design encoding and decoding strategies to symbolically represent the solution: Encoding is a method of representing a solution. The task of hybrid flow shop scheduling is to determine the sequence of workpieces and machine allocation for each stage, using a... OK Two-dimensional matrix of columns Indicate the solution, For the number of stages, Given the number of workpieces, with 3 stages and 5 workpieces in the workshop, the solution is as follows: ； Decoding transforms the solution into a practical scheduling scheme, for each stage of the workpiece sequence, workpiece In the matrix No. The position of the row indicates its position in the row. The processing sequence of each stage, according to the matrix The first stage of the workpiece sequence is as follows: In other words, the processing order of the workpieces in the first stage is workpiece 3, workpiece 2, workpiece 1, workpiece 5 and workpiece 4. For machine allocation, the "first idle" rule is adopted, that is, the workpieces are assigned to the machine that is first idle for processing. If there are multiple idle machines at the same time, the workpieces are randomly assigned to one of them for processing. Step 3: Construct an agent based on Markov decision processes, and design the state space, action space, state transition, reward function, and neural network architecture: Two intelligent agents were constructed based on Markov decision processes: and ,in, Based on the characteristics of the workpiece to be processed at each decision point, the workpiece sequence for each stage is determined, thereby quickly generating an initial solution. Based on the current workshop environment, the optimal neighborhood perturbation strategy is selected to drive the solution to evolve in a promising direction; Step 4: Introduce an adaptive exploration mechanism and a dynamic acceptance criterion to balance the algorithm's development and exploration capabilities: In intelligent agents and During training, to prevent the agent from always choosing The actions with the highest values ​​are overly reliant on specific state-action combinations, thus limiting their exploration of other strategies and affecting their ability to identify potential optimal solutions. A method based on... The policy employs an adaptive exploration mechanism to balance the probabilities of the agent's action choices. Generate a random number at each decision point. ,if Greater than the threshold The agent will choose the action with the highest Q value in the current state, and conversely, if Less than or equal to The agent will randomly select an action from the available actions, and the threshold will be set. It changes linearly during training. ,in This represents the total number of training iterations. Indicates the current number of training iterations; as training progresses, By gradually decreasing the size of the algorithm, it encourages the algorithm to make full use of the knowledge it has learned while still retaining its exploratory capabilities. This adaptive exploration mechanism ensures a balance in the learning process, promoting the identification of the optimal strategy and encouraging the exploration of new possibilities. To balance the local and global search capabilities of the algorithm, a dynamic acceptance criterion is constructed. At the end of each iteration, the current solution is compared with the optimal solution. If the current solution is better than the optimal solution, the current solution is used to update the optimal solution and is used as the initial solution for the next iteration. Otherwise, a random number is generated. ; Step 5: Train the agent and construct a two-agent neighborhood search algorithm: Training agents and Their parameter settings differ, but the training steps are the same. First, the experience replay buffer is initialized. and reward buffer Build an evaluation network and target network , Next, the training cycle begins. In each training round, the agent must first reset the environment and initialize the accumulated rewards. At each decision point The agent is based on the current state Select Action Execute actions To make the agent enter the next state Calculate instant rewards and transition state Stored in the experience playback buffer, once Once a predefined threshold is reached, a batch of data is randomly sampled. Using evaluation networks and target network calculate Value and loss function Then, minimize using the Adam optimizer. and update the evaluation network. At the end of each training session, calculate the average reward over the most recent 10 rounds. and compared it with the best average reward recorded. Compare, if Then save the current target network. ; Step Six: Generate an efficient scheduling scheme to achieve real-time scheduling and dynamic adjustment of production, and compare the proposed method with commonly used methods to verify the efficiency of the proposed method: During the workshop production process, sensors monitor workshop equipment, workpieces, and environmental data in real time and feed this data back to the workshop scheduling system. The scheduling system analyzes the current real-time data and provides a dual-agent neighborhood search algorithm to obtain the optimal operation under the current workshop environment, thereby generating the optimal scheduling decision scheme. Subsequently, the workshop scheduling system executes the decision scheme to achieve efficient scheduling planning of workshop production and shorten the production cycle of workpieces. To further verify the efficiency of the scheduling method based on the dual-agent neighborhood search algorithm in solving the scheduling of hybrid flow shops, it was compared with the genetic algorithm (GA) and iterative greedy algorithm (IG), which are commonly used to solve such problems, on 100 simulation instances.

2. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: The steps described in step three and The state-space design approach is as follows: In HFSP, there exists Each stage and There are [number] workpieces, totaling [number] pieces. At each decision point, Only workpieces that have reached the current stage can be scheduled, according to The processing time, arrival time, and waiting time of the workpiece are: The design includes 12 states; Based on the current solution's workshop environment, the optimal neighborhood search strategy is selected. Six states are designed based on the workpiece and machine state characteristics. Indicates arrival A collection of workpieces that are still in the process of being processed.

3. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: The steps described in step three and The motion space design method is as follows: The action is from the set The strategy for selecting workpieces is based on the set. Based on the processing time, arrival time, and waiting time of the workpiece, nine workpiece selection rules (JSR) are constructed. Based on the current workshop environment, an optimal neighborhood search strategy is selected to generate a new solution. These neighborhood search strategies constitute... Based on the swap and insert operators, four neighborhood search strategies (NS) are designed for the action space. Indicates arrival A collection of workpieces that are still in the process of being processed.

4. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: The steps described in step three and The reward function is designed to minimize the difference in system completion time between two consecutive decisions, at time step [time value missing]. Rewards The negative value of the difference between these two completion times, i.e. ,in, Indicates the moment of decision Completion time, Indicates the moment of decision Completion time, express and At any moment The reward function.

5. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: The steps described in step three and The state transition design is as follows: at each decision point t, the agent, based on the current state... , Execution of actions and get rewards from the environment. In response, this interaction causes the system to transition to a new state. ,exist and In, quadruplets are used This is used to represent state transitions and stored in the experience pool for subsequent training and decision-making.

6. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: The steps described in step three and The network structure design method is as follows: and Both use a five-layer fully connected neural network. The network input is the agent's state features, while the output corresponds to the action space.

7. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: In the dynamic acceptance criteria described in step four, ,in, Represents the total running time of the algorithm. Current running time.

8. The hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 1, characterized in that: The specific comparison method described in step six for the 100 simulation instances is as follows: 100 simulation instances were collected, with the following stage sizes: The number of workpieces are respectively ,common There are 100 instances in total, with 5 different instances generated by each combination. In each instance, the number of machines in each stage is randomly generated in the range of [1-5], and the processing time is randomly generated in the range of [10-50]. For each instance, the proposed method, genetic algorithm and iterative greedy algorithm are solved 10 times respectively. The average target value of each algorithm is calculated in 10 solutions, and the average value (Ave) of the 5 instances in each combination is calculated. The relative percentage growth (RPI) index is introduced for measurement.

9. A hybrid assembly line scheduling method based on a dual-agent neighborhood search algorithm according to claim 8, characterized in that: The formula for the Relative Percentage Growth (RPI) indicator is as follows: ； in, The optimal target value obtained by all algorithms, The average value of 10 solutions for each algorithm.

Images