Flexible production line scheduling method based on particle swarm optimization and reinforcement learning

Abstract

The invention discloses a flexible production line scheduling method based on a particle swarm algorithm and reinforcement learning. The method comprises the following steps: S1, collecting state data in a flexible production line and preprocessing the state data; s2, performing global optimization through an improved adaptive particle swarm optimization algorithm, and generating a preliminary flexible production line scheduling solution set; s3, inputting the preliminary flexible production line scheduling solution set into a state evaluation network for multi-target optimization; S4, optimizing a learning process of a multi-target scheduling decision through a state-target causal path tracking mechanism; s5, executing the optimized flexible production line scheduling solution set, and continuously collecting production line operation feedback data; s6, screening an optimal flexible production line scheduling scheme through a population resampling mechanism; and S7, executing an actual production task according to the optimal flexible production line scheduling scheme. According to the method, the intelligence and the real-time performance of flexible production line scheduling are improved, and the method has good adaptability and wide application prospects.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and production scheduling optimization technology, and in particular to a flexible production line scheduling method based on particle swarm optimization and reinforcement learning. Background Technology

[0002] With the continuous development of intelligent manufacturing, flexible production lines are increasingly widely used in the manufacturing industry. Flexible production lines can flexibly adjust production tasks, equipment resources, and material allocation according to different production needs, thereby improving production efficiency and resource utilization. However, existing scheduling methods have certain limitations when dealing with dynamic production environments, mainly manifested in low scheduling efficiency, serious resource waste, and difficulty in quickly adapting to changes in the production process. In particular, traditional scheduling methods have poor optimization effects under complex and variable production tasks.

[0003] Traditional flexible production line scheduling methods typically rely solely on particle swarm optimization (PSO). However, PSO is prone to getting trapped in local optima in practical applications and lacks the ability to quickly adapt to dynamic changes in the production environment, limiting its widespread application in flexible production lines. Meanwhile, reinforcement learning, through trial-and-error learning and adaptive optimization, can provide strong adaptability in dynamic environments. However, due to its complex training process and slow convergence speed, reinforcement learning still faces problems of insufficient stability and low efficiency in practical applications. Although both PSO and reinforcement learning have their advantages, they still have certain limitations when used alone.

[0004] Furthermore, traditional scheduling methods mostly focus on optimizing a single objective, such as maximizing output or minimizing energy consumption, while real-world production tasks typically require considering multiple objectives simultaneously. Balancing these multiple objectives and achieving comprehensive optimization remains a significant challenge for existing scheduling methods. Current methods fail to effectively balance various objectives, resulting in unsatisfactory scheduling performance and an inability to achieve optimal resource allocation in real-world production environments.

[0005] Therefore, how to provide a flexible production line scheduling method based on particle swarm optimization and reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a flexible production line scheduling method based on particle swarm optimization and reinforcement learning. This invention fully utilizes the global search capability of the improved adaptive particle swarm optimization algorithm and the adaptive optimization capability based on two-layer attention reinforcement learning. It combines dynamic fitness evaluation, local and global search, and random perturbation techniques, and describes in detail how to optimize flexible production line scheduling in complex production environments. It can adjust the scheduling scheme in real time to adapt to changes in production status, and has high resource utilization, flexible multi-objective optimization capability, and good dynamic adaptability.

[0007] A flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to an embodiment of the present invention includes the following steps: S1. Collect and preprocess the status data in the flexible production line to form a dynamic status vector; S2. Based on the dynamic state vector, the production tasks, equipment resources and material allocation are globally optimized by the improved adaptive particle swarm optimization algorithm to generate a preliminary flexible production line scheduling solution set. S3. Input the initial flexible production line scheduling solution set into a state evaluation network based on two-layer attention reinforcement learning for multi-objective optimization to obtain the optimized flexible production line scheduling solution set: S4. Optimize the learning process of multi-objective scheduling decision-making in the state evaluation network through a state-objective causal path tracing mechanism; S5. Execute the optimized flexible production line scheduling and continuously collect production line operation feedback data to adjust the status assessment network in real time. S6. By using a population resampling mechanism, the flexible production line scheduling solution set during the execution process is screened to obtain the optimal flexible production line scheduling scheme. S7. Transform the optimal flexible production line scheduling scheme into production scheduling instructions and feed them back to the production line to execute actual production tasks.

[0008] Optionally, step S1 specifically includes: S11. Real-time collection of status data of workstations, equipment, workpieces and materials in the flexible production line through sensors and data acquisition devices. The status data includes equipment start-up and shutdown status, equipment workload, task execution progress, resource occupancy, order priority, production progress, material supply status, equipment fault information, process execution time, process scheduling, temperature, humidity, pressure and environmental parameters. S12. Preprocess the collected status data, including data cleaning, data denoising, and timestamp alignment. S13. Normalize the preprocessed state data to convert state data of different magnitudes and units into a unified range, forming a dynamic state vector.

[0009] Optionally, step S2 specifically includes: S21. Based on dynamic state vectors, the production tasks, equipment resources, workstation allocation and material supply are globally optimized by adaptive particle swarm optimization algorithm to generate a preliminary flexible production line scheduling solution set. In the improved adaptive particle swarm optimization algorithm, each particle represents a scheduling scheme. The dependence of the particle on the historical optimal solution is dynamically adjusted through a dynamic memory reliability mechanism. The dynamic memory reliability mechanism adjusts the influence of the particle's historical solution through a time decay factor and evaluates the quality of the particle's scheduling scheme based on a real-time fitness evaluation function based on the production state. S22. Based on the current production status and environmental changes, calculate the credibility value of the particle's historical optimal solution, and dynamically adjust the particle's dependence on the historical solution. The credibility value of the historical optimal solution includes the causal influence at the current moment and the historical contribution. The causal influence at the current moment represents the difference between the current production status and the historical optimal solution, and the historical contribution is the contribution at the previous moment. The weight ratio between the causal influence at the current moment and the historical contribution is adjusted by a balance factor. S23. The dependence of particles on historical optimal solutions is dynamically adjusted by the time decay factor. The change of inertia weight is controlled by the difference between the maximum and minimum inertia weights. The inertia weight gradually decreases with the number of iterations during the particle swarm optimization process. S24. A fitness function is introduced to evaluate the scheduling scheme of each particle, and the individual optimal solution and global optimal solution of the particle are updated according to the evaluation results. The fitness function comprehensively considers three scheduling objectives: task completion, resource utilization efficiency and energy consumption. S25. Through multiple iterations of optimization, and by updating the individual optimal position and the global optimal position based on the fitness value of the particles, until the preset convergence condition is met or the maximum number of iterations is reached, the preliminary flexible production line scheduling solution set is output.

[0010] Optionally, step S3 specifically includes: S31. The state evaluation network based on two-layer attention reinforcement learning includes a production process attention mechanism, a scheduling target attention mechanism, and a multi-target scheduling decision network. S32. The production process attention mechanism uses a weighted adjacency matrix to model the dependency relationship between each production process, analyzes the mutual influence between the current production state and the historical scheduling scheme, quantifies the cooperation and conflict between production processes, and obtains the production process attention weight coefficient. S33. The scheduling target attention mechanism dynamically adjusts the priority by monitoring the real-time changes in production status to adjust the priority of each scheduling target and evaluates the degree of influence of each status in multi-target scheduling optimization to obtain the scheduling target attention weight coefficient. S34. Based on the evaluation results of the attention mechanism of the production process and the attention mechanism of the scheduling target, integrate the dynamic influence of each production process and the scheduling target to generate a comprehensive optimization coefficient. S35. Based on the comprehensive optimization coefficient, the initial flexible production line scheduling solution is input into the multi-objective scheduling decision network and optimized through the multi-objective optimization function. Taking into account the maximization of output, the minimization of energy consumption, the optimization of delivery time, and the maximization of resource utilization, the optimized flexible production line scheduling scheme is output.

[0011] Optionally, the multi-objective optimization function in step S34 comprehensively considers maximizing output, minimizing energy consumption, optimizing delivery time, and maximizing resource utilization. Maximizing output means maximizing the total output of production tasks completed in flexible production line scheduling. Minimizing energy consumption means minimizing the total energy consumed during scheduling. Optimizing delivery time means ensuring that each task can be completed within the predetermined time as much as possible. Maximizing resource utilization means maximizing the efficiency of all resources used in the production process.

[0012] Optionally, step S4 specifically includes: S41. The state-target causal path tracing mechanism tracks and analyzes changes in production state in real time, quantifying the impact of these changes on scheduling targets. S42. Based on the causal relationship between the current production status and the scheduling objective, calculate the direct or indirect impact of each production status change on the achievement of the scheduling objective, and obtain the contribution of each status change to the scheduling objective. S43. Based on the contribution of each state change to the scheduling objective, when the contribution exceeds the set contribution threshold, the objective weight in the multi-objective scheduling decision is increased and the priority is improved. S44. Introduce an experience replay mechanism to store the data of historical state-target causal path in the experience pool. The data includes production status, scheduling decisions, rewards or penalties and subsequent production status. S45. Periodically, randomly select a set of historical experiences from the experience pool and optimize the current flexible production line scheduling plan through replay learning. S46. By replaying historical experience, the multi-objective scheduling decision network adjusts the flexible production line scheduling scheme according to changes in production status.

[0013] Optionally, step S5 specifically includes: S51. Execute the optimized flexible production line scheduling solution set, which includes scheduling schemes suitable for the current production environment, inefficient solutions, and invalid solutions; S52. Continuously collect feedback data on production line operation, including production progress, equipment load, target completion rate, task execution status, resource utilization, equipment fault information, and production line efficiency. S53. Input the collected feedback data into the state evaluation network based on two-layer attention reinforcement learning to evaluate the fitness of each scheduling scheme in the flexible production line scheduling solution set in real time, and adjust the scheduling decision according to the current production status.

[0014] Optionally, step S6 specifically includes: S61. The flexible production line scheduling solution set during the execution process is screened through a population resampling mechanism. The population resampling mechanism introduces dynamic fitness evaluation, combines local search and global search, and incorporates random perturbation. S62. Based on the real-time collected feedback data, dynamically calculate the fitness of each scheduling scheme and eliminate scheduling schemes with fitness below the set threshold. S63. Perform local and global searches on the selected flexible production line scheduling solution set: optimize the solution set details by making small-scale adjustments, and increase the diversity of the solution set by expanding the search range of the solution space; S64. Randomly perturb the flexible production line scheduling solution set retained after the search to increase population diversity, prevent getting trapped in local optima, and select the scheduling scheme with the highest fitness as the optimal flexible production line scheduling scheme based on the perturbed flexible production line scheduling solution set.

[0015] Optionally, step S7 specifically includes: converting the optimal flexible production line scheduling scheme into a production scheduling instruction and feeding it back to the production line to execute the actual production task; continuously collecting production line operation status data during the execution process; evaluating the effectiveness of the scheduling scheme; and adjusting the scheduling scheme based on the evaluation results to ensure that the requirements of each production target are met.

[0016] The beneficial effects of this invention are: First, this invention introduces an improved adaptive particle swarm optimization algorithm. By dynamically adjusting the dependence of particles on historical optimal solutions and combining local and global search strategies, it addresses the shortcomings of traditional particle swarm optimization algorithms, which are prone to getting trapped in local optima. By dynamically adjusting the influence of particles on historical solutions through a time decay factor, particles can flexibly adjust scheduling schemes according to the current production state, improving scheduling efficiency and enhancing adaptability to changes in the production environment. Second, this invention further enhances the multi-objective scheduling optimization capability through a state evaluation network based on two-layer attention reinforcement learning. A two-layer attention mechanism is designed to optimize production processes and scheduling objectives: each production process is optimized based on the dependency between the current production state and historical scheduling results; and a weighted evaluation is performed based on the priority of different scheduling objectives and the impact of dynamic changes in production state on the objectives, thereby achieving comprehensive optimization of multiple scheduling objectives. Furthermore, a state-objective causal path tracking mechanism is introduced. By tracking the causal relationship between changes in production state and scheduling objectives in real time, the weight and priority of objectives are dynamically adjusted. By quantifying the direct or indirect impact of changes in production state on each scheduling objective, scheduling decisions are optimized to ensure that key objectives in the production process are prioritized. During the optimization process, the flexible production line scheduling solution will be screened through a population resampling mechanism to retain and actually execute the optimal scheduling scheme.

[0017] In summary, this invention effectively improves the accuracy and flexibility of flexible production line scheduling through an improved adaptive particle swarm optimization algorithm, a state evaluation network based on two-layer attention reinforcement learning, a state-target causal path tracking mechanism, and a population resampling optimization method. It can optimize multi-objective scheduling decisions in the face of complex and dynamically changing production environments, ensuring efficient allocation of production resources and task completion. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0019] Figure 1 This is a schematic diagram of a flexible production line scheduling method based on particle swarm optimization and reinforcement learning proposed in this invention. Figure 2 This is a flowchart of the flexible production line scheduling based on the adaptive particle swarm optimization algorithm in this invention; Figure 3 This is a flowchart of the flexible production line scheduling optimization based on two-layer attention reinforcement learning in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figure 1-3 A flexible production line scheduling method based on particle swarm optimization and reinforcement learning includes the following steps: S1. Collect and preprocess the status data in the flexible production line to form a dynamic status vector; S2. Based on the dynamic state vector, the production tasks, equipment resources and material allocation are globally optimized by the improved adaptive particle swarm optimization algorithm to generate a preliminary flexible production line scheduling solution set. S3. Input the initial flexible production line scheduling solution set into a state evaluation network based on two-layer attention reinforcement learning for multi-objective optimization to obtain the optimized flexible production line scheduling solution set: S4. Optimize the learning process of multi-objective scheduling decision-making in the state evaluation network through a state-objective causal path tracing mechanism; S5. Execute the optimized flexible production line scheduling and continuously collect production line operation feedback data to adjust the status assessment network in real time. S6. By using a population resampling mechanism, the flexible production line scheduling solution set during the execution process is screened to obtain the optimal flexible production line scheduling scheme. S7. Transform the optimal flexible production line scheduling scheme into production scheduling instructions and feed them back to the production line to execute actual production tasks.

[0022] In this embodiment, step S1 specifically includes: S11. Real-time collection of status data of workstations, equipment, workpieces and materials in the flexible production line through sensors and data acquisition devices. The status data includes equipment start-up and shutdown status, equipment workload, task execution progress, resource occupancy, order priority, production progress, material supply status, equipment fault information, process execution time, process scheduling, temperature, humidity, pressure and environmental parameters. S12. Preprocess the collected status data, including data cleaning, data denoising, and timestamp alignment. S13. Normalize the preprocessed state data to convert state data of different magnitudes and units into a unified range, forming a dynamic state vector.

[0023] In this embodiment, step S2 specifically includes: S21. Based on dynamic state vectors, the production tasks, equipment resources, workstation allocation and material supply are globally optimized by adaptive particle swarm optimization algorithm to generate a preliminary flexible production line scheduling solution set. In the improved adaptive particle swarm optimization algorithm, each particle represents a scheduling scheme. The dependence of the particle on the historical optimal solution is dynamically adjusted through a dynamic memory reliability mechanism. The dynamic memory reliability mechanism adjusts the influence of the particle's historical solution through a time decay factor and evaluates the quality of the particle's scheduling scheme based on a real-time fitness evaluation function based on the production state. S22. Based on the current production state and environmental changes, calculate the confidence value of the particle's historical optimal solution, and dynamically adjust the particle's dependence on the historical solution. The confidence value of the historical optimal solution includes the causal influence at the current moment and the historical contribution. The causal influence at the current moment represents the difference between the current production state and the historical optimal solution, and the historical contribution is the contribution at the previous moment. The weight ratio between the causal influence at the current moment and the historical contribution is adjusted by a balance factor. ; in, For the first Particles in time The reliability of the historical best solution at any given time. For the first The time difference between the historical optimal solutions for each particle indicates the degree to which the historical solutions are outdated. Current production status With the The historical optimal solution for each particle The state offset between the production state and the historical solution measures the difference between the production state and the historical solution. and As balancing factors, they control the effects of time decay and state shift, respectively. For the first The fitness value of a particle's historical solution represents the optimization effect of the historical solution; In this invention, when the production state changes, particles calculate the reliability value of the historical optimal solution based on the difference between the current production state and the historical optimal solution, as well as the time difference between the historical solutions. As the production state and time change, the particles' dependence on the historical optimal solution gradually decreases, ensuring that the particles can always make decisions based on the latest production environment and optimize the production scheduling scheme. Through a dynamic memory reliability mechanism, particles continuously adjust their trust in the historical optimal solution during the search process, avoiding over-reliance on invalid historical information and making the particle swarm more focused on the needs and constraints of the current production task, thereby improving the overall scheduling efficiency and effectiveness.

[0024] S23. The dependence of particles on historical optimal solutions is dynamically adjusted through a time decay factor. The change in inertia weight is controlled by the difference between the maximum and minimum inertia weights. Furthermore, the inertia weights during the particle swarm optimization process... With the number of iterations Gradually decrease: ; in, For maximum inertia weight, For minimum inertia weight, The maximum number of iterations, This represents the current iteration number; In this invention, a time decay factor is introduced, allowing particles to gradually reduce their dependence on earlier historical solutions over time, thus focusing more on current production tasks and constraints and avoiding over-reliance on inapplicable information. This is achieved by gradually reducing the inertia weight. The time decay factor prompts particles to transition from the initial global search to the later local search, thereby improving the efficiency of particle swarm optimization and accelerating the convergence process.

[0025] S24. A fitness function is introduced to evaluate the scheduling scheme for each particle, and the individual optimal solution and the global optimal solution of the particle are updated according to the evaluation results. The fitness function comprehensively considers three scheduling objectives: task completion, resource utilization efficiency, and energy consumption. ; in, For the first The fitness value of each particle. Indicates the first The task completion rate under each particle scheduling scheme is evaluated by comparing the completion rate of actual production tasks with that of expected tasks. Indicates the first Resource utilization efficiency under a particle scheduling scheme is evaluated based on the degree of optimization according to the usage of production resources. Indicates the first Energy consumption under a single particle scheduling scheme is calculated by evaluating the energy consumed by equipment and resources during the production process. The weighting coefficients for scheduling optimization objectives are adjusted based on the specific objectives. S25. Through multiple iterations of optimization, and by updating the individual optimal position and the global optimal position based on the fitness value of the particles, until the preset convergence condition is met or the maximum number of iterations is reached, the preliminary flexible production line scheduling solution set is output.

[0026] In this embodiment, step S3 specifically includes: S31. The state evaluation network based on two-layer attention reinforcement learning includes a production process attention mechanism, a scheduling target attention mechanism, and a multi-target scheduling decision network. S32. The production process attention mechanism uses a weighted adjacency matrix to model the dependency relationship between each production process, analyzes the mutual influence between the current production state and the historical scheduling scheme, quantifies the cooperation and conflict between production processes, and obtains the production process attention weight coefficient. ; in, For the first The production process affects the first The weight coefficients of each state, The degree of dependency between production status and historical scheduling results. As the attenuation factor, This is a dynamic adjustment coefficient for the production process, adjusted based on production priority or resource availability. This is the interaction coefficient in the production process, reflecting the dynamic cooperation or conflict between different stages of production. This represents the total number of production stages. S33. The scheduling target attention mechanism dynamically adjusts the priority by monitoring the real-time changes in production status to adjust the priority of each scheduling target and evaluates the degree of influence of each status in multi-target scheduling optimization to obtain the scheduling target attention weight coefficient. ; in, In the scheduling target attention mechanism, the first The scheduling target is the first The weight coefficients of each state, For the first The scheduling target is the first The degree of influence of each state The priority factor is dynamically adjusted, and the weight of each objective is dynamically adjusted according to the current production status. As the importance weight of the target, The interaction coefficient represents the mutual influence and balance between different objectives. The total number of scheduling targets; S34. Based on the evaluation results of the attention mechanism in the production process and the attention mechanism in the scheduling target, integrate the dynamic influence of each production process and the scheduling target to generate a comprehensive optimization coefficient. : ; in, and Adjusting the contribution levels of the two-tiered mechanism; S35, Based on comprehensive optimization coefficients The initial flexible production line scheduling solution is input into a multi-objective scheduling decision network and optimized through a multi-objective optimization function. Taking into account maximizing output, minimizing energy consumption, optimizing delivery time, and maximizing resource utilization, the optimized flexible production line scheduling scheme is output.

[0027] In this embodiment, the multi-objective optimization function in step S34 comprehensively considers maximizing output, minimizing energy consumption, optimizing delivery time, and maximizing resource utilization. Maximizing output means maximizing the total output of production tasks completed in flexible production line scheduling. Minimizing energy consumption means minimizing the total energy consumed during scheduling. Optimizing delivery time means ensuring that each task can be completed within the predetermined time as much as possible. Maximizing resource utilization means maximizing the efficiency of all resources used in the production process.

[0028] In this embodiment, step S4 specifically includes: S41. The state-target causal path tracing mechanism tracks and analyzes changes in production state in real time and quantifies the impact of these changes on scheduling targets. S42. Based on the causal relationship between the current production state and the scheduling objective, calculate the direct or indirect impact of each production state change on the achievement of the scheduling objective, and obtain the contribution of each state change to the scheduling objective: ; in, For the first Each scheduling target in time The contribution of the state at any given moment to the objective. As a balancing factor, it determines the weight of current causal influence and historical contribution. For the causal effect at the current moment, The contribution level at the previous moment; S43. Based on the contribution of each state change to the scheduling objective, when the contribution exceeds the set contribution threshold, the objective weight in the multi-objective scheduling decision is increased and the priority is improved. For example, if equipment failure occurs during production, leading to a significant drop in output, the contribution of the production status to the output target decreases by 30%. Based on this production status, the weight of the output target is reduced from 0.6 to 0.4, while the weight of the energy consumption target increases from 0.3 to 0.5. The increase in energy consumption is because backup equipment will be used during equipment failure. Furthermore, if equipment failure causes production delays, with an estimated delivery time delay of 10 hours, the system will increase the priority of the delivery time target from 0.2 to 0.4 to ensure on-time delivery. Therefore, changes in production status directly affect the weight and priority of scheduling targets, enabling scheduling decisions to focus more on urgent needs in the current production environment and ensuring that resource allocation prioritizes meeting critical production objectives.

[0029] S44. Introduce an experience replay mechanism to store the data of historical state-target causal path in the experience pool. The data includes production status, scheduling decisions, rewards or penalties and subsequent production status. S45. Periodically, randomly select a set of historical experiences from the experience pool and optimize the current flexible production line scheduling plan through replay learning. S46. By replaying historical experience, the multi-objective scheduling decision network adjusts the flexible production line scheduling scheme according to changes in production status.

[0030] For example, when equipment failure causes production line shutdown, the system extracts experience from historical data on similar situations through an experience playback mechanism. In a particular equipment failure, the system pauses low-priority tasks and schedules backup equipment to reduce downtime, recording this experience in the experience pool: Production status: Equipment failure, production line shutdown, task paused; Scheduling decision: Pause low-priority tasks, schedule backup equipment; Penalty: Output decreases by 30%, energy consumption increases by 15%, delivery time is delayed by 10 hours; Subsequent production status: Backup equipment is activated, tasks continue to execute, delivery time is delayed.

[0031] When the same equipment failure occurs again, the system extracts the historical experience from the experience pool and adjusts the current scheduling strategy based on replay learning: Production target: Due to equipment failure, the priority of the production target is reduced, and its weight decreases from 0.6 to 0.4; Delivery time target: To ensure on-time delivery as much as possible, the priority of the delivery time target is increased, and its weight increases from 0.2 to 0.4; Energy consumption target: Due to increased energy consumption caused by the use of backup equipment, the system increases the weight of the energy consumption target from 0.2 to 0.3 to optimize resource usage and reduce unnecessary energy consumption increases. By optimizing task allocation and resource scheduling, the system can ensure that production delays are minimized in the event of equipment failure, while avoiding excessive energy consumption, ultimately maintaining efficient production scheduling.

[0032] In this embodiment, step S5 specifically includes: S51. Execute the optimized flexible production line scheduling solution set, which includes scheduling schemes suitable for the current production environment, inefficient solutions, and invalid solutions; S52. Continuously collect feedback data on production line operation, including production progress, equipment load, target completion rate, task execution status, resource utilization, equipment fault information, and production line efficiency. S53. Input the collected feedback data into the state evaluation network based on two-layer attention reinforcement learning to evaluate the fitness of each scheduling scheme in the flexible production line scheduling solution set in real time, and adjust the scheduling decision according to the current production status.

[0033] In this embodiment, step S6 specifically includes: S61. The flexible production line scheduling solution set during the execution process is screened through a population resampling mechanism. The population resampling mechanism introduces dynamic fitness evaluation, combines local search and global search, and incorporates random perturbation. S62. Based on the real-time collected feedback data, dynamically calculate the fitness of each scheduling scheme and eliminate scheduling schemes with fitness below the set threshold. S63. Perform local and global searches on the selected flexible production line scheduling solution set: optimize the solution set details by making small-scale adjustments, and increase the diversity of the solution set by expanding the search range of the solution space; S64. Randomly perturb the flexible production line scheduling solution set retained after the search to increase population diversity, prevent getting trapped in local optima, and select the scheduling scheme with the highest fitness as the optimal flexible production line scheduling scheme based on the perturbed flexible production line scheduling solution set.

[0034] For example, suppose the flexible production line scheduling solution set includes Scheme 1, Scheme 2, Scheme 3, Scheme 4, and Scheme 5. Scheme 1 aims to maximize output with moderate energy consumption and a 10-hour delivery delay; Scheme 2 focuses on minimizing energy consumption with moderate output and an 8-hour delivery delay; Scheme 3 achieves balanced task allocation with moderate output and energy consumption and a 5-hour delivery delay; Scheme 4 has higher output and energy consumption and a 12-hour delivery delay; Scheme 5 has the shortest delivery time, lower output, and higher energy consumption. By providing feedback on the real-time execution of each scheme, production progress, equipment load, and target completion rate are collected.

[0035] Based on real-time feedback data, the fitness of each scheduling scheme is dynamically evaluated. Key evaluation indicators include output, energy consumption, and delivery time. Fitness is calculated for each scheme using preset weights. Assuming the fitness of scheme 1 is 0.8, scheme 2 is 0.85, scheme 3 is 0.83, scheme 4 is 0.69, and scheme 5 is 0.73, schemes with lower fitness are eliminated based on the fitness evaluation results; that is, schemes 4 and 5 are discarded.

[0036] The remaining solutions 1, 2, and 3 are optimized through local and global searches. Local searches fine-tune each solution, such as adjusting equipment load and task scheduling to optimize solution set details; global searches expand the scope of the solution space by reconfiguring task and material allocation to explore potential better solutions. This search process further improves the quality and diversity of solutions, avoiding getting trapped in local optima.

[0037] To further increase the diversity of the optimized solution set, a random perturbation mechanism is introduced. By randomly adjusting schemes 1, 2, and 3, such as fine-tuning equipment allocation and task priorities, the breadth of the search is enhanced, ensuring that the trap of local optima is avoided. After perturbation, the fitness of each scheme is re-evaluated, and the scheme most suitable for the current production environment is selected.

[0038] After all optimizations and perturbations, Scheme 2 was selected as the optimal scheduling scheme. The advantage of Scheme 2 lies in its ability to optimize energy consumption and delivery time while meeting production requirements. Finally, Scheme 2 was translated into specific production scheduling instructions and fed back to the production line for execution, ensuring the smooth completion of production tasks. Simultaneously, based on real-time collected feedback data, the scheduling scheme was continuously adjusted to ensure the continuous optimization of production targets.

[0039] In this embodiment, step S7 specifically includes: converting the optimal flexible production line scheduling scheme into a production scheduling instruction and feeding it back to the production line to execute the actual production task; continuously collecting production line operation status data during the execution process; evaluating the effectiveness of the scheduling scheme; and adjusting the scheduling scheme based on the evaluation results to ensure that the requirements of each production target are met.

[0040] Example 1: To verify the practical application effect of the invention, it was applied to a flexible production line in an automotive parts manufacturing plant. This production line mainly produces automotive parts such as engine parts, doors, and windows, and the production tasks frequently change, involving multiple production processes and equipment, resulting in problems such as uneven production task allocation, low resource utilization, and long production cycles.

[0041] During implementation, sensors and data acquisition devices are used to collect real-time status data of the production line, such as equipment start / stop status, workload, task progress, and resource utilization. The data is then cleaned, denoised, and timestamped to form a dynamic state vector. This dynamic state vector is input into an improved adaptive particle swarm optimization algorithm for global optimization, generating a preliminary flexible production line scheduling solution set. This preliminary flexible production line scheduling solution set is then input into a state evaluation network based on two-layer attention reinforcement learning for multi-objective optimization, comprehensively considering output, energy consumption, delivery time, and resource utilization. To address equipment failures and production delays, a state-objective causal path tracking mechanism is introduced. By tracking and analyzing changes in production status in real time, the weights and priorities of scheduling objectives are dynamically adjusted, and historical experience is applied to scheduling decision optimization through an experience replay mechanism. During optimization, a population resampling mechanism is used to filter the scheduling solution set during execution, eliminating inefficient solutions, performing local and global searches to ensure the diversity of the solution set, and a random perturbation mechanism is used to avoid local optima, selecting the scheduling scheme with the highest fitness for actual execution.

[0042] To further verify the effectiveness of this invention, four different scheduling methods were compared: traditional scheduling methods, particle swarm optimization methods, reinforcement learning methods, and a flexible production line scheduling method based on particle swarm optimization and reinforcement learning proposed in this invention. Traditional methods rely on classic heuristic scheduling rules and do not consider dynamic adjustment and optimization; particle swarm optimization methods perform global optimization of production scheduling through particle swarm optimization, but lack adaptability to changes in production status; reinforcement learning methods utilize reinforcement learning algorithms for adaptive optimization scheduling, but their training process is relatively slow when production status changes rapidly, making it difficult to quickly adapt to actual needs. Table 1 shows the comparison results of the proposed method with the above three methods in terms of production time, equipment utilization, output, energy consumption, and delivery time delay performance indicators.

[0043] Table 1. Performance Comparison of the Invention Method with Three Scheduling Methods

[0044] As shown in Table 1, the method of this invention outperforms traditional methods, particle swarm optimization (PSO), and reinforcement learning methods in all key performance indicators. Regarding production time, the method of this invention reduces production time by 24 hours compared to traditional methods, PSO by 10 hours, and reinforcement learning, through adaptive optimization, reduces production time by 5 hours. This demonstrates that the invention can efficiently adapt to dynamic changes in the production line, significantly shortening the production cycle and improving the overall operating efficiency of the production line. In terms of equipment utilization, the method of this invention achieves an equipment utilization rate of 88%, significantly higher than the 75% of traditional methods. The equipment utilization rates of PSO and reinforcement learning are 80% and 85%, respectively. This indicates that the invention can more effectively schedule resources, significantly reducing equipment idle time and waste, and improving production efficiency.

[0045] Furthermore, in terms of output, the method of this invention improves output by 6% compared to traditional methods. Especially under conditions of equipment failure and frequent changes in production tasks, this invention significantly improves scheduling flexibility and capacity by combining particle swarm optimization and reinforcement learning. Regarding energy consumption, traditional methods have the highest energy consumption, while the method of this invention reduces energy consumption by 6.7% through optimized scheduling, demonstrating that this invention not only improves production efficiency but also reduces energy consumption, meeting the energy conservation and emission reduction requirements of modern manufacturing. In terms of delivery time delay, the method of this invention reduces delivery time delay by 66.7%, significantly improving delivery timeliness. In contrast, other methods are less effective in this regard, especially traditional methods, which fail to effectively handle priority adjustments leading to longer delivery delays during production delays.

[0046] In summary, the method of the present invention demonstrates significant advantages in key performance indicators such as production time, equipment utilization, output, energy consumption, and delivery time delay, fully showcasing its high efficiency and practical application value in flexible production line scheduling.

[0047] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A flexible production line scheduling method based on particle swarm optimization and reinforcement learning, characterized in that, Includes the following steps: S1. Collect and preprocess the status data in the flexible production line to form a dynamic status vector; S2. Based on the dynamic state vector, the production tasks, equipment resources and material allocation are globally optimized by the improved adaptive particle swarm optimization algorithm to generate a preliminary flexible production line scheduling solution set. S3. Input the initial flexible production line scheduling solution set into a state evaluation network based on two-layer attention reinforcement learning for multi-objective optimization to obtain the optimized flexible production line scheduling solution set: S4. Optimize the learning process of multi-objective scheduling decision-making in the state evaluation network through a state-objective causal path tracing mechanism; S5. Execute the optimized flexible production line scheduling and continuously collect production line operation feedback data to adjust the status assessment network in real time. S6. By using a population resampling mechanism, the flexible production line scheduling solution set during the execution process is screened to obtain the optimal flexible production line scheduling scheme. S7. Transform the optimal flexible production line scheduling scheme into production scheduling instructions and feed them back to the production line to execute actual production tasks.

2. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S1 specifically includes: S11. Real-time collection of status data of workstations, equipment, workpieces and materials in the flexible production line through sensors and data acquisition devices. The status data includes equipment start-up and shutdown status, equipment workload, task execution progress, resource occupancy, order priority, production progress, material supply status, equipment fault information, process execution time, process scheduling, temperature, humidity, pressure and environmental parameters. S12. Preprocess the collected status data, including data cleaning, data denoising, and timestamp alignment. S13. Normalize the preprocessed state data to convert state data of different magnitudes and units into a unified range, forming a dynamic state vector.

3. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S2 specifically includes: S21. Based on dynamic state vectors, the production tasks, equipment resources, workstation allocation and material supply are globally optimized by adaptive particle swarm optimization algorithm to generate a preliminary flexible production line scheduling solution set. In the improved adaptive particle swarm optimization algorithm, each particle represents a scheduling scheme. The dependence of the particle on the historical optimal solution is dynamically adjusted through a dynamic memory reliability mechanism. The dynamic memory reliability mechanism adjusts the influence of the particle's historical solution through a time decay factor and evaluates the quality of the particle's scheduling scheme based on a real-time fitness evaluation function based on the production state. S22. Based on the current production status and environmental changes, calculate the credibility value of the particle's historical optimal solution, and dynamically adjust the particle's dependence on the historical solution. The credibility value of the historical optimal solution includes the causal influence at the current moment and the historical contribution. The causal influence at the current moment represents the difference between the current production status and the historical optimal solution, and the historical contribution is the contribution at the previous moment. The weight ratio between the causal influence at the current moment and the historical contribution is adjusted by a balance factor. S23. The dependence of particles on historical optimal solutions is dynamically adjusted by the time decay factor. The change of inertia weight is controlled by the difference between the maximum and minimum inertia weights. The inertia weight gradually decreases with the number of iterations during the particle swarm optimization process. S24. A fitness function is introduced to evaluate the scheduling scheme of each particle, and the individual optimal solution and global optimal solution of the particle are updated according to the evaluation results. The fitness function comprehensively considers three scheduling objectives: task completion, resource utilization efficiency and energy consumption. S25. Through multiple iterations of optimization, and by updating the individual optimal position and the global optimal position based on the fitness value of the particles, until the preset convergence condition is met or the maximum number of iterations is reached, the preliminary flexible production line scheduling solution set is output.

4. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S3 specifically includes: S31. The state evaluation network based on two-layer attention reinforcement learning includes a production process attention mechanism, a scheduling target attention mechanism, and a multi-target scheduling decision network. S32. The production process attention mechanism uses a weighted adjacency matrix to model the dependency relationship between each production process, analyzes the mutual influence between the current production state and the historical scheduling scheme, quantifies the cooperation and conflict between production processes, and obtains the production process attention weight coefficient. S33. The scheduling target attention mechanism dynamically adjusts the priority by monitoring the real-time changes in production status to adjust the priority of each scheduling target and evaluates the degree of influence of each status in multi-target scheduling optimization to obtain the scheduling target attention weight coefficient. S34. Based on the evaluation results of the attention mechanism of the production process and the attention mechanism of the scheduling target, integrate the dynamic influence of each production process and the scheduling target to generate a comprehensive optimization coefficient. S35. Based on the comprehensive optimization coefficient, the initial flexible production line scheduling solution is input into the multi-objective scheduling decision network. The network is then optimized through a multi-objective optimization function, taking into account maximizing output, minimizing energy consumption, optimizing delivery time, and maximizing resource utilization. The optimized flexible production line scheduling scheme is then output.

5. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 4, characterized in that, The multi-objective optimization function in step S34 comprehensively considers maximizing output, minimizing energy consumption, optimizing delivery time, and maximizing resource utilization. Maximizing output means maximizing the total output of production tasks completed in flexible production line scheduling. Minimizing energy consumption means minimizing the total energy consumed during scheduling. Optimizing delivery time means ensuring that each task can be completed within the predetermined time as much as possible. Maximizing resource utilization means maximizing the efficiency of all resources used in the production process.

6. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S4 specifically includes: S41. The state-target causal path tracing mechanism tracks and analyzes changes in production state in real time, quantifying the impact of these changes on scheduling targets. S42. Based on the causal relationship between the current production status and the scheduling objective, calculate the direct or indirect impact of each production status change on the achievement of the scheduling objective, and obtain the contribution of each status change to the scheduling objective. S43. Based on the contribution of each state change to the scheduling objective, when the contribution exceeds the set contribution threshold, the objective weight in the multi-objective scheduling decision is increased and the priority is improved. S44. Introduce an experience replay mechanism to store the data of historical state-target causal path in the experience pool. The data includes production status, scheduling decisions, rewards or penalties and subsequent production status. S45. Periodically, randomly select a set of historical experiences from the experience pool and optimize the current flexible production line scheduling plan through replay learning. S46. By replaying historical experience, the multi-objective scheduling decision network adjusts the flexible production line scheduling scheme according to changes in production status.

7. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S5 specifically includes: S51. Execute the optimized flexible production line scheduling solution set, which includes scheduling schemes suitable for the current production environment, inefficient solutions, and invalid solutions; S52. Continuously collect feedback data on production line operation, including production progress, equipment load, target completion rate, task execution status, resource utilization, equipment fault information, and production line efficiency. S53. Input the collected feedback data into the state evaluation network based on two-layer attention reinforcement learning to evaluate the fitness of each scheduling scheme in the flexible production line scheduling solution set in real time, and adjust the scheduling decision according to the current production status.

8. The flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S6 specifically includes: S61. The flexible production line scheduling solution set during the execution process is screened through a population resampling mechanism. The population resampling mechanism introduces dynamic fitness evaluation, combines local search and global search, and incorporates random perturbation. S62. Based on the real-time collected feedback data, dynamically calculate the fitness of each scheduling scheme and eliminate scheduling schemes with fitness below the set threshold. S63. Perform local and global searches on the selected flexible production line scheduling solution set: optimize the solution set details by making small-scale adjustments, and increase the diversity of the solution set by expanding the search range of the solution space; S64. Randomly perturb the flexible production line scheduling solution set retained after the search to increase population diversity, prevent getting trapped in local optima, and select the scheduling scheme with the highest fitness as the optimal flexible production line scheduling scheme based on the perturbed flexible production line scheduling solution set.

9. A flexible production line scheduling method based on particle swarm optimization and reinforcement learning according to claim 1, characterized in that, Step S7 specifically includes: converting the optimal flexible production line scheduling scheme into production scheduling instructions and feeding them back to the production line to execute actual production tasks. During the execution process, the production line operation status data is continuously collected to evaluate the effectiveness of the scheduling scheme and adjust the scheduling scheme according to the evaluation results to ensure that the requirements of each production target are met.

Images