Genetic algorithm of fusion graph attention reinforcement learning for solving steelmaking continuous casting production scheduling problem

Abstract

The invention provides a novel method for fusing graph attention reinforcement learning and a genetic algorithm for a steelmaking continuous casting production scheduling problem, and belongs to the technical field of hybrid flow shop scheduling. According to the algorithm, the attention mechanism of the graph neural network and the reinforcement learning strategy are combined on the basis of the genetic algorithm, and the solving effect of the actual manufacturing problem is remarkably improved. The specific process is as follows: in an initialization stage, using a random initialization mode to produce an initial population; then, two excellent individuals are selected according to the crossover probability to perform crossover operation to generate a filial generation, and intelligent agent-based mutation operation is executed according to the set mutation probability. In the variation process, a graph attention network is used for extracting production scheduling features, extracted feature data serve as input of a reinforcement learning algorithm, an intelligent agent outputs an action decision aiming at a specific position of a code, and local search optimization is carried out on the specific position of the code according to the action decision. According to the method, a variation mechanism driven by production scheduling characteristics is utilized, and the scheduling efficiency and the solution quality are improved.

Description

Technical Field

[0001] This invention belongs to the field of production scheduling optimization and intelligent computing technology, and relates to a genetic algorithm that integrates graph attention mechanism network (GAT) and proximal policy optimization (PPO) to solve the steelmaking continuous casting production scheduling problem. Background Technology

[0002] In modern manufacturing, production efficiency and resource optimization are crucial indicators of a company's core competitiveness. The mixed flow shop scheduling problem, a typical complex scheduling challenge in manufacturing systems, particularly in steelmaking and continuous casting, requires the scientific arrangement of multiple equipment and processes. The scheduling objectives include minimizing production cycles, reducing operating costs, and improving resource utilization while strictly meeting production constraints. However, traditional scheduling algorithms, failing to incorporate detailed production data such as deconstruction graphs, struggle to achieve precise optimization decisions, resulting in limited overall performance. In-depth analysis of production process characteristics allows for more accurate predictions of processing times and equipment availability for each process. Utilizing graph convolutional networks to extract production process features not only provides more effective guidance for optimization algorithms but also enhances the scientific rigor of scheduling decisions. The optimization algorithm proposed in this invention achieves multi-dimensional information fusion processing, significantly improving overall production efficiency, shortening delivery cycles, and promoting optimal resource allocation, providing crucial technical support for solving complex mixed flow shop scheduling problems.

[0003] With the rise of intelligent manufacturing, genetic algorithms, as typical evolutionary algorithms, drive the population to search for the optimal solution space through crossover and mutation operations. However, for the specific production scheduling problem of steelmaking continuous casting, the high randomness of crossover and mutation operations in traditional genetic algorithms leads to performance bottlenecks, generating a large number of invalid solutions and causing serious waste of computational resources. Deep reinforcement learning, as a data-driven intelligent algorithm, can efficiently explore promising state spaces by interacting with the environment and relying on reward mechanisms. Therefore, in this context, a genetic algorithm integrating graph attention reinforcement learning shows significant advantages for the steelmaking continuous casting production scheduling problem. This method extracts key features in the scheduling process through a graph attention network, uses deep reinforcement learning to guide the mutation direction of the genetic algorithm, effectively reduces invalid computation, avoids performance waste, and can quickly generate high-quality scheduling schemes. Summary of the Invention

[0004] This invention proposes a fusion graph attention reinforcement learning genetic algorithm for the production scheduling problem in steelmaking continuous casting, applied to the field of hybrid flow shop scheduling. This algorithm, through in-depth analysis of production process characteristics, guides the search direction in a targeted manner, effectively reducing the consumption of ineffective computational resources. Addressing the problem of insufficient utilization of feature information in steelmaking continuous casting production scheduling, this invention achieves more efficient feature mining and application, optimizing the problem of excessive energy consumption in actual processes. The specific implementation steps can be completed sequentially on a computer system as follows:

[0005] : Construct production information for workpieces to be scheduled and build a task scheduling architecture;

[0006] Initialize and set the following basic variables;

[0007] Each workpiece that needs to be processed, ;

[0008] The number of castings for the workpiece. ;

[0009] Number of processing machines on the production line ;

[0010] Number of production stages The first stage is the steelmaking stage, the second stage is the refining stage, and the third stage is the continuous casting stage.

[0011] : workpiece Processing time in stage P;

[0012] Pouring time Preparation time before processing begins;

[0013] : workpiece In the stage Start processing time;

[0014] Set the relevant parameters for the algorithm:

[0015] Population size;

[0016] Crossover probability;

[0017] : Probability of mutation;

[0018] : Agent learning rate;

[0019] Individual code: The code indicates the order in which workpieces are processed, and each element represents a furnace batch;

[0020] Graph convolutional state representation transforms the individual encoding of the scheduling scheme into a heterogeneous graph structure containing operation nodes, workpiece nodes, and encoding nodes;

[0021] Population initialization, randomly generated individual;

[0022] Based on fitness, use roulette wheel to select superior individuals;

[0023] : Select two superior individuals based on the crossover probability to perform cross-mapping to produce offspring;

[0024] : Mutate superior individuals according to the set mutation probability;

[0025] : Utilize graph attention networks to extract key features in the production process, and capture the complex relationships between nodes such as operations and workpieces in the scheduling scheme through attention mechanisms;

[0026] The feature data extracted by GAT is used as the input to the reinforcement learning algorithm, and the agent outputs action decisions for specific encoded locations.

[0027] Based on the guidance information output by the intelligent agent, local search optimization is performed on the specific location of the code to accurately improve the scheduling scheme and enhance the overall scheduling performance;

[0028] During the iteration process, the fitness of each agent is evaluated and the agent's reward is calculated, while the agent learns online.

[0029] If the algorithm reaches the termination condition, it outputs the optimal individual as the final solution; otherwise, it returns... Continue iterating. Attached Figure Description

[0030] Figure 1 A schematic diagram of the system architecture for fusion graph attention reinforcement learning to solve the steelmaking continuous casting production scheduling problem.

[0031] Figure 2 A flowchart of a genetic algorithm for fusion graph attention reinforcement learning to solve the steelmaking continuous casting production scheduling problem.

[0032] Figure 3 The convergence curve of the genetic algorithm for fusion graph attention reinforcement learning used in Example 1 after 400 iterations is shown in the figure.

[0033] Figure 4 Gantt graph of the results of the genetic algorithm for fusion graph attention reinforcement learning in solving the steelmaking continuous casting production scheduling problem in Example 3. Detailed Implementation

[0034] The present invention will now be described in detail with reference to specific embodiments.

[0035] A genetic algorithm for fusion graph attention reinforcement learning is proposed to solve the production scheduling problem in steelmaking continuous casting. The algorithm is implemented using the following steps:

[0036] : Construct production information for workpieces to be scheduled and build a task scheduling architecture;

[0037] : Set algorithm-related parameters;

[0038] Population size;

[0039] Crossover probability;

[0040] : Probability of mutation;

[0041] : Agent learning rate;

[0042] Population initialization, randomly generated individual;

[0043] Based on fitness, use roulette wheel to select superior individuals;

[0044] : Select two superior individuals based on the crossover probability to perform cross-mapping to produce offspring;

[0045] : Mutate superior individuals according to the set mutation probability;

[0046] During the iteration process, the fitness of each agent is evaluated and the agent's reward is calculated, while the agent learns online.

[0047] If the algorithm reaches the termination condition, output the optimal individual as the final solution; otherwise, return. Continue iterating;

[0048] The population size is 100;

[0049] The deep reinforcement learning rate is 1e-5;

[0050] The crossover probability is 0.7;

[0051] The mutation probability is 0.5;

[0052] Through simulation experiments, the improved algorithm of this invention ( Verification and genetic algorithm for solving the steel production scheduling problem. ), Artificial bee colony algorithm ( To compare, the ten cases 1-5 were run, and the average value of each case was compared 10 times.

[0053] Table 1 , Result comparison:

[0054] IGA GA ABC 1 11573.48 34255.61 61936.66 2 10015.47 34148.43 58840.26 3 19676.98 34500.26 62216.11 4 11369.43 31700.95 61743.24 5 12351.09 34440.83 62212.39

[0055] By analyzing Table 1 and Figure 3 We can see that the method proposed in this invention can achieve better standards in terms of both convergence speed and the degree of excellence of the solution, because the improved algorithm has strong local search ability and global search ability.

Claims

1. A genetic algorithm for fusion graph attention reinforcement learning to solve the steelmaking continuous casting production scheduling problem is characterized by: A scheduling method based on genetic algorithms is proposed for solving the production scheduling problem of continuous casting in steelmaking. This method is implemented on a computer following these steps: : Construct production information for workpieces to be scheduled and build a task scheduling architecture; Initialize and set the following basic variables; Each workpiece that needs to be processed, ; The number of castings for the workpiece. ; Number of processing machines on the production line ; Number of production stages The first stage is the steelmaking stage, the second stage is the refining stage, and the third stage is the continuous casting stage. : workpiece Processing time in stage P; Pouring time Preparation time before processing begins; : workpiece In the stage Start processing time; Set the relevant parameters for the algorithm: Population size; Crossover probability; : Probability of mutation; : Agent learning rate; Individual code: The code indicates the order in which workpieces are processed; each element represents a workpiece. Population initialization, randomly generated individual; Graph convolutional state representation transforms the individual encoding of the scheduling scheme into a heterogeneous graph structure containing operation nodes, workpiece nodes, and encoding nodes; Based on fitness, use roulette wheel to select superior individuals; : Select two superior individuals based on the crossover probability to perform cross-mapping to produce offspring; Based on a set mutation probability, superior individuals are mutated to produce offspring. During the iteration process, the fitness of each agent is evaluated and the agent's reward is calculated, while the agent learns online. If the algorithm reaches the termination condition, the optimal individual is output as the final solution. Otherwise return Continue iterating.

2. The method according to claim 1, characterized in that: To handle information at different levels of granularity, the construction of the heterogeneous graph includes a node set V and an edge set E. The node set V includes three types of nodes: operation nodes, workpiece nodes, and encoding nodes. The edge set E includes four types of edges: operation predecessor edges, operation and workpiece belonging edges, encoding sequence edges, and same machine edges.

3. The method according to claim 1, characterized in that: The operation precursor edge is used to represent the sequential relationship between operations; the operation and workpiece affiliation edge is used to represent the affiliation relationship between the workpiece and its corresponding operation node; and the encoding sequence edge is used to represent the sequential relationship of the encoding nodes in the scheduling scheme.

4. The method according to claim 1, characterized in that: The graph attention network aggregates node features of heterogeneous graphs through a multi-layer graph attention mechanism.

5. The method according to claim 1, characterized in that: The feature data extracted by the graph attention network is used as the input to the reinforcement learning algorithm, and the agent outputs action decisions for encoding specific locations.

Images