A multi-objective flexible job shop scheduling optimization method and system fusing graph embedding and reinforcement learning

Abstract

This invention belongs to the field of multi-objective flexible production scheduling technology in manufacturing, specifically a method and system for optimizing multi-objective flexible job shop scheduling that integrates graph embedding and reinforcement learning. The method includes: acquiring an initial population for a multi-objective flexible job shop scheduling problem, where each individual in the initial population is represented using a three-layer hybrid encoding mechanism, including an operation sequence, a machine sequence, and a speed sequence; converting the individual information in the population into a binary fuzzy disjunctive graph structure, where the graph structure includes operation nodes and edges representing process constraints and machine allocation relationships; extracting graph embedding features from the binary fuzzy disjunctive graph using a graph neural network; inputting the graph embedding features into a policy network and iteratively optimizing the population based on the optimization actions; and implementing a two-stage energy-saving strategy on the optimized population, reconstructing the machine sequence and optimizing the speed sequence to reduce the total fuzzy energy consumption while satisfying the fuzzy maximum completion time constraint.

Description

Technical Field

[0001] This invention belongs to the field of multi-objective flexible production scheduling technology in the manufacturing industry. Specifically, it is a multi-objective flexible job shop scheduling optimization method and system that integrates graph embedding and reinforcement learning, aiming to solve the complexity and uncertainty in multi-objective flexible job shop scheduling problems with two types of fuzzy time. Background Technology

[0002] With the deepening of economic globalization and the intensification of market competition, modern manufacturing is facing unprecedented challenges. Manufacturing enterprises not only need to continuously expand production scale to meet demand, but also must actively respond to increasing environmental pressures, especially the requirements of green energy management and carbon emission reduction targets. Against this backdrop, reducing total energy consumption and improving production efficiency have become core concerns for the survival and development of enterprises.

[0003] As a key component of modern manufacturing, the aluminum industry, especially its energy-intensive electrolytic aluminum production process, has become a focus of optimization research. To systematically address its production scheduling problem, this process has been abstracted as the Flexible Job Shop Scheduling Problem (FJSP). Given the significant energy consumption and cost pressures brought about by the expansion of manufacturing scale, single-objective optimization is insufficient. Therefore, this research needs to model the electrolytic aluminum process as a Multi-Objective Flexible Job Shop Scheduling Problem (MOFJSP), aiming to simultaneously optimize interrelated and even conflicting key objectives such as maximum completion time and total energy consumption.

[0004] The real-world production environment is highly complex, with factors such as machine wear, excessive temperatures, and material supply delays making accurate prediction of task processing time extremely difficult and exhibiting significant uncertainty. To more realistically reflect the realities of flexible manufacturing, introducing the concept of fuzzy time is crucial.

[0005] Existing technical solutions mainly rely on traditional heuristic and metaheuristic algorithms (such as genetic algorithms, particle swarm optimization, etc.). These methods usually construct models that simultaneously optimize the maximum completion time and total energy consumption, and introduce two types of fuzzy time theory to characterize the uncertainty of processing time caused by factors such as machine wear and material delay. However, such traditional multi-objective optimization algorithms generally have limitations when dealing with NP-hard problems with large solution spaces and uncertainty constraints: (1) Low computational efficiency and serious time consumption: Traditional algorithms are limited by the large solution space and NP-hard characteristics, and need to traverse a large number of invalid solutions, resulting in slow convergence speed and difficulty in quickly obtaining high-quality solutions in large-scale problems. (2) Insufficient dynamic adjustment and response capabilities: Existing methods lack real-time state perception capabilities and cannot dynamically adjust the search direction during the optimization process, resulting in blind and inefficient search processes and delayed response to disturbances such as machine failures and emergency orders. (3) Unstable optimization quality and high cost: Traditional methods are prone to getting trapped in local optima, and the solution quality fluctuates greatly. In addition, the failure to effectively coordinate machine processing speed and fuzzy time constraints leads to actual energy consumption exceeding expectations and increases production costs. (4) Deficiencies in solution set quality and diversity: Existing methods struggle to obtain high-quality and diverse non-dominated solution sets. In particular, schemes combining reinforcement learning and heuristics generally suffer from the problem of balancing global exploration and local development.

[0006] To address the aforementioned technical bottlenecks, this invention proposes a reinforcement learning framework (RLFGEM) based on graph embedding mechanism to effectively solve the key challenges in MOT2FJSP, and demonstrates significant application value, especially in high-energy-consuming industrial scenarios. Summary of the Invention

[0007] This invention aims to solve the aforementioned problems in the prior art and proposes a multi-objective flexible job shop scheduling optimization method and system that integrates graph embedding and reinforcement learning. By integrating graph neural networks, reinforcement learning, and a two-stage energy-saving strategy, this system can efficiently and intelligently handle multi-objective flexible job shop scheduling problems with two types of fuzzy time, achieve a good balance between global and local search, and generate high-quality and diverse scheduling schemes.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows:

[0009] A multi-objective flexible job shop scheduling optimization method integrating graph embedding and reinforcement learning includes the following steps:

[0010] Step 1: Obtain an initial population for the multi-objective flexible job shop scheduling problem. Each individual in the initial population is represented using a three-layer hybrid encoding mechanism, which includes an operation sequence, a machine sequence, and a speed sequence. Step 2: Transform the individual information in the population into a binary fuzzy disjunctive graph structure. The graph structure includes operation nodes and edges representing process constraints and machine allocation relationships. Step 3: Extract graph embedding features from the binary fuzzy disjunctive graph using a graph neural network. Step 4: Input the graph embedding features into a policy network to generate optimization actions for each individual, and iteratively optimize the population based on these optimization actions. Step 5: Implement a two-stage energy-saving strategy on the optimized population, reconstructing the machine sequence and optimizing the speed sequence respectively, to reduce the total fuzzy energy consumption while satisfying the fuzzy maximum completion time constraint.

[0011] Preferably, in step one, the initial population is generated through multiple initialization rules, including a maximum speed rule, a minimum speed rule, a load balancing rule, and a random allocation rule; wherein, the maximum speed rule is used to minimize the maximum completion time, the minimum speed rule is used to minimize total energy consumption, the load balancing rule is used to balance machine load, and the random allocation rule is used to increase population diversity. Specifically:

[0012] 1) Maximum speed rule: Initialize the process sequence and machine sequence randomly, and execute all operations at the maximum speed to minimize the fuzzy completion time.

[0013] 2) Minimum speed rule: Randomly initialize the process sequence and machine sequence, and execute all operations at the minimum speed to minimize fuzzy energy consumption.

[0014] 3) Load balancing rule: Sort all operations in ascending order of the number of available machines and assign them to the machines with the least current workload.

[0015] 4) Random allocation rules: Randomly initialize the process sequence, machine sequence, and speed sequence.

[0016] Preferably, in step two, the two types of fuzzy parsing graph structure specifically includes: a start node, an end node, an operation node, a solid directed edge representing the sequence of operations, and a colored dashed edge representing the connection between operations on the same machine; the operation node is attached with a two types of fuzzy processing time weight after speed normalization.

[0017] Preferably, in step three, the graph neural network is a graph isomorphic network.

[0018] Preferably, in step four, the policy network is a multi-layer fully connected network, and the policy network is trained based on the REINFORCE algorithm, the objective of which is to maximize the maximum completion time determined by fuzzy conditions (…). ) and fuzzy total energy consumption ( The cumulative reward is calculated based on the changes. The strategy model uses parameters... Define a random mapping from state to action probability. An individual's state is encoded by its disjunctive graph, while its actions are sampled from the output distribution of the policy network. For each individual in the population, the actions before and after policy application are calculated separately. and

[0019] Differences and And by using a normalization function, the overall improvement degree is... Mapped to the [0,1] interval. As shown in formula (1). Reward function The reward function is mapped to a range of 0-100 based on the improvement of the target. This reward function effectively incentivizes strategies to evolve towards Pareto improvement.

[0020] (1)

[0021] A multi-objective flexible job shop scheduling optimization system integrating graph embedding and reinforcement learning includes an encoding / decoding module, a population initialization module, a graph construction module, a strategy model module, and a two-stage energy-saving optimization module;

[0022] The encoding / decoding module is used to represent the scheduling scheme for a multi-objective flexible job shop scheduling problem with two types of fuzzy time using a three-layer hybrid encoding mechanism. The three-layer hybrid encoding includes operation sequences, machine sequences, and speed sequences. An executable scheduling scheme and corresponding fuzzy maximum completion time and fuzzy total energy consumption index are generated through dynamic insertion strategy decoding. The population initialization module generates an initial population using multiple initialization rules. The graph construction module transforms the workpiece sequence, machine assignment, and speed selection information contained in the individuals in the population into a two-type fuzzy disjunctive graph structure. The graph structure includes operation nodes and edges representing process constraints and machine allocation relationships. The strategy model module includes a graph neural network and a strategy network. The graph neural network extracts graph embedding features from the two-type fuzzy disjunctive graph, and the strategy network generates customized optimization actions for each individual based on the graph embedding features and updates the network parameters through a reinforcement learning algorithm. The two-stage energy-saving optimization module optimizes machine allocation and speed selection without changing the workpiece processing sequence. The first stage reconstructs the machine sequence, and the second stage optimizes the speed sequence.

[0023] Preferably, the population initialization module employs multiple initialization rules, including: maximum speed rule: randomly initializing the work sequence and machine sequence and selecting the maximum speed for all operations; minimum speed rule: randomly initializing the work sequence and machine sequence and selecting the minimum speed for all operations; load balancing rule: arranging all operations in ascending order of the number of available machines and assigning them to the machine with the lowest current workload; and random allocation rule: randomly initializing the job sequence, machine sequence, and speed sequence.

[0024] Preferably, in the second-class fuzzy parsing diagram, the numerical values ​​attached to the operation nodes are obtained by dividing the second-class fuzzy processing time by the corresponding speed and then normalizing it; the edges representing the sequence of operations are solid directed edges that strictly correspond to the workpiece process flow; the edges representing the machine allocation relationship are colored dashed edges, with different colors used to distinguish different processing machines.

[0025] Preferably, in the strategy model module, the graph neural network adopts a graph isomorphic network, the strategy network is a multi-layer fully connected network, and the parameters are updated based on the REINFORCE algorithm, guided by the cumulative reward calculated from the changes in fuzzy maximum completion time and fuzzy total energy consumption.

[0026] Preferably, in the two-stage energy-saving optimization module: the first stage reconstructs machine allocation without changing the job sequence to reduce idle time and processing time; the second stage optimizes the speed sequence while maintaining the job and machine sequences, reducing system energy consumption by slowing down the operation without extending the fuzzy maximum completion time.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] (1) Significantly improve computational efficiency: By integrating the reward mechanism and the REINFORCE algorithm, the global search capability and efficiency are greatly enhanced, and the blind search time is effectively reduced.

[0029] (2) Enhance dynamic adjustment capability: Introduce reinforcement learning technology to enhance local search capability, so that the algorithm can dynamically adjust the search direction according to the real-time system state and the feedback of individuals in the population during the iteration process, thereby improving search efficiency and accuracy.

[0030] (3) Optimize the quality and diversity of the solution: A two-stage energy-saving strategy is adopted. For the population individuals obtained by encoding and decoding, the workpiece sequence, machine allocation sequence and speed selection sequence are finely adjusted respectively, which significantly improves the quality of the solution and ensures the diversity of the solution set.

[0031] (4) Achieving a balance between global and local search: By using a hybrid search optimization algorithm, graph neural networks and REINFORCE algorithm are organically combined to effectively integrate global exploration capabilities and local development capabilities, thus avoiding getting trapped in local optima. Attached Figure Description

[0032] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0033] Figure 1 This is a framework diagram of the optimization system of the present invention.

[0034] Figure 2 This is a diagram illustrating the representation of each solution in the population in this invention.

[0035] Figure 3 This is a specific example diagram of the two types of fuzzy disjunctive graphs in this invention.

[0036] Figure 4 This is an example diagram of the two-stage energy-saving strategy in the algorithm framework of this invention.

[0037] Figure 5 This is a graph showing the ANOVA experimental results of the present invention.

[0038] Figure 6 This is a range chart for analyzing the effectiveness of the strategy of this invention.

[0039] Figure 7 This is a graph showing the comparison algorithm results of the present invention.

[0040] Figure 8 This is a Pareto front plot of the comparison algorithm results of this invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1

[0043] A multi-objective flexible job shop scheduling optimization method and system integrating graph embedding and reinforcement learning, comprising the following core modules:

[0044] 1) Problem Encoding / Decoding Module: For the multi-objective flexible job shop scheduling problem under fuzzy time conditions, a three-layer hybrid encoding mechanism is proposed, representing the scheduling scheme from three levels: operation sequence, machine sequence, and velocity sequence. The operation sequence determines the processing order of jobs, the machine sequence specifies the equipment used for each operation, and the velocity sequence controls the processing rate of the corresponding equipment. During decoding, a dynamic insertion strategy is employed to transform the encoded structure into an executable scheduling scheme, thereby generating quantitative indicators for fuzzy maximum completion time and total energy consumption.

[0045] 2) Hybrid Population Initialization Module: This module designs four initialization rules for constructing the initial population. These include: Maximum Speed ​​Rule: Randomly initializes the job sequence and machine sequence, selecting the maximum speed for all operations to minimize the maximum completion time. Minimum Speed ​​Rule: Randomly initializes the job sequence and machine sequence, selecting the minimum speed for all operations to minimize total energy consumption. Load Balancing Rule: Sorts all operations in ascending order of the number of selectable machines, assigning all operations to the machine with the lowest current workload. Random Allocation Rule: Randomly initializes the job sequence, machine sequence, and speed sequence to increase the randomness and diversity of the population.

[0046] 3) Binary Fuzzy Disjunctive Graph Construction Module: To convert sequence data into a graph structure to support deep learning processing, this invention constructs a binary fuzzy disjunctive graph. This graph contains two virtual nodes, a start node and an end node. Operation nodes are accompanied by fuzzy temporal weights. Solid arrows represent the sequential relationship between processes, and dashed lines of different colors are used to distinguish different processing machines, providing a structured graph representation foundation for subsequent feature extraction.

[0047] 4) Policy Model Module: The policy model module consists of a graph neural network and a policy network. It uses a graph isomorphic network to extract features from the disjunctive graph, effectively capturing information from the graph structure and converting it into vector form. Through multiple neural network modules, it generates feature vectors representing the entire graph. The policy network is a multi-layered fully connected network. The extracted graph embedding features are fed into the policy network to generate a probability distribution. Based on these probabilities, selections are made and transformations are applied to each individual to form a new individual.

[0048] 5) Two-stage energy-saving strategy module: This module addresses the characteristics of the problem by reconstructing the machine sequence and optimizing the speed sequence to reduce the fuzzy maximum completion time and total fuzzy energy consumption from different perspectives. The first stage reconstructs machine allocation without changing the job sequence to reduce idle time and processing time; the second stage optimizes the speed sequence while maintaining the job and machine sequences, effectively reducing system energy consumption through a deceleration strategy, while ensuring that the maximum completion time is not extended.

[0049] The overall process of the RLFGEM framework is as follows: First, an initial population is generated through diversified rules; then, the sequence information is transformed into a graph structure using a binary fuzzy disjunctive graph; next, features are extracted using graph embedding methods and input into the policy network to generate an optimized policy for each individual and update the population; finally, the network parameters are iteratively updated using the policy gradient method. Through continuous training, this framework can adaptively find the optimal scheduling policy for each individual, thereby achieving multi-objective scheduling optimization in fuzzy and uncertain environments.

[0050] In the scheduling environment of this invention, each process can be processed on multiple selectable machines, and the processing time varies on different machines. Simultaneously, due to the introduction of Type II fuzzy time, the job completion time is represented as an interval Type II fuzzy number, which needs to be divided by the corresponding speed to determine the actual processing time. This allows for the simultaneous evaluation of two conflicting objectives: the maximum fuzzy completion time and the total fuzzy energy consumption.

[0051] Example 2

[0052] like Figure 1 As shown, this invention proposes a reinforcement learning framework based on graph embedding mechanisms to handle the multi-objective flexible job shop scheduling problem with fuzzy time. This framework aims to address the inherent complexity and uncertainty of this type of problem by designing an efficient and intelligent optimization algorithm to achieve an effective balance between global exploration and local optimization, thereby efficiently obtaining high-quality non-dominated solutions. Its core architecture mainly includes population initialization, binary fuzzy disjunctive graph representation, policy model construction, and a two-stage energy-saving strategy, forming a unified reinforcement learning optimization process.

[0053] In the population initialization phase, this invention evenly divides the initial population into four parts, and generates corresponding individuals according to the maximum speed rule, minimum speed rule, load balancing rule, and random allocation rule, respectively. For example... Figure 2 As shown, each individual contains workpiece sequence, machine assignment, and speed selection information. This diverse initialization mechanism helps improve the algorithm's convergence efficiency while reducing computational resource consumption.

[0054] Subsequently, the information contained in each individual is uniformly represented using a type-two fuzzy disjunctive graph. For example... Figure 3 As shown, the diagram uses black arrows to indicate the sequence of processes, strictly corresponding to the workpiece's technological flow; colored lines represent the connection between processes on the same machine, reflecting the machine allocation scheme. The numerical value corresponding to each operation node is obtained by dividing the second-class fuzzy processing time by the corresponding speed and then normalizing it, thereby achieving the integration and accurate expression of various complex information.

[0055] The policy model, the core of the entire framework, is built upon Markov decision processes. This model treats the entire population as a whole and each individual as a state node in a reinforcement learning iteration. By applying customized actions (i.e., optimized policies for their sequences) output by the policy network to each individual, the changes in maximum fuzzy completion time and total fuzzy energy consumption before and after policy implementation are evaluated. This is used to calculate cumulative rewards, guiding the updates of the policy network and the graph embedding network. The policy network is trained using the REINFORCE algorithm, enabling it to assign appropriate optimized actions to different individuals based on the embedding features extracted from the fuzzy disjunctive graph.

[0056] To further reduce energy consumption, this invention also proposes a two-stage energy-saving strategy. Since the total energy consumption is highly sensitive to machine idle time and processing speed, this strategy optimizes machine allocation and speed selection separately without changing the workpiece processing sequence.

[0057] like Figure 4 As shown, the two-stage energy-saving strategy module is the key module of the entire framework, because The high sensitivity to processing time and idle time necessitates the design of a two-stage adjustment strategy. This strategy optimizes the machine sequence and speed sequence separately without altering the overall structure, thereby significantly reducing processing time and idle time. The first stage reduces idle and processing time in each process by refactoring the machine sequence. Its core idea is to reorganize the machine sequence, for example, in... Figure 4 In (A), in the allocated operation , , Based on this, for operation Reassign positions among its selectable machines m1, m2, and m3. And calculate separately. and The results showed that although m3 had the shortest completion time, its longest processing time resulted in higher energy consumption, while m2 achieved a better balance between the two, thus taking into account both energy consumption and completion time. The second stage optimized the speed sequence by employing deceleration to further reduce total energy consumption without prolonging the maximum fuzzy completion time. The main idea was to optimize the speed sequence through a deceleration strategy, such as... Figure 4 As shown in (B), by adjusting the operation The speed is kept constant so that the start time remains unchanged. Although the end time increases, it is controlled within the overall maximum completion time (26), thereby achieving an effective reduction in energy consumption while maintaining the completion time unchanged.

[0058] Figure 5The ANOVA experimental results of this invention are presented. The main effects plot based on hypervolume (HV) and inversion generation distance (IGD) validates the statistical analysis results, indicating that RLFGEM, with the parameter configuration of... , , The optimal performance is achieved when the population size is too large. An excessively large population size increases the computational burden of local search and limits the effective learning ability of the policy network, while an excessively small population weakens the global search ability. Both of these factors have an adverse effect on the overall algorithm performance. and Deviating from the optimal parameter settings can also lead to performance degradation, reflecting the sensitivity of policy gradient-based neural network structures to these parameters. Suboptimal parameter combinations can disrupt the algorithm's balance: lower... With higher It is prone to causing unstable and short-sighted updates, and has a higher probability of causing such problems. With lower This could lead to overly conservative updates and slow convergence. Inappropriate parameter selection ultimately results in slower convergence, unstable training, and consequently, reduced solution quality. The comparative results show that the optimization system proposed in this invention exhibits significant advantages across test instances of varying scales.

[0059] Figure 6 The effectiveness of the policy model module, the two-stage energy-saving policy module, and the population initialization module was verified separately. Three variants of RLFGEM were introduced in the experiment: S1 (replacing the post-training policy with a randomly selected policy), S2 (eliminating the two-stage energy-saving policy and proceeding directly to the next iteration), and S3 (using random rules instead of the rules designed for the problem in the two-stage energy-saving policy module). The original RLFGEM was denoted as S, and tests were conducted under the same conditions. The interval plot shows that S consistently performed best across all evaluation metrics, fully demonstrating the combined effect of the three modules in guiding the search toward the Pareto front while generating well-distributed non-dominated solutions. S consistently outperformed S1, S2, and S3, indicating that the policy model module, the two-stage energy-saving policy module, and the population initialization module played a crucial role in improving solution coverage, convergence, diversity, and dominance quality.

[0060] Figure 7The interval plots show the performance of this invention compared to other comparative algorithms. RLFGEM outperforms the comparative algorithms in both HV and IGD metrics, and the interval plots further validate the statistical significance of this performance advantage. When the instance size is small, the limited runtime due to the need for a wider training of the policy network may result in a performance improvement slightly below the optimal level. As the instance size increases, the advantage of RLFGEM becomes more pronounced. The well-trained policy network can effectively select high-quality policies for each individual, thereby accelerating population evolution, while baseline algorithms typically require longer computation time to explore the solution space. The two-stage energy-saving strategy integrated in RLFGEM provides additional performance gains through further improvement of the obtained solutions. As can be seen from the interval plots, in all evaluation metrics, the average performance interval of RLFGEM is significantly separated from and superior to the comparative algorithms at the 95% confidence level, exhibiting a higher median and a more concentrated and stable distribution, fully demonstrating its superior performance in convergence and solution set diversity. When solving MOT2FJSP, RLFGEM outperforms existing technologies in both the quantity and quality of solutions, and can obtain better Pareto solutions, further highlighting its significant advantages over the comparison algorithms.

[0061] Figure 8 The Pareto front plot provides direct macroscopic validation of the algorithm's effectiveness. The solution set generated by RLFGEM is consistently located closer to the origin, representing the dual objectives of minimizing completion time and energy consumption. A broader and more uniform solution distribution is observed on the tradeoff curve, visually confirming the better HV (larger dominance region) and lower IGD (closer to the ideal front) quantified in earlier analyses. This highlights RLFGEM's ability to produce high-quality, diverse Pareto solution sets. RLFGEM's superior performance is attributed to its collaborative architecture, particularly evident in medium to large problem instances. The policy network's ability to learn and generate scheduling policies enables efficient and targeted exploration of complex search spaces. This learned guidance is subsequently improved through a specialized two-stage energy-saving strategy to perform precise local optimizations. In contrast, baseline algorithms relying on fixed evolutionary operators require greater computational effort for undirected exploration, a limitation reflected in their poorer metrics and longer convergence times.

[0062] The above content elaborates on the basic principles, core features, and technical advantages of the optimization system of this invention. It should be emphasized that this invention is not limited to the above examples; the descriptions in the specification are merely illustrative. Regarding the technical solutions proposed in Part IV of this invention, there are multiple alternative technical paths that can achieve the same inventive objective. For example, in the population initialization stage, more heuristic rules can be introduced; in the construction and representation of the disjunctive graph, heterogeneous graph neural networks or sequence models such as Transformers can be used; in the policy model, graph attention networks or Actor-Critic architectures can be used; and in the two-stage energy-saving strategy, it can be adjusted to single-stage joint optimization. These alternative solutions can all achieve similar performance through different technical paths while maintaining the core objective of the invention. The application scope of this invention can be extended to any field requiring a scheduling optimization system. Without departing from the core idea of ​​this invention, those skilled in the art can make various improvements and modifications to this invention, and these improvements and modifications all fall within the protection scope defined by the claims of this invention and their equivalent technical solutions.

[0063] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A multi-objective flexible job shop scheduling optimization method integrating graph embedding and reinforcement learning, characterized in that, Includes the following steps: Step 1: Obtain the initial population for the multi-objective flexible job shop scheduling problem. Each individual in the initial population is represented by a three-layer hybrid coding mechanism, which includes operation sequence, machine sequence, and velocity sequence. Step 2: Transform the individual information in the population into a two-class fuzzy disjunctive graph structure, wherein the graph structure includes operation nodes and edges representing process constraints and machine allocation relationships; Step 3: Extract the graph embedding features of the two types of fuzzy disjunctive graphs using a graph neural network; Step 4: Input the graph embedding features into the policy network to generate optimization actions for each individual, and iteratively optimize the population based on the optimization actions; Step 5: Implement a two-stage energy-saving strategy on the optimized population, reconstructing the machine sequence and optimizing the speed sequence respectively, in order to reduce the total fuzzy energy consumption while satisfying the fuzzy maximum completion time constraint.

2. The method according to claim 1, characterized in that, In step one, the initial population is generated through multiple initialization rules, including a maximum speed rule, a minimum speed rule, a load balancing rule, and a random allocation rule. The maximum speed rule is used to minimize the maximum completion time, the minimum speed rule is used to minimize the total energy consumption, the load balancing rule is used to balance the machine load, and the random allocation rule is used to increase population diversity.

3. The method according to claim 1, characterized in that, In step two, the two types of fuzzy parsing graph structure specifically includes: a start node, an end node, an operation node, a solid directed edge representing the sequence of operations, and a colored dashed edge representing the connection between operations on the same machine; the operation node is attached with a two types of fuzzy processing time weight after speed normalization.

4. The method according to claim 1, characterized in that, In step three, the graph neural network is a graph isomorphic network.

5. The method according to claim 1, characterized in that, In step four, the policy network is a multi-layer fully connected network, which is trained based on the REINFORCE algorithm to maximize the cumulative reward calculated from the changes in fuzzy maximum completion time and fuzzy total energy consumption.

6. A multi-objective flexible job shop scheduling optimization system integrating graph embedding and reinforcement learning, characterized in that, It includes an encoding / decoding module, a population initialization module, a graph construction module, a strategy model module, and a two-stage energy-saving optimization module; The encoding and decoding module is used to represent the scheduling scheme for the multi-objective flexible job shop scheduling problem with two types of fuzzy time using a three-layer hybrid encoding mechanism. The three-layer hybrid encoding includes operation sequence, machine sequence and speed sequence. An executable scheduling scheme and the corresponding fuzzy maximum completion time and fuzzy total energy consumption index are generated by decoding through a dynamic insertion strategy. The population initialization module is used to generate an initial population through various initialization rules; The graph construction module is used to transform the work sequence, machine assignment and speed selection information contained in the individuals in the population into a two-class fuzzy disjunctive graph structure. The graph structure includes operation nodes and edges representing process constraints and machine allocation relationships. The strategy model module includes a graph neural network and a policy network. The graph neural network is used to extract graph embedding features from a binary fuzzy disjunctive graph. The policy network is used to generate customized optimization actions for each individual based on the graph embedding features and update the network parameters through a reinforcement learning algorithm. The two-stage energy-saving optimization module is used to optimize machine allocation and speed selection without changing the workpiece processing sequence. The first stage reconstructs the machine sequence, and the second stage optimizes the speed sequence.

7. The system according to claim 6, characterized in that, The population initialization module employs several initialization rules, including: Maximum speed rule: Initialize the process sequence and machine sequence randomly, and execute all operations at the maximum speed to minimize the fuzzy completion time; Minimum speed rule: Randomly initialize the process sequence and machine sequence, and execute all operations at the minimum speed to minimize fuzzy energy consumption; Load balancing rule: Sort all operations in ascending order of the number of available machines and assign them to the machines with the least current workload. Random allocation rules: Randomly initialize the process sequence, machine sequence, and speed sequence.

8. The system according to claim 6, characterized in that, In the second-class fuzzy parsing diagram, the values ​​attached to the operation nodes are obtained by dividing the second-class fuzzy processing time by the corresponding speed and then normalizing it; the edges representing the sequence of operations are solid directed edges, which strictly correspond to the workpiece process flow; the edges representing the machine allocation relationship are colored dashed edges, with different colors used to distinguish different processing machines.

9. The system according to claim 5, characterized in that, In the strategy model module, the graph neural network adopts a graph isomorphic network, the strategy network is a multi-layer fully connected network, and the parameters are updated based on the REINFORCE algorithm, guided by the cumulative reward calculated from the changes in fuzzy maximum completion time and fuzzy total energy consumption.

10. The system according to claim 5, characterized in that, In the two-stage energy-saving optimization module: The first phase involves reconfiguring machine allocation without changing the job sequence in order to reduce idle time and processing time. The second stage optimizes the speed sequence while maintaining the job and machine sequence, reducing system energy consumption by slowing down the operation without extending the fuzzy maximum completion time.

Images