Multi-objective collaborative optimization flexible job shop scheduling method and system

Abstract

The invention provides a multi-objective collaborative optimization flexible job shop scheduling method and system. Relates to the technical field of workshop scheduling. The method comprises the following steps: acquiring a production state characteristic at a rescheduling moment; the production state features at the rescheduling moment are input into a pre-trained encoder for encoding dimension reduction, and dimension reduction features are obtained; inputting the dimension reduction feature into a pre-trained SOM network, and outputting a production state tag associated with a neuron node having the smallest difference with the dimension reduction feature in an output layer as a production state tag of the dimension reduction feature; inputting the dimension reduction feature with the production state label into a pre-trained target selection agent, and selecting an optimization target most matched with the production state from a plurality of preset optimization targets by the pre-trained target selection agent as a temporary optimization target at the rescheduling moment for output; and scheduling the target flexible job shop based on the temporary optimization target. And the rescheduling temporary optimization target matched with the current workshop state can be quickly output.

Description

Technical Field

[0001] This invention relates to the field of workshop scheduling technology, and in particular to a flexible workshop scheduling method and system with multi-objective collaborative optimization. Background Technology

[0002] Flexible workshop scheduling is a core issue in intelligent manufacturing systems. It mainly studies how to rationally allocate machines and determine the processing sequence for processes in a production environment with multiple devices, multiple processes, multiple objectives, multiple constraints, and dynamic disturbances, in order to optimize production efficiency, improve resource utilization, and ensure on-time order delivery.

[0003] Existing flexible job shop scheduling methods mainly include heuristic rules, metaheuristic algorithms, and reinforcement learning-based scheduling methods. However, these methods still have significant limitations when facing complex dynamic environments: when disturbances such as equipment failure, emergency order insertion, order cancellation, and workpiece rework occur, the shop state data exhibits heterogeneous high-dimensionality, strong coupling, and nonlinearity. This not only contains a large amount of redundant information irrelevant to the current decision, causing a dimensionality explosion problem that exponentially increases the computational load for decision-making and makes it difficult to guarantee real-time performance; more importantly, the redundant features can mask effective information, interfering with the accurate extraction of the current shop state pattern, making it difficult for the generated decision to accurately match the current shop state, leading to suboptimal or even ineffective scheduling schemes. Summary of the Invention

[0004] Based on this, it is necessary to provide a flexible job shop scheduling method and system with multi-objective collaborative optimization to address the above-mentioned technical problems, so as to solve the defects of high-dimensional redundancy of state data under dynamic disturbances, weak adaptability of scheduling strategies and poor real-time performance in the existing technology, and achieve efficient and accurate scheduling in complex dynamic job shop environments.

[0005] The following technical solution is adopted in this specification: This specification provides a flexible job shop scheduling method with multi-objective collaborative optimization, including: When the target flexible work shop detects a disturbance event affecting the process, it triggers rescheduling and obtains the production status data at the time of rescheduling. The production status data at the time of rescheduling is then input into a pre-trained encoder for encoding and dimensionality reduction to obtain dimensionality-reduced features. The dimensionality reduction feature is input into the pre-trained SOM network. Each neuron node in the output layer of the pre-trained SOM network is associated with a pre-labeled production state label. The production state label associated with the neuron node in the output layer of the pre-trained SOM network that has the smallest difference from the dimensionality reduction feature is output as the production state label of the dimensionality reduction feature. The dimensionality reduction features with production state labels are input into the pre-trained target selection agent. The pre-trained target selection agent selects the optimization target that best matches the production state from multiple preset optimization targets as the temporary optimization target at the rescheduling time and outputs it. The target flexible operation workshop is scheduled based on the temporary optimization target.

[0006] The beneficial effect is that the pre-trained encoder enables adaptive extraction and reconstruction of production state data features, which solves the problem of dimensionality explosion and reduces the computational load of decision-making, while removing noise components and redundant information that are irrelevant to scheduling decisions. Furthermore, the pre-trained SOM network supplements the dimensionality-reduced features with semantically clear scene labels, so that the input of the final pre-trained target selection agent has the dual advantages of low-dimensional simplification and semantic clarity, providing reliable support for the subsequent generation of the optimal scheduling scheme.

[0007] Furthermore, the scheduling of the target flexible workshop based on the temporary optimization objective includes: Gene expression programming is used to generate corresponding composite scheduling rule sets for multiple preset optimization objectives. The composite scheduling rule sets include workpiece sorting rules for determining the priority of workpieces to be processed and machine allocation rules for allocating processing machines to workpieces to be processed. The dimensionality reduction features with production status labels and the temporary optimization objective are input into the pre-trained workpiece sorting agent. The pre-trained workpiece sorting agent selects the workpiece sorting rule corresponding to the temporary optimization objective from the set of composite scheduling rules to determine the priority of each workpiece to be processed at the rescheduling time. The dimensionality reduction features with production status labels and the temporary optimization objective are input into the pre-trained machine allocation agent. The pre-trained machine allocation agent selects the machine allocation rule corresponding to the temporary optimization objective from the set of composite scheduling rules and allocates processing machines to the workpieces to be processed. Based on the workpiece sorting rules, select the workpiece with the highest priority at the current rescheduling time. Based on the machine allocation rules, assign a processing machine to the workpiece with the highest priority at the current rescheduling time, and process the workpiece with the highest priority through the processing machine.

[0008] The beneficial effects lie in the fact that the composite rule set generated by gene expression programming provides multi-faceted support for scheduling. The dual-agent approach, combining scenario-based dimensionality reduction features with precise selection rules for temporary targets, solves the problem of blind matching in traditional scheduling rules. The accurate determination of workpiece priority and machine allocation avoids decision-making biases caused by redundant information interference, ensuring a high degree of adaptation between rescheduling and workshop status, reducing process blockages and machine idle time, and significantly improving scheduling efficiency and reliability in dynamic scenarios.

[0009] Furthermore, the step of generating corresponding composite scheduling rule sets for multiple preset optimization objectives through gene expression programming includes: Define the function set and terminal set for gene expression programming; wherein the function set includes arithmetic and logical operators for constructing compound rules, and the terminal set includes workpiece attributes and machine states in the target flexible workshop; For each of the multiple preset optimization objectives, a gene expression programming evolution process is executed. Through random combination of function sets and terminal sets, crossover mutation, and fitness screening, a composite scheduling rule set containing workpiece sorting rules and machine allocation rules is generated for each optimization objective.

[0010] The beneficial effect lies in using the actual workpiece attributes and machine states in the workshop as the terminal set, ensuring that the rules fit the actual production situation. Through random combination and screening in the evolutionary process, a unique composite rule set is generated for each optimization objective. The rules cover sorting and allocation logic, solving the problem of weak adaptability of traditional rules to dynamic working conditions, and providing reliable rule support for precise scheduling.

[0011] Furthermore, the steps for obtaining the pre-trained target selection agent, the pre-trained job sorting agent, and the pre-trained machine allocation agent include: A target selection agent, a job sorting agent, and a machine allocation agent are constructed. Each agent is equipped with a policy network and a value network. The policy network is used to generate policies, and each agent executes actions in the action environment according to the policies. The value network is used to evaluate the value of the actions executed by the agent. Specifically, the action environment of the target selection agent is a set of preset optimization goals; the action environment of the job sorting agent is the job sorting rules in a composite scheduling rule set; and the action environment of the machine allocation agent is the machine allocation rules in a composite scheduling rule set. The target selection agent, job sorting agent, and machine allocation agent, along with the configured policy network and value network, are integrated into a heterogeneous multi-agent system. A centralized training and decentralized execution architecture is used to train this heterogeneous multi-agent system. During the centralized training phase, the policy networks and value networks of each agent are updated at different frequencies on the digital twin simulation platform of the target flexible workshop: the policy networks and value networks of the workpiece sorting agent and the machine allocation agent are updated at a high frequency with a first preset update frequency, so that the workpiece sorting agent and the machine allocation agent can adapt to the dynamic changes of the process; the policy networks and value networks of the target selection agent are updated at a low frequency with a second preset update frequency lower than the first preset update frequency, so that the target selection agent can focus on long-period disturbance identification to maintain the stability of the optimization target. In the decentralized execution phase, the trained policy networks and corresponding agents are deployed to the edge computing nodes of the target flexible workshop to obtain pre-trained target selection agents, pre-trained workpiece sorting agents, and pre-trained machine allocation agents. Each pre-trained agent independently calls the trained policy network to obtain decisions and executes actions according to the decisions.

[0012] The beneficial effects lie in the fact that configuring agents, policy networks, and value networks for different scheduling tasks enables precise division of labor in target selection, workpiece sorting, and machine allocation, solving the problems of strong coupling and poor adaptability in traditional scheduling decisions. Centralized training combined with a frequency-divided network update strategy allows workpiece sorting and machine allocation agents to respond frequently to dynamic changes in processes, while ensuring that the target selection agent focuses on long-cycle disturbances and maintains decision stability. Decentralized edge deployment enables each agent to execute decisions independently, significantly improving the efficiency of scheduling command issuance and effectively addressing the technical pain points of insufficient real-time scheduling and decision lag in dynamic scenarios, providing efficient and reliable intelligent support for workshop rescheduling.

[0013] Furthermore, the value network evaluates the value of the agent's actions through a reward function that includes positive, neutral, and negative reward signals; the triggering conditions for each reward signal are as follows: Within the current rescheduling decision cycle, when the temporary optimization target value generated by the agent's action is better than the historical best value, the agent is given a positive reward signal; When the temporary optimization target value generated by the agent's action is worse than the historical best value, the agent is given a negative reward signal; When the temporary optimization target value generated by the agent's action is equal to the historical best target value, the agent is given a neutral reward signal. Specifically, after the current rescheduling decision cycle ends, the historical best value of the target will be updated to a temporary optimized target value that is better than the historical best value of the target.

[0014] The beneficial effect lies in using a three-tiered reward function (positive, neutral, and negative) to quantify the value of the agent's decisions, accurately reflecting the impact of decisions on the optimization objective. Positive rewards reinforce high-quality decisions, negative rewards avoid mismatched decisions, and neutral rewards ensure objective evaluation. Combined with a periodic optimal value dynamic update mechanism, the agent is driven to continuously iterate its strategy, effectively solving the problem of poor decision adaptability under dynamic perturbations and avoiding scheduling scheme inefficiencies or even failures due to ambiguous state recognition.

[0015] Furthermore, the steps for obtaining the pre-trained SOM network include: A SOM network is constructed, using the historical dimensionality reduction features of the target flexible workshop as training data, and the SOM network is trained through a competitive learning mechanism. Extract the weight vectors of all neurons in the output layer of the trained SOM network, cluster the weight vectors of all neurons in the output layer, and assign a production state label to each cluster result to obtain the pre-trained SOM network.

[0016] The beneficial effects are that the SOM network competitive learning mechanism maps high-dimensional state data to a low-dimensional topology space, effectively eliminating redundant information and ensuring real-time scheduling; by clustering the output layer weight vector and assigning production state labels, it accurately extracts the masked key pattern features, realizes rapid and accurate identification of working condition types, and ensures that scheduling decisions are accurately adapted to the current workshop state.

[0017] Furthermore, the step of inputting the production status data at the rescheduling time into a pre-trained encoder for encoding and dimensionality reduction to obtain dimensionality-reduced features includes: Construct an autoencoder that includes an encoder and a decoder; The autoencoder is trained using the historical production state features of the target flexible work workshop as training data: the encoder maps the historical production state features to a low-dimensional latent space representation, and the decoder reconstructs the low-dimensional latent space representation into the original input; the parameters of the encoder and decoder are iteratively optimized by minimizing the reconstruction error between the historical production state features and the reconstructed output, and training stops when the reconstruction error converges. After training is completed, the encoder parameters are fixed, and the production state data at the current rescheduling time is input into the pre-trained encoder. The output low-dimensional latent space representation is used as the dimensionality reduction feature.

[0018] The beneficial effects are that by using autoencoders to encode dimensionality reduction, heterogeneous high-dimensional data is mapped to a low-dimensional potential space, effectively eliminating redundant information, alleviating dimensionality explosion, significantly reducing the computational load of decision-making, and ensuring real-time performance. At the same time, key nonlinear feature correlations are preserved to avoid the obscuring of effective information, enabling accurate extraction of workshop state patterns and ensuring that scheduling decisions are precisely adapted to the current working conditions.

[0019] Furthermore, the policy network is updated using the near-end policy optimization PPO-Clip algorithm; the value network is updated by minimizing the temporal difference error.

[0020] The beneficial effects are that the policy network is updated using the PPO-Clip algorithm, which can improve exploration efficiency while ensuring the stability of policy iteration and avoid decision oscillations caused by dynamic disturbances in the workshop; the value network is updated by minimizing the temporal difference error, which can quickly and accurately evaluate the decision value and reduce the interference of redundant information.

[0021] Furthermore, the preset multiple optimization objectives include: maximizing completion time, minimizing average weighted delay time, and minimizing average flow time.

[0022] This specification provides a flexible job shop scheduling system with multi-objective collaborative optimization, including: The data acquisition and dimensionality reduction module is used to trigger rescheduling when the target flexible work shop detects a disturbance event affecting the process, acquire the production state features at the time of rescheduling, and input the production state features at the time of rescheduling into a pre-trained encoder for encoding and dimensionality reduction to obtain dimensionality-reduced features. The production state label acquisition module is used to input the dimensionality reduction features into the pre-trained SOM network. Each neuron node in the output layer of the pre-trained SOM network is associated with a pre-labeled production state label. The production state label associated with the neuron node in the output layer of the pre-trained SOM network that has the smallest difference from the dimensionality reduction features is output as the production state label of the dimensionality reduction features. The temporary optimization target acquisition module is used to input the dimensionality reduction features with production state labels into the pre-trained target selection agent. The pre-trained target selection agent selects the optimization target that best matches the production state from a number of preset optimization targets as the temporary optimization target for the rescheduling time and outputs it. The scheduling module is used to schedule the target flexible work workshop based on temporary optimization objectives.

[0023] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: This invention performs encoding dimensionality reduction on production state data at rescheduling time using an encoder. Compared to traditional dimensionality reduction methods such as principal component analysis, this encoding process achieves adaptive extraction and reconstruction of production state data features through pre-training. While solving the dimensionality explosion problem and reducing the computational load of decision-making, it accurately removes noise components and redundant information irrelevant to scheduling decisions, highlighting the effective workshop state information obscured by messy data. On this basis, a pre-trained SOM network is introduced, in which each neuron node in the output layer is pre-labeled with a production state label with clear semantics. By calculating the difference between the dimensionality reduction features and each neuron node, semantically clear scene labels are added to the dimensionality reduction features, so that the input of the final pre-trained target selection agent has the dual advantages of low dimensionality and semantic clarity. The pre-trained target selection agent, relying on the decision mapping relationship learned during pre-training, can quickly output a temporary rescheduling optimization target that is highly consistent with the current workshop state, providing reliable support for the subsequent generation of the optimal scheduling scheme. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1This document presents a flowchart illustrating a flexible job shop scheduling method based on multi-objective collaborative optimization. Figure 2 This specification provides a schematic diagram illustrating the relationship between the policy network and the value network in an A2C model. Figure 3 This specification provides a schematic diagram of a scheduling decision-making process based on HMAPPO-GEP. Figure 4 This document provides a schematic diagram of a heterogeneous multi-agent scheduling decision-making mechanism based on HMAPPO-GEP. Figure 5 This specification provides a schematic diagram of a SOM neural network clustering process for production status data. Figure 6 This document provides a comparative diagram of training and decision-making for three different architectures. Figure 7 This document provides a schematic diagram of the HMAPPO-GEP algorithm flowchart. Figure 8 This is an iterative diagram illustrating the training process of an HMAPPO-GEP method provided in this specification under three scheduling objectives; Figure 9 This specification provides a box-line diagram of the scheduling targets for MS, MWT, and MFT under different C values. Figure 10a This document provides a schematic diagram of the mean MS values ​​of HMAPPO-GEP and the classic scheduling rule for three typical instances. Figure 10b This document provides a schematic diagram of the mean MWT of HMAPPO-GEP and the classic scheduling rule in three typical examples. Figure 10c This document provides a schematic diagram illustrating the average values ​​of HMAPPO-GEP and classic scheduling rules for three typical examples. Figure 11 This diagram illustrates the ratio of target values ​​for MS, MWT, and MFT under different N values ​​for an HMAPPO-GEP method and a classic scheduling rule provided in this specification. Figure 12 The HMAPPO-GEP method provided in this specification differs from classic scheduling rules in different ways. A schematic diagram illustrating the target value ratios of MS, MWT, and MFT; Figure 13 This diagram illustrates the ratio of target values ​​of MS, MWT, and MFT under different e values ​​for an HMAPPO-GEP method and a classical scheduling rule provided in this specification. Figure 14This document provides a schematic diagram of the distribution points after 20 runs of HMAPPO-GEP and the classic scheduling rule. Figure 15 This is a schematic diagram of a flexible job shop scheduling system with multi-objective collaborative optimization, provided in this specification. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0027] The technical solution provided by this invention can be applied to flexible job shop scheduling scenarios with dynamic disturbances such as machine failures and random order arrivals. Existing flexible job shop scheduling methods mainly include heuristic rules, metaheuristic algorithms, and reinforcement learning-based scheduling methods. However, these methods still have significant limitations when facing complex dynamic environments: traditional heuristic rule methods have poor adaptability: rules such as shortest processing time priority and earliest delivery date priority, although fast in scheduling, have fixed strategies and cannot adaptively adjust according to the real-time status of the production system. When dynamic events such as machine failures or urgent order insertions occur, their scheduling performance will drop sharply. Metaheuristic algorithms have weak real-time response capabilities: although global optimization algorithms such as genetic algorithms and particle swarm optimization can obtain better scheduling solutions, they are computationally time-consuming and difficult to meet the real-time requirements of scheduling decisions in the production site.

[0028] Therefore, there is an urgent need for a novel intelligent scheduling method that possesses online adaptive decision-making capabilities, ensures excellent scheduling performance and fast convergence speed, and has a certain degree of interpretability, enabling collaborative optimization of multiple objectives such as maximum completion time, average weighted delay time, and average transit time. Based on this, and addressing the technical challenges of traditional scheduling methods, such as poor adaptive capability, weak real-time response, and insufficient convergence and interpretability, this invention provides a flexible job shop scheduling method based on multi-objective collaborative optimization.

[0029] The following is combined with Figures 1-14 The present invention describes a flexible job shop scheduling method based on multi-objective collaborative optimization, the method comprising: S100. When the target flexible work shop detects a disturbance event affecting the process, it triggers rescheduling and obtains the production status data at the time of rescheduling. The production status data at the time of rescheduling is input into the pre-trained encoder for encoding and dimensionality reduction to obtain dimensionality reduction features.

[0030] For example, rescheduling trigger conditions include: workpiece completing its current process, machine entering idle state, machine malfunction, or arrival of a new urgent order. Production data from scheduling decision points is acquired in real time using RFID readers and status sensors deployed in the workshop, constructing an initial feature vector including multi-dimensional indicators such as machine quantity, average equipment utilization rate, and utilization rate standard deviation. And calculate the difference between the initial eigenvector of the multidimensional index at the current scheduling time and the initial eigenvector of the multidimensional index at the previous time. ,Will and After splicing, the data is normalized to obtain the production status data at the rescheduling time.

[0031] For example, in an embodiment of this application, the step of obtaining dimensionality reduction features may include: constructing an autoencoder including an encoder and a decoder; training the autoencoder using historical production state features of the target flexible workshop as training data: the encoder maps the historical production state features to a low-dimensional latent space representation, and the decoder reconstructs the low-dimensional latent space representation into the original input; iteratively optimizing the parameters of the encoder and decoder by minimizing the reconstruction error between the historical production state features and the reconstructed output, and stopping training when the reconstruction error converges; after training is completed, fixing the encoder parameters, inputting the production state features at the current rescheduling time into the pre-trained encoder, and using the output low-dimensional latent space representation as the dimensionality reduction feature.

[0032] S200. Input the dimensionality reduction feature into the pre-trained SOM network. Each neuron node in the output layer of the pre-trained SOM network is associated with a pre-labeled production state label. The production state label associated with the neuron node in the output layer of the pre-trained SOM network that has the smallest difference from the dimensionality reduction feature is output as the production state label of the dimensionality reduction feature.

[0033] For example, the steps for obtaining the pre-trained SOM network include: constructing an SOM (Self-organizing map) network with a two-dimensional grid topology, whose output layer contains a predetermined number of neurons, each of which is initialized with a weight vector of the same dimension as the input vector; the input vector is a low-dimensional state feature vector collected during the historical operation of the flexible work workshop and obtained by encoder dimensionality reduction, which represents multi-dimensional operational status such as equipment availability, work-in-process backlog, process completion progress, intensity of urgent orders, and delivery urgency. Using the historical dimensionality-reduced features of the target flexible work workshop as training data, the SOM network is trained through a competitive learning mechanism; the weight vectors of all neurons in the output layer of the trained SOM network are extracted, and K-means clustering is performed on the weight vectors of all neurons in the output layer. Each weight clustering result corresponds to a typical production line operating state, so a production state label can be assigned to each clustering result to obtain the pre-trained SOM network.

[0034] S300. Input the dimensionality reduction features with production state labels into the pre-trained target selection agent. The pre-trained target selection agent selects the optimization target that best matches the production state from multiple preset optimization targets as the temporary optimization target for the rescheduling time and outputs it.

[0035] S400: Schedule the target flexible work workshop based on temporary optimization objectives.

[0036] For example, gene expression programming is used to generate corresponding composite scheduling rule sets for multiple preset optimization objectives. These composite scheduling rule sets include workpiece sorting rules for determining the priority of workpieces to be processed and machine allocation rules for assigning processing machines to the workpieces. Specifically:

[0037] Define the function set and terminal set for gene expression programming; wherein the function set includes arithmetic and logical operators for constructing composite rules, and the terminal set includes workpiece attributes and machine states in the target flexible workshop; for each of the preset multiple optimization objectives, execute the gene expression programming evolution process respectively, and through random combination, crossover mutation and fitness screening of the function set and terminal set, generate a composite scheduling rule set containing workpiece sorting rules and machine allocation rules for each optimization objective.

[0038] For example, the dimensionality reduction features with production status labels and the temporary optimization objective are input into a pre-trained workpiece sorting agent. The pre-trained workpiece sorting agent selects the workpiece sorting rule corresponding to the temporary optimization objective from the set of composite scheduling rules to determine the priority of each workpiece to be processed at the rescheduling time. The dimensionality reduction features with production status labels and the temporary optimization objective are input into a pre-trained machine allocation agent. The pre-trained workpiece sorting agent selects the machine allocation rule corresponding to the temporary optimization objective from the set of composite scheduling rules to allocate processing machines to the workpieces to be processed. According to the workpiece sorting rule, the workpiece with the highest priority at the current rescheduling time is selected, and according to the machine allocation rule, a processing machine is allocated to the workpiece with the highest priority at the current rescheduling time. The processing machine then processes the workpiece with the highest priority.

[0039] For example, the steps of acquiring a pre-trained target selection agent, a pre-trained artifact sorting agent, and a pre-trained machine assignment agent may include: A target selection agent, a job sorting agent, and a machine allocation agent are constructed. Each agent is equipped with a policy network and a value network. The policy network is used to generate policies, and each agent executes actions in the action environment according to the policies. The value network is used to evaluate the value of the actions executed by the agent. Specifically, the action environment of the target selection agent is a set of preset optimization goals; the action environment of the job sorting agent is the job sorting rules in the composite scheduling rule set; and the action environment of the machine allocation agent is the machine allocation rules in the composite scheduling rule set.

[0040] The target selection agent, job sorting agent, and machine allocation agent, along with the configured policy network and value network, are integrated into a heterogeneous multi-agent system. A centralized training and decentralized execution architecture is used to train this heterogeneous multi-agent system. During the centralized training phase, the policy networks and value networks of each agent are updated at different frequencies on the digital twin simulation platform of the target flexible workshop: the policy networks and value networks of the workpiece sorting agent and the machine allocation agent are updated at a high frequency with a first preset update frequency, so that the workpiece sorting agent and the machine allocation agent can adapt to the dynamic changes of the process; the policy networks and value networks of the target selection agent are updated at a low frequency with a second preset update frequency lower than the first preset update frequency, so that the target selection agent can focus on long-period disturbance identification to maintain the stability of the optimization target. In the decentralized execution phase, the trained policy networks and corresponding agents are deployed to the edge computing nodes of the target flexible workshop, resulting in pre-trained target selection agents, pre-trained workpiece sorting agents, and pre-trained machine allocation agents. Each pre-trained agent independently invokes the trained policy network to obtain decisions and executes actions based on those decisions. Specifically, the policy network is updated using the Proximal Policy Optimization (PPO-Clip) algorithm, and the value network is updated by minimizing temporal difference errors.

[0041] For example, the value network evaluates the value of an agent's actions through a reward function that includes positive, neutral, and negative reward signals; the triggering conditions for each reward signal are as follows: Within the current rescheduling decision cycle, when the temporary optimization target value generated by the agent's action is better than the historical best value, the agent is given a positive reward signal; When the temporary optimization target value generated by the agent's action is worse than the historical best value, the agent is given a negative reward signal; When the temporary optimization target value generated by the agent's action is equal to the historical best target value, the agent is given a neutral reward signal. Specifically, after the current rescheduling decision cycle ends, the historical best value of the target will be updated to a temporary optimized target value that is better than the historical best value of the target.

[0042] It can be seen that the above technical solution can achieve the following effects: Enhance adaptive capabilities: By constructing a heterogeneous multi-agent system, the scheduling strategy can perceive changes in the workshop status in real time and adaptively adjust and optimize targets and scheduling rules to effectively cope with dynamic disturbances.

[0043] Ensuring optimization effectiveness and real-time performance: By integrating gene expression programming and deep reinforcement learning, the composite scheduling rules generated by gene expression programming serve as the agent's action space. This approach combines the efficiency of heuristic rules with the optimization capabilities of reinforcement learning, ensuring scheduling quality while meeting real-time decision-making requirements.

[0044] Enhanced interpretability: By performing feature selection on scheduling rules, the contribution of key state features to scheduling decisions is clarified, thereby improving the transparency and understandability of the agent's decision-making process and enhancing the credibility of engineering applications.

[0045] Achieving multi-objective collaborative optimization: Designing a multi-objective-oriented reward function and evaluation system enables the system to effectively balance and optimize multiple key performance indicators such as maximum completion time (MS), mean weighted tardiness (MWT), and mean flow time (MFT) based on actual production conditions.

[0046] The following is a detailed description of the solution provided in the above embodiments. A flexible job shop dynamic adaptive scheduling method integrating Heterogeneous Multi-Agent Proximal Policy Optimization (HMAPPO), Gene Expression Programming (GEP), Auto-Encoder (AE), and Self-Organizing Map (SOM) feature extraction includes: system parameter and simulation environment initialization; construction and initialization of the heterogeneous multi-agent system; production state feature processing based on AE-SOM; construction of a composite scheduling rule base and action mapping; design of a multi-objective-oriented reward function; multi-agent training based on a centralized training with decentralized execution (CTDE) architecture; adaptive scheduling execution in a digital twin environment; performance evaluation indicators; experimental verification and comparative analysis. The scheduling method of this invention dynamically optimizes multi-objective scheduling problems by combining agent collaboration and reinforcement learning techniques with Gene Expression Programming (GEP) to generate scheduling rules. Each agent is trained using the HMAPPO algorithm to continuously optimize scheduling decisions, thereby effectively improving the scheduling efficiency and resource utilization of the production workshop. Specifically, this includes the following steps:

[0047] S1, System parameters and scheduling environment initialization.

[0048] S11, set the basic parameters of the HMAPPO-GEP algorithm, including: total number of training iterations (ITER), learning rates for the policy network and value network, and discount factor. , Pruning parameters of the algorithm Network update batch size Target selection agent's target adjustment step size In one specific embodiment, the total number of training iterations of the HMAPPO-GEP algorithm Set the learning rate to 2000. Discount factor , Pruning parameters of the algorithm Network update batch size Target selection agent's target adjustment step size .

[0049] S12, Set the parameters for the flexible workshop simulation environment, including: number of devices. Total number of workpieces Workpiece dynamic arrival time interval Mean time between failures (MTBF) Average machine repair time Urgent delivery time factor a In one specific embodiment, the simulation environment parameters utilize a Python simulation platform to construct a flexible workshop containing 10 machines. Workpieces arrive dynamically, with arrival intervals following parameters. The exponential distribution. Mean time between machine failures (MTBF). Set to 500 time units, mean repair time The time unit is 50. Delivery urgency factor. a The set of values ​​is This is implemented within the simulation environment settings. The workpiece delivery date is set to... ,in For the arrival time of the workpiece, This represents the total processing time for the workpiece. a That is, from the set Random selection.

[0050] S13, Define the objective set of scheduling optimization as O= ,in To maximize the completion time, For the average weighted delay time, This represents the average flow time.

[0051] S2, Initialization of network parameters for the heterogeneous multi-agent system; scheduling decision agents are divided into target selection agents based on their functions. Workpiece sorting intelligent agent Assigning intelligent agents to machines These three elements together constitute a heterogeneous multi-agent system. The initialization process is performed according to the following steps:

[0052] S21, Initialization of the target selection agent: Initialize the policy network and value network used for shop floor optimization target decision-making. Randomly initialize the policy function of the target selection agent. and value function Internal parameters. Policy function. The input is the workshop state feature vector, and the output is the probability distribution of the temporary optimization objective; the value function Used to evaluate the value of a state. In one specific embodiment, both the policy network and the value network of each agent employ... Framework construction and adoption The method initializes the weights. For example, Figure 2 This document provides a schematic diagram illustrating the relationship between the policy network and the value network in A2C, explaining the core mechanism of the A2C framework as the cornerstone of the algorithm. It explains the basic principle that the policy network (actors) is responsible for action decisions and the value network (commentators) is responsible for state evaluation, laying the foundation for understanding subsequent advanced policy optimization algorithms such as PPO. Figure 3 This specification provides a schematic diagram of a scheduling decision-making process based on HMAPPO-GEP, illustrating the overall closed-loop operation of the method of this invention and describing the complete process from "state awareness" to "decision execution" and then to "environmental feedback." It embodies the interaction between the digital twin environment and the heterogeneous multi-agent system, and serves as the general framework for understanding the technical solution of this invention. Figure 3 The multi-agent collaborative shop floor rescheduling process shown includes: the shop floor production state at rescheduling time t is... The target selection agent is based on the state feature vector at the rescheduling time. and the preset optimization goals Through target selection strategy A temporary scheduling target is determined; subsequently, the workpiece sorting agent determines the task based on the state feature vector at the rescheduling time. Based on the NGEP-FS workpiece sorting rule base, the workpiece sorting rule at rescheduling time t is given. Through workpiece selection strategy Obtain the workpiece sorting. The machine assigns an agent based on the state feature vector. Based on the machine allocation rules for rescheduling time t provided by the NGEP-FS machine allocation rule base Through machine allocation strategy Obtain machine allocation; in each subsequent step, the workpiece sorting agent and the machine allocation agent will respectively base their actions on the state feature vector at the corresponding time and the rules given by the rule base, such as... , The strategy involves continuously executing workpiece sorting and machine allocation until step t+C-1), with the entire process continuously adjusting the target interval for C steps, and the process is cyclically advanced. Figure 4 This specification provides a schematic diagram of a heterogeneous multi-agent scheduling decision-making mechanism based on HMAPPO-GEP, detailing the collaborative working mechanism and information flow among the three agents: target selection, job sorting, and machine allocation. This is crucial for understanding how the "heterogeneous multi-agent" mechanism of this invention achieves hierarchical and division-of-labor decision-making. Figure 4As shown, the workshop production environment above includes machines M1, M2, and M3 and workpieces J1, J2, J3, J4, and J5, and outputs the workshop status. With reward r t In the Actor (Online Execution) module, Generate the state feature vector of the target selection policy network Through the target selection strategy network Output scheduling target , and then combine Generate the state feature vector of the workpiece strategy network State feature vectors of machine policy networks The workpiece strategy network is then fed into the network. Machine Policy Networks This corresponds to calling the NGEP-FS workpiece sorting rule base and the NGEP-FS machine allocation rule base to output actions. , In the commentator (for training) module, the value function corresponds to the target value network. Workpiece value network Machine Value Network And calculate the timing difference error respectively. = - , = - It is used for training and optimization of agent networks.

[0053] S22, the old policy network is synchronously created, creating parameter copies for policy gradient updates. The policy function is then created. Parameter copy copy The importance sampling ratio is kept constant throughout the training period for calculating policy updates. This ensures stability during policy updates and avoids training oscillations.

[0054] S23, Multi-agent collaborative initialization: Extend the above initialization process to include the policy functions of each agent for sorting the workpiece. Value function And the policy function for machine-assigned agents. Value function Perform the initialization and replica creation process as described in steps S21 and S22 to obtain and A complete and independently updatable multi-agent decision-making system was constructed, laying the foundation for subsequent collaborative training and decision-making.

[0055] S24, Training Support System Initialization: Initializes the auxiliary modules required for training. Initializes the Elite library to store the optimal set of network parameters during iteration; initializes the iteration counter. Set to 0; This sets the global optimum for each scheduling objective. value, , The value is set to infinity to facilitate minimum value replacement during subsequent optimization. An efficient training record and optimal solution tracking mechanism is established to ensure that the algorithm converges to a high-quality solution.

[0056] S3, based on AE-SOM state feature processing, includes the following steps: S31, Workshop Status Data Acquisition and Feature Vector Construction: Production data from scheduling decision points is acquired in real time using RFID readers and status sensors deployed in the workshop. This data is used to construct an initial feature vector containing indicators such as machine quantity, average equipment utilization rate, and utilization rate standard deviation. Furthermore, calculate the current scheduling state. State compared to the previous moment difference vector .Will and The data is then spliced ​​together to form a complete workshop state feature vector. In one specific embodiment, at each scheduling point, 16-dimensional raw features such as average equipment utilization and workpiece completion rate are collected, and the difference with the previous time step is calculated to form a 32-dimensional feature vector S.

[0057] S32, State feature normalization processing, for the complete state feature vector Normalization is performed, mapping the data to the [0,1] interval to eliminate the influence of different feature scales on model training. The normalization process uses the Min-Max standardization method, and its calculation formula is as follows:

[0058] in, , These represent the minimum and maximum values ​​of each feature.

[0059] S33, feature dimensionality reduction based on autoencoders. The normalized feature vectors... The input is fed into the trained autoencoder model. The autoencoder consists of two parts: an encoder and a decoder; the encoder converts the 32-dimensional data into a decoder. Compressed to a low-dimensional latent space representation The decoder then attempts to... The original input is reconstructed. This is achieved by minimizing the reconstruction error. To optimize the autoencoder parameters until the loss function is optimized. Training is stopped when the time is right, and the reduced-dimensional feature vectors are output. .

[0060] S34, based on Feature clustering and discretization, reducing the dimensionality of the feature vectors Enter to The hybrid model performs cluster analysis and outputs discrete category labels to discretize the state space, facilitating understanding by the agent. This process includes:

[0061] S341, SOM Network Training: Initialize SOM network parameters, including the number of output layer nodes and the initial learning rate. Initial neighborhood radius and number of training sessions .Will The input network iterates and updates its weights through a competitive learning mechanism until training is complete. In one specific embodiment, the SOM output layer is a 5x5 grid. The number of mixed clusters is set to 10 to obtain discrete state category labels. The value range is 1-10.

[0062] S342, Clustering: After training, extract the weight vectors of all nodes in the output layer of the SOM network and use them... The algorithm clusters these weight vectors, setting the number of clusters.

[0063] S343, Label Mapping: Low-dimensional features for any input Find the winning neuron in the SOM network that is closest to it, and assign the weights of that neuron to... Cluster category as Final clustering labels For example, Figure 5 This specification provides a schematic diagram of a SOM neural network clustering process for production status data, as shown below. Figure 5 As shown, the PAV vector corresponding to the "workshop production attribute" on the left is [1.23, 0.85, 3.24, 1.69]. This vector is input into the SOM neural network. The SOM neural network processes the input workshop production attribute vector through competitive learning of its neurons. After a "clustering" operation, the winning neuron is matched to the corresponding category in the category identifier on the right, ultimately realizing the mapping of workshop production attributes to their corresponding categories. This demonstrates how to use the SOM neural network to reduce and discretize high-dimensional, continuous production state features, transforming complex workshop states into discrete signals that can be efficiently understood by an intelligent agent.

[0064] S4, Construction of composite scheduling rules and mapping of action space.

[0065] S41, based on multi-objective composite scheduling rule generation, a genetic expression programming method is used to generate corresponding composite scheduling rule sets for multiple optimization objectives of flexible job shop scheduling. The optimization objectives include at least maximum completion time, average weighted delay time, and average transit time. Simultaneously, for each optimization objective, genetic expression programming evolves and generates a set of job sorting rules and a set of machine allocation rules through a function set containing arithmetic and logical operators, and a terminal set containing job attributes and machine states. The job sorting rules are used to determine the priority of jobs to be processed, and the machine allocation rules are used to select processing machines for each operation. In a specific embodiment, the rule base is constructed using genetic expression programming. Each of the three objectives generates 7 composite scheduling rules, forming a rule base containing 21 rules. The function set is... The terminal set represents the workshop state characteristics (such as processing time, waiting queue length, etc.). These 21 rules are directly mapped to the discrete action spaces (action indices 1-21) of the workpiece sorting agent and the machine allocation agent. The action space of the target selection agent is 3 (corresponding to three optimization objectives).

[0066] S42, Mapping the scheduling rules to the agent's action space: The multiple sets of scheduling rules generated in step S41 are mapped to the discrete action spaces of the corresponding agents in the heterogeneous multi-agent system, specifically including: S421, Construction of the action space of the workpiece sorting agent: Gather all workpiece sorting rules for different optimization objectives to form the set of optional actions for the agent.

[0067] S422, Machine-assigned agent action space construction: Gather all machine assignment rules for different optimization objectives to form the agent's set of optional actions.

[0068] S423, Through the mapping, each agent can adaptively select the optimal composite scheduling rule from the corresponding action space according to the current scheduling goal.

[0069] S5, a reward function generation method based on multi-objective optimization, includes the following steps: S51, The differentiated reward mechanism defines a differentiated reward signal to guide the optimization direction of the agent's policy. The numerical relationship between positive reward, negative reward, and neutral reward satisfies: positive reward > neutral reward > negative reward, to ensure that the agent has the motivation to continuously optimize its policy. The reward signal includes:

[0070] S511, Positive Reward: When the temporary optimization target value obtained in the current scheduling cycle is better than the known historical best value, a positive reward is given to the agent; for example, a positive reward can be set. =1.0.

[0071] S512, Negative Reward: When the temporary optimization target value is worse than the historical best value, a negative reward is given to the agent; for example, a neutral reward can be set. = 0.0.

[0072] S513, Neutral Reward: When the temporary optimization target value is equal to the historical best value, a neutral reward is given to the agent; for example, a negative reward can be set. =-0.5.

[0073] S52, Adaptive Reward Calculation and Optimal Value Update: After each scheduling decision cycle, the following steps are performed to generate a reward signal: S521, Reward Calculation: Obtain the temporary target value generated by this scheduling decision for the current optimization objective. ;Will The historical best value corresponding to the stored target The comparison is performed, and the corresponding reward signal is output based on the comparison result.

[0074] S522, Historical Best Value Update: During the comparison process, if and better Then, while outputting a positive reward signal, use With Update The value of ; otherwise, keep . Unchanged. For example, if (For minimization objectives such as MS, MWT, and MFT), a reward is given. and update .like A reward will be given. .like A reward will be given. .

[0075] S6, a multi-agent training method based on the CTDE architecture. Figure 6 This document provides a comparative diagram of training and decision-making for three different architectures. Figure 6As shown, in a fully centralized architecture, the value network and policy network need to communicate during training, and the decision-making process also requires communication. In a fully decentralized architecture, the value network and policy network do not need to communicate during training, and neither does the decision-making process. In a centralized training + decentralized decision-making architecture, the value network and policy network need to communicate during training, but the decision-making process does not. By comparing these three multi-agent architectures—fully centralized, fully decentralized, and centralized training + decentralized execution—the significant advantages of the CTDE architecture adopted in this invention in balancing training efficiency and execution flexibility are highlighted. The heterogeneous multi-agent system is trained using a centralized training + decentralized execution architecture through a proximal policy optimization algorithm, including the following steps:

[0076] S61, Training loop initialization, sets the upper limit of the total number of iterations, initializes the iteration counter, and initializes an elite library to store the excellent network parameters generated during training.

[0077] S62, Simulation Environment Interaction and Data Acquisition Loop. Within the current iteration cycle, the complete workshop production scheduling process is simulated until all workpieces are processed. This process includes handling each rescheduling point.

[0078] S621, Dynamic Event Response and Status Update: Detects and responds to dynamic events such as the arrival of new workpieces, machine malfunctions, and emergency order insertions, and updates the status of workshop resources based on preset rules.

[0079] S622, State Awareness and Feature Processing: Collect the current workshop state and obtain low-dimensional feature vectors and discrete state categories.

[0080] S623, Collaborative Decision Making and Scheduling Execution: ① The target selection agent outputs a temporary optimized target at a fixed frequency based on the current state. ② The workpiece sorting agent and the machine allocation agent select rules from their respective action spaces based on the current state and the temporary optimized target, and execute scheduling decisions, thus transitioning the environment to the next state.

[0081] S624, Training Iteration Process: Reward Calculation and Experience Storage: Calculate the immediate reward and store the state, action, reward, and next state data generated in this interaction as an experience trajectory in the experience replay pool.

[0082] S63, Frequency-Divided Policy Optimization. During the simulation interaction, the policy networks and value networks of various agents are updated at different frequencies: ① High-frequency update: The policy networks and value networks of the workpiece sorting agent and the machine allocation agent are updated at a first frequency. ② Low-frequency update: The policy networks and value networks of the target selection agent are updated at a second frequency lower than the first frequency. The policy network is updated using the Proximal Policy Optimization (PPO-Clip) algorithm, and the stability of the policy update is ensured through a pruning mechanism; the value network is updated by minimizing the mean square error between its output and the target value. In a specific embodiment, the frequency-divided policy update training architecture adopts a centralized training (CT) and decentralized execution (DT) architecture. The training process is attached. Figure 6 As shown. During frequency-based updates, the network for the job sorting and machine allocation agents is updated every time a scheduling point (M=1) is reached; the network for the target selection agent is updated every time 10 scheduling points (C=12). During network updates, the policy network enables... The algorithm updates the value network by minimizing the temporal difference error. The optimizer uses Adam. The loss function of the policy network is implemented as follows:

[0083] in, This represents the probability ratio between the old and new strategies. For the dominant function, [ ]for, .

[0084] S64, Network Parameter Saving and Iteration Termination. After completing a full simulation interaction cycle, the iteration counter is incremented by 1, and the network parameters with the best performance obtained in the current iteration cycle are saved to the elite library; steps S62 to S64 are repeated until the preset total number of iterations is reached, training is completed, and the agent model with the best overall performance in the elite library is output.

[0085] Figure 7 This manual provides a flowchart illustrating the HMAPPO-GEP algorithm, showcasing its complete training lifecycle from initialization, interactive sampling, and experience replay to network parameter updates and model saving in a standard flowchart format. Figure 7 As shown, from the start of the process, at each rescheduling decision time t, multiple events are triggered: new workpiece arrives, machine M... k The i-th fault occurs, and the machine completes the current process; if the workpiece In machine M k The upper processing will execute C. i，j ←C i，j +btk，r and CT k (t)←C i，j , where C i，j It is a workpiece The completion time of the i-th process, bt k，r It is machine M k Standard processing time for the rth process, CT k (t) is the completion time of the i-th process. Otherwise, execute CT. k (t)←BS k，r +bt k，r , among which, BS k，r机器 M k During the initial idle time of the r-th process; when a new workpiece arrives, it will be placed into the workpiece set UC. job (t)), then obtain the workshop state features. , It is a state feature extraction function; then, at every (C) rescheduling steps, according to the target policy... Select optimization objective And construct the state features of the workpiece sorting agent respectively. State characteristics of machine-assigned agents Then, based on the workpiece strategy... Select workpiece sorting rules Based on machine strategy Select machine sorting rules And select the workpiece to be processed using these two rules respectively. Allocate an idle machine M k Then the workpiece The next process The next process is arranged on machine M. k The process is then processed; finally, it is determined that all processes have been scheduled. If not, the process returns to the rescheduling decision time (t) and repeats. If it is, the process ends.

[0086] Once the agent model is trained, steps S7 (online scheduling), S8 (performance evaluation), and S9 (experimental verification) can be executed. The purpose of online scheduling (S7) is to solidify the optimal model parameters after training and deploy them to the online system. Figure 8 This document provides an iterative diagram illustrating the training process of the HMAPPO-GEP method under three scheduling objectives: MS, MWT, and MFT. (See attached diagram.) Figure 8As shown, the training curves visually demonstrate the dynamic convergence process of the algorithm on the three optimization objectives, proving the superior performance of the method in terms of learning stability and convergence speed. The system listens for rescheduling events (such as machine idleness or failure) and triggers the agent's decision-making loop. Performance evaluation (S8) aims to evaluate the scheduling results using GD, IGD, and Spread metrics. The evaluation results are attached. Figures 9-14 As shown, Figure 9 This document provides a box plot of the scheduling targets for MS, MWT, and MFT under different C values. The box plot illustrates the sensitivity analysis results of the key parameter C, providing an intuitive and reliable experimental basis for algorithm parameter tuning.

[0087] Figure 10a This document provides a schematic diagram of the mean MS values ​​of HMAPPO-GEP and the classic scheduling rule for three typical instances. Figure 10b This document provides a schematic diagram of the mean MWT of HMAPPO-GEP and the classic scheduling rule in three typical examples. Figure 10c This document presents a series of diagrams illustrating the average performance of HMAPPO-GEP and classic scheduling rules in three typical examples. Through comparative experiments in three representative production scenarios, these diagrams demonstrate that the method of this invention achieves significantly better average performance across multiple performance indicators compared to classic scheduling rules, highlighting its scenario adaptability.

[0088] Figure 11 This diagram illustrates the ratio of target values ​​for MS, MWT, and MFT under different N values ​​for an HMAPPO-GEP method and a classic scheduling rule provided in this specification. Figure 12 The HMAPPO-GEP method provided in this specification differs from classic scheduling rules in different ways. A schematic diagram illustrating the target value ratios of MS, MWT, and MFT; Figure 13 This document presents a schematic diagram showing the ratio of target values ​​of MS, MWT, and MFT under different e values ​​for the HMAPPO-GEP method and the classical scheduling rule provided in this specification. This series of diagrams systematically verifies the performance stability of the method of this invention under different production condition parameters from multiple dimensions such as workpiece size, dynamic event level, and equipment failure level, fully demonstrating its robustness and wide applicability.

[0089] Figure 14 This document provides a schematic diagram of the distribution points after 20 runs of HMAPPO-GEP and the classic scheduling rule. Figure 14 The value of 'a' corresponds to N=30. =25, e=30; Figure 14 b corresponds to N=50. =50, e=50; Figure 14 c corresponds to N=200, =100, e=70; Figure 14 Through statistical distribution analysis of multiple independent experiments, the results objectively demonstrate, from both convergence and distributional perspectives, that the Pareto solution set obtained by the method of this invention is comprehensively superior to the comparative algorithms in terms of both quality and diversity. (See attached figure.) Figures 9-14 This demonstrates that the method of the present invention is significantly superior to the comparative algorithm. The purpose of experimental verification (S9) is to verify the method on multiple examples of different sizes. The results all show that the present invention has advantages in convergence and solution set quality.

[0090] S7, Adaptive Scheduling Execution in a Digital Twin Simulation Environment, deploys the trained optimal agent model in the digital twin environment to achieve online adaptive scheduling for actual production, including the following steps: S71, Construction of a Digital Twin System. A digital twin simulation environment is constructed that maps in real-time to the physical workshop. This environment includes a virtual workshop model, a scheduling and decision-making module, and a real-time data communication interface connecting the physical workshop and the virtual environment.

[0091] S72, Rescheduling Trigger Event Monitoring. The status monitoring module monitors dynamic events during the production process in real time. When a preset rescheduling trigger condition is detected, the online scheduling decision process is initiated. The rescheduling trigger conditions include: workpiece completing the current process, machine entering an idle state, machine malfunction, or a new urgent order arriving.

[0092] S73, Online Adaptive Scheduling Decision Loop, executes the following online decision steps when rescheduling is triggered: S731, Real-time Status Awareness and Processing: Collects real-time production status data from the physical workshop. And obtain the processed state feature information.

[0093] S732, Optimal Policy Model Invocation: Load the trained optimal target selection policy model. The model outputs a temporary objective to be optimized based on the state feature information. .

[0094] S733, Collaborative Rule-Based Decision Making: Synchronous Loading of Optimal Workpiece Sorting Strategy Model With the optimal machine allocation strategy model The workpiece sorting model is based on the state feature information and the temporary target. Output the optimal workpiece sorting rule The machine allocation model outputs the optimal machine allocation rule based on the same input. .

[0095] S734, Scheduling scheme generation and execution: Based on the workpiece sorting rules... Determine the priority order of the workpieces to be processed, and according to the machine allocation rules. Allocate appropriate processing machines to each process, generate specific scheduling instructions, and issue them to the physical workshop for execution.

[0096] S735, Performance Recording: Records the actual execution effect of this scheduling decision and updates the historical best performance record of the corresponding target. This is used to guide subsequent rescheduling decisions and knowledge updates. S8, a scheduling performance evaluation method based on multi-objective optimization, uses common evaluation metrics in the field of multi-objective evolutionary algorithms to quantitatively evaluate the comprehensive performance of the scheduling scheme set generated by the methods described in S1-S7. These evaluation metrics include: S81, Convergence evaluation, using the intergenerational distance (GD) index to measure the closeness of the scheduling scheme set to the ideal Pareto front. The smaller the intergenerational distance value, the better the convergence of the scheduling scheme set.

[0097] S82, Distribution Evaluation: The Inverted Generational Distance (IGD) index is used to measure the distribution range of the scheduling scheme set in the target space. The smaller the IGD value, the wider the distribution and the stronger the representativeness of the scheduling scheme set.

[0098] S83, diversity evaluation, using the diversity index (Spread) to measure the uniformity of the distribution of the scheduling scheme set on the Pareto front. The smaller the value of the diversity index, the more uniform the distribution of the scheduling scheme set.

[0099] S9, Experimental Verification and Comparative Analysis S91. Experimental Environment and Parameter Settings: A flexible workshop simulation model was constructed based on a digital twin simulation platform to verify the effectiveness of the methods described in S1-S8. The simulation model includes multiple processing devices and simulates dynamic arrival of workpieces and random machine failures. The training iteration number and learning rate parameters of the HMAPPO-GEP algorithm were set, and the method was compared with traditional scheduling rules and benchmark algorithms.

[0100] S92, Performance Comparison and Result Analysis: The method of this invention is compared with traditional heuristic rules (including SPT, FIFO, and EDD rules), metaheuristic algorithms (including the GRASP algorithm), and deep reinforcement learning algorithms (including the DDQN algorithm) through comparative experiments. The comparison results show that the method of this invention significantly outperforms the compared algorithms in terms of intergenerational distance, inverse intergenerational distance, and diversity metrics, demonstrating its comprehensive advantages in convergence, solution set distribution, and diversity when solving the dynamic multi-objective flexible job shop scheduling problem.

[0101] The following describes the flexible job shop scheduling system with multi-objective collaborative optimization provided by the present invention. The flexible job shop scheduling system with multi-objective collaborative optimization described below can be referred to in correspondence with the flexible job shop scheduling method with multi-objective collaborative optimization described above.

[0102] Figure 15 This is a schematic diagram of the structure of a flexible job shop scheduling system with multi-objective collaborative optimization provided by the present invention. For example, please refer to [link to schematic diagram]. Figure 15 As shown, the flexible job shop scheduling system with multi-objective collaborative optimization can include: The data acquisition and dimensionality reduction module is used to trigger rescheduling when the target flexible work shop detects a disturbance event affecting the process, and to acquire the production state features at the time of rescheduling. The production state features at the time of rescheduling are then input into a pre-trained encoder for encoding and dimensionality reduction to obtain dimensionality-reduced features.

[0103] The production state label acquisition module is used to input the dimensionality reduction features into the pre-trained SOM network. Each neuron node in the output layer of the pre-trained SOM network is associated with a pre-labeled production state label. The production state label associated with the neuron node in the output layer of the pre-trained SOM network that has the smallest difference from the dimensionality reduction features is output as the production state label of the dimensionality reduction features.

[0104] The temporary optimization target acquisition module is used to input the dimensionality reduction features with production state labels into the pre-trained target selection agent. The pre-trained target selection agent selects the optimization target that best matches the production state from a number of preset optimization targets as the temporary optimization target for the rescheduling time and outputs it.

[0105] The scheduling module is used to schedule the target flexible work workshop based on temporary optimization objectives.

[0106] Specific limitations regarding the flexible job shop scheduling system based on multi-objective collaborative optimization can be found in the limitations of the flexible job shop scheduling method based on multi-objective collaborative optimization mentioned above, and will not be repeated here. Each module in the aforementioned flexible job shop scheduling system based on multi-objective collaborative optimization can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0107] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A flexible job shop scheduling method for multi-objective collaborative optimization, characterized in that, The method comprises the following steps: triggering rescheduling when the target flexible job shop detects a disturbance event affecting the process, obtaining production state data at the rescheduling time; inputting the production state data at the rescheduling time into a pre-trained encoder for encoding and dimension reduction, and obtaining dimension-reduced features; inputting the dimension-reduced features into a pre-trained SOM network, each neuron node in the output layer of the pre-trained SOM network being associated with a pre-labeled production state label, and outputting the production state label associated with the neuron node with the smallest distance to the dimension-reduced features in the output layer of the pre-trained SOM network as the production state label of the dimension-reduced features; inputting the dimension-reduced features with the production state label into a pre-trained target selection agent, and outputting, by the pre-trained target selection agent, a temporary optimization target at the rescheduling time by selecting, from a plurality of preset optimization targets, an optimization target that best matches the production state; scheduling the target flexible job shop based on the temporary optimization target.

2. The method of claim 1, wherein, The scheduling of the target flexible job shop based on the temporary optimization target comprises the following steps: generating a corresponding composite scheduling rule set for each of the plurality of preset optimization targets by genetic expression programming, the composite scheduling rule set including a workpiece sequencing rule for determining the priority of a workpiece to be processed and a machine allocation rule for allocating a processing machine to the workpiece to be processed; inputting the dimension-reduced features with the production state label and the temporary optimization target into a pre-trained workpiece sequencing agent, and selecting, by the pre-trained workpiece sequencing agent, a workpiece sequencing rule corresponding to the temporary optimization target from the composite scheduling rule set to determine the priority of each workpiece to be processed at the rescheduling time; inputting the dimension-reduced features with the production state label and the temporary optimization target into a pre-trained machine allocation agent, and selecting, by the pre-trained machine allocation agent, a machine allocation rule corresponding to the temporary optimization target from the composite scheduling rule set to allocate a processing machine to the workpiece to be processed; selecting, according to the workpiece sequencing rule, a workpiece to be processed with the highest priority at the current rescheduling time, allocating, according to the machine allocation rule, a processing machine to the workpiece to be processed with the highest priority at the current rescheduling time, and processing the workpiece to be processed with the highest priority by the processing machine.

3. The method of claim 2, wherein, The method comprises the following steps: defining a function set and a terminal set of genetic expression programming, wherein the function set includes arithmetic operators and logical operators for constructing composite rules, and the terminal set includes workpiece attributes and machine states in the target flexible job shop; for each of the plurality of preset optimization targets, respectively performing a genetic expression programming evolution process, and generating, by random combination, crossover, mutation and fitness selection of the function set and the terminal set, a composite scheduling rule set including a workpiece sequencing rule and a machine allocation rule for each optimization target.

4. The method of claim 2, wherein, The pre-trained target selection agent, the pre-trained workpiece sequencing agent and the pre-trained machine allocation agent are obtained by the following steps: The target selection agent, the workpiece sequencing agent and the machine allocation agent are constructed, and each agent is configured with a policy network and a value network; the policy network is used to generate a policy, and each agent executes an action in an action environment according to the policy; and the value network is used to evaluate the value of the action executed by the agent; the action environment of the target selection agent is a plurality of preset optimization targets; the action environment of the workpiece sequencing agent is a workpiece sequencing rule in a composite scheduling rule set; and the action environment of the machine allocation agent is a machine allocation rule in the composite scheduling rule set; The target selection agent, the workpiece sequencing agent and the machine allocation agent, and the configured policy network and value network are integrated into a heterogeneous multi-agent, and the heterogeneous multi-agent is trained using a centralized training and decentralized execution architecture; in the centralized training stage, In the target flexible job shop digital twin simulation platform, the policy network and the value network of each agent are updated at different frequencies: the policy network and the value network of the workpiece sequencing agent and the machine allocation agent are updated at a high frequency with a first preset update frequency, so that the workpiece sequencing agent and the machine allocation agent adapt to the dynamic changes of the process; and the policy network and the value network of the target selection agent are updated at a low frequency with a second preset update frequency lower than the first preset update frequency, so that the target selection agent focuses on long-period disturbance identification to maintain the stability of the optimization target; In the decentralized execution stage, 5. The method of claim 4, wherein, The value network evaluates the value of the action executed by the agent through a reward function including a positive reward signal, a neutral reward signal and a negative reward signal; the triggering conditions of each reward signal are as follows: When the temporary optimization target value generated by the action executed by the agent is better than the target historical optimal value in the current rescheduling decision cycle, the agent is given a positive reward signal; When the temporary optimization target value generated by the action executed by the agent is worse than the target historical optimal value, the agent is given a negative reward signal; When the temporary optimization target value generated by the action executed by the agent is equal to the target historical optimal value, the agent is given a neutral reward signal; Wherein, after the end of the current rescheduling decision cycle, the target historical optimal value is updated to the temporary optimization target value better than the target historical optimal value.

6. The method of claim 1, wherein, The obtaining step of the pre-trained SOM network comprises: Constructing a SOM network, using the historical dimensionality reduction features of the target flexible job shop as training data, and training the SOM network through a competitive learning mechanism; Extracting the weight vectors of all neuron nodes in the output layer of the trained SOM network, clustering the weight vectors of all neuron nodes in the output layer, and assigning a production state label to each clustering result to obtain a pre-trained SOM network.

7. The method of claim 1, wherein, The production state data at the rescheduling time is input into the pre-trained encoder for encoding and dimensionality reduction to obtain a dimensionality reduction feature, comprising: Constructing an autoencoder comprising an encoder and a decoder; The historical production state features of the target flexible job shop are taken as training data to train the autoencoder: the encoder maps the historical production state features to a low-dimensional latent space representation, and the decoder reconstructs the low-dimensional latent space representation to the original input; the parameters of the encoder and the decoder are iteratively optimized by minimizing the reconstruction error between the historical production state features and the reconstruction output, and the training is stopped when the reconstruction error converges. After training, the encoder parameters are fixed, and the production state data at the current rescheduling time is input into the pre-trained encoder, and the output low-dimensional latent space representation is taken as the dimensionality reduction feature.

8. The method of claim 4, wherein, The policy network is updated by the proximal policy optimization PPO-Clip algorithm, and the value network is updated by minimizing the temporal difference error.

9. The method of claim 1, wherein, The preset multiple optimization objectives include: maximum completion time, minimum average weighted delay time, and minimum average flow time.

10. A flexible job shop scheduling system for multi-objective collaborative optimization, characterized in that, It comprises: The data acquisition and dimensionality reduction module is used to trigger rescheduling when the target flexible job shop detects a disturbance event affecting the process, and to acquire the production state features at the rescheduling time; The production state features at the rescheduling time are input into the pre-trained encoder for encoding and dimensionality reduction to obtain the dimensionality reduction features; The production state label acquisition module is used to input the dimensionality reduction features into the pre-trained SOM network, and each neuron node in the output layer of the pre-trained SOM network is associated with a pre-calibrated production state label. The production state label associated with the neuron node with the smallest distance in the output layer of the pre-trained SOM network is output as the production state label of the dimensionality reduction features. The temporary optimization objective acquisition module is used to input the dimensionality reduction features with the production state label into the pre-trained target selection agent, and the pre-trained target selection agent selects the optimization objective that best matches the production state from the preset multiple optimization objectives as the temporary optimization objective at the rescheduling time. The scheduling module is used to schedule the target flexible job shop based on the temporary optimization objective.

Images