Configurable flexible job shop scheduling method based on multi-agent game model

Abstract

The invention discloses a configurable flexible job shop scheduling method based on a multi-agent game model, and the method comprises the steps: mapping a scheduling problem into a dynamic disjunction graph model to describe the relation between a process and equipment, representing process nodes, equipment nodes and the mutual relation through a node set and an edge set, and combining the flexibility of a dynamic disjunction graph to obtain a flexible job shop scheduling model. And matching of equipment and processes is dynamically adjusted according to changes of production requirements, so that the equipment utilization rate is optimized. And a clustering method based on structure entropy is adopted, and process nodes are clustered into sub-graphs, so that the search space of a scheduling problem is reduced, and the calculation complexity is reduced. Construction of an order agent and an equipment agent and a game and cooperation process between the order agent and the equipment agent are covered. And each agent optimizes a decision through a deep Q network, and cooperates or competes according to a respective target. An automatic optimization configurator is used for optimizing hyper-parameters and a network structure of a deep Q network in a multi-agent system, and scheduling performance and adaptability are further improved.

Description

Technical Field

[0001] The invention belongs to the technical field of production workshop scheduling, and in particular relates to a configurable flexible job shop scheduling method based on a multi-agent game model. Background Art

[0002] With the rapid development of intelligent manufacturing technology, production systems are gradually shifting towards flexibility and intelligence, and production modes are tending towards multi-task parallelism and resource sharing. In this context, the production workshops of manufacturing enterprises are facing scheduling problems with increasing dynamics and uncertainty, such as frequent changes in equipment status and order requirements, which makes it difficult for traditional static scheduling methods to meet actual needs. In this ever-changing production environment, the production system must not only consider how to reasonably arrange the sequence of various processes, but also optimize resource allocation to maximize equipment utilization and minimize total completion time. In addition, with the increasing market demand for high-quality, short-cycle products, enterprises have put forward higher requirements for scheduling systems. The workshop scheduling system must have the ability to quickly adapt to diversified needs and be able to flexibly adjust production plans in emergencies such as equipment failures and urgent order insertions. To this end, how to achieve efficient scheduling of production resources in this dynamic environment through multi-agent collaboration, parameter automatic configuration and other methods has become one of the key technical challenges in the field of intelligent manufacturing.

[0003] Most of the existing flexible job shop scheduling methods are based on static models, which make it difficult to quickly respond to dynamic changes in the production environment, such as the random arrival of orders, equipment failures, and module replacements. This seriously affects the real-time and accuracy of scheduling decisions, and cannot effectively adapt to frequently changing production needs. In addition, traditional methods often lack efficient coordination mechanisms when dealing with multi-task competition and equipment resource management, making it difficult to achieve the best balance between task conflicts and resource allocation, thereby reducing resource utilization and overall production efficiency.

[0004] As the number of tasks and equipment in the job shop increases, the computational complexity of traditional global optimization scheduling methods increases exponentially, making it difficult to achieve real-time scheduling in large-scale and complex systems. When dealing with real-time scheduling needs, these methods often fail to provide optimized scheduling solutions within a reasonable time, limiting their application in actual production environments. At the same time, the lack of an effective computational complexity control mechanism makes the scheduling decision process time-consuming and unable to meet the requirements of efficient response.

[0005] The existing scheduling system mainly relies on static configuration, which makes it difficult to cope with the dynamic changes of tasks and equipment status in flexible workshops, resulting in the inability to adjust the scheduling strategy in real time, affecting the accuracy and efficiency of decision-making. In addition, with the increase in the number of tasks and equipment, the computational complexity of traditional methods has increased significantly, resulting in a long scheduling decision, which limits the real-time application of the system in large-scale and complex production environments and reduces the overall scheduling efficiency. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a configurable flexible job shop scheduling method based on a multi-agent game model, which is an advanced scheduling method that can dynamically adapt to changes in the production environment and efficiently manage computational complexity; it can quickly respond to dynamic changes in the production environment, improve the efficiency of multi-task collaboration and resource management, adapt to scheduling methods in complex and dynamically changing environments, and explore more flexible solutions to achieve faster and scalable scheduling decisions.

[0007] To achieve the above object, the technical solution adopted by the present invention is: a configurable flexible job shop scheduling method based on a multi-agent game model, comprising the steps of:

[0008] S10: Establish a dynamic scheduling model for a flexible job shop;

[0009] S20: Collect all kinds of production data in the workshop in real time, and use data preprocessing technology to clean and standardize the data;

[0010] S30: The flexible job shop scheduling model is converted into a dynamic disjunctive graph, and the structural entropy clustering method is used to divide the dynamic disjunctive graph into several subgraphs; this process optimizes the collaborative work between intelligent agents by analyzing the correlation between processes and resources;

[0011] S40: Based on the dynamic disjunctive graph, a multi-agent system is constructed, including order agents and equipment agents, which are responsible for scheduling and resource management respectively; hyperparameters are initialized and a multi-agent network structure is established, including multiple deep Q networks, each agent has a deep Q network, and the deep Q network is trained, using experience replay and target network mechanisms to improve learning effects; based on the training results, the network structure and hyperparameters are optimized through the configurator;

[0012] S50: Each agent selects its own action according to the current state through the output of the deep Q network and selects the current optimal action in combination with the game cooperation strategy;

[0013] S60: Apply the parameter and structure configurator of multiple optimization operators to adapt to different optimization requirements and environmental requirements, and perform online tuning of the agent's hyperparameters and deep Q network structure to ensure that the agent maintains efficient decision-making ability and scheduling performance in a changing environment.

[0014] Furthermore, in step S10, a flexible job shop dynamic scheduling model is established, including:

[0015] Model assumptions and symbol definitions: The set and attributes of jobs, equipment and their processes are clarified, including the dynamic arrival time of jobs, the processing capacity of equipment, the processing time of processes, the failure of equipment and the module replacement time. Each parameter is defined in detail through the symbol system to provide a basis for the scheduling model;

[0016] Optimization objectives and constraints: With the goal of minimizing the maximum completion time, multiple constraints such as process sequence, equipment conflict, equipment failure, and module replacement are set to ensure the feasibility of the scheduling plan and the efficiency of equipment utilization, and to ensure the continuity of the process and the reasonable allocation of equipment.

[0017] Furthermore, in step S30, the modeling of the dynamic disjunctive graph includes: mapping the scheduling problem into a dynamic disjunctive graph model, updating the process and equipment nodes and their relationships in real time, handling the dynamic changes of new job arrival, equipment failure and module replacement, optimizing equipment utilization through the graph model and dynamically adjusting the scheduling strategy.

[0018] Furthermore, in step S30, the dynamic disjunctive graph modeling includes:

[0019] Initialize clusters: Treat each node in the dynamic disjunctive graph as an independent cluster, and quantify the distribution state of each node by calculating the initial structural entropy;

[0020] Structural entropy optimization: With the goal of minimizing structural entropy, clusters with strong correlation are gradually merged; in each merging operation, the cluster pair that minimizes the structural entropy increment is selected, the structural entropy change ΔH after merging is calculated, and the overall structural entropy is updated;

[0021] Cluster stabilization: Continue to iterate the cluster merging operation until the structural entropy reaches a stable state, forming a set of highly correlated subgraphs.

[0022] Furthermore, in step S40, the multi-agent system construction method includes:

[0023] Agent construction: Order agents and equipment agents are constructed according to scheduling requirements, responsible for order process allocation and equipment resource management respectively; order agents are scheduled according to process clusters divided by structural entropy clustering, and equipment agents accept or reject tasks with the goal of maximizing utilization;

[0024] State space construction: define the state space for the agent; the state of the order agent includes the process completion status and the optional equipment status; the state of the equipment agent includes utilization, task queue and fault status, ensuring that the agent can obtain the environmental information required for scheduling decisions;

[0025] Action space construction: define the action selection space of the agent; the action of the order agent is to assign the process cluster to the appropriate equipment; the action of the equipment agent is whether to accept the cluster task, thereby realizing dynamic resource allocation.

[0026] Furthermore, in step S50, each agent selects its own action according to the output of the deep Q network according to the current state and selects the current optimal action in combination with the game cooperation strategy. Game and cooperation mechanism: game and cooperation between agents, using deep Q network and greedy strategy to optimize the scheduling strategy, and using a customized reward mechanism to promote the cooperation of similar agents and regulate the competition between different types of agents.

[0027] Furthermore, in step S60, the parameter and structure configurator of the multi-optimization operator includes:

[0028] A heuristic-based hyperparameter optimization module for dynamically optimizing the hyperparameters of deep Q networks in multi-agent systems using genetic algorithms, particle swarm algorithms, and ant colony algorithms;

[0029] And a structural optimization module based on neural architecture search, which is used to automatically optimize the deep Q network structure through fish swarm algorithm, bat algorithm, and firefly algorithm.

[0030] Furthermore, the hyperparameter optimization module based on heuristic algorithms includes genetic algorithm, particle swarm algorithm and ant colony algorithm;

[0031] The genetic algorithm iteratively optimizes the hyperparameter configuration of the population individuals through selection, crossover, and mutation steps;

[0032] The particle swarm algorithm searches for the optimal configuration based on the global and individual historical optimal positions by updating the particle speed and position;

[0033] The ant colony algorithm selects the hyperparameter path through pheromone concentration and heuristic information, and continuously optimizes the hyperparameters using the pheromone update mechanism;

[0034] Each algorithm is based on fitness evaluation and iteratively solves the optimal hyperparameter configuration suitable for dynamic flexible job shops.

[0035] Furthermore, the structural optimization module based on neural architecture search includes fish school optimization operator, bat optimization operator and firefly optimization operator;

[0036] The structural update mechanism based on the fish school optimization operator simulates the foraging behavior of fish schools and uses position, speed and linear attraction functions to adjust the configuration of each layer of the network structure to adapt to different scheduling tasks and environmental changes;

[0037] The structural search mechanism based on the bat optimization operator uses the bat echolocation behavior, combines the adjustment factor and the noise factor, and dynamically optimizes the network structure to achieve effective exploration to adapt to different scheduling tasks and environmental changes;

[0038] Based on the brightness attraction mechanism of the firefly optimization operator, by simulating the luminous attraction behavior of fireflies, the network hierarchy configuration is adjusted according to the architecture fitness and attraction parameters to adapt to different scheduling tasks and environmental changes.

[0039] The beneficial effects of adopting this technical solution are:

[0040] The present invention proposes a scheduling method for dynamic flexible job shop scheduling. First, the scheduling problem is mapped into a dynamic disjunctive graph model to describe the relationship between processes and equipment. The model represents process nodes, equipment nodes and their mutual relationships through node sets and edge sets, and combines the flexibility of the dynamic disjunctive graph to dynamically adjust the matching of equipment and processes according to changes in production demand to optimize equipment utilization. Then, a clustering method based on structural entropy is adopted to cluster process nodes into subgraphs, thereby reducing the search space of scheduling problems and reducing computational complexity. In addition, the construction of order agents and equipment agents and the game and collaboration process between them are covered. Each agent optimizes decisions through a deep Q network and collaborates or competes according to their respective goals. Finally, an automatic optimization configurator is designed to optimize the hyperparameters and network structure of the deep Q network in a multi-agent system to further improve scheduling performance and adaptability.

[0041] The present invention maps each process flow, resource allocation and job execution status in the scheduling problem to nodes in the graph structure, and uses the weight of the edge to represent the dependency between tasks, dynamically updates the constraints and relationships between tasks, and enhances the ability to handle the interaction between jobs in a complex scheduling environment. Specifically, as time goes by, the graph structure can reflect the changes in the workshop status and the dynamic adjustment of task priorities in real time, thereby achieving more efficient and flexible scheduling decisions.

[0042] The present invention designs a structural entropy clustering method based on a dynamic disjunctive graph, which effectively reduces the search space by dividing the scheduling graph into multiple subgraphs, thereby reducing the computational complexity of the algorithm. Specifically, structural entropy clustering discovers the correlation between process nodes and aggregates closely related processes together, thereby effectively reducing the search space of the scheduling problem. This method continuously optimizes the clustering results through iterative merging operations, so that the scheduling process is simplified from the traditional complex decision-making based on all process combinations to the independent processing of each cluster, reducing the computational complexity of the scheduling algorithm. In addition, structural entropy clustering enhances the utilization of equipment resources by the scheduling algorithm and improves the scheduling efficiency and flexibility of the system.

[0043] The present invention can optimize the production scheduling strategy: by introducing scheduling based on a multi-agent system, the present invention can achieve efficient collaboration and competition between agents in a dynamic and flexible job shop environment, thereby significantly improving scheduling efficiency and utilization of production resources. Each agent makes real-time decisions based on its tasks and goals, based on shared information and resources, and can adapt to complex dynamic changes in the workshop. In addition, the present invention enables order agents and equipment agents to find the optimal scheduling solution in local games and ultimately achieve global optimization by designing an adaptable reward mechanism and deep Q network training. This dynamic, collaborative-competitive parallel scheduling mechanism improves production flexibility and response speed, and optimizes the overall production scheduling strategy.

[0044] The present invention can realize automatic optimization of hyperparameters and Q network structure: by introducing an optimization module based on heuristic algorithms and neural architecture search, the present invention can automatically adjust the hyperparameters and network structure of the deep Q network according to the dynamic changes of the production environment. Specifically, the hyperparameter optimization module adopts classic heuristic operators such as genetic algorithm, particle swarm algorithm and ant colony algorithm, and optimizes the key parameters such as learning rate and discount factor of the Q network in the multi-agent system by simulating evolution, group intelligence and foraging behavior in nature. These algorithms can efficiently explore the hyperparameter space in complex scheduling tasks and improve scheduling performance and adaptability. At the same time, the structural optimization module based on neural architecture search combines bionic optimization algorithms such as fish schools, bats and fireflies, and automatically searches and optimizes the hierarchical structure and neuron configuration of the Q network by simulating the behavior of animals in nature to match the scheduling requirements under different production environments. Through the collaborative work of these two optimization modules, the present invention can dynamically adjust the hyperparameters and architecture of the deep Q network, thereby realizing continuous optimization and performance improvement of the scheduling strategy, ensuring that efficient and accurate scheduling capabilities are always maintained in a constantly changing production environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flow chart of a configurable flexible job shop scheduling method based on a multi-agent game model. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings.

[0047] In this embodiment, see Figure 1 As shown, the present invention proposes a configurable flexible job shop scheduling method based on a multi-agent game model, comprising the steps of:

[0048] S10: Establish a dynamic scheduling model for a flexible job shop;

[0049] S20: Collect all kinds of production data in the workshop in real time, and use data preprocessing technology to clean and standardize the data;

[0050] S30: The flexible job shop scheduling model is converted into a dynamic disjunctive graph, and the structural entropy clustering method is used to divide the dynamic disjunctive graph into several subgraphs to improve scheduling efficiency and reduce computational difficulty. This process optimizes the collaborative work between intelligent agents by analyzing the correlation between processes and resources.

[0051] S40: Based on the dynamic disjunctive graph, a multi-agent system is constructed, including order agents and equipment agents, which are responsible for scheduling and resource management respectively; hyperparameters are initialized and a multi-agent network structure is established, including multiple deep Q networks, each agent has a deep Q network, and the deep Q network is trained, using experience replay and target network mechanisms to improve learning effects; based on the training results, the network structure and hyperparameters are optimized through the configurator;

[0052] S50: In the step S50, each agent selects its own action according to the current state through the output of the deep Q network and selects the current optimal action in combination with the game cooperation strategy;

[0053] S60: Apply the parameter and structure configurator of multiple optimization operators to adapt to different optimization requirements and environmental requirements, and perform online tuning of the agent's hyperparameters and deep Q network structure to ensure that the agent maintains efficient decision-making ability and scheduling performance in a changing environment.

[0054] As an optimization solution of the above embodiment, in step S10, a flexible job shop dynamic scheduling model is established, including:

[0055] Model assumptions and symbol definitions: The set and attributes of jobs, equipment and their processes are clarified, including the dynamic arrival time of jobs, the processing capacity of equipment, the processing time of processes, the failure of equipment and the module replacement time. Each parameter is defined in detail through the symbol system to provide a basis for the scheduling model;

[0056] Model Assumptions:

[0057] ①There are J jobs in total, each job contains several processes.

[0058] ② Each job j contains a set of processes O that are executed in sequence j .

[0059] ③There are M devices in the system, each of which can process a specific set of processes, defined as O m .

[0060] ④ The set of equipment available for process (j,o) is represented by M j,o , that is, some processes can only be processed by specific equipment.

[0061] ⑤The arrival time of the job is dynamic. Job j arrives at time α j Arrival system.

[0062] ⑥ The device is reconfigurable, that is, the device m can replace the module when necessary, and each replacement takes time δ m .

[0063] ⑦The device may fail, that is, device m fails at time f m A failure occurs and the repair time is r m .

[0064] ⑧Once the process starts, it cannot be interrupted and must be completed on the same equipment.

[0065] ⑨Each process can only be processed by one device at any time.

[0066] ⑩ The process can be assigned to different equipment before starting processing. If it needs to be transferred from equipment m to equipment m', the transfer time t needs to be considered. j,o,m,m' .

[0067] The processes have priority constraints, and the previous process must be completed before the next process can be completed.

[0068] Symbol definition and explanation: Table 4-1 gives the parameter table of the flexible job shop scheduling problem modeled by this patent.

[0069] Table 4-1 Parameters of flexible job shop scheduling problem

[0070]

[0071] Optimization objectives and constraints: With the goal of minimizing the maximum completion time, multiple constraints such as process sequence, equipment conflict, equipment failure, and module replacement are set to ensure the feasibility of the scheduling plan and the efficiency of equipment utilization, and to ensure the continuity of the process and the reasonable allocation of equipment.

[0072] Based on the above basic assumptions about the scheduling problem, the optimization objective and constraint equations of the scheduling problem are as follows:

[0073]

[0074] Sj, o ≥C j',o' -ρ(1-y j,o,j',o',m )+ρ(2-χ j,o,m -x j',o',m ) (5)

[0075] S j',o' ≥C j,o -ρyj,o,j',o',m +ρ(2-χ j,o,m -x j',o',m ) (6)

[0076]

[0077] S j,o ≥t fm +r m -ρ(1-χ j,o,m ) (8)

[0078] C j,o ≤t fm -∈+ρ(1-χ j,o,m ) (9)

[0079] S j,o ≥t sm +δ m -ρ(1-χ j,o,m ) (10)

[0080]

[0081] Where ρ represents a sufficiently large positive number, ∈ represents a very small positive number; formula (1) represents the optimization objective; formula (2) represents the process sequence constraint; formulas (3) and (4) represent the process start time constraint; formulas (5), (6) and (7) represent the equipment conflict constraint, y j,o,h',o',m ∈{0,1} means that any two processes processed on the same equipment cannot be performed at the same time; Formulas (8) and (9) represent equipment failure constraints; Formula (10) represents equipment module replacement constraints; Formulas (11) and (12) represent non-negative constraints; χ j,o,m ∈{0,1} means that if the oth process of job j is processed by equipment m, then χ j,o,m =1, otherwise 0.

[0082] As an optimization solution for the above embodiment, in step S30, the modeling of the dynamic disjunctive graph includes: mapping the scheduling problem into a dynamic disjunctive graph model, updating the process and equipment nodes and their relationships in real time, handling the dynamic changes of new job arrival, equipment failure and module replacement, optimizing equipment utilization through the graph model and dynamically adjusting the scheduling strategy.

[0083] Specific:

[0084] The dynamic flexible job shop scheduling problem is mapped into a dynamic disjunctive graph model. The scheduling process of the multi-agent system is modeled, where t represents the time.

[0085] The node set V(t) contains all the process nodes (j, o) and equipment nodes m∈M that are currently to be scheduled, that is,

[0086] V(t)={(j,o)|j∈J(t),o j ∈O j}∪{m / m∈M},

[0087] Where J(t) represents the set of jobs arriving at time t.

[0088] Deterministic edge set E c (t) represents the sequential relationship of the processes within the same operation, that is,

[0089] E c (t)={(j,o),(j,o+1) / j∈J(t),o j ∈O j}.

[0090] Disjunctive edge set E d (t) represents the mutually exclusive relationship between processes that need to be processed on the same equipment, that is,

[0091] E d (t)={(j,o),(j',o') / m∈M j',o' ∩M j,o ,j≠j'},

[0092] The order of the processes is represented by the bidirectional edges ((j,o)→(j',o')) and ((j',o')→(j,o)). The equipment utilization is indirectly represented by the weight of each disjunctive edge. The processing time of each process is used as the weight of the disjunctive edge to measure the time that the process occupies the equipment, that is,

[0093] W (j,o),(j',o') =p j,o ,

[0094] Then the utilization rate of equipment m is It is represented by the sum of the processing time of all processes assigned to equipment m, that is,

[0095]

[0096] When a new job arrives, add the process node of the job to the node set V(t) and add the corresponding deterministic edge E c (t) and the disjunctive edge E d (t) represents the relationship between processes. If a device fails or needs module replacement at a certain moment, the disjunctive edges of the relevant processes can be temporarily removed to ensure that the device will not be assigned new tasks during the failure or replacement period. After the failure is resolved or the replacement is completed, the relevant disjunctive edges are restored.

[0097] In the dynamic flexible job shop scheduling problem, the complexity of the scheduling graph increases rapidly with the random arrival of orders and the change of equipment status. In order to improve scheduling efficiency and simplify the calculation process, it is an effective strategy to cluster nodes of the disjunctive graph using structural entropy clustering technology. The main goal of implementing structural entropy clustering in the dynamic disjunctive graph model is to discover the correlation between processes by minimizing structural entropy, so as to group closely related processes together.

[0098] In the step S30, the dynamic disjunctive graph modeling includes:

[0099] Initialize clusters: Treat each node in the dynamic disjunctive graph as an independent cluster, and quantify the distribution state of each node by calculating the initial structural entropy;

[0100] Structural entropy optimization: With the goal of minimizing structural entropy, clusters with strong correlation are gradually merged; in each merging operation, the cluster pair that minimizes the structural entropy increment is selected, the structural entropy change ΔH after merging is calculated, and the overall structural entropy is updated;

[0101] Cluster stabilization: Continue to iterate the cluster merging operation until the structural entropy reaches a stable state, forming a set of highly correlated subgraphs.

[0102] Specifically, the clustering objective can be expressed as:

[0103]

[0104] Where |C| is the total number of clusters, |C k | is a class cluster C k In the initial stage of clustering, all nodes are considered as independent clusters, that is:

[0105]

[0106] At this time, the number of clusters is equal to the total number of nodes |V(t)|. The structural entropy H(G(t)) of the initial cluster can be calculated by the following formula:

[0107]

[0108] Among them, |E c (C k )| represents the class cluster C k The number of deterministic edges in |E d (C k )| represents the class cluster C k The number of disjunctive edges in |E c (t)| and |E d (t)| The total number of all deterministic edges and disjunctive edges in . Then, by selecting the merge operation, the structural entropy is reduced. Select two clusters C i and C j The merge is performed to minimize the increase in structural entropy. The structural entropy after the merge can be expressed as:

[0109] H ij =H(G(t))+ΔH

[0110] Where ΔH represents the combined C i and C j The change in the post-structural entropy. This change can be calculated by the following formula:

[0111] ΔH=H(C i ∪C j )-(H(C i )+H(C j )).

[0112] The merging operation is iterated until the structural entropy becomes stable. After structural entropy clustering, the dynamic disjunction graph Decomposed into multiple subgraphs Each of the C k Represents a group of highly related process nodes that share the same equipment during the scheduling process; E c (t),E c (t) represent the deterministic edges and mutually exclusive edges between processes in the cluster. This structural feature enables clustering to more clearly reflect the relationship between processes, thereby improving the utilization of equipment resources by the scheduling algorithm. The structural entropy clustering process effectively reduces the search space of the scheduling problem by dividing the node set V(t) into multiple groups. In traditional scheduling methods, scheduling decisions are usually based on the combination of all processes, resulting in an exponential increase in computational complexity. Through structural entropy, the present invention simplifies the scheduling process to a cluster of each cluster C. k The processing reduces the complexity of the scheduling algorithm from O(n!) to O(k!).

[0113] As an optimization scheme for the above embodiment, in order to solve the above-mentioned flexible job shop dynamic scheduling problem, the present invention proposes to use a multi-agent system for solving it. Each agent represents a scheduling unit, which can autonomously perceive, make decisions and collaborate to optimize the scheduling scheme. The following describes the multi-agent system from four aspects: agent construction, state space, action space and game process.

[0114] In step S40, the multi-agent system construction method includes:

[0115] Agent construction: Order agents and equipment agents are constructed according to scheduling requirements, which are responsible for order process allocation and equipment resource management respectively; the order agent is scheduled according to the process clusters divided by structural entropy clustering, and the equipment agent accepts or rejects tasks with the goal of maximizing utilization.

[0116] Specifically, in this scheduling problem, agents are divided into two types: order agents and equipment agents. Each order corresponds to an order agent. j , whose goal is to complete all processes and minimize the total completion time. The decision-making process of the order agent is based on the result of the structural entropy clustering module division. The basic unit of each order agent operation is the process cluster C k , rather than a single process node, improves the efficiency of scheduling decisions. Each device corresponds to an equipment agent m , whose goal is to maximize its own utilization U m , to avoid idle resources. The main function of the equipment agent is to manage the accepted process tasks and allocate tasks based on clustering. j ,Agent m / j∈J,m∈M}. The order agent and the equipment agent are connected through the nodes V(t) and edges E of the disjunction graph. c (t),E d (t) Interaction, achieving coordination among multiple agents.

[0117] State space construction: define the state space for the agent; the state of the order agent includes the completion of the process and the status of the optional equipment; the state of the equipment agent includes the utilization rate, task queue and fault status, ensuring that the agent can obtain the environmental information required for scheduling decisions.

[0118] Specifically, in a multi-agent system, the state space st describes the environmental information required by each agent in the scheduling process, including all relevant state variables at the current time t. The state space can be divided into the state parts of the order agent and the equipment agent. j The status reflects the process completion status, optional equipment status and other related information of the order. It can be expressed as:

[0119]

[0120] in, Represents cluster C k When all processes in the cluster are completed, Otherwise, 0; Represents cluster C k The number of processes included in the task provides information on the size of the task; Represents cluster C k the urgency of the Represents the cluster set at the current time t. m The status includes the utilization rate, task queue and fault status of the device. It can be expressed as:

[0121]

[0122] Among them, U m (t) represents the utilization rate of equipment m at time t; Indicates the cluster of devices m currently waiting to be processed; F m (t) indicates the equipment failure state. When the equipment fails, F m (t) = 1, otherwise 0.

[0123] Action space construction: define the action selection space of the agent; the action of the order agent is to assign the process cluster to the appropriate equipment; the action of the equipment agent is whether to accept the cluster task, thereby realizing dynamic resource allocation.

[0124] Specifically, in the dynamic flexible job shop scheduling problem, the action space at contains the set of actions that each agent can choose at the scheduling time t, which is specifically divided into the actions of the order agent and the equipment agent. j Responsible for managing the process scheduling of job j, its main action is to select and assign the processes in the cluster to the appropriate equipment for processing. Specifically, the action space of the order agent can be expressed as:

[0125]

[0126] in, Indicates that device m is selected to complete the corresponding cluster. When the cluster C k Assigned to equipment m for processing; Indicates that it can process cluster C k Each device agent m The main goal of is to decide whether to accept a new clustering task. The action space of the device agent is cluster level, which is specifically expressed as:

[0127]

[0128] Among them, ap(C k ) indicates that when the device accepts cluster C k All the tasks in the process are added to the task queue and U is updated. m (t), rj(C k) indicates that when the device rejects new tasks, it maintains the current state.

[0129] In step S50, each agent selects its own action according to the output of the deep Q network according to the current state and selects the current optimal action in combination with the game cooperation strategy. Game and cooperation mechanism: game and cooperation between agents, using deep Q network and greedy strategy (ε-greedy strategy can be used) to optimize the scheduling strategy, and using customized reward mechanism to promote the cooperation of similar agents and regulate the competition between different types of agents.

[0130] In this dynamic and flexible job shop scheduling system, the game process is not only to optimize the action selection of independent agents, but also to regulate the cooperation and competition among multiple agents based on clustering. The structural entropy clustering results provide the associated groups of divided jobs and equipment, allowing agents to play games in a smaller decision space. Through the deep Q network and greedy strategy, each agent can dynamically select actions to optimize the scheduling plan.

[0131] In a deep Q-network, each agent uses a Q-network based on a deep learning model to fit its Q-value function Where θ is the parameter of the deep learning model. For each agent i, the Q network receives the state st i As input, and output each action at i The Q network is trained with the following loss function:

[0132]

[0133] Among them, r represents the immediate reward; st' and at' represent the next state and action respectively; θ' represents the parameters of the target network; γ represents the discount factor.

[0134] In order to achieve collaboration and competition, the present invention designs different reward mechanisms, which enables agents of the same type to work together to achieve common optimization goals, while agents of different types may exhibit non-cooperative behaviors due to differences in goals. In this game model, agents of the same type cooperate to optimize the overall scheduling, while agents of different types compete due to differences in goals. Specifically, order agents of the same type can share information and resources to jointly optimize the scheduling effect. For example, multiple order agents can negotiate to select the best equipment to reduce the total completion time. Different types of agents compete with each other due to different optimization goals (such as order agents hope to complete all processes quickly, while equipment agents hope to maximize the utilization rate of equipment). The reward functions of the two types of agents are designed separately, reflecting the rewards for performing actions in specific states. Specifically, the reward function of the order agent is:

[0135]

[0136] Among them, η1 and η2 represent weight coefficients, which are used to balance the impact of completion time and urgency. The reward function of the equipment agent is:

[0137]

[0138] Among them, μ1 and μ2 represent weight coefficients, which are used to balance the impact of utilization and the number of tasks. In each step, the agent j With Agent m Select actions based on the value output by the Q network and use the ε-greedy strategy for exploration:

[0139]

[0140] Through the design of deep Q network and reward mechanism, agents of the same type can collaborate within the cluster, while agents of different types compete according to their optimization goals. Based on the structural entropy clustering grouping, different types of agents compete with each other and finally reach the optimal solution for scheduling in the local subgraph.

[0141] Collaboration Mechanism:

[0142] Order Agent j and Agent j' In the same cluster C k The collaboration is to optimize the overall completion time. The collaboration process is achieved by sharing the cluster state. Implementation, the specific collaboration mechanism can be expressed as:

[0143]

[0144] in, represents the collaboration weight, reflecting the relationship between agents j and j' on cluster C k The degree of dependence. m and Agent m' In the same cluster C k The collaboration is designed to maximize overall equipment utilization. The collaboration process is achieved through a shared task queue. Implementation, the specific collaboration mechanism can be expressed as:

[0145]

[0146] in, represents the collaboration weight, reflecting the relationship between agents m and m' on cluster C k degree of dependence.

[0147] Competition Mechanism:

[0148] Order Agent j Agent m Competition between agents is reflected in resource allocation, especially when multiple order agents compete for the same device resources. The competition mechanism is realized through the mutual influence of utility functions:

[0149]

[0150] Among them, λ1 and λ2 represent competition weight coefficients; represents the set of order agents that apply to use equipment m; represents the indicator function. In the game process, the multi-agent system seeks to reach the Nash equilibrium, that is, in the equilibrium state, no agent can improve its utility by unilaterally changing its strategy. Mathematically, for all agents i,

[0151]

[0152] in, represents the optimal action of agent i; represents the optimal action of all agents except agent i. Through deep Q network training, each agent i learns its optimal strategy To maximize its expected utility Gradually approaching Nash equilibrium.

[0153] The overall algorithm flow is as follows:

[0154] Step 1: According to the structural entropy clustering results, map the agent to the corresponding cluster subgraph.

[0155] Step 2: Each agent perceives the current state of its cluster

[0156] Step 3: The agent is based on the current state Action selection via a deep Q-network.

[0157] Step 4: The order agent assigns cluster tasks to the equipment agent, and the equipment agent decides to accept or reject the task.

[0158] Step 5: Calculate the cooperative and competitive utility functions of each agent based on the action results and

[0159] Step 6: Through the loss function Update the deep Q-network parameters to minimize the prediction error.

[0160] Step 7: Repeat steps 2-6 until the termination condition is met.

[0161] As an optimization solution for the above-mentioned embodiment, the parameter and structure configurator is used for automatic structure configuration and hyperparameter optimization of the deep Q network of agents in a multi-agent system, aiming to automatically optimize the hyperparameters and structure of the deep Q network model according to the dynamic changes of the production environment. Its core modules include: a hyperparameter optimization module based on heuristic algorithms and an automatic Q network structure optimization module based on neural architecture search.

[0162] In step S60, the parameter and structure configurator of the multi-optimization operator includes:

[0163] The hyperparameter optimization module based on heuristic algorithms is used to dynamically optimize the hyperparameters of the deep Q network in a multi-agent system through genetic algorithms, particle swarm algorithms and ant colony algorithms to improve the performance and adaptability of the scheduling strategy in dynamic flexible job shops.

[0164] This module aims to optimize the hyperparameters of the deep Q network of the multi-agent system through a variety of heuristic algorithms to improve the performance and adaptability of the scheduling strategy in the dynamic flexible job shop. This module includes three classic heuristic operators: genetic algorithm, particle swarm algorithm and ant colony algorithm.

[0165] Preferably, the hyperparameter optimization module based on the heuristic algorithm includes a genetic algorithm, a particle swarm algorithm and an ant colony algorithm;

[0166] The genetic algorithm iteratively optimizes the hyperparameter configuration of the population individuals through selection, crossover, and mutation steps;

[0167] The particle swarm algorithm searches for the optimal configuration based on the global and individual historical optimal positions by updating the particle speed and position;

[0168] The ant colony algorithm selects the hyperparameter path through pheromone concentration and heuristic information, and continuously optimizes the hyperparameters using the pheromone update mechanism;

[0169] Each algorithm is based on fitness evaluation and iteratively solves the optimal hyperparameter configuration suitable for dynamic flexible job shops.

[0170] Hyperparameter optimization based on genetic algorithm:

[0171] Genetic algorithms search for optimal solutions through population evolution, which can effectively explore and find the best configuration in the hyperparameter space. In the hyperparameter optimization process, the algorithm first initializes a randomly generated population Each individual represents a hyperparameter configuration. Next, the algorithm evaluates the fitness of each individual and calculates the fitness value. This value is obtained by scheduling a task in a specific The Q network model of the agent is obtained by training it, that is:

[0172]

[0173] in, is the loss function of the model. The higher the fitness value, the better the hyperparameter configuration effect. In order to select the parent individuals, the genetic algorithm adopts the tournament selection method. Randomly select from the population individuals, compare their fitness values, and select the individual with the highest fitness as the parent:

[0174]

[0175] Where S is a set of randomly selected individuals. In the crossover process, two parent individuals are selected and The crossover factor α is generated randomly ga To generate offspring individuals Right now:

[0176]

[0177] At the same time, the mutation operation has a certain mutation probability The random changes of individuals can be expressed as:

[0178]

[0179] Among them, ∈ ga is a random variable sampled from a normal distribution; Represents the mutated individual. After fitness evaluation, selection, crossover and mutation, the algorithm continues to iterate and evolve until the preset stop condition is met.

[0180] Hyperparameter optimization based on particle swarm algorithm:

[0181] The particle swarm algorithm is an optimization algorithm based on swarm intelligence, simulating the foraging behavior of bird flocks. In hyperparameter optimization, the particle swarm algorithm searches for the optimal hyperparameter configuration through the movement and update of particles. First, N particles are generated, each particle represents a hyperparameter configuration, the position of the particle represents the hyperparameter value, and the speed represents the direction and amplitude of the parameter update. Then, the fitness value of each particle is calculated. Right now:

[0182]

[0183] According to the fitness evaluation results, the speed and position of the particle are updated, which can be expressed as:

[0184]

[0185] Among them, c1 represents the inertia weight, c2 and c3 represent the learning factors; r1 and r2 represent random numbers; pbesti represents the historical optimal position of the particle; gbest represents the global optimal position of the group. The particle swarm algorithm continuously iterates and evolves until the preset stop condition is met.

[0186] Hyperparameter optimization based on ant colony algorithm:

[0187] The ant colony algorithm is an optimization algorithm that simulates the foraging behavior of ants. It guides the search path through the deposition and volatilization of pheromones. In the hyperparameter optimization process, the ant colony algorithm first initializes the initial pheromone amount τ of each path. ij , which is used to guide the direction of hyperparameter selection. Each ant selects a hyperparameter configuration based on pheromone and heuristic information, and its selection probability is determined by the following formula:

[0188]

[0189] Among them, τ ij represents the pheromone concentration of path (i, j); η ij Represents the heuristic information of path (i, j); α1 represents the importance parameter of pheromone, which controls the influence of pheromone in the selection process. The larger its value, the greater the influence of pheromone on path selection; α2 represents the importance parameter of heuristic information, which controls the influence of pheromone in the selection process. The larger its value, the greater the influence of heuristic information on path selection. After each ant completes the search, it needs to evaluate the fitness of the path it has passed through and calculate its fitness value Then, the pheromone concentration of the high-quality path is enhanced by updating the pheromone amount:

[0190] τ' ij =(1-c4)τ ij +Δτ ij ,

[0191] Among them, c4 represents the pheromone volatility factor, Δτ ij Indicates the pheromone added by the ants passing through the path. The ant colony algorithm continuously optimizes the hyperparameter configuration by simulating the ant movement and pheromone update process until the preset stop condition is met.

[0192] And a structural optimization module based on neural architecture search, which is used to automatically optimize the deep Q network structure through fish swarm algorithm, bat algorithm, and firefly algorithm to adapt to different production environments and scheduling requirements, and achieve continuous improvement in scheduling strategy performance.

[0193] Preferably, the structure optimization module based on neural architecture search includes a fish school optimization operator, a bat optimization operator and a firefly optimization operator;

[0194] The structural update mechanism based on the fish school optimization operator simulates the foraging behavior of fish schools and uses position, speed and linear attraction functions to adjust the configuration of each layer of the network structure to adapt to different scheduling tasks and environmental changes;

[0195] The structural search mechanism based on the bat optimization operator uses the bat echolocation behavior, combines the adjustment factor and the noise factor, and dynamically optimizes the network structure to achieve effective exploration to adapt to different scheduling tasks and environmental changes;

[0196] Based on the brightness attraction mechanism of the firefly optimization operator, by simulating the luminous attraction behavior of fireflies, the network hierarchy configuration is adjusted according to the architecture fitness and attraction parameters to adapt to different scheduling tasks and environmental changes.

[0197] Specifically, in dynamic flexible job shop scheduling, the structural optimization of the Q network is a key link to improve the performance of the scheduling strategy. This module is based on the neural architecture search framework and uses three different bionic optimization operators to achieve automatic Q network structure optimization to adapt to different production environments and scheduling requirements. This goal can be mathematically expressed as:

[0198]

[0199] in, represents the set of candidate network architectures, Indicates the reward of the selected architecture under a given scheduling task. In the initial stage of the search, the present invention uses a random generation method to construct a network architecture. Each network architecture can be represented by a parameter vector in, represents the number of neurons in the i-th layer, Indicates the type of the i-th layer. During the optimization process, this module combines three bionic optimization operators: fish school, bat, and firefly, aiming to efficiently search and optimize the Q network architecture. The fish school operator optimizes the network structure by imitating the foraging behavior of fish schools. The key to this operator is to evaluate the performance of each architecture and update the architecture based on the performance. Specifically, its update rules are as follows:

[0200]

[0201] Among them, χ i is the position of the i-th architecture; represents the speed at which the architecture moves in the search space; Φ(·) represents the linear attraction function; d1 represents the adjustment factor; χ best represents the position of the current best architecture; foodIndicates the location of the food vector. The bat operator performs architecture search by simulating the echolocation behavior of bats. The algorithm updates the exploration operator based on the exploration and evaluation of the architecture, and then updates the network structure. The specific update process is:

[0202]

[0203] Among them, d2 represents the adjustment factor and d3 represents the noise factor, which is used to simulate the search behavior of bats. The firefly operator optimizes the architecture through the attraction factor by imitating the flashing behavior of fireflies. The attraction of the firefly operator is proportional to its brightness, which is related to the fitness of the selected architecture. The optimization process is described by the following formula:

[0204]

[0205] Among them, χ j Indicates the location of the light-emitting structure; ω ij Representation architecture χ i and χ j The distance between them is calculated by Euclidean distance; d4 and d5 represent two parameters that control attraction. By combining these three bionic optimization algorithms, this module can search and optimize the architecture of the deep Q network to adapt to different scheduling tasks and production environments.

[0206] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A configurable flexible job shop scheduling method based on a multi-agent game model, characterized in that: Includes steps: S10: Establish a dynamic scheduling model for a flexible job shop; S20: Collect all kinds of production data in the workshop in real time, and use data preprocessing technology to clean and standardize the data; S30: The flexible job shop scheduling model is converted into a dynamic disjunctive graph, and the structural entropy clustering method is used to divide the dynamic disjunctive graph into several subgraphs; this process optimizes the collaborative work between intelligent agents by analyzing the correlation between processes and resources; S40: Based on the dynamic disjunctive graph, a multi-agent system is constructed, including order agents and equipment agents, which are responsible for scheduling and resource management respectively; hyperparameters are initialized and a multi-agent network structure is established, including multiple deep Q networks, each agent has a deep Q network, and the deep Q network is trained, using experience replay and target network mechanisms to improve learning effects; According to the training results, the network structure and hyperparameters are optimized through the configurator; S50: Each agent selects its own action according to the current state through the output of the deep Q network and selects the current optimal action in combination with the game cooperation strategy; S60: Apply the parameter and structure configurator of multiple optimization operators to adapt to different optimization requirements and environmental requirements, and perform online tuning of the agent's hyperparameters and deep Q network structure to ensure that the agent maintains efficient decision-making ability and scheduling performance in a changing environment.

2. A configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S10, a flexible job shop dynamic scheduling model is established, including: Model assumptions and symbol definitions: The set and attributes of jobs, equipment and their processes are clarified, including the dynamic arrival time of jobs, the processing capacity of equipment, the processing time of processes, the failure of equipment and the module replacement time. Each parameter is defined in detail through the symbol system to provide a basis for the scheduling model; Optimization objectives and constraints: With the goal of minimizing the maximum completion time, multiple constraints such as process sequence, equipment conflict, equipment failure, and module replacement are set to ensure the feasibility of the scheduling plan and the efficiency of equipment utilization, and to ensure the continuity of the process and the reasonable allocation of equipment.

3. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S30, the modeling of the dynamic disjunctive graph includes: mapping the scheduling problem into a dynamic disjunctive graph model, updating the process and equipment nodes and their relationships in real time, handling the dynamic changes of new job arrival, equipment failure and module replacement, optimizing equipment utilization through the graph model and dynamically adjusting the scheduling strategy.

4. A configurable flexible job shop scheduling method based on a multi-agent game model according to claim 3, characterized in that: In the step S30, the dynamic disjunctive graph modeling includes: Initialize clusters: Treat each node in the dynamic disjunctive graph as an independent cluster, and quantify the distribution state of each node by calculating the initial structural entropy; Structural entropy optimization: With the goal of minimizing structural entropy, clusters with strong correlation are gradually merged; in each merging operation, the cluster pair that minimizes the structural entropy increment is selected, the structural entropy change ΔH after merging is calculated, and the overall structural entropy is updated; Cluster stabilization: Continue to iterate the cluster merging operation until the structural entropy reaches a stable state, forming a set of highly correlated subgraphs.

5. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S40, the multi-agent system construction method includes: Agent construction: Order agents and equipment agents are constructed according to scheduling requirements, responsible for order process allocation and equipment resource management respectively; order agents are scheduled according to process clusters divided by structural entropy clustering, and equipment agents accept or reject tasks with the goal of maximizing utilization; State space construction: define the state space for the agent; the state of the order agent includes the process completion status and the optional equipment status; the state of the equipment agent includes utilization, task queue and fault status, ensuring that the agent can obtain the environmental information required for scheduling decisions; Action space construction: define the action selection space of the agent; the action of the order agent is to assign the process cluster to the appropriate equipment; the action of the equipment agent is whether to accept the cluster task, thereby realizing dynamic resource allocation.

6. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S50, each agent selects its own action according to the output of the deep Q network according to the current state and selects the current optimal action in combination with the game cooperation strategy. Game and cooperation mechanism: game and cooperation between agents, using deep Q network and greedy strategy to optimize the scheduling strategy, using customized reward mechanism to promote the cooperation of similar agents and regulate the competition between different types of agents.

7. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S60, the parameter and structure configurator of the multi-optimization operator includes: A heuristic-based hyperparameter optimization module for dynamically optimizing the hyperparameters of deep Q networks in multi-agent systems using genetic algorithms, particle swarm algorithms, and ant colony algorithms; And a structural optimization module based on neural architecture search, which is used to automatically optimize the deep Q network structure through fish swarm algorithm, bat algorithm, and firefly algorithm.

8. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 7, characterized in that: The hyperparameter optimization module based on heuristic algorithms includes genetic algorithm, particle swarm algorithm and ant colony algorithm; The genetic algorithm iteratively optimizes the hyperparameter configuration of the population individuals through selection, crossover, and mutation steps; The particle swarm algorithm searches for the optimal configuration based on the global and individual historical optimal positions by updating the particle speed and position; The ant colony algorithm selects the hyperparameter path through pheromone concentration and heuristic information, and continuously optimizes the hyperparameters using the pheromone update mechanism; Each algorithm is based on fitness evaluation and iteratively solves the optimal hyperparameter configuration suitable for dynamic flexible job shops.

9. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 7, characterized in that: A structural optimization module based on neural architecture search, including fish school optimization operator, bat optimization operator and firefly optimization operator; The structural update mechanism based on the fish school optimization operator simulates the foraging behavior of fish schools and uses position, speed and linear attraction functions to adjust the configuration of each layer of the network structure to adapt to different scheduling tasks and environmental changes; The structural search mechanism based on the bat optimization operator uses the bat echolocation behavior, combines the adjustment factor and the noise factor, and dynamically optimizes the network structure to achieve effective exploration to adapt to different scheduling tasks and environmental changes; Based on the brightness attraction mechanism of the firefly optimization operator, by simulating the luminous attraction behavior of fireflies, the network hierarchy configuration is adjusted according to the architecture fitness and attraction parameters to adapt to different scheduling tasks and environmental changes.

Images