Configurable flexible job shop scheduling method based on multi-agent game model

Abstract

The present invention discloses a configurable flexible job shop scheduling method based on a multi-agent game model, which maps the scheduling problem into a dynamic disjunctive graph model to describe the relationship between processes and equipment. The process nodes, equipment nodes and their mutual relationships are represented by node sets and edge sets. Combined with the flexibility of the dynamic disjunctive graph, the matching of equipment and processes is dynamically adjusted according to changes in production demand to optimize equipment utilization. A clustering method based on structural entropy is adopted to cluster process nodes into subgraphs, thereby reducing the search space of the scheduling problem and reducing the computational complexity. It covers the construction of order agents and equipment agents and the game and collaboration process between them. Each agent optimizes decisions through a deep Q network and collaborates or competes according to their respective goals. An automatic optimization configurator is used to optimize the hyperparameters and network structure of the deep Q network in the multi-agent system, further improving scheduling performance and adaptability.

Description

Technical Field

[0001] The present invention belongs to the technical field of production workshop scheduling, and in particular relates to a configurable flexible job shop scheduling method based on a multi-agent game model. Background Art

[0002] With the rapid development of intelligent manufacturing technology, production systems are gradually shifting towards flexibility and intelligence, and production models are increasingly moving towards multi-tasking and resource sharing. Against this backdrop, manufacturing companies' production workshops face increasingly dynamic and uncertain scheduling problems. For example, frequent changes in equipment status and order requirements make traditional static scheduling methods difficult to meet real-world needs. In this dynamic production environment, production systems must not only consider how to rationally arrange the sequence of various processes but also optimize resource allocation to maximize equipment utilization and minimize total completion time. Furthermore, the increasing market demand for high-quality, short-cycle products places higher demands on scheduling systems. Workshop scheduling systems must be able to rapidly adapt to diverse demands and flexibly adjust production plans in the event of unexpected situations, such as equipment failures and urgent order insertions. Therefore, how to efficiently schedule production resources in this dynamic environment through methods such as multi-agent collaboration and automated parameter configuration has become a key technical challenge in the field of intelligent manufacturing.

[0003] Existing flexible job shop scheduling methods are mostly based on static models, making them difficult to quickly respond to dynamic changes in the production environment, such as the random arrival of orders, equipment failures, and module replacements. This severely impacts the real-time and accuracy of scheduling decisions, making it impossible to effectively adapt to frequently changing production demands. Furthermore, traditional methods often lack efficient coordination mechanisms when dealing with multi-task competition and equipment resource management, making it difficult to achieve an optimal balance between task conflicts and resource allocation, thereby reducing resource utilization and overall production efficiency.

[0004] As the number of tasks and equipment on a job shop increases, the computational complexity of traditional global optimization scheduling methods increases exponentially, making real-time scheduling difficult to achieve in large-scale and complex systems. These methods often fail to provide optimized scheduling solutions within a reasonable timeframe, limiting their effectiveness in real-world production environments. Furthermore, the lack of effective computational complexity control mechanisms makes the scheduling decision-making process time-consuming and unable to meet the requirements for efficient response.

[0005] Existing scheduling systems primarily rely on static configurations, making them difficult to adapt to the dynamic changes in task and equipment status in flexible job shops. This results in an inability to adjust scheduling strategies in real time, impacting the accuracy and efficiency of decision-making. Furthermore, as the number of tasks and equipment increases, the computational complexity of traditional methods increases significantly, resulting in lengthy scheduling decisions. This limits the system's real-time application in large-scale and complex production environments and reduces overall scheduling efficiency. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes a configurable flexible job shop scheduling method based on a multi-agent game model, which is an advanced scheduling method that can dynamically adapt to changes in the production environment and efficiently manage computational complexity; it can quickly respond to dynamic changes in the production environment, improve the efficiency of multi-task collaboration and resource management, adapt to complex and dynamically changing environments, and explore more flexible solutions to achieve faster and scalable scheduling decisions.

[0007] To achieve the above object, the present invention adopts a technical solution: a configurable flexible job shop scheduling method based on a multi-agent game model, comprising the following steps:

[0008] S10: Establish a dynamic scheduling model for a flexible job shop;

[0009] S20: Collects all kinds of production data in the workshop in real time and uses data preprocessing technology to clean and standardize the data;

[0010] S30: Convert the flexible job shop scheduling model into a dynamic disjunctive graph and use the structural entropy clustering method to divide the dynamic disjunctive graph into several subgraphs. This process optimizes the collaborative work between intelligent agents by analyzing the correlation between processes and resources.

[0011] S40: Based on the dynamic disjunctive graph, a multi-agent system is constructed, including an order agent and a device agent, responsible for scheduling and resource management, respectively. Hyperparameters are initialized and the multi-agent network structure is established, including multiple deep Q networks, one for each agent. The deep Q networks are trained, using experience replay and target network mechanisms to improve learning. Based on the training results, the network structure and hyperparameters are optimized through the configurator.

[0012] S50: Each agent selects its own action based on the current state through the output of the deep Q network and selects the current optimal action in combination with the game cooperation strategy;

[0013] S60: Apply parameter and structure configurators for multiple optimization operators to adapt to different optimization and environmental requirements, and perform online tuning of the agent's hyperparameters and deep Q network structure to ensure that the agent maintains efficient decision-making and scheduling performance in a changing environment.

[0014] Furthermore, in step S10, a flexible job shop dynamic scheduling model is established, including:

[0015] Model Assumptions and Symbol Definitions: This defines the set and attributes of jobs, equipment, and their processes, including the dynamic arrival time of jobs, the processing capacity of equipment, the processing time of processes, and the time for equipment failure and module replacement. The symbolic system defines each parameter in detail, providing a foundation for the scheduling model.

[0016] Optimization objectives and constraints: With the goal of minimizing the maximum completion time, multiple constraints are set, including process sequence, equipment conflict, equipment failure, and module replacement, to ensure the feasibility of the scheduling plan and equipment utilization efficiency, as well as process continuity and reasonable equipment allocation.

[0017] Furthermore, in step S30, the modeling of the dynamic disjunctive graph includes: mapping the scheduling problem into a dynamic disjunctive graph model, updating the process and equipment nodes and their relationships in real time, handling the dynamic changes of new job arrival, equipment failure and module replacement, optimizing equipment utilization through the graph model and dynamically adjusting the scheduling strategy.

[0018] Furthermore, in step S30, the dynamic disjunctive graph modeling includes:

[0019] Initialize clusters: Treat each node in the dynamic disjunctive graph as an independent cluster and quantify the distribution of each node by calculating the initial structural entropy;

[0020] Structural entropy optimization: With the goal of minimizing structural entropy, clusters with strong correlations are gradually merged. In each merging operation, the cluster pair that minimizes the structural entropy increment is selected, the structural entropy change ΔH after the merger is calculated, and the overall structural entropy is updated.

[0021] Cluster stabilization: Continuously iterate the cluster merging operation until the structural entropy reaches a stable state, forming a set of highly correlated subgraphs.

[0022] Furthermore, in step S40, the multi-agent system construction method includes:

[0023] Agent construction: Order agents and equipment agents are constructed based on scheduling requirements, responsible for order process allocation and equipment resource management, respectively. The order agent performs scheduling based on process clusters divided by structural entropy clustering, while the equipment agent accepts or rejects tasks with the goal of maximizing utilization.

[0024] State space construction: Define the state space for the agent. The state of the order agent includes the completion status of the process and the status of the optional equipment. The state of the equipment agent includes utilization, task queue and fault status, ensuring that the agent can obtain the environmental information required for scheduling decisions.

[0025] Action space construction: Define the action selection space of the agent; the action of the order agent is to assign the process cluster to the appropriate equipment; the action of the equipment agent is whether to accept the cluster task, thereby realizing dynamic resource allocation.

[0026] Furthermore, in step S50, each agent selects its own action according to the output of the deep Q network based on the current state and selects the current optimal action in combination with the game cooperation strategy. Game and cooperation mechanism: game and cooperation between agents, using deep Q network and greedy strategy to optimize scheduling strategy, using customized reward mechanism to promote the cooperation of similar agents and regulate the competition between different types of agents.

[0027] Furthermore, in step S60, the parameter and structure configurator of the multiple optimization operators includes:

[0028] A heuristic-based hyperparameter optimization module for dynamically optimizing the hyperparameters of deep Q networks in multi-agent systems using genetic algorithms, particle swarm algorithms, and ant colony algorithms;

[0029] And a structural optimization module based on neural architecture search, which is used to automatically optimize the deep Q network structure through fish swarm algorithm, bat algorithm, and firefly algorithm.

[0030] Furthermore, the hyperparameter optimization module based on heuristic algorithms includes genetic algorithm, particle swarm algorithm and ant colony algorithm;

[0031] The genetic algorithm iteratively optimizes the hyperparameter configuration of the individuals in the population through selection, crossover, and mutation steps;

[0032] The particle swarm optimization algorithm searches for the optimal configuration based on the global and individual historical optimal positions by updating the particle speed and position;

[0033] The ant colony algorithm selects the hyperparameter path through pheromone concentration and heuristic information, and continuously optimizes the hyperparameters using the pheromone update mechanism;

[0034] Each algorithm is based on fitness evaluation and iteratively solves the optimal hyperparameter configuration suitable for dynamic and flexible job shops.

[0035] Furthermore, the structural optimization module based on neural architecture search includes fish school optimization operator, bat optimization operator and firefly optimization operator;

[0036] The structural update mechanism based on the fish school optimization operator simulates the foraging behavior of fish schools and uses position, speed and linear attraction functions to adjust the configuration of each layer of the network structure to adapt to different scheduling tasks and environmental changes;

[0037] The structure search mechanism based on the bat optimization operator leverages bat echolocation behavior, combines adjustment factors with noise factors, and dynamically optimizes the network structure to achieve effective exploration to adapt to different scheduling tasks and environmental changes.

[0038] Based on the brightness attraction mechanism of the firefly optimization operator, by simulating the luminous attraction behavior of fireflies, the network hierarchy configuration is adjusted according to the architecture fitness and attraction parameters to adapt to different scheduling tasks and environmental changes.

[0039] The beneficial effects of adopting this technical solution are:

[0040] The present invention proposes a scheduling method for dynamic flexible job shop scheduling. First, the scheduling problem is mapped into a dynamic disjunctive graph model to describe the relationship between processes and equipment. The model represents process nodes, equipment nodes and their mutual relationships through node sets and edge sets, and combines the flexibility of the dynamic disjunctive graph to dynamically adjust the matching of equipment and processes according to changes in production demand to optimize equipment utilization. Then, a clustering method based on structural entropy is adopted to cluster process nodes into subgraphs, thereby reducing the search space of the scheduling problem and reducing the computational complexity. In addition, it covers the construction of order agents and equipment agents and the game and collaboration process between them. Each agent optimizes decisions through a deep Q network and collaborates or competes according to their respective goals. Finally, an automatic optimization configurator is designed to optimize the hyperparameters and network structure of the deep Q network in the multi-agent system to further improve the scheduling performance and adaptability.

[0041] This method maps the various process flows, resource allocations, and job execution statuses in a scheduling problem to nodes in a graph structure. It uses edge weights to represent dependencies between tasks, dynamically updating constraints and relationships between tasks. This enhances the ability to handle interactions between tasks in complex scheduling environments. Specifically, over time, the graph structure can reflect changes in shop floor status and the dynamic adjustment of task priorities in real time, enabling more efficient and flexible scheduling decisions.

[0042] This paper designs a structural entropy clustering method based on a dynamic disjunctive graph. By dividing the scheduling graph into multiple subgraphs, it effectively reduces the search space and thus the algorithm's computational complexity. Specifically, structural entropy clustering discovers the correlations between process nodes and clusters closely related processes together, effectively reducing the search space of the scheduling problem. This method continuously optimizes the clustering results through iterative merging operations, simplifying the scheduling process from traditional complex decisions based on all process combinations to independent processing of each cluster, reducing the computational complexity of the scheduling algorithm. In addition, structural entropy clustering enhances the scheduling algorithm's utilization of equipment resources, improving the system's scheduling efficiency and flexibility.

[0043] The present invention can optimize production scheduling strategies: By introducing scheduling based on a multi-agent system, the present invention can achieve efficient collaboration and competition among agents in a dynamic and flexible job shop environment, thereby significantly improving scheduling efficiency and the utilization of production resources. Each agent makes real-time decisions based on its tasks and goals, sharing information and resources, and can adapt to the complex dynamic changes in the workshop. In addition, the present invention, through the design of a highly adaptable reward mechanism and deep Q-network training, enables order agents and equipment agents to find the optimal scheduling solution in local games, ultimately achieving global optimization. This dynamic, collaborative and competitive parallel scheduling mechanism improves production flexibility and responsiveness, and optimizes the overall production scheduling strategy.

[0044] The present invention can achieve automatic optimization of hyperparameters and Q network structure: by introducing an optimization module based on heuristic algorithms and neural architecture search, the present invention can automatically adjust the hyperparameters and network structure of the deep Q network according to the dynamic changes of the production environment. Specifically, the hyperparameter optimization module adopts classic heuristic operators such as genetic algorithm, particle swarm algorithm and ant colony algorithm to optimize the key parameters such as learning rate and discount factor of the Q network in the multi-agent system by simulating evolution, swarm intelligence and foraging behavior in nature. These algorithms can efficiently explore the hyperparameter space in complex scheduling tasks and improve scheduling performance and adaptability. At the same time, the structural optimization module based on neural architecture search combines biomimetic optimization algorithms such as fish schools, bats and fireflies, and automatically searches and optimizes the hierarchical structure and neuron configuration of the Q network by simulating the behavior of animals in nature to match the scheduling requirements in different production environments. Through the collaborative work of these two optimization modules, the present invention can dynamically adjust the hyperparameters and architecture of the deep Q network, thereby achieving continuous optimization and performance improvement of the scheduling strategy, ensuring that efficient and accurate scheduling capabilities are always maintained in a constantly changing production environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The figure is a flow chart of a configurable flexible job shop scheduling method based on a multi-agent game model of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings.

[0047] In this embodiment, see Figure 1 As shown, the present invention proposes a configurable flexible job shop scheduling method based on a multi-agent game model, comprising the following steps:

[0048] S10: Establish a dynamic scheduling model for a flexible job shop;

[0049] S20: Collects all kinds of production data in the workshop in real time and uses data preprocessing technology to clean and standardize the data;

[0050] S30: Convert the flexible job shop scheduling model into a dynamic disjunctive graph and use the structural entropy clustering method to divide the dynamic disjunctive graph into several subgraphs to improve scheduling efficiency and reduce computational complexity. This process optimizes the collaborative work between agents by analyzing the correlation between processes and resources.

[0051] S40: Based on the dynamic disjunctive graph, a multi-agent system is constructed, including an order agent and a device agent, responsible for scheduling and resource management, respectively. Hyperparameters are initialized and the multi-agent network structure is established, including multiple deep Q networks, one for each agent. The deep Q networks are trained, using experience replay and target network mechanisms to improve learning. Based on the training results, the network structure and hyperparameters are optimized through the configurator.

[0052] S50: In step S50, each agent selects its own action according to the output of the deep Q network based on the current state and selects the current optimal action in combination with the game cooperation strategy;

[0053] S60: Apply parameter and structure configurators for multiple optimization operators to adapt to different optimization and environmental requirements, and perform online tuning of the agent's hyperparameters and deep Q network structure to ensure that the agent maintains efficient decision-making and scheduling performance in a changing environment.

[0054] As an optimization solution of the above embodiment, in step S10, a flexible job shop dynamic scheduling model is established, including:

[0055] Model Assumptions and Symbol Definitions: This defines the set and attributes of jobs, equipment, and their processes, including the dynamic arrival time of jobs, the processing capacity of equipment, the processing time of processes, and the time for equipment failure and module replacement. The symbolic system defines each parameter in detail, providing a foundation for the scheduling model.

[0056] Model assumptions:

[0057] ①There are J jobs in total, and each job contains several processes.

[0058] ② Each job j contains a set of processes O that are executed in sequence j .

[0059] ③ There are M devices in the system, each of which can process a specific set of processes, defined as O m .

[0060] ④ The set of equipment available for process (j,o) is represented by M j,o , that is, some processes can only be processed by specific equipment.

[0061] ⑤ The arrival time of the job is dynamic. Job j arrives at time α j Arrival system.

[0062] ⑥ The device is reconfigurable, that is, the device m can replace the module when necessary, and each replacement takes time δ m .

[0063] ⑦The device may fail, that is, the device m fails at time f m A failure occurs and the repair time is r m .

[0064] ⑧Once the process starts, it cannot be interrupted and must be completed on the same equipment.

[0065] ⑨Each process can only be processed by one device at any time.

[0066] ⑩ The process can be assigned to different equipment before starting processing. If it needs to be transferred from equipment m to equipment m', the transfer time t needs to be considered. j,o,m,m' .

[0067] The processes have priority constraints, and the previous process must be completed before the next process can be completed.

[0068] Symbol definition and explanation: Table 4-1 gives the parameter table of the flexible job shop scheduling problem modeled by this patent.

[0069] Table 4-1 Parameters of flexible job shop scheduling problem

[0070] symbol meaning J Workpiece set, j∈J M Device set, m∈M <![CDATA[O j ]]> <![CDATA[Process set of workpiece j, o j ∈O j > <![CDATA[o m ]]> Each device can process a specific set of processes <![CDATA[p j,o ]]> The processing time of the oth step of job j <![CDATA[M j,o ]]> The set of optional equipment for the oth process of job j <![CDATA[α j ]]> Arrival time of job j <![CDATA[δ m ]]> Equipment m module replacement time <![CDATA[t fm ]]> Time when the equipment m failure occurred <![CDATA[t sm ]]> Device m module replacement start time <![CDATA[r m ]]> Maintenance time of equipment m <![CDATA[t j,o,m,m' ]]> Process transfer time <![CDATA[s j,o ]]> The start time of the oth process of job j <![CDATA[c j,o ]]> The end time of the oth process of job j

[0071] Optimization objectives and constraints: With the goal of minimizing the maximum completion time, multiple constraints are set, including process sequence, equipment conflict, equipment failure, and module replacement, to ensure the feasibility of the scheduling plan and equipment utilization efficiency, as well as process continuity and reasonable equipment allocation.

[0072] Based on the above basic assumptions about the scheduling problem, the optimization objective and constraint equations of the scheduling problem are as follows:

[0073]

[0074] S j,o ≥C j',o' -ρ(1-y j,o,j',o',m )+ρ(2-x j,o,m -x j',o',m ) (5)

[0075] S j',o' ≥C j,o -ρyj,o,j',o',m +ρ(2-x j,o,m -x j',o',m ) (6)

[0076]

[0077] S j,o ≥t fm +r m -ρ(1-x j,o,m ) (8)

[0078] C j,o ≤t fm -∈+ρ(1-x j,o,m ) (9)

[0079] S j,o ≥t sm +δ m -ρ(1-x j,o,m ) (10)

[0080]

[0081] Where ρ represents a sufficiently large positive number, ∈ represents a very small positive number; Formula (1) represents the optimization objective; Formula (2) represents the process sequence constraint; Formulas (3) and (4) represent the process start time constraint; Formulas (5), (6) and (7) represent the equipment conflict constraint, y j,o,j',o',m ∈{0,1} means that any two processes processed on the same equipment cannot be carried out at the same time; Formulas (8) and (9) represent equipment failure constraints; Formula (10) represents equipment module replacement constraints; Formulas (11) and (12) represent non-negative constraints; x j,o,m ∈{0,1} means that if the o-th process of job j is processed by equipment m, then x j,o,m =1, otherwise 0.

[0082] As an optimization solution for the above embodiment, in step S30, the modeling of the dynamic disjunctive graph includes: mapping the scheduling problem into a dynamic disjunctive graph model, updating the process and equipment nodes and their relationships in real time, handling the dynamic changes of new job arrival, equipment failure and module replacement, optimizing equipment utilization through the graph model and dynamically adjusting the scheduling strategy.

[0083] Specifically:

[0084] The dynamic flexible job shop scheduling problem is mapped into a dynamic disjunctive graph model. Model the scheduling process of a multi-agent system, where t represents the time.

[0085] The node set V(t) contains all the process nodes (j,o) and equipment nodes m∈M that are currently scheduled, that is,

[0086] V(t)={(j,o) / j∈J(t),o j ∈O j}∪{m / m∈M},

[0087] Here, J(t) represents the set of jobs arriving at time t.

[0088] Deterministic edge set E c (t) represents the sequence relationship of the processes within the same operation, that is,

[0089] E c (t)={(j,o),(j,o+1) / j∈J(t),o j ∈O j}.

[0090] Disjunctive edge set E d (t) represents the mutually exclusive relationship between processes that need to be processed on the same equipment, that is,

[0091] E d (t)={(j,o),(j',o') / m∈M j',o' ∩M j,o'j ≠j'},

[0092] The order of the processes is represented by the bidirectional edges ((j,o)→(j',o')) and ((j',o')→(j,o)). The equipment utilization rate is indirectly represented by the weight of each disjunctive edge. The processing time of each process is used as the weight of the disjunctive edge to measure the time the process occupies the equipment, that is,

[0093] W (j,o),(j',o') =p j,o ,

[0094] Then the utilization rate of equipment m It is represented by the sum of the processing time of all processes assigned to equipment m, that is,

[0095]

[0096] When a new job arrives, the process node of the job is added to the node set V(t) and the corresponding deterministic edge E is added c (t) and the disjunctive edge E d (t) represents the relationship between processes. If a device fails or requires module replacement at a certain point, the disjunctive edges associated with the relevant processes can be temporarily removed to ensure that the device is not assigned new tasks during the failure or replacement period. Once the failure is resolved or the replacement is complete, the relevant disjunctive edges are restored.

[0097] In the dynamic flexible job shop scheduling problem, the complexity of the scheduling graph increases rapidly with the random arrival of orders and changes in equipment status. To improve scheduling efficiency and simplify the computational process, clustering nodes in the disjunctive graph using structural entropy clustering is an effective strategy. The main goal of implementing structural entropy clustering in a dynamic disjunctive graph model is to discover the correlations between processes by minimizing structural entropy, thereby grouping closely related processes together.

[0098] In step S30, dynamic disjunctive graph modeling includes:

[0099] Initialize clusters: Treat each node in the dynamic disjunctive graph as an independent cluster and quantify the distribution of each node by calculating the initial structural entropy;

[0100] Structural entropy optimization: With the goal of minimizing structural entropy, clusters with strong correlations are gradually merged. In each merging operation, the cluster pair that minimizes the structural entropy increment is selected, the structural entropy change ΔH after the merger is calculated, and the overall structural entropy is updated.

[0101] Cluster stabilization: Continuously iterate the cluster merging operation until the structural entropy reaches a stable state, forming a set of highly correlated subgraphs.

[0102] Specifically, the clustering objective can be expressed as:

[0103]

[0104] Where |C| is the total number of clusters, |C k | is a class cluster C k In the initial stage of clustering, all nodes are considered as independent clusters, that is:

[0105]

[0106] At this point, the number of clusters is equal to the total number of nodes |V(t)|. The structural entropy H(G(t)) of the initial cluster can be calculated by the following formula:

[0107]

[0108] Among them, |E C (C k )| represents the cluster C k The number of deterministic edges in |E d (C k )| represents the cluster C k The number of disjunctive edges in |E c (t)| and |E d (t)| The total number of all deterministic edges and disjunctive edges in . Then, by selecting the merge operation, the structural entropy is reduced. Select two clusters C i and C j Merge to minimize the increase in structural entropy. The structural entropy after merging can be expressed as:

[0109] H ij =H(G(t))+ΔH

[0110] Where ΔH represents the combined C i and C j The change in the post-structural entropy can be calculated using the following formula:

[0111] ΔH=(H(C i ∪C j )-(H(C i )+(H(C i )).

[0112] The merging operation is performed iteratively until the structural entropy becomes stable. After the structural entropy clustering, the dynamic disjunction graph Decomposed into multiple subgraphs Each of the C k Represents a group of highly related process nodes that share the same equipment during the scheduling process; E c (t),E d (t) represent the deterministic edges and mutually exclusive edges between processes in the cluster respectively. This structural feature enables clustering to more clearly reflect the relationship between processes, thereby improving the utilization of equipment resources by the scheduling algorithm. The structural entropy clustering process effectively reduces the search space of the scheduling problem by dividing the node set V(t) into multiple groups. In traditional scheduling methods, scheduling decisions are usually based on the combination of all processes, resulting in an exponential increase in computational complexity. Through structural entropy, the present invention simplifies the scheduling process to a cluster of each cluster C k The processing reduces the complexity of the scheduling algorithm from O(n!) to O(k!).

[0113] As an optimization solution to the aforementioned embodiment, this invention proposes employing a multi-agent system to solve the flexible job shop dynamic scheduling problem described above. Each agent represents a scheduling unit, capable of autonomous perception, decision-making, and collaboration to optimize the scheduling solution. The following describes this multi-agent system from four perspectives: agent construction, state space, action space, and game process.

[0114] In step S40, the multi-agent system construction method includes:

[0115] Agent construction: Order agents and equipment agents are constructed based on scheduling requirements, responsible for order process allocation and equipment resource management, respectively. The order agent performs scheduling based on process clusters divided by structural entropy clustering, and the equipment agent accepts or rejects tasks with the goal of maximizing utilization.

[0116] Specifically, in this scheduling problem, agents are divided into two types: order agents and equipment agents. Each order corresponds to an order agent. j , whose goal is to complete all processes and minimize the total completion time. The decision-making process of the order agent is based on the results of the structural entropy clustering module division. The basic unit of each order agent operation is the process cluster C k , rather than a single process node, improves the efficiency of scheduling decisions. Each device corresponds to an agent m , whose goal is to maximize its own utilization U m , to avoid idle resources. The main function of the equipment agent is to manage the accepted process tasks and to allocate tasks based on clustering. j ,Agent m / j∈J,m∈M}. Order agent and equipment agent are connected through the nodes V(t) and edges E of the disjunction graph. c (t),E d (t) Interaction, achieving coordination among multiple agents.

[0117] State space construction: Define the state space for the agent; the state of the order agent includes the completion status of the process and the status of optional equipment; the state of the equipment agent includes utilization, task queue and fault status, ensuring that the agent can obtain the environmental information required for scheduling decisions.

[0118] Specifically, in a multi-agent system, the state space st describes the environmental information required by each agent in the scheduling process, including all relevant state variables at the current time t. The state space can be divided into the state parts of the order agent and the equipment agent. j The status reflects the order's process completion status, optional equipment status, and other related information. It can be expressed as:

[0119]

[0120] in, Represents cluster C k When all processes in the cluster are completed, Otherwise, 0; Represents cluster C k The number of processes included in the task provides information on the scale of the task; Represents cluster C k the urgency of the Represents the cluster set at the current time t. m The status includes the utilization rate, task queue and fault status of the device. It can be expressed as:

[0121]

[0122] Among them, U m (t) represents the utilization rate of equipment m at time t; Indicates the cluster of devices m currently waiting to be processed; F m (t) indicates the equipment failure state. When the equipment fails, F m (t) = 1, otherwise 0.

[0123] Action space construction: Define the action selection space of the agent; the action of the order agent is to assign the process cluster to the appropriate equipment; the action of the equipment agent is whether to accept the cluster task, thereby realizing dynamic resource allocation.

[0124] Specifically, in the dynamic flexible job shop scheduling problem, the action space at contains the set of actions that each agent can choose at the scheduling time t, which is specifically divided into the actions of the order agent and the equipment agent. j Responsible for managing the process scheduling of job j, its main action is to select and assign the processes in the cluster to the appropriate equipment for processing. Specifically, the action space of the order agent can be expressed as:

[0125]

[0126] in, Indicates that device m is selected to complete the corresponding cluster. When the cluster C k Assigned to equipment m for processing; Indicates that it can process cluster C k Each device agent m The main goal of is to decide whether to accept a new clustering task. The action space of the device agent is cluster level, specifically expressed as:

[0127]

[0128] Among them, ap(C k ) indicates that when the device accepts cluster C k All the tasks in the process are added to the task queue and U is updated. m (t), rj(C k) indicates that when the device rejects new tasks, it maintains the current state.

[0129] In step S50, each agent selects its own action according to the output of the deep Q network based on the current state and selects the current optimal action in combination with the game cooperation strategy. Game and cooperation mechanism: game and cooperation between agents, using deep Q network and greedy strategy (ε-greedy strategy can be used) to optimize the scheduling strategy, and using a customized reward mechanism to promote the cooperation of similar agents and regulate the competition between different types of agents.

[0130] In this dynamic, flexible job shop scheduling system, the game process not only optimizes the action selection of independent agents but also regulates the collaboration and competition among multiple agents based on clustering. Structural entropy clustering provides clusters of associated tasks and equipment, allowing agents to compete within a smaller decision space. Using a deep Q-network and a greedy strategy, each agent can dynamically select actions to optimize the scheduling solution.

[0131] In a deep Q-network, each agent uses a Q-network based on a deep learning model to fit its Q-value function Where θ is the parameter of the deep learning model. For each agent i, the Q network receives the state st i As input, and output each action at i The Q network is trained with the following loss function:

[0132]

[0133] Among them, r represents the immediate reward; st' and at' represent the next state and action respectively; θ' represents the parameters of the target network; γ represents the discount factor.

[0134] In order to achieve collaboration and competition, the present invention designs different reward mechanisms, which enables agents of the same type to work together to achieve common optimization goals, while agents of different types may exhibit non-cooperative behavior due to differences in goals. In this game model, agents of the same type cooperate to optimize the overall scheduling, while agents of different types compete due to differences in goals. Specifically, order agents of the same type can share information and resources to jointly optimize the scheduling effect. For example, multiple order agents can negotiate to select the best equipment to reduce the total completion time. Different types of agents have different optimization goals (such as order agents hope to complete all processes quickly, while equipment agents hope to maximize the utilization rate of equipment), which leads to competition between each other. Reward functions for the two types of agents are designed separately, reflecting the rewards for performing actions in specific states. Specifically, the reward function of the order agent is:

[0135]

[0136] Among them, η1 and η2 represent weight coefficients, which are used to balance the impact of completion time and urgency. The reward function of the equipment agent is:

[0137]

[0138] Among them, μ1 and μ2 represent weight coefficients, which are used to balance the influence of utilization and number of tasks. In each step, the agent j Agent m Select actions based on the output value of the Q network and use the ε-greedy strategy for exploration:

[0139]

[0140] Through the design of a deep Q-network and reward mechanism, agents of the same type can collaborate within their clusters, while agents of different types compete based on their optimization objectives. Based on structural entropy clustering and grouping, different types of agents compete with each other to ultimately reach the optimal scheduling solution within a local subgraph.

[0141] Collaboration Mechanism:

[0142] Order Agent j and Agent j' In the same cluster C k The collaboration is to optimize the overall completion time. The collaboration process is achieved by sharing cluster status. Implementation, the specific collaboration mechanism can be expressed as:

[0143]

[0144] in, Represents the collaboration weight, reflecting the relationship between agents j and j' on cluster C k The degree of dependence. Device Agent m and Agent m' In the same cluster C k The collaboration is designed to maximize overall equipment utilization. The collaboration process is achieved through a shared task queue. Implementation, the specific collaboration mechanism can be expressed as:

[0145]

[0146] in, Represents the collaboration weight, reflecting the relationship between agents m and m' on cluster C k degree of dependence.

[0147] Competition Mechanism:

[0148] Order Agent j and device agent m Competition between agents is reflected in resource allocation, especially when multiple order agents compete for the same device resources. The competition mechanism is achieved through the mutual influence of utility functions:

[0149]

[0150] Among them, λ1 and λ2 represent the competition weight coefficients; represents the set of order agents that apply to use equipment m; In the game process, the multi-agent system seeks to reach the Nash equilibrium, that is, in the equilibrium state, no agent can improve its utility by unilaterally changing its strategy. Mathematically, for all agents i,

[0151]

[0152] in, represents the optimal action of agent i; Represents the optimal action of all agents except agent i. Through deep Q network training, each agent i learns its optimal strategy To maximize its expected utility Gradually approaching Nash equilibrium.

[0153] The overall algorithm flow is as follows:

[0154] Step 1: According to the structural entropy clustering results, map the agent to the corresponding cluster subgraph.

[0155] Step 2: Each agent perceives the current state of its cluster

[0156] Step 3: The agent is based on the current state Action selection via a deep Q-network.

[0157] Step 4: The order agent assigns cluster tasks to the equipment agent, and the equipment agent decides to accept or reject the tasks.

[0158] Step 5: Calculate the cooperative and competitive utility functions of each agent based on the action results and

[0159] Step 6: Pass the loss function Update the deep Q-network parameters to minimize the prediction error.

[0160] Step 7: Repeat steps 2-6 until the termination condition is met.

[0161] As an optimization solution for the aforementioned embodiment, the parameter and structure configurator is used for automatic structure configuration and hyperparameter optimization of the Deep Q Network of agents in a multi-agent system. This tool aims to automatically optimize the hyperparameters and structure of the Deep Q Network model based on dynamic changes in the production environment. Its core modules include a hyperparameter optimization module based on heuristic algorithms and an automatic Q Network structure optimization module based on neural architecture search.

[0162] In step S60, the parameters and structure configurator of the multiple optimization operators include:

[0163] The hyperparameter optimization module based on heuristic algorithms is used to dynamically optimize the hyperparameters of the deep Q network in a multi-agent system through genetic algorithms, particle swarm algorithms, and ant colony algorithms to improve the performance and adaptability of the scheduling strategy in dynamic and flexible job shops.

[0164] This module aims to optimize the hyperparameters of the deep Q-network of a multi-agent system through a variety of heuristic algorithms to improve the performance and adaptability of the scheduling strategy in a dynamic and flexible job shop. This module includes three classic heuristic operators: genetic algorithm, particle swarm algorithm, and ant colony algorithm.

[0165] Preferably, the hyperparameter optimization module based on the heuristic algorithm includes a genetic algorithm, a particle swarm algorithm and an ant colony algorithm;

[0166] The genetic algorithm iteratively optimizes the hyperparameter configuration of the individuals in the population through selection, crossover, and mutation steps;

[0167] The particle swarm optimization algorithm searches for the optimal configuration based on the global and individual historical optimal positions by updating the particle speed and position;

[0168] The ant colony algorithm selects the hyperparameter path through pheromone concentration and heuristic information, and continuously optimizes the hyperparameters using the pheromone update mechanism;

[0169] Each algorithm is based on fitness evaluation and iteratively solves the optimal hyperparameter configuration suitable for dynamic and flexible job shops.

[0170] Hyperparameter optimization based on genetic algorithm:

[0171] Genetic algorithms search for optimal solutions through population evolution, which can effectively explore and find the best configuration in the hyperparameter space. In the hyperparameter optimization process, the algorithm first initializes a randomly generated population Each individual represents a hyperparameter configuration. Next, the algorithm evaluates the fitness of each individual and calculates the fitness value. This value is obtained by scheduling a task in a specific The Q network model of the training agent is obtained, that is:

[0172]

[0173] in, is the loss function of the model. A higher fitness value indicates a better hyperparameter configuration. To select parent individuals, the genetic algorithm uses a tournament selection method. K individuals are randomly selected from the population, their fitness values ​​are compared, and the individual with the highest fitness is selected as the parent:

[0174]

[0175] Where S is a set of randomly selected individuals. In the crossover process, two parent individuals are selected and By randomly generating the crossover factor α ga To generate offspring individuals Right now:

[0176]

[0177] At the same time, the mutation operation has a certain mutation probability Random changes to individuals can be expressed as:

[0178]

[0179] Among them, ∈ ga is a random variable sampled from a normal distribution; Represents the mutated individual. After fitness evaluation, selection, crossover, and mutation, the algorithm continues to iterate and evolve until the preset stopping condition is met.

[0180] Hyperparameter optimization based on particle swarm optimization:

[0181] The particle swarm optimization algorithm is an optimization algorithm based on swarm intelligence that simulates the foraging behavior of bird flocks. In hyperparameter optimization, the particle swarm optimization algorithm searches for the optimal hyperparameter configuration through the movement and update of particles. First, N particles are generated, each particle represents a hyperparameter configuration, the particle position represents the hyperparameter value, and the velocity represents the direction and magnitude of the parameter update. Then, the fitness value of each particle is calculated. Right now:

[0182]

[0183] According to the fitness evaluation results, the speed and position of the particle are updated, which can be expressed as:

[0184]

[0185] Among them, c1 represents the inertia weight, c2 and c3 represent the learning factors; r1 and r2 represent random numbers; pbest irepresents the historical optimal position of the particle; gbest represents the global optimal position of the swarm. The particle swarm algorithm continuously iterates and evolves until the preset stopping condition is met.

[0186] Hyperparameter optimization based on ant colony algorithm:

[0187] The ant colony algorithm is an optimization algorithm that simulates the foraging behavior of ants. It guides the search path through the deposition and volatilization of pheromones. In the hyperparameter optimization process, the ant colony algorithm first initializes the initial pheromone amount τ for each path. ij , which is used to guide the direction of hyperparameter selection. Each ant selects a hyperparameter configuration based on pheromone and heuristic information, and its selection probability is determined by the following formula:

[0188]

[0189] Among them, τ ij represents the pheromone concentration of path (i, j); η ij Represents the heuristic information of path (i, j); α1 represents the importance parameter of pheromone, which controls the influence of pheromone in the selection process. The larger its value, the greater the influence of pheromone on path selection; α2 represents the importance parameter of heuristic information, which controls the influence of pheromone in the selection process. The larger its value, the greater the influence of heuristic information on path selection. After each ant completes the search, it needs to evaluate the fitness of the path it has passed and calculate its fitness value. Then, the pheromone concentration of the high-quality path is enhanced by updating the pheromone amount:

[0190] τ' ij =(1-c4)τ ij +Δτ ij ,

[0191] Among them, c4 represents the pheromone volatility factor, Δτ ij The ant colony algorithm continuously optimizes the hyperparameter configuration by simulating the ant movement and pheromone update process until the preset stopping condition is met.

[0192] And a structural optimization module based on neural architecture search, which is used to automatically optimize the deep Q network structure through the fish swarm algorithm, bat algorithm, and firefly algorithm to adapt to different production environments and scheduling requirements, and achieve continuous improvement in scheduling strategy performance.

[0193] Preferably, the structure optimization module based on neural architecture search includes a fish swarm optimization operator, a bat optimization operator, and a firefly optimization operator;

[0194] The structural update mechanism based on the fish school optimization operator simulates the foraging behavior of fish schools and uses position, speed and linear attraction functions to adjust the configuration of each layer of the network structure to adapt to different scheduling tasks and environmental changes;

[0195] The structure search mechanism based on the bat optimization operator leverages bat echolocation behavior, combines adjustment factors with noise factors, and dynamically optimizes the network structure to achieve effective exploration to adapt to different scheduling tasks and environmental changes.

[0196] Based on the brightness attraction mechanism of the firefly optimization operator, by simulating the luminous attraction behavior of fireflies, the network hierarchy configuration is adjusted according to the architecture fitness and attraction parameters to adapt to different scheduling tasks and environmental changes.

[0197] Specifically, in dynamic flexible job shop scheduling, optimizing the Q network structure is a key step in improving the performance of scheduling strategies. This module, based on the neural architecture search framework, uses three different biomimetic optimization operators to automatically optimize the Q network structure to adapt to different production environments and scheduling requirements. This goal can be mathematically expressed as:

[0198]

[0199] in, represents the set of candidate network architectures, Indicates the reward of the selected architecture under a given scheduling task. In the initial stage of the search, the present invention uses a random generation method to construct the network architecture. Each network architecture can be represented by a parameter vector in, represents the number of neurons in the i-th layer, Represents the type of layer i. During the optimization process, this module combines three biomimetic optimization operators: fish school, bat, and firefly, aiming to efficiently search for and optimize the Q network architecture. The fish school operator optimizes the network structure by mimicking the foraging behavior of fish. The key to this operator is to evaluate the performance of each architecture and update it accordingly. Specifically, its update rules are as follows:

[0200]

[0201] in, is the position of the i-th architecture; represents the speed at which the architecture moves in the search space; Φ(·) represents the linear attraction function; d1 represents the adjustment factor; Indicates the position of the current best architecture; The bat operator performs architecture search by simulating the echolocation behavior of bats. The algorithm updates the exploration operator based on the exploration and evaluation of the architecture, and then updates the network structure. The specific update process is as follows:

[0202]

[0203] Here, d2 represents the adjustment factor and d3 represents the noise factor, which are used to simulate the search behavior of bats. The firefly operator mimics the flashing behavior of fireflies and optimizes the architecture using the attraction factor. The attraction of the firefly operator is proportional to its brightness, which in turn is related to the fitness of the selected architecture. The optimization process is described by the following formula:

[0204]

[0205] in, Indicates the location of the luminous structure; ω ij Representation Architecture and The distance between them is calculated using the Euclidean distance; d4 and d5 represent two parameters that control attraction. By combining these three bionic optimization algorithms, this module can search and optimize the architecture of deep Q networks to adapt to different scheduling tasks and production environments.

[0206] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A configurable flexible job shop scheduling method based on a multi-agent game model, characterized in that: Including steps: S10: Establish a dynamic scheduling model for a flexible job shop; S20: Collects all kinds of production data in the workshop in real time and uses data preprocessing technology to clean and standardize the data; S30: Convert the flexible job shop scheduling model into a dynamic disjunctive graph and use the structural entropy clustering method to divide the dynamic disjunctive graph into several subgraphs. This process optimizes the collaborative work between intelligent agents by analyzing the correlation between processes and resources. S40: Based on the dynamic disjunctive graph, a multi-agent system is constructed, including order agents and equipment agents. The agents are mapped to corresponding cluster subgraphs. Each order corresponds to an order agent. The basic unit of operation for each order agent is a process cluster, not a single process node. Each device corresponds to an equipment agent. The main function of the equipment agent is to manage the accepted process tasks and assign tasks based on cluster division. Initialize hyperparameters and establish the multi-agent network structure. Each agent has a deep Q network, and the deep Q network is trained. Experience replay and target network mechanisms are used to improve learning effects. Based on the training results, the network structure and hyperparameters are optimized through the configurator; S50: Each agent perceives the current state of its cluster; based on the current state, the agent selects an action through the deep Q network; the order agent assigns cluster tasks to the device agent, and the device agent decides whether to accept or reject the task; based on the action results, the collaborative and competitive utility functions of each agent are calculated; the deep Q network parameters are updated through the loss function to minimize the prediction error; the cycle repeats until the termination condition is met; S60: Apply parameter and structure configurators for multiple optimization operators to adapt to different optimization and environmental requirements, and perform online tuning of the agent's hyperparameters and deep Q network structure to ensure that the agent maintains efficient decision-making and scheduling performance in a changing environment.

2. A configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S10, a flexible job shop dynamic scheduling model is established, including: Model Assumptions and Symbol Definitions: This defines the set and attributes of jobs, equipment, and their processes, including the dynamic arrival time of jobs, the processing capacity of equipment, the processing time of processes, and the time for equipment failure and module replacement. The symbolic system defines each parameter in detail, providing a foundation for the scheduling model. Optimization objectives and constraints: With the goal of minimizing the maximum completion time, multiple constraints are set, including process sequence, equipment conflict, equipment failure, and module replacement, to ensure the feasibility of the scheduling plan and equipment utilization efficiency, as well as process continuity and reasonable equipment allocation.

3. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S30, the modeling of the dynamic disjunctive graph includes: mapping the scheduling problem into a dynamic disjunctive graph model, updating the process and equipment nodes and their relationships in real time, handling the dynamic changes of new job arrival, equipment failure and module replacement, optimizing equipment utilization through the graph model and dynamically adjusting the scheduling strategy.

4. A configurable flexible job shop scheduling method based on a multi-agent game model according to claim 3, characterized in that: In step S30, dynamic disjunctive graph modeling includes: Initialize clusters: Treat each node in the dynamic disjunctive graph as an independent cluster and quantify the distribution of each node by calculating the initial structural entropy; Structural entropy optimization: With the goal of minimizing structural entropy, clusters with strong correlations are gradually merged. In each merging operation, the cluster pair that minimizes the structural entropy increment is selected, the structural entropy change ΔH after the merger is calculated, and the overall structural entropy is updated. Cluster stabilization: Continuously iterate the cluster merging operation until the structural entropy reaches a stable state, forming a set of highly correlated subgraphs.

5. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S40, the multi-agent system construction method includes: Agent construction: Order agents and equipment agents are constructed based on scheduling requirements, responsible for order process allocation and equipment resource management, respectively. The order agent performs scheduling based on process clusters divided by structural entropy clustering, while the equipment agent accepts or rejects tasks with the goal of maximizing utilization. State space construction: Define the state space for the agent. The state of the order agent includes the completion status of the process and the status of the optional equipment. The state of the equipment agent includes utilization, task queue and fault status, ensuring that the agent can obtain the environmental information required for scheduling decisions. Action space construction: Define the action selection space of the agent; the action of the order agent is to assign the process cluster to the appropriate equipment; the action of the equipment agent is whether to accept the cluster task, thereby realizing dynamic resource allocation.

6. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S50, each agent selects its own action according to the output of the deep Q network based on the current state and selects the current optimal action in combination with the game cooperation strategy. Game and cooperation mechanism: game and cooperation between agents, using deep Q network and greedy strategy to optimize scheduling strategy, using customized reward mechanism to promote the cooperation of similar agents and regulate the competition between different types of agents.

7. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 1, characterized in that: In step S60, the parameter and structure configurator of the multi-optimization operator includes: A heuristic-based hyperparameter optimization module for dynamically optimizing the hyperparameters of deep Q networks in multi-agent systems using genetic algorithms, particle swarm algorithms, and ant colony algorithms; And a structural optimization module based on neural architecture search, which is used to automatically optimize the deep Q network structure through fish swarm algorithm, bat algorithm, and firefly algorithm.

8. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 7, characterized in that: The hyperparameter optimization module based on heuristic algorithms includes genetic algorithm, particle swarm algorithm and ant colony algorithm; The genetic algorithm iteratively optimizes the hyperparameter configuration of the individuals in the population through selection, crossover, and mutation steps; The particle swarm optimization algorithm searches for the optimal configuration based on the global and individual historical optimal positions by updating the particle speed and position; The ant colony algorithm selects the hyperparameter path through pheromone concentration and heuristic information, and continuously optimizes the hyperparameters using the pheromone update mechanism; Each algorithm is based on fitness evaluation and iteratively solves the optimal hyperparameter configuration suitable for dynamic and flexible job shops.

9. The configurable flexible job shop scheduling method based on a multi-agent game model according to claim 7, characterized in that: A structural optimization module based on neural architecture search, including fish swarm optimization operators, bat optimization operators, and firefly optimization operators; The structural update mechanism based on the fish school optimization operator simulates the foraging behavior of fish schools and uses position, speed and linear attraction functions to adjust the configuration of each layer of the network structure to adapt to different scheduling tasks and environmental changes; The structure search mechanism based on the bat optimization operator leverages bat echolocation behavior, combines adjustment factors with noise factors, and dynamically optimizes the network structure to achieve effective exploration to adapt to different scheduling tasks and environmental changes. Based on the brightness attraction mechanism of the firefly optimization operator, by simulating the luminous attraction behavior of fireflies, the network hierarchy configuration is adjusted according to the architecture fitness and attraction parameters to adapt to different scheduling tasks and environmental changes.

Images