Multi-robot collaborative production scheduling method and system for flexible manufacturing scenarios

Abstract

The application discloses a multi-robot cooperative production scheduling method and system for a flexible manufacturing scene, and relates to the technical field of intelligent manufacturing, comprising: constructing a global state representation; constructing a multi-robot cooperative scheduling optimization model based on the global state representation, and abstracting a flexible manufacturing line into a heterogeneous cooperative scheduling graph, performing feature extraction on the heterogeneous cooperative scheduling graph based on a graph neural network to generate global state embedding; modeling the multi-robot cooperative scheduling process as a Markov decision process, defining a state space, an action space and a reward function; based on the Markov decision process, taking the global state embedding as input, and training a multi-robot scheduling strategy in a digital twin environment by using a double-layer reinforcement learning method; and deploying the trained multi-robot scheduling strategy to an actual flexible manufacturing line to realize real-time scheduling and disturbance response; the method realizes dynamic task allocation, path cooperative optimization and adaptive response to production disturbances.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing technology, and in particular to a multi-robot collaborative production scheduling method and system for flexible manufacturing scenarios. Background Technology

[0002] In the field of advanced manufacturing technology, flexible manufacturing systems have become an important production mode to adapt to the demands of multi-variety, small-batch, and dynamic orders. With the continuous improvement of automation and intelligence levels, various types of robots, such as automated guided vehicles (AGVs) and industrial robotic arms, are widely used in production lines to complete tasks such as material handling, processing, and assembly. How to achieve efficient collaborative scheduling among multiple robots in complex and ever-changing production environments has become one of the key technical issues in flexible manufacturing systems.

[0003] In actual production processes, flexible manufacturing production lines typically feature diverse task types, complex process dependencies, strong resource heterogeneity, and frequent production disturbances. For example, dynamic order insertion, equipment failure, and path congestion can all impact production rhythm. Against this backdrop, traditional scheduling methods based on static rules or centralized planning struggle to respond promptly to changes in production line status, easily leading to extended production completion times, reduced equipment utilization, and frequent robot path conflicts, failing to meet the real-time and robustness requirements of modern flexible manufacturing.

[0004] Existing research has explored methods to solve production scheduling problems using heuristic algorithms or mathematical programming models. However, these methods typically rely on precise modeling of the production environment, resulting in high computational complexity. Furthermore, they struggle to obtain feasible solutions within a limited timeframe when dealing with large-scale, multi-robot collaborative scenarios. On the other hand, some studies have introduced reinforcement learning methods to learn scheduling strategies. However, these methods are mostly focused on single-agent or homogeneous robot scenarios, lacking effective modeling of heterogeneous robot collaborative relationships and the global production state. Consequently, the learned strategies have limited generalization ability in complex, flexible manufacturing environments.

[0005] Furthermore, most existing scheduling systems lack adaptive mechanisms to handle production disturbances, often requiring manual intervention or replanning of the overall scheduling scheme. This not only increases system response latency but also reduces the continuity and stability of the production system. Therefore, how to achieve collaborative scheduling and rapid reconfiguration of multiple robots in dynamic and uncertain environments while ensuring real-time performance remains a pressing technical challenge in this field.

[0006] A dynamic flexible shop floor scheduling method based on multi-rule combination, published in CN118331185A, first obtains the state variables of each machine and workpiece in the shop floor, establishes a dynamic flexible shop floor scheduling model, and calculates the priority values ​​of machines and workpieces based on sequence rule functions and routing rule functions to complete the scheduling. Subsequently, multiple sets of initial scheduling rules are constructed through a genetic programming algorithm, and the weights of each rule in the multi-rule combination scheduling model are optimized by combining a hybrid differential evolution algorithm, thereby improving the shop floor scheduling process. This scheme mainly relies on manually designed rule functions and their weight combinations, and performs offline optimization of rules through evolutionary algorithms. Its scheduling decision-making process is centered on rule priority calculation, lacks explicit modeling capabilities for multi-robot cooperative behavior and path conflicts, and does not consider the problem of online learning and adaptive decision-making based on global state in complex dynamic environments. Summary of the Invention

[0007] Based on the technical problems existing in the background technology, this invention proposes a multi-robot collaborative production scheduling method and system for flexible manufacturing scenarios, which realizes dynamic task allocation, path collaborative optimization and adaptive response to production disturbances.

[0008] The multi-robot collaborative production scheduling method for flexible manufacturing scenarios proposed in this invention includes: Acquire real-time operating status and production task information of flexible manufacturing production lines from multiple sources, and construct a global state representation; Based on the global state representation, a multi-robot collaborative scheduling optimization model is constructed with the objectives of minimizing completion time, maximizing equipment utilization, minimizing path conflicts and energy consumption. The flexible manufacturing production line is abstracted into a heterogeneous collaborative scheduling graph containing task nodes, robot nodes and workstation nodes. Based on the graph neural network, feature extraction is performed on the heterogeneous collaborative scheduling graph to generate a global state embedding. The multi-robot collaborative scheduling process is modeled as a Markov decision process, defining the state space, action space, and reward function; Based on the Markov decision process, and with the global state embedding as input, a two-layer reinforcement learning method is used to train a multi-robot scheduling strategy in a digital twin environment. Specifically, the two-layer reinforcement learning method is as follows: the upper-layer policy network takes the global state embedding as input and outputs long-term task allocation and resource planning decisions; the lower-layer policy takes the local observation information of each robot, the global state embedding, and the upper-layer decisions as input and outputs the real-time scheduling actions of each robot. The trained multi-robot scheduling strategy is deployed to the actual flexible manufacturing production line to achieve real-time scheduling and disturbance response.

[0009] Furthermore, the process of constructing the global state representation is as follows: The collected multi-source heterogeneous data is processed for time synchronization, anomaly removal and normalization, and a unified global state vector is generated as the global state representation through a data fusion function.

[0010] Furthermore, the step of extracting features from the heterogeneous collaborative scheduling graph based on the graph neural network to generate a global state embedding specifically includes: inputting the initial features of task nodes, robot nodes, and workstation nodes into the graph neural network, updating the node features through multi-layer neighborhood aggregation, and aggregating all node features using a readout function to obtain the global state embedding.

[0011] Furthermore, the state space of the Markov decision process is composed of the global state embedding and the local observation information of each robot; wherein, the local observation information includes at least one of the robot's current position, motion state, load state, task execution state, and availability state; The actions of each robot in the action space include task selection, path adjustment, waiting or cooperation request; The reward function evaluates the task completion status, time consumption, path conflicts, and energy consumption.

[0012] Furthermore, the reward function is specifically as follows: ; in, For the first A robot at time step Instant rewards received For the first A robot at time step The increase in the number of tasks completed within the period For the first A robot at time step The time consumed by executing tasks within the timeframe. For the first A robot at time step The number of path conflicts that occurred with other robots within the area. For the first A robot at time step Energy consumed internally These are the weighting coefficients.

[0013] Furthermore, the lower-level strategy is trained using a near-end strategy optimization algorithm, with the optimization objective being to minimize the expected value of the product of the cutoff probability ratio and the advantage function.

[0014] Furthermore, the digital twin environment maintains consistency with the actual flexible manufacturing production line in terms of structural layout, equipment capabilities, and operating rules, and simulates multi-robot parallel operation, dynamic changes in orders, and equipment failure scenarios.

[0015] Furthermore, the disturbance response includes: when at least one production disturbance such as equipment failure, order insertion, or path blockage is detected, updating the production line status and re-outputting the scheduling action based on the deployed multi-robot scheduling strategy to achieve adaptive rescheduling.

[0016] A computer system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0017] The advantages of the multi-robot collaborative production scheduling method and system for flexible manufacturing scenarios provided by this invention are as follows: It acquires multi-source heterogeneous data from automated guided vehicles, robotic arms, workstations, and production tasks on the production line in real time, forming a unified global state representation. Then, based on this global state representation, a multi-robot collaborative scheduling optimization model is constructed, introducing a heterogeneous collaborative scheduling graph structure and graph neural networks to extract global features of the production line as global state embeddings. Next, the multi-robot collaborative scheduling problem is modeled as a Markov decision process, and a two-layer reinforcement learning framework is used to train the multi-robot scheduling strategy in a digital twin environment highly consistent with the real production line. Finally, the trained multi-robot scheduling strategy is deployed to the actual flexible manufacturing production line, realizing dynamic task allocation, path collaborative optimization, and adaptive response to production disturbances, thereby improving production line operating efficiency and system flexibility. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 Flowchart of a multi-agent reinforcement learning scheduling framework. Detailed Implementation

[0019] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0020] like Figure 1 and Figure 2 As shown, the multi-robot collaborative production scheduling method for flexible manufacturing scenarios proposed in this invention includes: S1. Obtain real-time operating status and production task information of the flexible manufacturing production line from multiple sources, and construct a global state representation; S2. Based on the global state representation, construct a multi-robot collaborative scheduling optimization model with the objectives of minimizing completion time, maximizing equipment utilization, minimizing path conflicts and energy consumption, and abstract the flexible manufacturing production line into a heterogeneous collaborative scheduling graph containing task nodes, robot nodes and workstation nodes. Based on the graph neural network, extract features from the heterogeneous collaborative scheduling graph to generate a global state embedding. S3. Model the multi-robot collaborative scheduling process as a Markov decision process, and define the state space, action space and reward function; S4. Based on the Markov decision process, and with the global state embedding as input, a two-layer reinforcement learning method is used to train a multi-robot scheduling strategy in a digital twin environment. Specifically, the two-layer reinforcement learning method is as follows: the upper-layer policy network takes the global state embedding as input and outputs long-term task allocation and resource planning decisions; the lower-layer policy takes the local observation information of each robot, the global state embedding, and the upper-layer decision as input and outputs the real-time scheduling actions of each robot. S5. Deploy the trained multi-robot scheduling strategy to the actual production line to achieve real-time scheduling and disturbance response.

[0021] The production scheduling method in this embodiment is applicable to intelligent manufacturing production lines that include automated guided vehicles (AGVs), robotic arms, and multiple workstations. First, a multi-source perception layer composed of sensors, RFID, industrial vision systems, and a production management information system is constructed to acquire real-time multi-source heterogeneous data from AGVs, robotic arms, workstations, and production tasks on the production line, forming a unified global state representation. Then, a multi-robot collaborative scheduling optimization model is built based on this global state representation, introducing a heterogeneous collaborative scheduling graph structure and graph neural networks to extract global features of the production line as global state embeddings. Next, the multi-robot collaborative scheduling problem is modeled as a Markov decision process. In a digital twin environment highly consistent with the real production line, a two-layer reinforcement learning framework is used to train the multi-robot scheduling strategy. Finally, the trained multi-robot scheduling strategy is deployed to the actual flexible manufacturing production line to achieve dynamic task allocation, path collaborative optimization, and adaptive response to production disturbances, thereby improving production line operating efficiency and system flexibility.

[0022] In one embodiment, step S1 involves acquiring multi-source real-time operating status and production task information of the flexible manufacturing production line and constructing a global state representation. Specifically, in step S1, the system first collects real-time operating status data of various resources in the flexible manufacturing production line through multiple sensing devices and information systems; then, it performs time synchronization, anomaly removal, and fusion processing on the collected multi-source heterogeneous data; finally, it integrates the scattered local state information into a global state representation that reflects the overall operating status of the production line, providing a state foundation for subsequent collaborative scheduling modeling, graph neural network feature extraction, and reinforcement learning strategy training. By constructing a multi-source sensing layer, real-time acquisition of the operating status and production task information of the flexible manufacturing production line is achieved, providing a unified and reliable state input vector for subsequent multi-robot collaborative scheduling decisions.

[0023] It should be noted that the robots involved in this embodiment specifically include AGVs and robotic arms.

[0024] Step S1 specifically includes steps S101 to S102: S101: Multi-source status information acquisition for the production line. The system acquires the status of various production resources in real time through a sensor network, RFID equipment, industrial vision system, and robot control interface deployed on the flexible manufacturing production line. Assume the system operates at discrete time steps... Global state set of flexible manufacturing production lines Represented as: ; in, For time steps The set of states of all automated guided vehicles (AGVs) For time steps The set of states of all robotic arms For time steps The status set of all workstations. For time steps The set of statuses for all order tasks (including processes).

[0025] It can be represented as: ; in, For the first Taiwan AGV in time step Two-dimensional coordinate position, This is the index number of the AGV. The total number of AGVs For the first Taiwan AGV in time step instantaneous speed, For the first Taiwan AGV in time step The load status.

[0026] It can be represented as: ; in, This represents the total number of robotic arms. For the index number of the robotic arm, For the first The robotic arm in time step The status of the work. For the first The robotic arm in time step Task progress, For the first The robotic arm in time step Availability (1 indicates available, 0 indicates faulty or under maintenance).

[0027] It can be represented as: ; in, This represents the total number of workstations. This is the index number for the workstation. For the first Each workstation in time step The length of the queue to be processed. For the first Each workstation in time step The remaining processing capacity For the first Each workstation in time step Availability (1 indicates available, 0 indicates occupied).

[0028] It can be represented as: ; in, This represents the total number of currently unfinished order tasks (including all processes). For the index number of the order task (or process), For the first A description vector for each order task (usually a specific process); For the first The order number to which each order task belongs is used to associate multiple processes of the same product; For the first The process dependencies of an order task, such as the preceding process, the following process, or the required resource types; For the first Estimated processing / handling time (or remaining processing time) for each order task. For the first The priority of each order task is indicated by a higher value, which is more urgent and is used to guide scheduling decisions.

[0029] The robot's operating status, equipment availability, and order process constraints are described respectively, providing basic information for subsequent collaborative scheduling.

[0030] S102: Multi-source data fusion and global state construction. The system performs time synchronization and fusion processing on the collected multi-source heterogeneous data. To eliminate the influence of different units, the state variables are normalized.

[0031] Subsequently, the system maps each normalized set of substates to a unified global state vector, which serves as the global state representation: ; in, For time steps The global state representation output after fusion is used to describe the overall operation status and resource distribution of the production line, and serves as the input for subsequent graph neural networks and reinforcement learning; This represents a data fusion function, such as concatenation, weighted summation, neural network mapping, or other fusion strategies, used to integrate multi-source heterogeneous data into a fixed-dimensional vector to generate a consistent representation of production line status. This is a normalized set of AGV states, including information such as position, speed, and load, with all units standardized to the same numerical range. This is a normalized set of robotic arm states, including information such as operational status, task progress, and availability. This is a normalized set of workstation states, including information such as queue length, remaining processing capacity, and availability. It is a normalized set of order task statuses, including information such as process dependencies, estimated time, and priority.

[0032] In one embodiment, in step S2, a multi-robot collaborative scheduling optimization model is constructed by uniformly modeling the collaborative relationships between order tasks, robots, and workstations in the production line, providing a structured description and optimization objectives for subsequent scheduling decisions. In this step S2, the system not only focuses on the execution efficiency of individual robots but also characterizes the collaborative relationships and resource competition constraints among multiple robots from a holistic perspective.

[0033] The specific steps for constructing a multi-robot cooperative scheduling optimization model include steps S201 to S203: S201: Collaborative scheduling optimization target modeling, with comprehensive consideration of completion time in the system. Equipment utilization rate Number of path conflicts and total system energy consumption Construct the scheduling optimization objective function: ; in, The overall scheduling cost (the smaller the better) is used to evaluate the overall performance of the scheduling scheme. These are weighting coefficients (all positive numbers) used to balance the relative importance of the four objectives: completion time, equipment utilization, conflict, and energy consumption. Their values ​​can be adjusted according to the actual production line requirements. They can be represented as: ; ; ; ; in, For the first The completion time of each order task, that is, the time from the start of scheduling to the end of the order task. To obtain the maximum value of the completion times of all tasks, i.e. the total completion time of the entire production batch, This is the index number of the equipment, which includes all schedulable resources such as AGVs, robotic arms, and workstations. This represents the total number of devices in the system. For equipment The actual working time within the scheduling cycle For equipment The total available time from the start to the end of the scheduling is typically equal to... , These are the indexes for two different robots. They represent robots and On the current travel path in the scheduling plan, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise. The total scheduling period is usually equal to... , For the first Taiwan AGV at all times Instantaneous power (related to factors such as speed, acceleration, and load).

[0034] The aforementioned scheduling optimization objective function is used to comprehensively evaluate the scheduling strategy and guide the system to achieve a balance among multiple objectives.

[0035] S202: Construction of Heterogeneous Collaborative Scheduling Graph: The system abstracts the flexible manufacturing production line into a heterogeneous collaborative scheduling graph. The node set is defined as: ,in, Heterogeneous Cooperative Scheduling Graph The set of nodes, Including task nodes Robot nodes and workstation nodes edge set This indicates the executable relationship and path constraint relationship between tasks and resources.

[0036] This heterogeneous collaborative scheduling graph can formally express the complex coupling relationships between multiple tasks and resources. By characterizing the matching relationships and path constraints between order tasks, robots, and workstations through nodes and edges, it enables complex scheduling problems to have a clear and structured expression.

[0037] S203: Global state embedding is generated based on graph neural networks. The system uses graph neural networks to extract features from heterogeneous collaborative scheduling graphs. Let the nodes... In the The characteristics of the layer are Then its update rule is: ; in, For nodes In the The characteristics of a layer correspond to nodes. After the first New features after layer update It is a non-linear activation function. For the first The layer's own weight matrix, For nodes The set of neighboring nodes, For nodes In the Features of the layer For the first The neighbor weight matrix of the layer, and It is a node index.

[0038] After multiple layers of propagation, the global state embedding is generated through the readout function: ,in, For time step global state embedding, Common implementations of the readout function include summing, averaging, or maximizing the features of all nodes, or using learnable mappings such as attention mechanisms or multilayer perceptrons. For the process After the layered graph neural network is updated, all nodes are at the 1st... The feature set of the layer For nodes In the Characteristics of the layer.

[0039] Global state embedding It reflects the overall collaborative state of the flexible manufacturing production line and can serve as a high-dimensional input for reinforcement learning algorithms to guide the decision-making of intelligent agents.

[0040] In one embodiment, step S3 specifically involves: The system models the multi-robot scheduling problem as a Markov decision process (MDP): ,in, For the tuple representation of a Markov decision process, For state space, For the action space, Let be the state transition probability. This is the reward function.

[0041] By formalizing the multi-robot collaborative production scheduling problem as a Markov decision process, the system can make continuous decisions and optimize strategies in a dynamic and uncertain production environment. The core of step S3 lies in abstracting the complex production scheduling behavior into a closed-loop process of state-action-feedback, thereby providing a standardized modeling foundation for reinforcement learning algorithms.

[0042] The specific steps involved in transforming the coordinated scheduling problem into a Markov decision process are S301 to S302: S301: State Space, Action Space, and State Transition Modeling: The system embeds the global state output by the graph neural network and the states of each robot to form the state of the Markov decision process. .

[0043] The system state is defined as follows: ; For the first A robot at time step The local state.

[0044] The motion space for each robot is: ; For the first A robot at any moment The chosen action Select a task, that is, choose a task (such as handling, processing, or assembly) from the queue of tasks to be processed. Route adjustment refers to modifying the current driving path or operation sequence to avoid conflicts or optimize the route. For the purpose of waiting (e.g., to avoid congestion, waiting for resources to be released). For collaboration requests, that is, initiating or responding to collaboration requests (such as dual-machine collaborative handling, handover of workpieces).

[0045] Within each decision cycle, each robot selects an action based on its current state, including task selection, path adjustment, waiting, or requesting cooperation. System state transitions are jointly determined by the robot's action execution results, changes in production resources, and random environmental disturbances, and satisfy the following conditions: ; For time steps state, The state transition probability is, specifically, given the current state. and joint actions The system was transferred to The probability of this transition is affected by factors such as the robot's motion model, task execution time, and random disturbances (such as equipment failure or order insertion).

[0046] S302: Reward Function Design: To guide multiple robots to form a cooperative scheduling strategy, the system defines a reward function for each robot: ; in, For the first A robot at time step The immediate reward received is categorized as either a positive reward (representing beneficial behavior) or a negative reward (representing punishment). For the first A robot at time step The increase in the number of tasks completed within the period For the first A robot at time step The time consumed by executing tasks within the timeframe. For the first A robot at time step The number of path conflicts that occurred with other robots within the area. For the first A robot at time step Energy consumed internally These are weighting coefficients, used to balance the relative importance of multiple objectives, and can be adjusted according to production line requirements.

[0047] This reward function guides the robot to achieve a balance between individual efficiency and system coordination. By comprehensively evaluating task completion, time delay, conflicts, and energy consumption, the reward function enables the robot to pursue local efficiency while also considering the overall system performance, thereby forming stable and efficient cooperative scheduling behavior.

[0048] In one embodiment, step S4 specifically involves: constructing a digital twin environment that is highly consistent with the real production line and introducing a two-layer reinforcement learning framework to systematically train the multi-robot scheduling strategy, so that the agent has good adaptability and generalization ability under complex production conditions.

[0049] Training an intelligent agent based on digital twins and a two-layer reinforcement learning algorithm specifically includes steps S401 to S402: Step S401: Digital Twin Simulation Environment Construction: Construct a digital twin simulation environment that is consistent with the actual production line in terms of structure, equipment capabilities, and operating rules. : , This digital twin simulation environment provides a real-world flexible manufacturing production line environment. It can simulate scenarios such as multiple robots operating in parallel, dynamic order changes, and equipment failures, offering a safe and controllable platform for reinforcement learning training.

[0050] Step S402: Two-layer reinforcement learning training process: During the training process, the system adopts a two-layer reinforcement learning structure: the upper-layer policy performs task allocation and resource planning on a longer time scale, and the lower-layer policy uses a proximal policy optimization algorithm to perform real-time scheduling training for each robot.

[0051] The input to the upper-level policy is the global state embedding. The output is Specifically: upper-level strategies Typically, this is a multilayer perceptron (MLP) or a neural network with a softmax output. This network uses... Given the input, output a probability distribution for long-term task allocation and resource planning decisions. It does not directly control the robot, but rather constrains the action space of the underlying strategy. For the upper-layer policy network (parameters to be learned), output the given... The probability of each candidate decision.

[0052] Targeting lower-level strategies; Each robot The input is: global state embedding ; probability distribution of the upper-level policy output Local observation information of the robot itself The robot's own status, such as position, speed, load, remaining battery power, and current task progress; Lower-level strategies Training is performed using Proximal Policy Optimization (PPO). Each robot (or shared policy network) concatenates the three inputs mentioned above and outputs the action through a policy network (such as an MLP). ,in For robots At time step The action.

[0053] The optimization objective of the lower-level strategy PPO is: ; in, The objective function for Proximal Policy Optimization (PPO) is to maximize this expected value during training in order to update the network parameters of the current policy. ; For mathematical expectation, For probability ratios, For the dominant function, To truncate the probability ratio to an interval To prevent excessively large policy updates, typically... Take 0.2.

[0054] and Specifically, it is expressed as follows: ; ; in, The current new strategy (currently being optimized) is in the state Take action below The probability, For the old strategy in state Take action below The probability of is fixed during sampling. As depreciation factor, The importance of balancing immediate rewards and future rewards. The closer the value is to 1, the more the robot focuses on long-term returns; For GAE (Generalized Advantage Estimation) parameters, control for the trade-off between bias and variance. ; For timing difference error, For time step indexing.

[0055] Two-layer reinforcement learning enables robots to learn global collaborative strategies while maintaining real-time execution through centralized training and distributed execution.

[0056] It is understood that the centralized training in this embodiment (in a digital twin environment) specifically refers to the following: During two-layer reinforcement learning training, the system can access the global information of all robots, including the state, actions, and global state embeddings of all AGVs and robotic arms. And the reward function. Algorithms such as Multi-Agent Proximal Policy Optimization (PPO) are employed, using a centralized Critic network (or advantage function) to evaluate the value of joint actions, thereby guiding the policy updates of each agent.

[0057] Distributed execution specifically means that during the two-layer reinforcement learning execution, each robot relies solely on its own local observation information. Global state embedding and the output of the upper-level strategy Calculate your own movements independently It does not require acquiring the real-time actions or private states of other robots. Without a centralized critic, each robot's policy network directly outputs actions, ensuring low latency and real-time performance. Therefore, the input during the execution phase does not contain internal information from other robots, and the decision-making process is completely distributed.

[0058] In one embodiment, step S5 specifically involves: deploying the trained multi-robot scheduling strategy to the actual flexible manufacturing production line to achieve real-time scheduling control of the flexible manufacturing system and adaptive response to production disturbances, thereby enabling the system to operate continuously and stably. Step S5 specifically includes S501 and S502.

[0059] S501: Strategy Deployment and Execution: The system will deploy and execute the trained multi-robot scheduling strategy. Deployed to the production line control system, it generates scheduling actions based on real-time status:

[0060] in, To achieve the optimal strategy Given a state Choose action The probability, For time steps Select the action to be performed (it can be an action of a single robot or a combined action). For time steps The current system state; To achieve the optimal strategy Below, the output is in the state. The following actions The probability distribution.

[0061] Actions The task is then distributed to AGVs and robotic arms for execution, enabling dynamic task allocation and path optimization.

[0062] S502: Disturbance Detection and Adaptive Rescheduling: When the system detects disturbance events such as equipment failure, order insertion, or path congestion... At that time, the scheduling system updates the state using the following function: This automatically triggers a rescheduling mechanism to adjust task allocation and path planning, thereby achieving rapid recovery without stopping the production line. This is the updated system state obtained after a disturbance. This corresponds to the current system state before the disturbance occurs, and is used for subsequent rescheduling. This is a state update function that modifies the corresponding fields in the state based on the type of disturbance.

[0063] This embodiment achieves efficient collaborative operation of multiple robots in a complex flexible manufacturing environment by integrating multi-source perception information, collaborative scheduling modeling, and a multi-agent reinforcement learning decision-making mechanism. Compared with traditional rule-based or static planning-based scheduling methods, this embodiment can quickly adjust the scheduling strategy under dynamic order changes and equipment disturbances, effectively shortening the overall production completion time, improving the utilization rate of AGVs and robotic arms, and significantly reducing path conflicts and unnecessary waiting during robot operation, thereby reducing system energy consumption and improving the overall operating efficiency, stability, and flexibility of the flexible manufacturing production line.

[0064] Example 1 This embodiment describes a multi-robot collaborative production scheduling method for flexible manufacturing scenarios, using a typical flexible manufacturing assembly line as an application scenario. The flexible manufacturing production line in this embodiment includes 3 automated guided vehicles (AGVs), 2 industrial robotic arms, and 4 workstations, used to complete the handling, processing, and assembly tasks of various product types.

[0065] In this embodiment, each AGV is responsible for transporting materials between workstations, and each robotic arm is responsible for gripping, assembling, and processing workpieces. Workstations are used to complete processing tasks at different stages. The system executes steps T1 to T5 sequentially, achieving collaborative scheduling of multiple robots in a dynamic task environment. Specific steps include: Step T1: Obtain multi-source real-time operating status and production task information of the flexible manufacturing production line and construct a global state representation.

[0066] Specifically, the system first collects real-time data on the operational status of various resources on the production line through multiple sensing devices and information systems; then, it performs time synchronization, anomaly removal, and fusion processing on the collected multi-source heterogeneous data; finally, it integrates the scattered local state information into a global state representation that reflects the overall operational status of the production line, providing a state foundation for the scheduling model and reinforcement learning algorithm. By constructing a multi-source sensing layer, the system achieves real-time acquisition of the operational status and production task information of the flexible manufacturing production line, providing a unified and reliable state input vector for subsequent multi-robot collaborative scheduling decisions.

[0067] T101: Multi-source status information acquisition for the production line. The system collects real-time status data on various production resources through a sensor network, RFID devices, industrial vision systems, and robot control interfaces deployed on the flexible manufacturing production line. The real-time collected information includes: the current position coordinates, operating speed, and current load status of 3 AGVs; the operating status, task execution progress, and availability of 2 robotic arms; the processing status, queue length, and remaining processing capacity of 4 workstations; and the current order task list, process sequence, and inter-process dependencies.

[0068] Suppose the system has discrete time steps The set of states is represented as: .

[0069] T102: Multi-source data fusion and global state construction. The system performs time synchronization and fusion processing on the collected multi-source heterogeneous data.

[0070] T2: Constructing a multi-agent scheduling optimization model for multi-robot collaborative operations based on global state information. This involves uniformly modeling the collaborative relationships between tasks, robots, and workstations on the production line to build a multi-robot collaborative scheduling optimization model.

[0071] The specific steps for constructing a multi-robot cooperative scheduling optimization model include: S201: Modeling the objective of collaborative scheduling optimization and constructing the objective function for scheduling optimization: ; S202: Construction of Heterogeneous Collaborative Scheduling Graph Model: The system abstracts the production line into a heterogeneous collaborative scheduling graph. .

[0072] Node set It includes task nodes, robot nodes, and workstation nodes. Task nodes represent the various processes within an order; robot nodes represent 3 AGVs and 2 robotic arms; and workstation nodes represent 4 processing or assembly workstations. (Edge set) This indicates the executable relationship and path constraint relationship between tasks and resources.

[0073] S203: Global state embedding based on graph neural network, which generates global state embedding through readout function after multi-layer propagation. .

[0074] T3: The multi-robot collaborative production scheduling problem is formally modeled as a Markov decision process. In step S3, the system models the multi-robot scheduling problem as a Markov decision process (MDP): .

[0075] T4: Training multi-robot scheduling strategies in a digital twin environment using a two-layer reinforcement learning approach. By constructing a digital twin environment highly consistent with the real production line and introducing a two-layer reinforcement learning framework, the multi-robot scheduling strategy is systematically trained, enabling the agent to possess good adaptability and generalization ability under complex production conditions.

[0076] T5: Deploy the trained agent strategy to the actual production line and achieve real-time scheduling and disturbance response.

[0077] This embodiment perceives and integrates the real-time status and production task information of the flexible manufacturing production line to construct a multi-robot collaborative scheduling optimization model. The multi-robot scheduling problem is transformed into a Markov decision process, and the scheduling strategy is trained and optimized by combining graph neural networks and multi-agent reinforcement learning algorithms. Then, the strategy is verified and iterated through a digital twin environment, and the trained scheduling strategy is deployed to the actual production control system to realize the dynamic allocation of multi-robot tasks and path collaborative optimization, thereby improving the production line operating efficiency, resource utilization, and overall system flexibility and stability under complex production disturbance conditions.

[0078] This embodiment also provides a multi-robot collaborative production scheduling system for flexible manufacturing scenarios. The multi-robot collaborative production scheduling system for flexible manufacturing scenarios can be implemented by executing the process steps of the production scheduling method. That is, those skilled in the art can understand the production scheduling method as a preferred implementation of the multi-robot collaborative production scheduling system for flexible manufacturing scenarios.

[0079] Specifically, a multi-robot collaborative production scheduling system for flexible manufacturing scenarios includes: Module M1: Production line status perception and task information acquisition module, used to acquire real-time operating status and production task information of flexible manufacturing production lines.

[0080] Module M2: Multi-agent collaborative scheduling modeling and state representation module, used to build a multi-robot collaborative scheduling optimization model and perform structured modeling of the global state of flexible manufacturing production lines.

[0081] Module M3: Cooperative Scheduling Markov Decision Modeling Module, used to transform the multi-robot cooperative scheduling problem into a Markov decision process.

[0082] Module M4: A two-layer reinforcement learning training module based on digital twins, used to train multi-robot scheduling strategies in a simulation environment.

[0083] Module M5: Scheduling strategy deployment and disturbance response module, used to deploy the trained multi-robot scheduling strategy to the actual flexible manufacturing production line and realize real-time scheduling and disturbance response.

[0084] Module M1 is used to implement step S1, which includes sensing and fusing the real-time status of various types of resources in the flexible manufacturing production line. Module M1 collects information in real time, such as the position, speed, and load status of the automated guided vehicle, the operating status and task progress of the robotic arm, the processing status and queuing status of the workstation, as well as the order task list and process dependencies, through sensor networks, RFID systems, industrial vision systems, and robot control interfaces. It also manages the collected multi-source heterogeneous data in a unified manner.

[0085] Module M1 includes the following sub-modules: Module M1.1: Multi-source status data acquisition submodule, used to collect the real-time operating status of AGV, robotic arm, workstation and task according to step S101; Module M1.2: State preprocessing and normalization submodule, used to perform time synchronization, anomaly removal and normalization on the collected data; Module M1.3: Global State Fusion Submodule, used to fuse various state information into a unified global state representation according to step S102, providing state input for subsequent scheduling decisions.

[0086] Module M2 is used to implement step S2. Based on the global state information output by module M1, it performs a unified modeling of the collaborative relationships between tasks, robots, and workstations in the production line. Module M2 constructs a collaborative scheduling optimization model with the objectives of minimizing completion time, maximizing equipment utilization, and minimizing path conflicts and energy consumption, enabling scheduling decisions to balance local efficiency with overall system performance.

[0087] Module M2 includes: Module M2.1: Scheduling optimization objective modeling submodule, used to construct a multi-objective cooperative scheduling optimization function according to step S201; Module M2.2: Heterogeneous collaborative scheduling graph construction submodule, used to abstract the production line into a heterogeneous graph structure containing task nodes, robot nodes and workstation nodes according to step S202; Module M2.3: Global State Embedding Generation Submodule, used to extract features from the collaborative scheduling graph based on graph neural network according to step S203, and generate an embedded representation reflecting the overall collaborative state of the production line.

[0088] Module M3 implements step S3, formally modeling the multi-robot collaborative production scheduling problem as a Markov decision process. Module M3 provides a unified definition of the state space, action space, state transition mechanism, and reward function in the scheduling process, enabling the multi-robot scheduling problem to be solved using reinforcement learning methods.

[0089] Module M3 includes: Module M3.1: State and Action Space Modeling Submodule, used to construct the state representation and action set of multiple agents according to step S301; Module M3.2: State transition modeling submodule, used to describe the evolution process of the production line state after the robot performs actions; Module M3.3: Reward function construction submodule, used to design a comprehensive reward signal according to step S302, to guide multiple agents to form a cooperative scheduling strategy.

[0090] Module M4 is used to implement step S4, training the multi-robot scheduling strategy by constructing a digital twin environment highly consistent with a real flexible manufacturing production line. Module M4 adopts a two-layer reinforcement learning framework, where the upper-layer strategy is used for long-term task allocation and resource planning, and the lower-layer strategy is used for real-time scheduling and path coordination of multiple robots.

[0091] The module M4 includes: Module M4.1: Digital Twin Simulation Environment Construction Submodule, used to construct a simulation environment consistent with the actual production line structure and operating rules according to step S401; Module M4.2: Two-layer reinforcement learning training submodule, used to train the scheduling strategy using the multi-agent proximal policy optimization algorithm according to step S402, and improve the stability and generalization ability of the strategy through centralized training and distributed execution.

[0092] Module M5 is used to implement step S5, deploying the trained multi-robot scheduling strategy to the actual flexible manufacturing production line, and realizing real-time scheduling and disturbance response of the production process. Module M5 can automatically trigger a rescheduling mechanism to adjust task allocation and path planning when disturbances such as equipment failure, order insertion, or path blockage occur, in order to ensure continuous and stable operation of the production line.

[0093] Module M5 includes: Module M5.1: Scheduling strategy deployment and execution submodule, used to embed the scheduling strategy into the production line control system and generate real-time scheduling instructions according to step S501; Module M5.2: Disturbance Detection and Rescheduling Submodule, used to detect production line disturbances and trigger adaptive rescheduling according to step S502.

[0094] Those skilled in the art will understand that the system can be implemented either through computer-readable program code or through hardware such as logic circuits, application-specific integrated circuits, programmable logic controllers, or embedded controllers. The modules described above can be implemented as independent software functional modules, or as hardware structures or a combination of software and hardware.

[0095] Based on the above description of the embodiments, those skilled in the art will understand that the multi-robot collaborative production scheduling method and system for flexible manufacturing scenarios described in this embodiment can be implemented in pure software or deployed and run on general-purpose or dedicated computing hardware platforms. Based on this essence, the technical solution of this embodiment can be specifically implemented in the form of a software product containing program instructions. This software product can be stored on various non-volatile storage media or directly deployed as a local or cloud service. The program instructions are used to cause computer devices with processing capabilities—including but not limited to personal computers, server clusters, mobile terminals, or other network devices—to execute the steps described in this embodiment.

[0096] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-robot collaborative production scheduling method for flexible manufacturing scenarios, including: Acquire real-time operating status and production task information of flexible manufacturing production lines from multiple sources, and construct a global state representation; Based on the global state representation, a multi-robot collaborative scheduling optimization model is constructed with the objectives of minimizing completion time, maximizing equipment utilization, minimizing path conflicts and energy consumption. The flexible manufacturing production line is abstracted into a heterogeneous collaborative scheduling graph containing task nodes, robot nodes and workstation nodes. Based on the graph neural network, feature extraction is performed on the heterogeneous collaborative scheduling graph to generate a global state embedding. The multi-robot collaborative scheduling process is modeled as a Markov decision process, defining the state space, action space, and reward function; Based on the Markov decision process, and with the global state embedding as input, a two-layer reinforcement learning method is used to train a multi-robot scheduling strategy in a digital twin environment. Specifically, the two-layer reinforcement learning method is as follows: the upper-layer policy network takes the global state embedding as input and outputs long-term task allocation and resource planning decisions; the lower-layer policy takes the local observation information of each robot, the global state embedding, and the upper-layer decisions as input and outputs the real-time scheduling actions of each robot. The trained multi-robot scheduling strategy is deployed to the actual flexible manufacturing production line to achieve real-time scheduling and disturbance response.

2. The method according to claim 1, characterized in that, The process of constructing the global state representation is as follows: The collected multi-source heterogeneous data is processed for time synchronization, anomaly removal and normalization, and a unified global state vector is generated as the global state representation through a data fusion function.

3. The method according to claim 1, characterized in that, The step of extracting features from the heterogeneous collaborative scheduling graph based on a graph neural network to generate a global state embedding specifically includes: inputting the initial features of task nodes, robot nodes, and workstation nodes into the graph neural network, updating the node features through multi-layer neighborhood aggregation, and aggregating all node features using a readout function to obtain the global state embedding.

4. The method according to claim 1, characterized in that, The state space of the Markov decision process is composed of the global state embedding and the local observation information of each robot. The local observation information includes at least one of the robot's current position, motion state, load state, task execution state, and availability state. The actions of each robot in the action space include task selection, path adjustment, waiting or cooperation request; The reward function evaluates the task completion status, time consumption, path conflicts, and energy consumption.

5. The method according to claim 1, characterized in that, The reward function is specifically as follows: in, For the first A robot at time step Instant rewards received For the first A robot at time step The increase in the number of tasks completed within the period For the first A robot at time step The time consumed by executing tasks within the timeframe. For the first A robot at time step The number of path conflicts that occurred with other robots within the area. For the first A robot at time step Energy consumed internally These are the weighting coefficients.

6. The method according to claim 1, characterized in that, The lower-level strategy is trained using a near-end strategy optimization algorithm, with the optimization objective being to minimize the expected value of the product of the cutoff probability ratio and the advantage function.

7. The method according to claim 1, characterized in that, The digital twin environment is consistent with the actual flexible manufacturing production line in terms of structural layout, equipment capabilities and operating rules, and simulates multi-robot parallel operation, dynamic changes in orders and equipment failure scenarios.

8. The method according to claim 1, characterized in that, The disturbance response includes: when at least one production disturbance such as equipment failure, order insertion, or path blockage is detected, updating the production line status and re-outputting the scheduling action based on the deployed multi-robot scheduling strategy to achieve adaptive rescheduling.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-8.

Images