A kind of isolation belt maintenance operation robot system based on multi-AGENT cooperation
By using improved QMIX and TimeSformer networks, the problems of task allocation and path coordination in the isolation zone maintenance robot system were solved, enabling efficient and low-energy multi-robot collaborative operation, thus improving work efficiency and the system's intelligence level.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAOYUN INTELLIGENT SYST (BEIJING) CO LTD
- Filing Date
- 2025-12-04
- Publication Date
- 2026-05-19
AI Technical Summary
Existing robot systems for maintaining isolation zones struggle to achieve reasonable task allocation and path coordination during multi-task parallel execution. Path planning and motion control neglect interaction, leading to path congestion, uneven energy consumption, and a lack of adaptive energy consumption optimization, making it difficult to meet the high-frequency, multi-cycle intelligent requirements.
By employing an improved QMIX network and an improved TimeSformer network, and through task graph convolutional fusion, state cooperation matrix, and dynamic task weight mechanism, it achieves task collaborative allocation, intelligent path planning, and task conflict avoidance among multiple robots, and has closed-loop control capabilities from environment mapping, state perception, path prediction to adaptive energy consumption adjustment.
It improves the intelligence level and work efficiency of multi-robot collaborative operations, enhances path prediction accuracy and obstacle avoidance robustness, realizes flexible adjustment of energy consumption and balanced allocation of resources, and improves the system's collaborative optimization capability in complex environments.
Smart Images
Figure CN121483067B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent transportation equipment technology, and in particular to a robot system for maintaining median strips based on multi-agent collaboration. Background Technology
[0002] With the continuous expansion of urban transportation infrastructure and the promotion of green and intelligent maintenance concepts, the demand for intelligent maintenance of linear green areas such as road medians is increasing. Existing median maintenance operations mostly rely on manual driving or individual robots to complete tasks such as spraying, pruning, and cleaning. This is inefficient, has limited coverage, and lacks optimization mechanisms for collaborative operations among multiple robots. In practical applications, the following problems are commonly encountered:
[0003] Maintenance robots often struggle to achieve reasonable task allocation and path coordination during multi-task parallel execution, frequently resulting in overlapping work areas, task conflicts, or blind spots. The path planning and motion control processes of each robot often neglect the interaction between them, leading to path congestion or uneven energy consumption in complex scenarios. Traditional path prediction and control methods are mostly based on a single agent decision model, lacking modeling and feedback mechanisms for the group state, resulting in a decline in overall work efficiency. Existing methods often lack an adaptive energy consumption optimization mechanism, failing to dynamically adjust task allocation weights based on historical energy consumption and performance, making it difficult to meet the needs of high-frequency, multi-cycle intelligent isolation zone maintenance operations.
[0004] Therefore, how to provide a robot system for maintaining isolation zones based on multi-Agent collaboration is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] One objective of this invention is to propose a multi-agent collaborative robot system for maintaining isolation zones. This system employs an improved QMIX network and an improved TimeSformer network to achieve collaborative task allocation, intelligent path planning, and conflict avoidance among multiple robots. It possesses closed-loop control capabilities from environmental mapping, state perception, path prediction to adaptive energy consumption adjustment. By introducing task graph convolutional fusion, state collaboration matrices, and dynamic task weight mechanisms, this invention improves the energy sensitivity and response flexibility of task allocation. By constructing a global interactive attention tensor and combining it with control command parameters, it significantly enhances the accuracy of path prediction and obstacle avoidance robustness among multiple agents. This system boasts advantages such as high operational efficiency, reasonable task division, low operating energy consumption, and strong collaborative capabilities.
[0006] A robot system for maintaining isolation zones based on multi-agent collaboration according to an embodiment of the present invention includes the following steps:
[0007] The environmental perception module is used to collect environmental data of the isolation zone operation scenario and generate environmental feature tensors and a global environmental map;
[0008] The state modeling module is used to collect the running state information of each agent and generate standardized state embedding vectors.
[0009] The task collaborative allocation module is used to input the global environment map and the standardized state embedding vectors of each agent into the improved QMIX network for task partitioning and collaborative planning, and generate the task allocation results for each agent.
[0010] The path planning module is used to input the task allocation results and environmental feature tensors into the improved TimeSformer network, perform spatiotemporal attention modeling and cross-AGENT interaction attention modeling, and generate the path target parameters and control command parameters for each AGENT at each time step.
[0011] The maintenance operation execution module is used to control the movement trajectory and operation range of the agent body according to the path target parameters, and drive the operation execution unit to complete the isolation zone maintenance operation task according to the control command parameters;
[0012] The communication and interaction module is used to exchange standardized status embedding vectors and job progress information of each agent through a wireless communication network during task execution, and to feed back the task allocation results to the improved QMIX network.
[0013] The feedback adjustment module is used to collect energy consumption assessment indicators of each agent after the task is completed, calculate the predicted value of energy consumption per unit path, and update the task weight of each agent in the next work cycle.
[0014] Optionally, modules can be integrated using the following methods:
[0015] Step 1: Each agent collects environmental data of the isolation zone operation scenario, performs preprocessing, and generates environmental feature tensors and a global environmental map;
[0016] Step 2: Collect the running status information of each agent, perform standardization processing, and generate standardized state embedding vectors;
[0017] Step 3: The global environment map and the standardized state embedding vectors of each agent are used to perform task partitioning and collaborative planning through an improved QMIX network to generate task allocation results for each agent;
[0018] Step 4: Input the task allocation results and environmental feature tensors into the improved TimeSformer network to perform spatiotemporal attention modeling and cross-AGENT interaction attention modeling, and generate path target parameters and control command parameters;
[0019] Step 5: Drive the maintenance operation execution module according to the control command parameters, and each agent executes the isolation strip maintenance operation task based on the path target parameters;
[0020] Step Six: During execution, each agent exchanges standardized state embedding vectors and job progress information through the wireless communication network and feeds back the task allocation results to the improved QMIX network.
[0021] Step 7: After the maintenance work on the isolation zone is completed, collect the energy consumption assessment indicators of each agent, calculate the predicted energy consumption per unit path, and update the task weights.
[0022] Optionally, step one specifically includes:
[0023] Environmental data is constructed by collecting terrain data and road boundary information through lidar sensors, vegetation image data through cameras, distance data of nearby obstacles through ultrasonic sensors, and attitude change data and acceleration data through inertial measurement units.
[0024] Synchronize environmental data according to a unified time base;
[0025] Based on the calibration parameters of the sensors in each agent, the vegetation image data, distance data, attitude change data and acceleration data are registered to the lidar coordinate system using the extrinsic parameter transformation matrix;
[0026] Voxel filtering and statistical outlier removal are performed on terrain data and road boundary information; median filtering and bilateral filtering are performed on vegetation image data; sliding window smoothing filtering is performed on distance data; and low-pass filtering is performed on attitude change data and acceleration data.
[0027] Data exceeding a set threshold range in the environmental data is removed, and differential detection and removal operations are performed on data whose variation exceeds a set threshold in consecutive sampling frames to generate standardized environmental data.
[0028] The standardized environmental data is mapped to the global environmental coordinate system constructed by the main agent through pose transformation, and the spatial range of the isolation zone operation area is discretized according to the set voxel resolution to obtain several spatial voxel units.
[0029] Calculate the channel features of each spatial voxel unit, and combine the channel features of all spatial voxel units in spatial order to generate the environment feature tensor.
[0030] Based on the environmental feature tensor, the main agent constructs a global environmental map using SLAM technology.
[0031] Optionally, step two specifically includes:
[0032] The operational status information includes three-dimensional position coordinate information, power information, load information, task progress information, and communication status information;
[0033] The running status information is concatenated into a running status vector. The minimum-maximum normalization method is used to map each dimension of the running status vector to the interval [0,1] to obtain a normalized running status vector.
[0034] The normalized running state vector is linearly mapped and its dimensions are compressed to generate a standardized state embedding vector.
[0035] Optionally, step three specifically includes:
[0036] The improved QMIX network includes several local policy networks, task graph convolutional fusion layers, and hierarchical hybrid networks. Each agent corresponds to a local policy network. The hierarchical hybrid network includes an inspection task hybrid network, a job task hybrid network, and an energy consumption scheduling hybrid network. The local policy network, inspection task hybrid network, job task hybrid network, and energy consumption scheduling hybrid network are all two-layer multilayer perceptron structures.
[0037] The main agent divides the operation area according to the global environment map and constructs a task candidate set. The task candidate set includes several task candidates. Each task candidate is generated into a candidate task action vector through feature concatenation and embedding mapping.
[0038] The normalized state embedding vector of each AGENT is concatenated with the action vector of each candidate task to form a joint state-action vector;
[0039] Introduce the task weight of each agent in the current job cycle, expand the dimension of the task weight and multiply it element-wise with the state-action joint vector to obtain the weighted state-action joint vector, and input it into the local policy network of each agent to calculate the task action Q value.
[0040] In the task graph convolutional fusion layer, the task graph is constructed through the main agent;
[0041] The task action Q values of all agents for the same task candidate are concatenated to obtain the node feature vector of the corresponding graph node, and all node feature vectors are arranged by node number to form a node feature matrix.
[0042] Construct a task graph adjacency matrix, wherein the corresponding elements of the task graph adjacency matrix are the weighted sum of the distance exponential decay term and the task type consistency term;
[0043] The adjacency matrix and the node feature matrix are propagated by a single-layer graph convolution to obtain the graph convolution node feature matrix;
[0044] Construct a state coordination matrix, the corresponding elements of which are: the weighted inner product between the normalized state embedding vectors of the AGENT and normalized by the Sigmoid function;
[0045] The state collaboration matrix and the graph convolution node feature matrix are jointly input into the hierarchical hybrid network to calculate the hybrid Q-values of inspection tasks, operation tasks, and energy consumption tasks, respectively, and then perform weighted combination to generate the global Q-value.
[0046] Under the premise that each task candidate can be assigned to at most one agent and each agent can receive at most one task candidate, the sum of all global Q values is maximized, and a greedy matching algorithm is used to generate the task assignment matrix.
[0047] Based on the task allocation matrix, the task allocation result corresponding to each AGENT is output. The task allocation result includes the task identifier, the spatial coordinates of the target task area, the task type code, and the task priority. The task priority is set according to the global Q value sorting result of each AGENT.
[0048] Optionally, step four specifically includes:
[0049] The main agent determines the target task region for each agent based on the task allocation results, and extracts a set of spatial voxel units that match the target task region from the environmental feature tensor. The task allocation results of each agent and the set of matching spatial voxel units are concatenated in the channel dimension and the path feature tensor is generated by organizing it according to the time step.
[0050] The path feature tensor of each agent is linearly mapped along the channel dimension to generate a path mapping tensor.
[0051] A temporal position coding tensor is introduced by sine and cosine coding, and a spatial position coding tensor is constructed by relative position offset.
[0052] The path mapping tensor is added element-wise to the temporal location encoding tensor and the spatial location encoding tensor to obtain the path embedding tensor for each agent.
[0053] In the time step dimension, attention operations are performed on the spatial location features of the path embedding tensor of each AGENT to generate a spatial attention tensor;
[0054] In the spatial location dimension, attention operations are performed on the temporal series features of the path embedding tensor for each AGENT to generate a temporal attention tensor;
[0055] Perform channel residual join and linear fusion with the path embedding tensor, spatial attention tensor, and temporal attention tensor to generate the path residual embedding tensor for each agent.
[0056] At each time step, the attention features between the path residual embedding tensors of any two AGENTs are calculated to generate a global interaction attention tensor;
[0057] Based on the global interaction attention tensor, extract the global interaction feature vector sequence for each agent;
[0058] The global interaction feature vector sequence of each agent is used to predict the path target parameters of each agent at each time step through a two-layer MLP structure. The path target parameters include the planar coordinates of the target path point, the velocity magnitude, and the orientation angle.
[0059] The global interaction feature vector sequence of each agent is used to predict the control command parameters of each agent at each time step through a one-dimensional convolutional structure. The control command parameters include execution status identifier, steering angle and energy consumption adjustment coefficient.
[0060] Optionally, step five specifically includes:
[0061] The maintenance operation execution module includes a spraying arm, a pruning arm, a cleaning arm, and an execution drive unit;
[0062] At each time step, each agent determines its current operating mode based on the execution status flag, adjusts its movement direction based on the steering angle, and sets its execution level based on the energy consumption adjustment coefficient.
[0063] Each agent controls the machine's movement trajectory and speed based on path target parameters, determines the current work space execution area, and sets the spraying angle, spraying speed and spraying flow rate of the spraying arm, the trimming angle and cutting speed of the trimming arm, and the sweeping direction and cleaning power of the cleaning arm in combination with control command parameters.
[0064] The maintenance operation module is driven by the drive unit to complete the maintenance operation task of the isolation zone.
[0065] Optionally, step six specifically includes:
[0066] At each time step, each agent exchanges standardized state embedding vectors and job progress information with each other via a wireless communication network;
[0067] The job progress information includes the current path execution status identifier, the current job task completion percentage, and the current path time step index;
[0068] The main agent receives standardized state embedding vectors and job progress information uploaded by all agents and fuses them to generate a global state matrix;
[0069] The main agent inputs the global state matrix into the improved QMIX network to perform network updates and generate updated task assignment results.
[0070] Optionally, step seven specifically includes:
[0071] Collect energy consumption assessment indicators for each agent in the current work cycle. The energy consumption assessment indicators include actual energy consumption value, remaining power value, path execution time and work path length.
[0072] Based on the ratio of actual energy consumption to work path length, calculate the unit path energy consumption prediction value for each agent, compare the unit path energy consumption prediction value with the set energy consumption threshold, and if the unit path energy consumption prediction value exceeds the set energy consumption threshold, adjust the task weight of the current agent in the next work cycle.
[0073] Based on the updated task weights, adjust the task allocation results for each agent in the next job cycle.
[0074] The beneficial effects of this invention are:
[0075] This invention significantly improves the intelligence level and operational efficiency of multi-robot collaborative operations in complex road environments by constructing a multi-agent collaborative isolation strip maintenance robot system.
[0076] First, state collaboration modeling among various agents is achieved through an improved QMIX network, enabling each agent to perceive the execution status and job progress of other agents, thereby dynamically adjusting the task allocation strategy. The improved QMIX network introduces a local policy network, a hierarchical hybrid network, and a task graph convolutional fusion layer, which effectively enhances the collaborative capabilities among different agents in the path prediction and job execution stages.
[0077] Secondly, by introducing an improved TimeSformer network to uniformly model the standardized state embedding vector sequence and the path residual embedding tensor, the modeling ability of multiple agents to depict changes in work state and path deviation trends within continuous time steps is significantly enhanced. The improved TimeSformer network integrates a temporal modeling module and a spatial attention adjustment mechanism, which can accurately identify small offsets and local conflict areas in the path residual while maintaining the global modeling capability for long sequences. Furthermore, through joint decoding, the path target parameters and control command parameters of each agent are generated, improving the spatiotemporal modeling accuracy of path prediction and effectively avoiding the collision risk caused by spatial overlap or path intersection between agents. This ensures safe collaboration and path non-overlapping among multiple agents in high-density work areas.
[0078] In addition, after each work cycle, energy consumption assessment indicators such as actual energy consumption, remaining power, path execution time and work path length of each agent are collected. Based on the predicted energy consumption per unit path and the set energy consumption threshold, task weight adjustment is performed, realizing energy consumption-driven task reallocation optimization and path scheduling adjustment, which enhances the system's adaptability to resource changes and environmental disturbances.
[0079] In summary, this invention achieves structural innovation and synergistic performance improvement in key modules such as task allocation optimization, path prediction modeling, and operation energy consumption adjustment, and has good promotional value and engineering applicability. Attached Figure Description
[0080] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0081] Figure 1 This is a schematic diagram of a robot system for maintaining isolation zones based on multi-agent collaboration, as proposed in this invention.
[0082] Figure 2 This is a flowchart of an improved QMIX network task allocation system for a multi-Agent collaborative isolation zone maintenance robot system proposed in this invention.
[0083] Figure 3 This is a flowchart of an improved TimeSformer network path prediction system for a multi-Agent collaborative isolation zone maintenance robot system proposed in this invention. Detailed Implementation
[0084] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0085] refer to Figures 1-3 A robot system for maintaining firebreaks based on multi-agent collaboration includes the following steps:
[0086] The environmental perception module is used to collect environmental data of the isolation zone operation scenario and generate environmental feature tensors and a global environmental map;
[0087] The state modeling module is used to collect the running state information of each agent and generate standardized state embedding vectors.
[0088] The task collaborative allocation module is used to input the global environment map and the standardized state embedding vectors of each agent into the improved QMIX network for task partitioning and collaborative planning, and generate the task allocation results for each agent.
[0089] The path planning module is used to input the task allocation results and environmental feature tensors into the improved TimeSformer network, perform spatiotemporal attention modeling and cross-AGENT interaction attention modeling, and generate the path target parameters and control command parameters for each AGENT at each time step.
[0090] The maintenance operation execution module is used to control the movement trajectory and operation range of the agent body according to the path target parameters, and drive the operation execution unit to complete the isolation zone maintenance operation task according to the control command parameters;
[0091] The communication and interaction module is used to exchange standardized status embedding vectors and job progress information of each agent through a wireless communication network during task execution, and to feed back the task allocation results to the improved QMIX network.
[0092] The feedback adjustment module is used to collect energy consumption assessment indicators of each agent after the task is completed, calculate the predicted value of energy consumption per unit path, and update the task weight of each agent in the next work cycle.
[0093] In this embodiment, the modules are interconnected using the following method:
[0094] Step 1: Each agent collects environmental data of the isolation zone operation scenario, performs preprocessing, and generates environmental feature tensors and a global environmental map;
[0095] Step 2: Collect the running status information of each agent, perform standardization processing, and generate standardized state embedding vectors;
[0096] Step 3: The global environment map and the standardized state embedding vectors of each agent are used to perform task partitioning and collaborative planning through an improved QMIX network to generate task allocation results for each agent;
[0097] Step 4: Input the task allocation results and environmental feature tensors into the improved TimeSformer network to perform spatiotemporal attention modeling and cross-AGENT interaction attention modeling, and generate path target parameters and control command parameters;
[0098] Step 5: Drive the maintenance operation execution module according to the control command parameters, and each agent executes the isolation strip maintenance operation task based on the path target parameters;
[0099] Step Six: During execution, each agent exchanges standardized state embedding vectors and job progress information through the wireless communication network and feeds back the task allocation results to the improved QMIX network.
[0100] Step 7: After the maintenance work on the isolation zone is completed, collect the energy consumption assessment indicators of each agent, calculate the predicted energy consumption per unit path, and update the task weights.
[0101] In this embodiment, step one specifically includes:
[0102] Environmental data is constructed by collecting terrain data and road boundary information through lidar sensors, vegetation image data through cameras, distance data of nearby obstacles through ultrasonic sensors, and attitude change data and acceleration data through inertial measurement units.
[0103] Synchronize environmental data according to a unified time base;
[0104] Based on the calibration parameters of the sensors in each agent, the vegetation image data, distance data, attitude change data and acceleration data are registered to the lidar coordinate system using the extrinsic parameter transformation matrix;
[0105] Voxel filtering and statistical outlier removal are performed on terrain data and road boundary information; median filtering and bilateral filtering are performed on vegetation image data; sliding window smoothing filtering is performed on distance data; and low-pass filtering is performed on attitude change data and acceleration data.
[0106] Data exceeding a set threshold range in the environmental data is removed, and differential detection and removal operations are performed on data whose variation exceeds a set threshold in consecutive sampling frames to generate standardized environmental data.
[0107] The standardized environmental data is mapped to the global environmental coordinate system constructed by the main agent through pose transformation, and the spatial range of the isolation zone operation area is discretized according to the set voxel resolution to obtain several spatial voxel units.
[0108] The channel features of each spatial voxel are calculated, including occupancy state, obstacle distance, color channel, image texture, normal vector and pose estimation information. The channel features of all spatial voxel units are combined in spatial order to generate an environment feature tensor.
[0109] Based on the environmental feature tensor, the main agent constructs a global environmental map using SLAM technology.
[0110] In this embodiment, step two specifically includes:
[0111] The operational status information includes three-dimensional position coordinate information, power information, load information, task progress information, and communication status information;
[0112] The running status information is concatenated into a running status vector. The minimum-maximum normalization method is used to map each dimension of the running status vector to the interval [0,1] to obtain a normalized running status vector.
[0113] The normalized running state vector is linearly mapped and its dimensions are compressed to generate a standardized state embedding vector.
[0114] In this embodiment, step three specifically includes:
[0115] The improved QMIX network includes several local policy networks, task graph convolutional fusion layers, and hierarchical hybrid networks. Each agent corresponds to a local policy network. The hierarchical hybrid network includes an inspection task hybrid network, a job task hybrid network, and an energy consumption scheduling hybrid network. The local policy network, inspection task hybrid network, job task hybrid network, and energy consumption scheduling hybrid network are all two-layer multilayer perceptron structures.
[0116] The main agent divides the operation area according to the global environment map and constructs a task candidate set. The task candidate set includes several task candidates. Each task candidate consists of a task identifier, task area spatial coordinates and task type code. Each task candidate is generated into a candidate task action vector through feature concatenation and embedding mapping.
[0117] The normalized state embedding vector of each AGENT is concatenated with the action vector of each candidate task to form a joint state-action vector;
[0118] Introduce the task weight of each agent in the current job cycle, expand the dimension of the task weight and multiply it element-wise with the state-action joint vector to obtain the weighted state-action joint vector, and input it into the local policy network of each agent to calculate the task action Q value.
[0119] In the task graph convolutional fusion layer, the task graph is constructed through the main agent. Specifically, each task candidate in the task candidate set is used as a graph node. If the Euclidean distance between two task candidates in spatial location is less than a set threshold, the task types are the same, or there is an execution order relationship, then a connection edge is established between the graph nodes.
[0120] The task action Q values of all agents for the same task candidate are concatenated to obtain the node feature vector of the corresponding graph node, and all node feature vectors are arranged by node number to form a node feature matrix.
[0121] Construct a task graph adjacency matrix. The corresponding elements of the task graph adjacency matrix are the weighted sum of the distance exponential decay term and the task type consistency term. The distance exponential decay term is the Euclidean distance between task candidates calculated by the exponential decay function. The task type consistency term is: if two task candidates have the same task type code, the task type consistency term is equal to 1; otherwise, the task type consistency term is equal to 0.
[0122] The adjacency matrix and the node feature matrix are propagated by a single-layer graph convolution to obtain the graph convolution node feature matrix;
[0123] In this invention, the main agent constructs a task graph based on task candidates and, by combining distance exponential decay and task type consistency, constructs an adjacency matrix for the task graph, ensuring that the task graph structure takes into account both physical layout and semantic constraints. Through graph convolution propagation operations, the Q-value information of different agents for the same task candidate is effectively integrated, improving the modeling ability of global task collaboration among multiple agents and enhancing the system's allocation discrimination and robustness in task-dense regions.
[0124] Construct a state coordination matrix, the corresponding elements of which are: the weighted inner product of the normalized state embedding vectors of the agent, and normalized using the Sigmoid function:
[0125] ;
[0126] in, This represents the state synergy between the i-th agent and the j-th agent. This represents the normalized state embedding vector of the i-th agent. This represents the normalized state embedding vector of the j-th agent. Represents the state-coordinated weighted matrix. Represents the Sigmoid function;
[0127] The state collaboration matrix and the graph convolution node feature matrix are jointly input into the hierarchical hybrid network to calculate the hybrid Q-values of inspection tasks, operation tasks, and energy consumption tasks, respectively, and then perform weighted combination to generate the global Q-value.
[0128] In this invention, by constructing a state collaboration matrix and jointly inputting it with the feature matrix of graph convolutional nodes into a hierarchical hybrid network, explicit modeling of state relationships among multiple agents and collaborative evaluation of global task value are achieved. The state collaboration matrix characterizes the state correlation among agents through weighted inner product and sigmoid normalization operations, dynamically reflecting the degree of cooperation among different agents in terms of location, power, load, and communication status. Based on this, the hierarchical hybrid network generates hybrid Q values for three dimensions: inspection, operation, and energy consumption, and performs weighted fusion. This ensures the multi-objective balance of task execution and enhances the adaptability and decision accuracy of global task allocation, thereby improving the system's collaborative optimization capability in complex dynamic environments.
[0129] Under the premise that each task candidate can be assigned to at most one AGENT and each AGENT can receive at most one task candidate, the sum of all global Q values is maximized, and a greedy matching algorithm is used to generate a task assignment matrix. If task candidate j is assigned to the i-th AGENT, the corresponding element of the task assignment matrix is equal to 1; otherwise, the corresponding element of the task assignment matrix is equal to 0.
[0130] Based on the task allocation matrix, the task allocation result corresponding to each AGENT is output. The task allocation result includes task identifier, spatial coordinates of the target task area, task type code and task priority. The task priority is set according to the global Q value sorting result of each AGENT. The larger the value, the higher the task priority. The task priority is an integer value label.
[0131] In this embodiment, step four specifically includes:
[0132] The main agent determines the target task region for each agent based on the task allocation results, and extracts a set of spatial voxel units that match the target task region from the environmental feature tensor. The task allocation results of each agent and the set of matching spatial voxel units are concatenated in the channel dimension and the path feature tensor is generated by organizing it according to the time step.
[0133] The path feature tensor of each agent is linearly mapped along the channel dimension to generate a path mapping tensor.
[0134] A temporal position coding tensor is introduced by sine and cosine coding, and a spatial position coding tensor is constructed by relative position offset.
[0135] The path mapping tensor is added element-wise to the temporal location encoding tensor and the spatial location encoding tensor to obtain the path embedding tensor for each agent.
[0136] In the time step dimension, attention operations are performed on the spatial location features of the path embedding tensor of each AGENT to generate a spatial attention tensor;
[0137] In the spatial location dimension, attention operations are performed on the temporal series features of the path embedding tensor for each AGENT to generate a temporal attention tensor;
[0138] Perform channel residual join and linear fusion with the path embedding tensor, spatial attention tensor, and temporal attention tensor to generate the path residual embedding tensor for each agent.
[0139] At each time step, the attention features between the path residual embedding tensors of any two agents are calculated to generate a global interaction attention tensor. The global interaction attention tensor is used to characterize the path interaction relationship and conflict association between all agents at all time steps.
[0140] In this invention, to characterize the interactions and potential conflicts between paths of all agents during execution, attention feature calculation is performed on the path residual embedding tensors of any two agents at each time step. For any two agents, their path residual embedding tensors represent the task region spatial features and target path embedding features of that agent at each time step, respectively. First, linear mapping is performed on the two path residual embedding tensors to generate corresponding query tensors, key tensors, and value tensors. At the current time step, the query tensor of one agent is similarly matched with the key tensor of another agent, and attention scores are obtained through dot product operations. All attention scores are then normalized, and the normalized attention scores are multiplied by the value tensor of the agent being followed to aggregate spatial path features and complete the extraction of interaction information. This process is performed once for each pair of agents at each time step, thereby constructing a global interaction attention tensor covering all agents and all time steps.
[0141] Based on the global interaction attention tensor, extract the global interaction feature vector sequence for each agent;
[0142] The global interaction feature vector sequence of each agent is used to predict the path target parameters of each agent at each time step through a two-layer MLP structure. The path target parameters include the planar coordinates of the target path point, the velocity magnitude, and the orientation angle.
[0143] The global interaction feature vector sequence of each agent is used to predict the control command parameters of each agent at each time step through a one-dimensional convolutional structure. The control command parameters include execution status identifier, steering angle and energy consumption adjustment coefficient.
[0144] In this embodiment, step five specifically includes:
[0145] The maintenance operation execution module includes a spraying arm, a pruning arm, a cleaning arm, and an execution drive unit;
[0146] At each time step, each agent determines its current operating mode based on the execution status flag, adjusts its movement direction based on the steering angle, and sets its execution level based on the energy consumption adjustment coefficient.
[0147] Each agent controls the machine's movement trajectory and speed based on path target parameters, determines the current work space execution area, and sets the spraying angle, spraying speed and spraying flow rate of the spraying arm, the trimming angle and cutting speed of the trimming arm, and the sweeping direction and cleaning power of the cleaning arm in combination with control command parameters.
[0148] The maintenance operation module is driven by the drive unit to complete the maintenance operation task of the isolation zone.
[0149] In this embodiment, step six specifically includes:
[0150] At each time step, each agent exchanges standardized state embedding vectors and job progress information with each other via a wireless communication network;
[0151] The job progress information includes the current path execution status identifier, the current job task completion percentage, and the current path time step index;
[0152] The main agent receives standardized state embedding vectors and job progress information uploaded by all agents and fuses them to generate a global state matrix;
[0153] The main agent inputs the global state matrix into the improved QMIX network to perform network updates and generate updated task assignment results.
[0154] In this embodiment, step seven specifically includes:
[0155] Collect energy consumption assessment indicators for each agent in the current work cycle. The energy consumption assessment indicators include actual energy consumption value, remaining power value, path execution time and work path length.
[0156] Based on the ratio of actual energy consumption to job path length, the predicted energy consumption per unit path for each agent is calculated. This predicted energy consumption per unit path is then compared to a set energy consumption threshold. If the predicted energy consumption per unit path exceeds the set threshold, the task weight of the current agent in the next job cycle is adjusted.
[0157] ;
[0158] in, Index representing the job cycle, Indicates the first In the first work cycle The task weight of each agent Indicates the first In the first work cycle The task weight of each agent Represents an exponential function. This represents the energy consumption weighting adjustment coefficient. Indicates the first Predicted energy consumption per path for each agent This indicates the setting of an energy consumption threshold;
[0159] Based on the updated task weights, adjust the task allocation results for each agent in the next job cycle.
[0160] Example 1
[0161] To verify the feasibility of this invention in practice, it was applied to a scenario of refined maintenance of urban greenbelts by a municipal department in a city. On-site deployment and simulation evaluation of multi-Agent robot collaborative spraying and pruning tasks were conducted. This urban greenbelt is densely distributed between main roads and expressways, with narrow work areas, numerous obstacles, and complex tasks, encompassing pest and disease control, pruning, irrigation, and other operations. Traditional manual or single-robot inspections suffer from path planning conflicts, inefficient collaboration, and unreasonable energy allocation, especially in high-temperature or nighttime operation scenarios where efficiency is extremely low, and the mixing of humans and robots poses significant safety hazards.
[0162] During implementation, six robots were deployed to cover approximately 2400 meters of main road. Each robot was equipped with a visual perception system, odometer, and battery monitoring module, and was capable of spraying, pruning, and watering. A standardized state embedding vector was constructed using a state modeling module. A task allocation matrix was generated based on work density, path residuals, and current energy consumption weights, and the task content was distributed to each agent. Each agent completed spraying and pruning operations according to instructions, uploading the current work frame number, path time step index, remaining battery percentage, and current location coordinates to the master agent every 30 seconds. The master agent summarized the status and updated task weights to optimize task allocation for the next work cycle. Feedback evaluation was conducted using path execution time, battery decrease rate, and energy consumption per unit path to achieve adaptive energy scheduling, ensuring a more balanced energy distribution and preventing overload of high-frequency operation agents.
[0163] To verify the overall performance of the system, the system of this invention was compared with traditional static centralized scheduling schemes, QMIX scheduling schemes without energy feedback, and path scheduling methods based on genetic algorithms. Six robots were selected to perform maintenance tasks for 12 consecutive hours in three typical isolation zone scenarios. The tasks were identical. The evaluation indicators included: average path overlap rate, energy consumption per path, task completion time, overall system power fluctuation rate, number of path conflicts, and number of scheduling convergence rounds. The experimental results are shown in Table 1.
[0164] Table 1. Comparison of the overall performance of the present invention and the comparative scheme.
[0165]
[0166] As shown in Table 1, this invention outperforms the comparative methods in multiple performance metrics. Regarding path overlap rate, this invention, based on an improved TimeSformer for path prediction modeling and combined with inter-Agent interaction attention, effectively avoids path intersections and redundant task execution, reducing the average path overlap rate to 4.2%. In terms of energy consumption per unit path, this invention achieves a more balanced energy allocation and path selection by dynamically adjusting task weights and energy consumption prediction feedback, reducing energy consumption per unit path to 2.31 Wh / m, significantly better than the comparative scheme. Simultaneously, task completion time is shortened by approximately 20% compared to the comparative scheme, and the power fluctuation rate is reduced to 8.1%, fully demonstrating the practicality and robustness of the energy consumption scheduling strategy. This invention can achieve multi-Agent strategy scheduling convergence within 12 rounds, far less than the 30 rounds required by the genetic algorithm-based path scheduling scheme, significantly improving strategy stability and scheduling efficiency. The number of path conflicts is zero, verifying the improved TimeSformer network's ability to guarantee the security of multi-Agent collaboration.
[0167] The proposed invention provides a multi-agent collaborative isolation strip maintenance robot system that exhibits significant advantages in intelligent scheduling, energy consumption optimization, and path conflict suppression in real-world operation scenarios. This system can effectively improve the intelligence level, energy efficiency, and system stability of urban maintenance operations.
[0168] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A robot system for maintaining isolation zones based on multi-agent collaboration, characterized in that, include: The environmental perception module is used to collect environmental data of the isolation zone operation scenario and generate environmental feature tensors and a global environmental map; The state modeling module is used to collect the running state information of each agent and generate standardized state embedding vectors. The task collaborative allocation module is used to input the global environment map and the standardized state embedding vectors of each agent into the improved QMIX network for task partitioning and collaborative planning, and generate the task allocation result for each agent. The improved QMIX network includes several local policy networks, task graph convolutional fusion layers and hierarchical hybrid networks. Each agent corresponds to a local policy network. The hierarchical hybrid network includes an inspection task hybrid network, an operation task hybrid network and an energy consumption scheduling hybrid network. The local policy network, inspection task hybrid network, operation task hybrid network and energy consumption scheduling hybrid network are all two-layer multilayer perceptron structures. The path planning module is used to input the task allocation results and environmental feature tensors into the improved TimeSformer network, perform spatiotemporal attention modeling and cross-AGENT interaction attention modeling, and generate the path target parameters and control command parameters for each AGENT at each time step. The maintenance operation execution module is used to control the movement trajectory and operation range of the agent body according to the path target parameters, and drive the operation execution unit to complete the isolation zone maintenance operation task according to the control command parameters; The communication and interaction module is used to exchange standardized status embedding vectors and job progress information of each agent through a wireless communication network during task execution, and to feed back the task allocation results to the improved QMIX network. The feedback adjustment module is used to collect energy consumption assessment indicators for each agent after the task is completed, calculate the predicted energy consumption per unit path, and update the task weight of each agent in the next job cycle. Specifically, it includes: Collect energy consumption assessment indicators for each agent in the current work cycle. The energy consumption assessment indicators include actual energy consumption value, remaining power value, path execution time and work path length. Based on the ratio of actual energy consumption to job path length, the predicted energy consumption per unit path for each agent is calculated. This predicted energy consumption per unit path is then compared to a set energy consumption threshold. If the predicted energy consumption per unit path exceeds the set threshold, the task weight of the current agent in the next job cycle is adjusted. ； in, Index representing the job cycle, Indicates the first In the first work cycle The task weight of each agent Indicates the first In the first work cycle The task weight of each agent Represents an exponential function. This represents the energy consumption weighting adjustment coefficient. Indicates the first Predicted energy consumption per path for each agent This indicates the setting of an energy consumption threshold; Based on the updated task weights, adjust the task allocation results for each agent in the next job cycle.
2. The multi-agent collaborative isolation zone maintenance robot system according to claim 1, characterized in that, The modules are connected in the following way: Step 1: Each agent collects environmental data of the isolation zone operation scenario, performs preprocessing, and generates environmental feature tensors and a global environmental map; Step 2: Collect the running status information of each agent, perform standardization processing, and generate standardized state embedding vectors; Step 3: The global environment map and the standardized state embedding vectors of each agent are used to perform task partitioning and collaborative planning through an improved QMIX network to generate task allocation results for each agent; Step 4: Input the task allocation results and environmental feature tensors into the improved TimeSformer network to perform spatiotemporal attention modeling and cross-AGENT interaction attention modeling, and generate path target parameters and control command parameters; Step 5: Drive the maintenance operation execution module according to the control command parameters, and each agent executes the isolation strip maintenance operation task based on the path target parameters; Step Six: During execution, each agent exchanges standardized state embedding vectors and job progress information through the wireless communication network and feeds them back to the improved QMIX network to update the task allocation results; Step 7: After the maintenance work on the isolation zone is completed, collect the energy consumption assessment indicators of each agent, calculate the predicted energy consumption per unit path, and update the task weights.
3. The isolation zone maintenance robot system based on multi-agent collaboration according to claim 2, characterized in that, Step one specifically includes: Environmental data is constructed by collecting terrain data and road boundary information through lidar sensors, vegetation image data through cameras, distance data of nearby obstacles through ultrasonic sensors, and attitude change data and acceleration data through inertial measurement units. Synchronize environmental data according to a unified time base; Based on the calibration parameters of the sensors in each agent, the vegetation image data, distance data, attitude change data and acceleration data are registered to the lidar coordinate system using the extrinsic parameter transformation matrix; Voxel filtering and statistical outlier removal are performed on terrain data and road boundary information; median filtering and bilateral filtering are performed on vegetation image data; sliding window smoothing filtering is performed on distance data; and low-pass filtering is performed on attitude change data and acceleration data. Data exceeding a set threshold range in the environmental data is removed, and differential detection and removal operations are performed on data whose variation exceeds a set threshold in consecutive sampling frames to generate standardized environmental data. The standardized environmental data is mapped to the global environmental coordinate system constructed by the main agent through pose transformation, and the spatial range of the isolation zone operation area is discretized according to the set voxel resolution to obtain several spatial voxel units. Calculate the channel features of each spatial voxel unit, and combine the channel features of all spatial voxel units in spatial order to generate the environment feature tensor. Based on the environmental feature tensor, the main agent constructs a global environmental map using SLAM technology.
4. The multi-agent collaborative isolation zone maintenance robot system according to claim 2, characterized in that, Step two specifically includes: The operational status information includes three-dimensional position coordinate information, power information, load information, task progress information, and communication status information; The running status information is concatenated into a running status vector. The minimum-maximum normalization method is used to map each dimension of the running status vector to the interval [0,1] to obtain a normalized running status vector. The normalized running state vector is linearly mapped and its dimensions are compressed to generate a standardized state embedding vector.
5. A robot system for maintaining isolation zones based on multi-agent collaboration according to claim 2, characterized in that, Step three specifically includes: The main agent divides the operation area according to the global environment map and constructs a task candidate set. The task candidate set includes several task candidates. Each task candidate is generated into a candidate task action vector through feature concatenation and embedding mapping. The normalized state embedding vector of each AGENT is concatenated with the action vector of each candidate task to form a joint state-action vector; Introduce the task weight of each agent in the current job cycle, expand the dimension of the task weight and multiply it element-wise with the state-action joint vector to obtain the weighted state-action joint vector, and input it into the local policy network of each agent to calculate the task action Q value. In the task graph convolutional fusion layer, the task graph is constructed through the main agent; The task action Q values of all agents for the same task candidate are concatenated to obtain the node feature vector of the corresponding graph node, and all node feature vectors are arranged by node number to form a node feature matrix. Construct a task graph adjacency matrix, wherein the corresponding elements of the task graph adjacency matrix are the weighted sum of the distance exponential decay term and the task type consistency term; The adjacency matrix and the node feature matrix are propagated by a single-layer graph convolution to obtain the graph convolution node feature matrix; Construct a state coordination matrix, the corresponding elements of which are: the weighted inner product between the normalized state embedding vectors of the AGENT and normalized by the Sigmoid function; The state collaboration matrix and the graph convolution node feature matrix are jointly input into the hierarchical hybrid network to calculate the hybrid Q-values of inspection tasks, operation tasks, and energy consumption tasks, respectively, and then perform weighted combination to generate the global Q-value. Under the premise that each task candidate can be assigned to at most one agent and each agent can receive at most one task candidate, the sum of all global Q values is maximized, and a greedy matching algorithm is used to generate the task assignment matrix. Based on the task allocation matrix, the task allocation result corresponding to each AGENT is output. The task allocation result includes the task identifier, the spatial coordinates of the target task area, the task type code, and the task priority. The task priority is set according to the global Q value sorting result of each AGENT.
6. The multi-agent collaborative isolation zone maintenance robot system according to claim 2, characterized in that, Step four specifically includes: The main agent determines the target task region for each agent based on the task allocation results, and extracts a set of spatial voxel units that match the target task region from the environmental feature tensor. The task allocation results of each agent and the set of matching spatial voxel units are concatenated in the channel dimension and the path feature tensor is generated by organizing it according to the time step. The path feature tensor of each agent is linearly mapped along the channel dimension to generate a path mapping tensor. A temporal position coding tensor is introduced by sine and cosine coding, and a spatial position coding tensor is constructed by relative position offset. The path mapping tensor is added element-wise to the temporal location encoding tensor and the spatial location encoding tensor to obtain the path embedding tensor for each agent. In the time step dimension, attention operations are performed on the spatial location features of the path embedding tensor of each AGENT to generate a spatial attention tensor; In the spatial location dimension, attention operations are performed on the time-series features of the path embedding tensor for each agent to generate a temporal attention tensor; Perform channel residual join and linear fusion with the path embedding tensor, spatial attention tensor, and temporal attention tensor to generate the path residual embedding tensor for each agent. At each time step, the attention features between the path residual embedding tensors of any two AGENTs are calculated to generate a global interaction attention tensor; Based on the global interaction attention tensor, extract the global interaction feature vector sequence for each agent; The global interaction feature vector sequence of each agent is used to predict the path target parameters of each agent at each time step through a two-layer MLP structure. The path target parameters include the planar coordinates of the target path point, the velocity magnitude, and the orientation angle. The global interaction feature vector sequence of each agent is used to predict the control command parameters of each agent at each time step through a one-dimensional convolutional structure. The control command parameters include execution status identifier, steering angle and energy consumption adjustment coefficient.
7. The isolation zone maintenance robot system based on multi-agent collaboration according to claim 2, characterized in that, Step five specifically includes: The maintenance operation execution module includes a spraying arm, a pruning arm, a cleaning arm, and an execution drive unit; At each time step, each agent determines its current operating mode based on the execution status flag, adjusts its movement direction based on the steering angle, and sets its execution level based on the energy consumption adjustment coefficient. Each agent controls the machine's movement trajectory and speed based on path target parameters, determines the current work space execution area, and sets the spraying angle, spraying speed and spraying flow rate of the spraying arm, the trimming angle and cutting speed of the trimming arm, and the sweeping direction and cleaning power of the cleaning arm in combination with control command parameters. The maintenance operation module is driven by the drive unit to complete the maintenance operation task of the isolation zone.
8. A robot system for maintaining isolation zones based on multi-agent collaboration according to claim 2, characterized in that, Step six specifically includes: At each time step, each agent exchanges standardized state embedding vectors and job progress information with each other via a wireless communication network; The job progress information includes the current path execution status identifier, the current job task completion percentage, and the current path time step index; The main agent receives standardized state embedding vectors and job progress information uploaded by all agents and fuses them to generate a global state matrix; The main agent inputs the global state matrix into the improved QMIX network to perform network updates and generate updated task assignment results.