A multi-robot cooperative handling method and system

Abstract

The application belongs to the field of reinforcement learning, and particularly relates to a multi-robot cooperative carrying method and system. The method comprises the following steps: obtaining team-in communication representations of each robot in each sub-team based on local observation embedding and relationship sub-graphs; setting an adaptive enhancement factor for key nodes screened out based on a comprehensive importance score to obtain enhanced enhanced local observation embedding; calculating a final score according to the comprehensive importance score and an original structure strength score, and selecting a robot with the highest final score as a sub-team representative; performing multi-head attention interaction on team representations of all sub-teams to obtain team-between communication representations of each robot; and splicing the enhanced local observation embedding, the team-between communication representations and the team-in communication representations of each robot to obtain final decision representations of each robot. The application has the advantages of reducing information loss and deviation and improving overall cooperation effect.

Description

Technical Field

[0001] This invention belongs to the field of reinforcement learning, and specifically relates to a multi-robot cooperative handling method and system. Background Technology

[0002] In automated warehousing operations involving parallel receiving and dispatching of goods at multiple locations and dynamic updates of handling tasks, the specifications and weights of goods awaiting transfer within the site vary, and there are significant differences in the load capacity and motion performance of individual robots in the cluster. This can easily lead to problems such as task mismatch, invalid information interaction, and imbalance in capacity allocation. Currently, the allocation is mainly achieved through agent reinforcement learning.

[0003] In multi-agent reinforcement learning systems, agents typically possess only local observation capabilities and require efficient team communication for collaborative decision-making. Existing team communication frameworks based on multi-agent reinforcement learning often employ a hierarchical communication structure. Agents are first divided into several sub-teams, and information is aggregated within each sub-team before representative nodes are selected for cross-team interaction, thus achieving a balance between communication overhead and decision-making performance.

[0004] However, existing methods generally suffer from the following problems: First, information aggregation within a team lacks dynamic identification of key nodes. Existing methods use fixed topology or single structural features (such as the static connection strength of robots) for information aggregation, failing to dynamically identify the robot performing the key function at the current moment based on the real-time task context (such as multiple obstacles or target goods appearing simultaneously in a robot's field of vision), resulting in low information aggregation efficiency. Second, a homogenized information processing method is used for all robots, without targeted perception enhancement for high-value key nodes, limiting decision-making performance in some observable and information-sparse scenarios. Third, the selection of team representative nodes relies excessively on static topological features, failing to integrate dynamic importance information, resulting in insufficient representativeness of cross-team information transmission. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a multi-robot collaborative handling method and system, which reduces information loss and deviation, improves the overall collaborative effect, and rationally arranges the handling scheme according to the handling capabilities of different robots to improve transportation efficiency.

[0006] A multi-robot cooperative handling method includes: The input data of the multi-robot system is obtained. The input data includes the local observation embedding of each robot and the multi-robot relationship graph. The local observation embedding is a feature generated by the robot's own position coordinates, current velocity vector, load capacity level, distance and direction of obstacles in the field of view, position and weight level of the target to be transported, and relative orientation of other robots. The multi-robot relationship graph is the initial communication topology graph between robots. Based on the multi-robot relationship graph, all robots are divided into teams to obtain several sub-teams and their corresponding relationship sub-graphs. For each robot within a subteam, a representation of intra-team communication is obtained based on local observation embeddings and relational subgraphs; For each subteam, the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength of each robot in the subteam are calculated separately. The information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength are weighted and fused to obtain a comprehensive importance score. The comprehensive importance score is normalized and mapped through the softmax function to obtain the importance probability distribution. Based on the statistical characteristics of the importance probability distribution, a dynamic threshold is calculated. Combining a hybrid selection strategy of probability sampling and deterministic selection, a set of key robot nodes is selected from the sub-team, where each robot is considered a node. For each node in the selected set of key robot nodes, an adaptive enhancement factor is set based on the comprehensive importance score to amplify the local observation embedding of the key node, resulting in an enhanced local observation embedding. Based on the comprehensive importance score and the original structural strength score, the scores are normalized and then weighted and fused to obtain the final score of each robot. The robot with the highest final score is selected as the representative of the sub-team. The original structural strength score is calculated from the static topological features extracted from the multi-robot relationship graph. If the set of key robot nodes selected from the sub-team is a non-empty set, then the intra-team communication representations of each key robot node set are weighted and aggregated based on the comprehensive importance score of the key robot node set to obtain the team representation of the sub-team; if the set of key robot nodes is an empty set, then the intra-team communication representation represented by the sub-team is directly used as the team representation of the sub-team. Input all team representations into the multi-head attention module to generate global interaction messages for each team, and broadcast the global interaction messages to all members within the team as their inter-team communication representation; The enhanced local observation embedding, inter-team communication representation, and intra-team communication representation of each robot are concatenated to obtain the final decision representation of each robot. The final decision representation is then input into the policy network to output the robot's handling result, which includes the robot's movement direction, movement position, and handling cargo level.

[0007] Optionally, the intra-team communication representation of each robot within each sub-team, based on local observation embeddings and relational subgraphs, via a graph attention network, includes: The local observations of each robot within the subteam are embedded and feature-mapped using a multilayer perceptron to obtain the mapped features; The mapping features and relational subgraphs are input into the graph attention network to obtain the adaptive attention weights for each robot; Based on the mapping features and the adaptive attention weights of each robot, the intra-team communication representation of each robot is obtained.

[0008] Optionally, for each sub-team, the information entropy metric, structure centrality metric, attention diversity metric, and robot intrinsic feature strength of each robot within the sub-team are calculated separately. These metrics are then weighted and fused to obtain a comprehensive importance score, including: Obtain the interaction matrix between robots; Based on the interaction matrix, the information entropy measure of each robot within the sub-team is calculated; Based on the interaction matrix, obtain the robot's in-degree, out-degree, and feature norm; The structural centrality measure is obtained by weighted summation of the in-degree, out-degree, and their characteristic norms. The attention diversity measure is calculated based on the information entropy measure. Obtain the inherent feature strength of the robot; The information entropy measure, structure centrality measure, attention diversity measure, and intrinsic feature strength are weighted and fused to obtain a comprehensive importance score.

[0009] Optionally, the step of calculating a dynamic threshold based on the statistical characteristics of the importance probability distribution, and combining a hybrid selection strategy of probability sampling and deterministic selection to select a set of key robot nodes from within the sub-team, includes: The mean and standard deviation are calculated based on the importance probability distribution of each robot node within the subteam and the number of robots. The importance probability distribution is composed of the importance probabilities of several robot nodes. The dynamic threshold is calculated based on the mean and the standard deviation. Robot nodes whose importance probability is greater than or equal to the dynamic threshold are selected as candidate nodes. Set the probability of a deterministic choice; Each time a key node is filtered, a list is generated. A random number that is uniformly distributed within an interval; If the random number is greater than the deterministic selection probability, then a deterministic selection method is used to select the candidate with the highest importance probability from the candidate set. A set of key robot nodes is defined as a set of robot nodes. If the random number is less than the deterministic selection probability, then a random sampling method is used, and multinomial sampling is performed based on the importance probability distribution of each robot node in the candidate set to extract... A set of key robot nodes is defined as a set of robot nodes. in, , The preset maximum critical node scaling factor. This represents the total number of robot nodes within the current subteam.

[0010] Optionally, an adaptive enhancement factor is set based on the comprehensive importance score to amplify the local observation embedding of key nodes, resulting in enhanced local observation embedding, including: The adaptive enhancement factor is calculated based on the comprehensive importance score and is expressed as follows:

[0011] in, As an adaptive enhancement factor, Basic enhancement factor, For the sigmoid function, A comprehensive importance score; The enhancement factor is multiplied element-wise with the local observation embedding to perform amplification, resulting in the enhanced local observation embedding, expressed as:

[0012] in, To enhance local observation embedding, Embedded for local observation.

[0013] Optionally, the robot's final score is obtained by normalizing and weighting the comprehensive importance score and the original structural strength score, respectively, and then selecting the robot with the highest final score as the representative of the sub-team, including: Extract the number of robot edges and edge strength features from the multi-robot relationship graph; The original structural strength score is calculated based on the number of robot edge connections and the edge strength characteristics. The overall importance score and the original structural strength score are normalized within the team to obtain the team's overall importance score and the team's original structural strength score. The team's overall importance score and the team's original structural strength score are weighted and fused to obtain the final score for each robot. The robot with the highest final score is selected as the representative of the sub-team.

[0014] Optionally, the enhanced local observation embedding, inter-team communication representation, and intra-team communication representation of each robot are concatenated to obtain the final decision representation of each robot as follows:

[0015] in, To indicate the final decision, To enhance local observation embedding, For inter-team communication, This refers to communication within the team.

[0016] A multi-robot collaborative handling system includes: The acquisition module is used to acquire input data of the multi-robot system. The input data includes local observation embeddings of each robot and a multi-robot relationship graph. The local observation embeddings are features generated by the robot's own position coordinates, current velocity vector, load capacity level, distance and direction of obstacles in the field of view, position and weight level of the target to be transported, and relative orientation of other robots. The multi-robot relationship graph is the initial communication topology graph between robots. The team division module is used to divide all robots into teams based on the multi-robot relationship graph, resulting in several sub-teams and the corresponding relationship sub-graphs of the sub-teams. The internal communication representation acquisition module is used to obtain the internal communication representation of each robot in each sub-team based on local observation embedding and relational subgraph; The calculation module is used to calculate the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength of each robot in each sub-team. The information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength are weighted and fused to obtain a comprehensive importance score. The comprehensive importance score is normalized and mapped through the softmax function to obtain the importance probability distribution. The screening module is used to calculate a dynamic threshold based on the statistical characteristics of the importance probability distribution, and to select a set of key robot nodes from the sub-team by combining a hybrid selection strategy of probability sampling and deterministic selection, wherein one robot is considered as a node. The enhancement module is used to set an adaptive enhancement factor for each node in the selected set of key robot nodes based on the comprehensive importance score, and to amplify the local observation embedding of the key nodes to obtain the enhanced local observation embedding. The weighting module is used to normalize and then weight and fuse the comprehensive importance score and the original structural strength score to obtain the final score of each robot. The robot with the highest final score is selected as the representative of the sub-team. The original structural strength score is calculated from the static topological features extracted from the multi-robot relationship graph. The team representation calculation module is used to perform weighted aggregation of the intra-team communication representations of each key robot node set based on the comprehensive importance score of the key robot node set if the set of key robot nodes selected from the sub-team is a non-empty set, so as to obtain the team representation of the sub-team; if the set of key robot nodes is an empty set, the intra-team communication representation represented by the sub-team is directly used as the team representation of the sub-team. The interaction module is used to input all team representations into the multi-head attention module, generate a global interaction message for each team, and broadcast the global interaction message to all members within the team as a representation for inter-team communication. The fusion module is used to splice together the enhanced local observation embedding, inter-team communication representation and intra-team communication representation of each robot to obtain the final decision representation of each robot. The final decision representation is input into the policy network and the robot's handling result is output. The handling result includes the robot's movement direction, movement position and handling cargo level.

[0017] A terminal device includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, a multi-robot cooperative handling method is used.

[0018] A computer-readable storage medium storing a computer program, which, when loaded and executed by a processor, employs a multi-robot cooperative handling method.

[0019] The beneficial effects of this invention are: When automating warehouse operations involving parallel receiving and dispatching of goods at multiple locations and dynamic updates of handling tasks, it offers the following advantages: 1. Improve information aggregation efficiency and decision-making accuracy. By integrating information entropy, structural centrality, attention diversity, and inherent feature strength to construct a multi-dimensional importance assessment system, it can accurately identify key nodes that play a core role in team collaboration decisions, avoid ineffective information aggregation, significantly improve the efficiency of information transmission and feature fusion within the team, and rationally arrange handling plans based on the handling capabilities of different robots to improve transportation efficiency.

[0020] 2. Enhance the perception capabilities of key nodes and improve adaptability to complex environments. Adaptively amplify and enhance the local observation embedding of key nodes to enable high-value robots to have more outstanding environmental perception and information expression capabilities. This effectively alleviates decision-making uncertainty in some observable and information-sparse scenarios and improves the robustness of the system in complex dynamic environments.

[0021] 3. Optimize the team representative selection mechanism to improve the quality of cross-team communication. Combine dynamic comprehensive importance score with static topology strength score to achieve more reasonable team representative selection. At the same time, generate team representation based on weighted aggregation of key nodes. This can take into account both the inherent importance of the robot in the global structure and the dynamic task value at the current moment, making the representative nodes of cross-team communication more representative, thereby improving the efficiency of inter-team collaboration.

[0022] 4. Enhanced System Robustness and Convergence Stability: The hybrid selection strategy introduced in this invention (combining deterministic selection and random sampling) effectively combats observation noise and random errors in scoring calculation, avoiding the omission of potential key nodes due to single scoring bias. Simultaneously, the scheduling of dynamic thresholds and hybrid probabilities allows for thorough exploration of the importance distribution of different nodes in the early stages of training, while stabilizing the use of the optimal estimate in later stages. Attached Figure Description

[0023] Figure 1 This is a framework diagram of a multi-robot cooperative handling method according to the present invention; Figure 2 Box plots showing the cumulative reward for the last 50 rounds for each algorithm under different environments are provided. Figure 2 (a) Cumulative reward box chart for the last 50 rounds in the Strategic_12 environment. Figure 2 (b) Round cumulative reward box plot for the last 50 rounds in the Strategic_15 environment; Figure 3 Box plots showing the cumulative reward for the last 50 rounds for each algorithm under different environments are provided. Figure 3 (a) Round-cumulative reward box plot for the last 50 rounds in the Large_Coop environment. Figure 3 (b) Round cumulative reward box plot for the last 50 rounds in the Coordinated_21 environment; Figure 4 Box plots showing the cumulative reward for the last 50 rounds for each algorithm under different environments are provided. Figure 4 (a) Round-cumulative reward box plot for the last 50 rounds in Massive environment. Figure 4 (b) Round cumulative reward box chart for the last 50 rounds in Extreme mode; Figure 5 The graph shows the average round cumulative reward curves for each algorithm under different environments. Figure 5 (a) Average round cumulative reward curve in the Strategic_12 environment Figure 5 (b) Average round cumulative reward curve in the Strategic_15 environment; Figure 6 The graph shows the average round cumulative reward curves for each algorithm under different environments. Figure 6 (a) Average round cumulative reward curve in Large_Coop environment Figure 6 (b) Average round cumulative reward curve in the Coordinated_21 environment; Figure 7 The graph shows the average round cumulative reward curves for each algorithm under different environments. Figure 7 (a) Average round cumulative reward curve in Massive environment Figure 7 (b) Average round cumulative reward curve in Extreme environment; Figure 8 The learning curves of Tiecomm-IntraObs and KnoCoM-TieComm under different ObsWeights are shown below. Figure 8 (a) shows the learning curve of Tiecomm-IntraObs. Figure 8 (b) shows the learning curve of KnoCoM-TieComm; Figure 9 This invention integrates the weighting effect on the performance of Tiecomm-Hybrid under different environments. Detailed Implementation

[0024] A multi-robot cooperative handling method includes: S1. Obtain input data for the multi-robot system. The input data includes local observation embeddings of each robot and a multi-robot relationship graph. The local observation embeddings are features generated by the robot's own position coordinates, current velocity vector, load capacity level, distance and direction of obstacles in the field of view, position and weight level of the target to be transported, and relative orientation of other robots. The multi-robot relationship graph is the initial communication topology graph between robots. Specifically, a multi-robot system includes several robots, each performing different functions. The robots collaborate to complete a task.

[0025] Specifically, the local observation embedding of each robot is a d-dimensional vector obtained by feature extraction after the robot perceives its local environment; the communication features of each robot are d-dimensional feature vectors formed during the communication process of the robot participating in the multi-robot system; the interaction matrix between robots is an s×s matrix generated by the underlying attention mechanism, where s is the number of robots in a single team, and the element in the i-th row and j-th column of the matrix represents the degree of attention of the i-th robot to the j-th robot; the multi-robot relationship graph is a topological graph composed of all robots in the multi-robot system as nodes and the connections between robots as edges.

[0026] For a containing Team of each node , recorded as ,Notice, This represents the total number of agents in the entire multi-agent system, while This indicates the number of nodes in the current team. For each node... With the following input: local observation embedding (Extracted from raw observations by the encoder), intra-team communication representation (This representation is obtained through aggregation within the team via a multi-head graph attention network (GAT)) and the attention interaction matrix. ,in Represents a node For nodes The level of attention, It is the set of real numbers.

[0027] S2. Based on the multi-robot relationship graph, all robots are divided into teams to obtain several sub-teams and the corresponding relationship sub-graphs of the sub-teams. Specifically, all robots in the system are divided into several relatively independent sub-teams (to facilitate refined communication and collaboration within the teams); at the same time, for each sub-team, a local subgraph containing only the robots in that sub-team and their connections is extracted from the global multi-robot relationship graph, that is, the relationship subgraph corresponding to the sub-team. This relationship subgraph is the core topological basis for subsequent communication representation aggregation within the sub-team, and all the relationship subgraphs of the sub-teams have no overlapping nodes and together completely cover all nodes and corresponding edges of the original multi-robot relationship graph.

[0028] In robot relationship graphs, the connecting edges between robots have edge strength weights, which are calculated from interaction features such as interaction frequency, communication correlation, and state similarity between robots. This is an inherent property of multi-robot relationship graphs. Determining the graph partitioning threshold: Setting the graph partitioning threshold This is an adjustable parameter, and its value is dynamically output by the upper-level inference strategy based on the current global observation state of the multi-robot system. Edge filtering and sub-team generation: Traverse all edges in the multi-robot relationship graph, and select the edge that satisfies the condition that the edge strength between any two robots in a sub-team is greater than or equal to a threshold. If the edge strength between robots in subteams is less than 1, then the edge is retained; if the edge strength between robots in subteams is less than 1, then the edge is retained. If an edge is not found in the graph, it is removed. After filtering the edges, connected components are found in the graph formed by the remaining edges. Each connected component is a sub-team, and all robot nodes within that connected component constitute the set of members of that sub-team, denoted as . Ultimately obtained There are several independent subteams, and the set of subteams is represented as... ,in This represents the number of teams obtained from the current division.

[0029] Relationship subgraph extraction: For each generated sub-team From the original multi-robot relationship diagram Extract all robot nodes contained in the sub-team, as well as all connecting edges (including edge strength weights) between these nodes. The topological graph formed by these local nodes and edges is the relational subgraph corresponding to the sub-team, denoted as . .

[0030] S3. For each robot in a subteam, obtain the team communication table based on local observation embedding and relational subgraph; For each robot within a subteam, a team communication table is obtained based on local observation embeddings and relational subgraphs, including: Team Each robot node in First, embed its local observation. The initial feature vector is obtained by feature mapping through a multilayer perceptron; The initial feature vector and relation subgraph The information from neighboring nodes is fed into a graph attention network, where a multi-head attention mechanism aggregates the information to obtain the robot's intra-team communication representation, denoted as . .

[0031] Specifically, team communication is represented as follows:

[0032] in, For team communication, Embedded for localized observation of the robot, It is a multilayer perceptron. For mapping features, For the team The corresponding relationship subgraph, For graph attention networks, Indicates the first A robot.

[0033] S4. For each subteam, calculate the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength of each robot in the subteam. Weight and fuse the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength to obtain a comprehensive importance score. After normalizing the comprehensive importance score, map it through the softmax function to obtain the importance probability distribution. For each sub-team, the information entropy metric, structure centrality metric, attention diversity metric, and robot intrinsic feature strength of each robot within the sub-team are calculated separately. These metrics are then weighted and fused to obtain a comprehensive importance score, including: Obtain the interaction matrix between robots; Based on the interaction matrix, the information entropy measure of each robot within the sub-team is calculated; Based on the interaction matrix, obtain the robot's in-degree, out-degree, and feature norm; The structural centrality measure is obtained by weighted summation of the in-degree, out-degree, and their characteristic norms. The attention diversity measure is calculated based on the information entropy measure. Obtain the inherent feature strength of the robot; The information entropy measure, structure centrality measure, attention diversity measure, and intrinsic feature strength are weighted and fused to obtain a comprehensive importance score.

[0034] Specifically, in team collaboration, the attention distribution characteristics of nodes directly affect the effectiveness of their information transmission. If a node's attention is highly focused on a few specific neighbors, it indicates that it plays a clear role as an "information filter" or "decision facilitator" in the current task; conversely, if the attention distribution is even, it may be undertaking a broader information gathering function. To quantify this characteristic, we introduce an information entropy metric to assess the degree of certainty in node information transmission. (Defining nodes...) The formula for the normalized attention distribution entropy is:

[0035]

[0036] in, For nodes Information entropy measurement For nodes For nodes The level of attention, It is a constant. For the number of nodes, This is a normalized measure of information entropy.

[0037] Beyond their informational characteristics, the position of nodes within a team's topology is equally crucial. Nodes at the center of the structure often play a key role in information transfer and coordination. A structural centrality metric is designed to comprehensively evaluate the importance of nodes from a graph structure perspective, considering three sub-indices: in-degree, out-degree, and eigennorm. This is expressed as:

[0038]

[0039]

[0040]

[0041] in, Let be the in-degree of the node. This represents the out-degree of the node. For characteristic norm, For the team the robot currently belongs to, As a measure of structural centrality, For the first The current intra-team communication of each node is represented as follows: For the first The current intra-team communication of each node is represented as follows: The biggest contributor to the team. The largest in-degree component in the team. Let d be the set of real numbers, and d be the dimension. , as well as As weight, The default configuration is , , . and The activity level of a node as both a receiver and sender of information is measured separately. Measure the relative strength of node feature vectors.

[0042] However, an overly concentrated distribution of attention can lead to information bottlenecks, causing teams to rely excessively on a few nodes while neglecting other potential information sources. To address this, we introduce a measure of attention diversity, defined as normalized entropy:

[0043] in, A measure of attention diversity. The larger the value, the more likely it is to be a node. The more evenly attention is distributed, the more comprehensive the information the team can obtain. In subsequent comprehensive scoring, it serves as a regularization factor, encouraging moderate distraction.

[0044] After obtaining the above multi-dimensional evaluation metrics, we need to integrate them into a unified comprehensive score. This integration is not a simple linear summation, but a weighted combination based on the relative importance of each dimension in a specific task. The final comprehensive score must reflect both the structural importance of the node and its informational value within the current task context. This is achieved by combining the above metrics with the inherent feature strength. Combined, a comprehensive importance score is obtained, expressed as:

[0045] in, For nodes The overall importance score, , , as well as As weight, The intensity of the inherent characteristics is normalized.

[0046] The normalization calculation method for intrinsic feature strength is as follows: Directly adopt the node static strength proposed in TieComm Its definition depends on the strength of the connection between nodes. Its formula is:

[0047] in For nodes and The strength of the connection between them is expressed by the formula:

[0048] in for The Jump to neighbor set, for j The Jump to neighbor set, for The reciprocal of the jump is used to reflect the effect of distance. Nodes were measured The structural importance within the global topology reflects its potential as an information hub and is a stable priori for team communication. Normalizing this strength within the team... Interval:

[0049] When all nodes in the team have the same strength, define .

[0050] To avoid the influence of extreme values, soft normalization is performed:

[0051] in, The overall importance score after soft normalization. and Indicates the current team Different robots inside, It is a constant. For nodes The overall importance score.

[0052] Finally, through temperature parameters The controlled softmax function transforms the soft-normalized overall importance score into an importance probability distribution:

[0053] in, This represents a probability distribution of importance. Temperature. Dynamically adjustable: In the initial stage of training Larger values ​​result in a smoother probability distribution, encouraging exploration; later stages Reduce the size to make the distribution more acute and encourage its use.

[0054] S5. Calculate the dynamic threshold based on the statistical characteristics of the importance probability distribution, and select the set of key robot nodes from the sub-team by combining a hybrid selection strategy of probability sampling and deterministic selection. The dynamic threshold is calculated based on the statistical characteristics of the importance probability distribution. A hybrid selection strategy combining probability sampling and deterministic selection is used to select a set of key robot nodes from within the sub-team, including: Based on the importance probability distribution and the number of robots, the mean and standard deviation are calculated. The dynamic threshold is calculated based on the mean and the standard deviation. Robot nodes whose importance probability is greater than or equal to the dynamic threshold are formed into a candidate set; Set the probability of a deterministic choice; Each time a key node is filtered, a list is generated. A random number that is uniformly distributed within an interval; If the random number is greater than the deterministic selection probability, then a deterministic selection method is used to select the candidate with the highest importance probability from the candidate set. The set of key robot nodes; If the random number is less than the deterministic selection probability, then a random sampling method is used, and multinomial sampling is performed based on the importance probability distribution of each robot node in the candidate set to extract... A set of key robot nodes is defined as a set of robot nodes. in, , The preset maximum critical node scaling factor. This represents the total number of robot nodes within the current subteam.

[0055] Specifically, obtain the probability distribution of node importance. Subsequently, this needs to be translated into actual critical node selection decisions. Traditional methods typically employ fixed thresholds or simple ranking strategies, but these methods struggle to adapt to dynamically changing team states and task requirements. To address this, we propose an adaptive thresholding method based on distribution statistics to filter out nodes with significantly lower importance at the current moment (each robot is considered a node), thereby reducing the computational burden of subsequent selections.

[0056] Define Team Mean of the probability distribution of importance of internal nodes and standard deviation :

[0057]

[0058]

[0059] in, The mean, Standard deviation This is a dynamic threshold.

[0060] in, It depends on the training phase (such as the training rounds of the team). The adaptive factor was initially set to a relatively large 1.5. (in Using a constant threshold to relax the threshold allows more nodes to enter the candidate set, helping the scoring network obtain diverse samples in the early stages of training; the threshold is then reduced to a constant value later. Tighten the threshold, focus on nodes of higher importance, and improve communication efficiency.

[0061] The set of candidate key nodes is as follows:

[0062] like Then take directly All nodes in the middle (if the actual number is insufficient, use the actual number); otherwise, start from... The final selection is achieved through a hybrid node selection strategy. A key node.

[0063] The threshold mechanism is responsible for excluding nodes with excessively low probabilities (i.e., nodes whose importance is significantly lower at the current moment), thereby reducing unnecessary computation.

[0064] Dynamic threshold It can automatically adjust according to the distribution. In the early stage of training, the threshold is set larger to select more key nodes for exploration; as training progresses, the threshold is gradually tightened to focus on the most important nodes.

[0065] Node Importance Score It is based on real-time observations and attention distribution calculations, and is inevitably affected by observation noise, partial observability, and finite sampling variance. If a completely deterministic selection method were used (selecting the option with the highest probability each time),... (If there are multiple nodes, nodes that actually perform key functions may be missed due to random bias in a single scoring. Conversely, if sampling is done randomly according to a probability distribution, nodes with lower importance may be selected, reducing communication efficiency.)

[0066] To strike a balance between utilizing the current best estimate and maintaining robustness to scoring uncertainty, we use the candidate set A hybrid selection strategy is implemented. This strategy uses probability... Make a deterministic choice (take a probability distribution) The highest front (Number of nodes), ensuring that at least the node with the highest current rating can be used each time, with probability. Perform random sampling (based on probability distribution) Perform polynomial sampling, select (Number of nodes) allows low-probability but potentially important nodes to be selected, thereby enhancing robustness to scoring noise. The formula for this hybrid selection strategy is:

[0067] in, , The proportion of the most critical nodes. Probability. With training steps From exploration to utilization The highest , For importance distribution Perform polynomial sampling.

[0068] Deterministic choice probability in mixed selection With training steps From initial value Linear growth to final value :

[0069] in, To set the minimum selection probability, To set the maximum selection probability, Let t be the maximum number of training steps, and t be the t-th step.

[0070] In the early stages of training ( (Smaller), the system tends to sample randomly to fully expose the chances of different nodes being selected as key nodes, helping the scoring network learn a more robust importance distribution; in the later stages of training ( (The score is relatively high), the rating tends to stabilize, and the system relies more on deterministic choices to ensure communication efficiency.

[0071] S6. For each node in the selected set of key robot nodes, an adaptive enhancement factor is set based on the comprehensive importance score to amplify the local observation embedding of the key node, thereby obtaining the enhanced local observation embedding. An adaptive enhancement factor is designed based on a comprehensive importance score to amplify the local observation embeddings of key nodes, resulting in enhanced local observation embeddings, including: The adaptive enhancement factor is calculated based on the comprehensive importance score and is expressed as follows:

[0072] in, As an adaptive enhancement factor, Basic enhancement factor, For the sigmoid function, A comprehensive importance score; The amplification process is performed by multiplying the enhancement factor element-wise with the local observation embedding, resulting in the enhanced local observation embedding, expressed as:

[0073] in, To enhance local observation embedding, Embedded for local observation.

[0074] S7. Based on the comprehensive importance score and the original structural strength score, normalize them respectively and then perform weighted fusion to obtain the final score of each robot. Select the robot with the highest final score as the representative of the sub-team. The original structural strength score is calculated from the static topological features extracted from the multi-robot relationship graph. The process involves normalizing and weighting the comprehensive importance score and the original structural strength score to obtain the final score for each robot. The robot with the highest final score is selected as the representative of the sub-team, including: Extract the number of robot edges and edge strength features from the multi-robot relationship graph; The original structural strength score is calculated based on the number of robot edge connections and the edge strength characteristics. The overall importance score and the original structural strength score are normalized within the team to obtain the team's overall importance score and the team's original structural strength score. The team's overall importance score and the team's original structural strength score are weighted and fused to obtain the final score for each robot. The robot with the highest final score is selected as the representative of the sub-team.

[0075] Specifically, to ensure that different metrics are integrated under the same scale, the key node scores and node strengths in the original paper are first normalized within the team using a min-max method. Based on this, a hybrid core selection strategy is proposed to balance structural features and dynamic importance. Candidate nodes final score The calculation is as follows:

[0076] in, The key node scores are normalized. The normalized original node static strength, A mixed weighting method was used. Each team was ultimately selected. Internal selection The largest node serves as the team's representative node. .

[0077] S8. If the set of key robot nodes selected from the sub-team is a non-empty set, then the team communication representation of each key robot node set is weighted and aggregated based on the comprehensive importance score of the key robot node set to obtain the team representation of the sub-team; if the set of key robot nodes is an empty set, then the team communication representation represented by the sub-team is directly used as the team representation of the sub-team. Specifically, the team The actual set of key nodes selected in the process is as follows: The team stated It can be uniformly expressed as: If there is at least one critical node, then the weighted aggregation of the critical nodes is used as the team representation:

[0078] If no key node is selected at the current moment, the system degenerates into using the representation of the team representative node: 。

[0079] in, For nodes The team's internal communication indicated that For temperature coefficient, This indicates the index of the team representative node. As the weight of key nodes, Indicates team The actual set of key nodes selected in the process (based on dynamic thresholds) Selected by a combination of methods, the number of elements does not exceed ), and Representing nodes respectively and nodes The overall importance score after soft normalization.

[0080] S9. Input all team representations into the multi-head attention module to generate global interaction messages for each team. and will Broadcast to all team members as a form of inter-team communication: ,in For nodes The team to which they belong.

[0081] S10. The enhanced local observation embedding, inter-team communication representation and intra-team communication representation of each robot are spliced ​​together to obtain the final decision representation of each robot. The final decision representation is input into the policy network and the robot's handling result is output. The handling result includes the robot's movement direction, movement position and handling cargo level.

[0082] The enhanced local observation embedding, inter-team communication representation, and intra-team communication representation of each robot are concatenated to obtain the final decision representation of each robot as follows:

[0083] in, To indicate the final decision, To enhance local observation embedding, For inter-team communication, This refers to communication within the team.

[0084] The KnoCoM module is a module integrated using the method of this application. The overall architecture of the KnoCoM module is as follows: Figure 1As shown, it comprises three core sub-modules: a node importance assessment module, which calculates the overall importance score for each node; an adaptive key node selection module, which dynamically selects key nodes based on their importance scores; and an information enhancement and aggregation module, which enhances the features of key nodes and constructs a team representation. This design follows a plug-and-play principle, allowing for seamless integration with existing frameworks' attention mechanisms and team structures.

[0085] Efficiency and Interpretability Analysis In addition to information theory guarantees, the communication efficiency, computational overhead, and interpretability of the KnoCoM module in actual deployment are also important dimensions for evaluating its value.

[0086] Communication efficiency analysis KnoCoM improves communication efficiency without increasing the amount of communication data through a critical node mechanism. Specifically, this is reflected in: At the team representation level: When a set of key nodes is identified, the team representation is generated by a weighted aggregation of multiple key nodes, with weights adaptively assigned based on importance scores. Compared to the team representation composed of a single representative node in the original design, the aggregated representation contains richer key information and has a higher "information density".

[0087] At the team representative level: KnoCoM offers pure critical node strategy and hybrid strategy as new options for representative node selection, enabling representative nodes in cross-team communication to reflect dynamic task context rather than relying solely on static structural strength.

[0088] In bandwidth-constrained scenarios, the team indicated that compression was necessary before transmission. Since the team representation generated by KnoCoM already contained richer key information, it resulted in less information loss at the same compression ratio, thus achieving rate-distortion performance superior to the original method.

[0089] Computational cost analysis Let the total number of nodes in the multi-agent system be . Divided into Teams, average team size It is a fixed value (not affected by the total number of nodes). (As it increases and changes), the dimension of the node's hidden state is... The core computational overhead of the original team communication algorithm comes from the attention network and intra / inter-team communication, with a computational complexity of O(n). The first term comes from the attention calculation of the GAT within the team, and the second term comes from the multi-head attention between teams. The additional computational overhead introduced by the KnoCoM module comes from the identification of key nodes. The core operations are: information entropy and structure centrality calculation, which requires calculating the complexity of the attention distribution entropy for each team. In-degree and out-degree complexity The complexity of the feature norm The complexity of the overall score and softmax The complexity of dynamic thresholds and hybrid selection Here, O() represents the time complexity. Therefore, the extra complexity is: .

[0090] The relative cost is:

[0091] When the hidden state dimension The increase of approaches 0 in practice. Alternatively, it can be 128, and matrix multiplication can be hardware accelerated, so the computational overhead of KnoCoM is negligible.

[0092] Interpretability Analysis KnoCoM's multi-dimensional importance scoring mechanism provides inherent interpretability. The physical meaning of each dimension is clear: Structural centrality measure This reflects the importance of a node's position in the team's communication topology. Highly central nodes typically serve as information relay nodes, and their failure has a significant impact on team collaboration.

[0093] Information entropy measurement The uncertainty of node observation is measured. High-entropy nodes contain richer information and have higher value in decision-making, playing the role of "information filter" or "decision guide".

[0094] Attention diversity measurement High values ​​indicate that node attention is scattered, which helps to collect diverse information.

[0095] Original node feature strength High values ​​indicate that the node itself has abundant observations and is located in an information-dense region.

[0096] By analyzing the weighted contribution of each dimension, we can trace the reasons why key nodes were selected. For example, if a node is selected because of... The fact that someone was chosen as a representative due to their height indicates the importance of their structural position; if it is because... Being selected based on a high score indicates that one's current attention is focused. This decomposition capability makes KnoCoM's decision-making process auditable and debuggable, which is particularly important for safety-sensitive multi-agent systems (such as autonomous driving and robot collaboration).

[0097] Experimental setup To comprehensively evaluate the algorithm's performance under different collaborative complexities, we conducted six level foraging simulation experiments on NVIDIA RTX 3090 GPU hardware. These environments covered multi-robot collaborative scenarios ranging from small- to large-scale, and the specific configurations are shown in Table 1.

[0098] Table 1. Parameters in different scenarios under horizontal foraging environment

[0099] The selected environments have two notable characteristics designed to test the algorithm's adaptability. First, they exhibit scalability diversity: the number of agents ranges from 12 to 24, and the grid size varies from 18*18 to 35*35, providing a natural transition from simple tasks to large, complex scenes. Second, all environments are partially observable, with each agent's field of view limited to 2 to 3 cells; this limitation simulates the incompleteness of information in the real world and forces the algorithm to learn effective information-sharing and coordination mechanisms.

[0100] To ensure fairness in the comparison, all algorithm variants shared a common set of hyperparameters, which were recorded using SwanLab. The training process consisted of 850 epochs, with a maximum of 100 time steps per epoch, and a batch size of 64 for empirical replay sampling. Optimization was performed using the RMSprop optimizer with a learning rate of 0.001, a discount factor (γ=0.9), and a network update frequency of one step. The hidden dimension of all neural networks was set to (d=64), and the graph partitioning threshold for team building was (ε=0.5). Variant-specific parameters included the observation enhancement factor (β=1.5) for the in-observation variant and the blending weights (β=1.5) for the hybrid variant. α =0.5).

[0101] Benchmark Algorithms and Variants To rigorously verify the effectiveness of the proposed dynamic key node identification and hierarchical information aggregation mechanism, four incrementally progressive variants of the algorithm were designed, using the standard Tiecomm algorithm as a benchmark. These variants were tested in isolation against a single core component. All variants shared the same infrastructure and hyperparameters to ensure fairness and reproducibility in the comparisons.

[0102] Benchmark Algorithm Tiecomm is a graph-based multi-robot communication benchmark algorithm. Its core is static team partitioning based on fixed structural metrics (such as edge strength). The algorithm uses a higher-level network to threshold-partition the robot relationship graph at fixed intervals, forming sub-teams. Communication is divided into two layers: intra-team and inter-team. Intra-team communication uses a graph attention network (GAT) for aggregation; inter-team communication involves selecting the node with the highest structural centrality (e.g., the node with the largest sum of edge strength) in each sub-team as the representative for interaction.

[0103] Tiecomm's team representative selection relies entirely on the assumption that "structurally important nodes are necessarily functionally critical." However, this assumption may not hold true in dynamic, goal-oriented tasks, thus limiting collaboration efficiency.

[0104] Algorithm Variant Design To overcome the aforementioned limitations and validate our core ideas, four contrasting variants were designed by adjusting the team representative selection strategy and communication enhancement mechanisms. The characteristics of the baseline and the four variants are shown in Table 2.

[0105] Tiecomm-KeyNode: This module removes the original representative selection mechanism based on structural centrality and replaces it with a key node identification module based on multi-dimensional scoring. This module analyzes the attention distribution of the team's communication GAT (Gaining Attention Target) and other factors to identify the most important node for the information flow in real time and uses it as the team representative. If this variant outperforms the benchmark Tiecomm, it demonstrates the effectiveness of multi-dimensional scoring.

[0106] Tiecomm-Hybrid: This method linearly fuses the key node scores of multi-dimensional scoring with the original structural importance scores using an adjustable weight (set to 0.5) in order to obtain the complementary advantages of structural and dynamic information sources.

[0107] Tiecomm-IntraObs: Building upon Tiecomm, this module introduces an information enhancement module. For identified key nodes, their local observational features are amplified before being aggregated within the team. This module simulates a scenario where leaders in a human team are given more comprehensive information, testing the effectiveness of allocating resources (attention) towards key nodes.

[0108] KnoCoM-TieComm: This variant uses a hybrid strategy to select representatives and enhance their observations. It is used to examine synergistic effects between modules.

[0109] Table 2. Comparison of variables between the baseline and the four variants.

[0110] In Table 2, "original" represents the original node, "key_node_only" represents the key node, "hybrid_weight" represents the hybrid weight, and "key_node_enhancement" represents the key node enhancement coefficient.

[0111] Controlled experimental design These variants were designed following the principle of controlled experiments, with each variant changing only one core component: Tiecomm vs. Tiecomm-KeyNode: Comparing the advantages and disadvantages of static structural metrics versus dynamic learning evaluation.

[0112] Tiecomm-KeyNode vs. Tiecomm-Hybrid: Validating the advantages of fusion strategies compared to single signal sources.

[0113] Tiecomm vs. Tiecomm-Intra Obs: Testing the independent contribution of information augmentation mechanisms to the underlying algorithm.

[0114] Progressive comparison: Quantify the marginal contribution of each module through progressive improvements from Tiecomm to KnoCoM-TieComm.

[0115] Through this systematic ablation study, we can accurately assess the actual utility of each innovative component and provide empirical evidence for the design of a team-based, hierarchical, multi-robot reinforcement learning communication mechanism.

[0116] Evaluation indicators To comprehensively and objectively evaluate the performance of Tiecomm and its variants in complex multi-robot collaborative environments, this study employs a multi-dimensional quantitative evaluation system, covering key aspects such as final performance, convergence speed, stability, and box-line distribution. The definitions, calculation methods, and theoretical basis of each index are as follows: Final Performance Final performance is the core metric for evaluating the collaborative efficiency an algorithm can achieve after sufficient training. To avoid the influence of random fluctuations at the end of training, we use the average cumulative reward over the last 50 training epochs as the measure of final performance:

[0117] in, This represents the cumulative reward for round t. T=850, The average cumulative reward over 50 rounds is used. This metric reflects the steady-state performance level achieved by the algorithm after sufficient exploration, directly representing the collaborative effectiveness under different communication mechanisms. Next, box plots are used to visualize the distribution characteristics of the cumulative reward over the last 50 rounds for each algorithm, and its mean, median, and interquartile range are calculated.

[0118] Convergence Speed Convergence speed is a key indicator for evaluating the learning efficiency of an algorithm, reflecting the time cost required for the algorithm to reach a high performance level during training. This paper uses a visualization analysis method of learning curve features to extract key information about convergence speed from the learning curve. Figures 2 to 4 As shown, we plotted smoothed learning curves for each algorithm in six foraging environments (sliding window of 100), and added shaded areas representing the mean ± standard deviation. By observing the early rising slopes of these learning curves and the location of the plateau points, we can visually analyze the convergence speed.

[0119] Stability The stability of an algorithm is reflected in the degree of performance fluctuation during training and is an important dimension for evaluating its robustness and reliability. We use the coefficient of variation (CV%) and comprehensively evaluate the performance-stability based on the CV.

[0120] To eliminate the impact of absolute performance differences between algorithms on stability evaluation, a unified metric is provided for cross-environment stability assessment. We define the coefficient of variation as the ratio of the standard deviation to the mean of the cumulative reward over the last 50 rounds, typically expressed as a percentage.

[0121] in, The standard deviation of the cumulative reward for the last 50 rounds. This represents the corresponding average value.

[0122] Overall Performance and Baseline Comparison To comprehensively evaluate the performance of each algorithm in diverse collaborative tasks, this section conducts a systematic analysis from five dimensions: round-based cumulative reward, convergence behavior, module contribution, robustness, and hyperparameter sensitivity, aiming to reveal the effectiveness of each improved module and its applicable scenarios.

[0123] Average cumulative reward per round in the last 50 rounds Table 3 summarizes the average cumulative reward for each algorithm across six environments (last 50 rounds). It can be seen that KnoCoM-TieComm performs best in most environments, with its average cumulative reward across environments (105.1) being approximately 11.5% higher than the original Tiecomm (94.3). Tiecomm-Hybrid and Tiecomm-Intra Obs also outperform the baseline in multiple environments, preliminarily validating the effectiveness of modular improvements. The average cumulative reward across environments for KnoCoM-TieComm is calculated as: (60.9+89.6+72.9+133.4+125.7+148.0) / 6=105.1, while the average cumulative reward across environments for the original Tiecomm is: (50.2+73.4+58.3+125.0+118.5+140.2) / 6=94.3.

[0124] Table 3. Average cumulative reward per round for each algorithm under different environments (last 50 rounds)

[0125] To further reveal the performance distribution characteristics of each algorithm, Figures 2 to 4 Box plots show the cumulative reward for the last 50 rounds of the algorithm under six environments. The plots clearly present the median, interquartile range (IQR), and outlier distribution of the cumulative reward for each algorithm, providing a visual basis for evaluating the stability and generalization ability of the algorithm.

[0126] from Figures 2 to 4 The following conclusions can be drawn from this: Consistency of performance improvements: In all six environments, the median of the bins for each improved version (KnoCoM-TieComm, Tiecomm-Hybrid, Tiecomm-KeyNode, Tiecomm-Intra Obs) was higher than that of the original Tiecomm, indicating that each module could improve the final performance of the algorithm to varying degrees. This evidence at the distribution level is consistent with the average round-cumulative reward results in Table 3.

[0127] Key advantages of KnoCoM-TieComm: In the Strategic_15 and Coordinated_21 environments, the median of KnoCoM-TieComm is significantly higher than that of other algorithms, and its interquartile range is narrower, indicating that the algorithm not only has superior average performance, but also exhibits high stability under different random seeds.

[0128] Performance in complex environments: In the three complex environments of Coordinated_21, Massive, and Extreme, the box positions of Tiecomm-Hybrid and KnoCoM-TieComm are both at a high level, demonstrating their potential in complex or large-scale environments. This fully demonstrates that the key node identification and dynamic information enhancement mechanism based on attention can effectively improve the collaborative efficiency of multi-robot systems under sparse rewards and high-dimensional observation.

[0129] Convergence Analysis Figures 5 to 7 The learning curves of each algorithm are shown in six environments (sliding window of 100 steps). Overall, the proposed variant outperforms the baseline in the early stages of most environments (such as Strategic_12 and Large_Coop), indicating higher sample efficiency. In Coordinated_21, the convergence speeds of the algorithms are similar, but the improved version has a higher average cumulative reward per round. In Strategic_15, although KnoCoM-TieComm converges slightly slower, its average cumulative reward per round is significantly higher, reflecting its stronger expressive power in complex policy tasks.

[0130] Module effectiveness and contribution analysis To quantify the independent contribution of each module, we conducted a comparative analysis of the four variants, combining Table 3, Figure 2 , Figure 3 and Figure 4 The following conclusions can be drawn: The key node identification module outperforms the baseline Tiecomm in most environments. In Strategic_15 and Large_Coop, its final average cumulative reward per round reaches 89.2 and 84.0, respectively, representing improvements of 21.5% and 44.1% over the baseline. In Strategic_12, Massive, and Extreme, it converges faster, with final cumulative rewards per round matching or slightly exceeding the baseline. This module replaces the traditional graph-based static structure center with a multi-dimensional score that integrates information entropy, structure centrality, and attention diversity, enabling a more accurate capture of dynamic information value and thus improving collaboration efficiency.

[0131] The local observation enhancement module performs exceptionally well in field-limited environments (Large_Coop, Strategic_12 / 15). For example, in Large_Coop, its average cumulative reward per round reaches 83.5, a 43.2% improvement over the baseline. By assigning higher observation feature weights to key nodes, this module enhances the signal-to-noise ratio of local perception in information-sparse scenarios, demonstrating the complementary value of feature-level enhancement over topology-level adjustment.

[0132] The converged communication module maintains stable and high performance in most environments, especially in the Extreme environment where the average cumulative reward per round reaches 153.1, the highest among all algorithms. Its box plots show that the median and interquartile range are generally better than the baseline, indicating that the hybrid mechanism combining structural robustness and content importance has advantages in long-range collaborative tasks. In vast maps, this mechanism helps key nodes simultaneously utilize both locational advantages and informational value, effectively mitigating the disorientation problem caused by limited observation.

[0133] As shown in Table 3, the average cumulative reward per round for collaborative modules was highest in Strategic_15 (89.6), surpassing Tiecomm-Hybrid (73.6), but slightly lower in Large_Coop (72.9 vs. 82.0). This phenomenon suggests that when the representative nodes selected by Tiecomm-Hybrid can capture most of the key information, further enhancement of their observations may introduce noise or increase the learning burden, leading to performance fluctuations. For example, in the later stages of Large_Coop training (around 850 rounds), the cumulative reward per round for KnoCoM-TieComm was still unstable, possibly due to the optimization difficulties brought about by the increased model complexity. Therefore, the advantage of KnoCoM-TieComm lies not in performing observation enhancements for all scenes, but in its potential for dynamically identifying key nodes and adaptive enhancement, which needs to be weighed in different tasks.

[0134] Robustness Analysis The coefficient of variation (CV%) was used to measure the stability of each algorithm in six types of environments, and the results are shown in Table 4. KnoCoM-TieComm had the lowest average CV (13.58%), indicating that it was the least sensitive to environmental changes and had the best consistency in performance across different scenarios. Tiecomm-Hybrid had an average CV of 13.90%, but it fluctuated significantly in small and medium-sized scenarios (Strategic_12 and Strategic_15) (CVs of 19.5% and 13.7%, respectively), while it showed good stability in large scenarios. Tiecomm-IntraObs had an average CV of 14.05%, but it experienced a significant performance drop after about 600 steps in Strategic_15, indicating that simply enhancing local observations may lead to policy instability in complex policy scenarios. Tiecomm-KeyNode had an average CV of 14.57%, which was a significant improvement over the baseline (17.85%). In summary, KnoCoM-TieComm maintained the highest average cumulative reward per round while exhibiting the best robustness, validating its environmental adaptability advantage.

[0135] Table 4: Comparison of coefficient of variation (CV%) of each algorithm in six scenarios

[0136] To evaluate the sensitivity of key modules to hyperparameters, we examined the impact of the observation enhancement factor (ObsWeight) and the fusion weight (hybrid_weight) on the algorithm performance in the Strategic_15 environment. The results are as follows: Figure 8 and Figure 9 As shown.

[0137] Observational Enhancement Factor (ObsWeight): Figure 8 The learning curves of Tiecomm-Intra Obs and KnoCoM-TieComm under different ObsWeights are shown. Figure 8 (a) Analysis of the independent role of observed enhancement in the Tiecomm-Intra Obs variant. Figure 8 (b) Analysis of the synergistic effect of observed enhancements in the KnoCoM-TieComm variant. As shown in 8(a), Tiecomm-Intra Obs has the highest final average cumulative reward per round at ObsWeight = 1.35, but its convergence speed is slightly slower than the baseline without enhancement (Tiecomm); the final average cumulative reward per round is similar at ObsWeight = 1.2 and 1.5, but the convergence speed is also relatively slow, indicating that the enhancement factor has a significant impact on the convergence process and that there exists a relatively optimal interval. Figure 8(b) It is evident that KnoCoM-TieComm achieves the highest cumulative average reward per round at ObsWeight=1.2 (slightly higher than Tiecomm-Hybrid), but converges the slowest. At ObsWeight=1.35 and 1.5, the convergence speed is comparable, and the cumulative average reward per round is on par with Tiecomm-Hybrid. This indicates that in the collaborative module, the selection of the observation enhancement factor requires a trade-off between convergence speed and final performance, and its impact is related to the base module.

[0138] Hybrid weight: Figure 9 The average cumulative reward per round in Tiecomm-Hybrid was compared under the fusion weights (hybrid_weight = 0.5, 0.6, 0.7) of multi-dimensional scores and original structural strength. Experimental results show that the average cumulative reward per round under the three weight groups is highly similar with no significant difference, indicating that the fusion weights in the range of 0.5–0.7 have no significant impact on the algorithm's performance in this environment, and Tiecomm-Hybrid has good robustness to this hyperparameter. In other words, multi-dimensional scores and structural importance are complementary in this task, and this complementary relationship can be maintained under certain weight shifts, thus simplifying the complexity of parameter tuning.

[0139] In summary, the observation enhancement factor needs to be carefully adjusted according to the specific module and task, while the fusion weights are stable over a wide range, which facilitates the practical deployment of the algorithm.

[0140] The KnoCoM module proposed in this application significantly improves the communication efficiency and collaborative performance of multi-robot reinforcement learning systems through multi-dimensional node importance evaluation, adaptive key node selection, and fine-grained information enhancement strategies. Experimental results validate the effectiveness of the design in the following dimensions: Verification of core hypotheses. Ablation experiments show that replacing static structure centrality with multi-dimensional importance assessment (Tiecomm-KeyNode) increases the average cumulative reward per round by 21.5% and 44.1% in Strategic_15 and Large_Coop, respectively, confirming the advantages of the multi-dimensional scoring mechanism (integrating information entropy, structure centrality, and attention diversity) in capturing the value of dynamic task contextual information. In scenarios with low information transmission efficiency and unclear objectives, nodes with high information entropy or attention diversity are often more representative than nodes with purely central structure, revealing the limitations of a single static metric.

[0141] The synergistic effect of key node identification and information augmentation. Tiecomm-IntraObs shows a significant performance improvement (+43.2%) in vision-constrained environments (such as Large_Coop), indicating that feature-level augmentation can effectively improve the signal-to-noise ratio in information-sparse environments, which is consistent with the reinforcement transmission mechanism of key decision-making information in human teams. Tiecomm-Hybrid performs best in large-scale, wide maps (such as Extreme) (final average cumulative reward of 153.1 rounds), demonstrating that the fusion of structural robustness and information value is crucial for long-range collaboration. However, in scenarios with relatively uniform information distribution (such as Large_Coop), further observation augmentation of KnoCoM-TieComm does not bring additional benefits, and is even slightly inferior to Tiecomm-Hybrid. This indicates that "more information augmentation is not always better": augmentation is effective in scenarios with highly concentrated information demand, but when information is uniformly distributed, excessive augmentation may introduce redundant information and learning costs, requiring adaptive adjustment of the augmentation intensity.

[0142] Boundary conditions for the collaborative module. KnoCoM-TieComm performs best in medium-complexity tasks such as Strategic_15 (89.6), but performs worse than Tiecomm-Hybrid in Large_Coop (72.9 vs. 82.0). This suggests that when the basic hybrid strategy can capture most of the key information, adding observational augmentations may increase model complexity and optimization difficulty, leading to performance fluctuations. This also reveals that the advantage of KnoCoM-TieComm lies in its potential for dynamic recognition and adaptive augmentation, rather than indiscriminate augmentation in all scenarios.

[0143] Robustness and Convergence. Robustness analysis (Table 4) shows that KnoCoM-TieComm has the lowest average coefficient of variation (13.58%) while maintaining the highest average cumulative reward per round (105.1), indicating that it has the best generalization ability and stability across scenarios and is suitable for deployment in real-world systems. In terms of convergence, most variants learn faster in the early stages, but in some scenarios (such as KnoCoM-TieComm in Strategic_15), although convergence is slightly slower, the final average cumulative reward per round is significantly higher, suggesting that adaptive step size or early stopping strategies can be combined in the future to alleviate local oscillations.

[0144] Hyperparameter sensitivity and limitations. Hyperparameter analysis ( Figure 8 and Figure 9The results show that the observation enhancement factor (ObsWeight) has a significant impact on convergence speed and final performance, requiring meticulous adjustment based on modules and tasks. The hybrid weight, however, remains stable within the range of 0.5–0.7, indicating that multi-dimensional scoring and structural importance are complementary, simplifying parameter tuning complexity. Currently, the weights for multi-dimensional scoring still require manual setting; future development could introduce meta-learning or adaptive mechanisms to dynamically change weights based on tasks and environments. Furthermore, this paper focuses on homogeneous agent scenarios. In heterogeneous or multi-task environments, key node identification requires adaptive allocation combining agent capabilities and roles. During large-scale deployments, computational and communication overhead will become a bottleneck, necessitating the exploration of sparse attention and hierarchical strategies.

[0145] A multi-robot collaborative handling system includes: The acquisition module is used to acquire input data of the multi-robot system. The input data includes local observation embeddings of each robot and a multi-robot relationship graph. The local observation embeddings are features generated by the robot's own position coordinates, current velocity vector, load capacity level, distance and direction of obstacles in the field of view, position and weight level of the target to be transported, and relative orientation of other robots. The multi-robot relationship graph is the initial communication topology graph between robots. The team division module is used to divide all robots into teams based on the multi-robot relationship graph, resulting in several sub-teams and the corresponding relationship sub-graphs of the sub-teams. The internal communication representation acquisition module is used to obtain the internal communication representation of each robot in each sub-team based on local observation embedding and relational subgraph; The calculation module is used to calculate the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength of each robot in each sub-team. The information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength are weighted and fused to obtain a comprehensive importance score. The comprehensive importance score is normalized and mapped through the softmax function to obtain the importance probability distribution. The screening module is used to calculate a dynamic threshold based on the statistical characteristics of the importance probability distribution, and to select a set of key robot nodes from the sub-team by combining a hybrid selection strategy of probability sampling and deterministic selection, wherein one robot is considered as a node. The enhancement module is used to set an adaptive enhancement factor for each node in the selected set of key robot nodes based on the comprehensive importance score, and to amplify the local observation embedding of the key nodes to obtain the enhanced local observation embedding. The weighting module is used to normalize and then weight and fuse the comprehensive importance score and the original structural strength score to obtain the final score of each robot. The robot with the highest final score is selected as the representative of the sub-team. The original structural strength score is calculated from the static topological features extracted from the multi-robot relationship graph. The team representation calculation module is used to perform weighted aggregation of the intra-team communication representations of each key robot node set based on the comprehensive importance score of the key robot node set if the set of key robot nodes selected from the sub-team is a non-empty set, so as to obtain the team representation of the sub-team; if the set of key robot nodes is an empty set, the intra-team communication representation represented by the sub-team is directly used as the team representation of the sub-team. The interaction module is used to input all team representations into the multi-head attention module, generate a global interaction message for each team, and broadcast the global interaction message to all members within the team as a representation for inter-team communication. The fusion module is used to splice together the enhanced local observation embedding, inter-team communication representation and intra-team communication representation of each robot to obtain the final decision representation of each robot. The final decision representation is input into the policy network and the robot's handling result is output. The handling result includes the robot's movement direction, movement position and handling cargo level.

[0146] This application also discloses a terminal device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, a multi-robot collaborative handling method is used.

[0147] The terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server. The terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may also include input / output devices, network access devices, and buses.

[0148] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0149] The memory can be an internal storage unit of the terminal device, such as a hard disk or RAM of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the terminal device. Furthermore, the memory can be a combination of internal storage units and external storage devices of the terminal device. The memory is used to store computer programs and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0150] In this terminal device, the multi-robot collaborative handling method described in the above embodiments is stored in the terminal device's memory and loaded and executed on the terminal device's processor for convenient use.

[0151] This application also discloses a computer-readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, it employs the multi-robot cooperative handling method described above.

[0152] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0153] The multi-robot collaborative handling method described in the above embodiments is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above method.

[0154] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0155] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. A multi-robot cooperative handling method, characterized in that it includes: The input data of the multi-robot system is obtained. The input data includes the local observation embedding of each robot and the multi-robot relationship graph. The local observation embedding is a feature generated by the robot's own position coordinates, current velocity vector, load capacity level, distance and direction of obstacles in the field of view, position and weight level of the target to be transported, and relative orientation of other robots. The multi-robot relationship graph is the initial communication topology graph between robots. Based on the multi-robot relationship graph, all robots are divided into teams to obtain several sub-teams and their corresponding relationship sub-graphs. For each robot within a subteam, a representation of intra-team communication is obtained based on local observation embeddings and relational subgraphs; Team communication is represented as: wherein, is a team internal communication, is a local observation embedding for the robot, is a multi-layer perceptron, is a mapping feature, is a sub-team is a corresponding relation subgraph, is a graph attention network, represents a first robot; For each subteam, the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength of each robot in the subteam are calculated separately. The information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength are weighted and fused to obtain a comprehensive importance score. The comprehensive importance score is normalized and mapped through the softmax function to obtain the importance probability distribution. Based on the statistical characteristics of the importance probability distribution, a dynamic threshold is calculated. Combining a hybrid selection strategy of probability sampling and deterministic selection, a set of key robot nodes is selected from the sub-team, where each robot is considered a node. For each node in the selected set of key robot nodes, an adaptive enhancement factor is set based on the comprehensive importance score to amplify the local observation embedding of the key node, resulting in an enhanced local observation embedding. Based on the comprehensive importance score and the original structural strength score, the scores are normalized and then weighted and fused to obtain the final score of each robot. The robot with the highest final score is selected as the representative of the sub-team. The original structural strength score is calculated from the static topological features extracted from the multi-robot relationship graph. If the set of key robot nodes selected from the sub-team is a non-empty set, then the intra-team communication representations of each key robot node set are weighted and aggregated based on the comprehensive importance score of the key robot node set to obtain the team representation of the sub-team; if the set of key robot nodes is an empty set, then the intra-team communication representation represented by the sub-team is directly used as the team representation of the sub-team. Input all team representations into the multi-head attention module to generate global interaction messages for each team, and broadcast the global interaction messages to all members within the team as their inter-team communication representation; The enhanced local observation embedding, inter-team communication representation, and intra-team communication representation of each robot are spliced ​​together to obtain the final decision representation of each robot. The final decision representation is input into the policy network to output the robot's handling result, which includes the robot's movement direction, movement position, and handling cargo level. The dynamic threshold is calculated based on the statistical characteristics of the importance probability distribution. A hybrid selection strategy combining probability sampling and deterministic selection is used to select a set of key robot nodes from within the sub-team, including: The mean and standard deviation are calculated based on the importance probability distribution of each robot node within the subteam and the number of robots. The importance probability distribution is composed of the importance probabilities of several robot nodes. The dynamic threshold is calculated based on the mean and the standard deviation. Robot nodes whose importance probability is greater than or equal to the dynamic threshold are selected as candidate nodes. Set the probability of a deterministic choice; Each time a key node is filtered, a list is generated. A random number that is uniformly distributed within an interval; If the random number is greater than the deterministic selection probability, then a deterministic selection method is used to select the candidate with the highest importance probability from the candidate set. A set of key robot nodes is defined as a set of robot nodes. If the random number is less than the deterministic selection probability, then a random sampling method is used, and multinomial sampling is performed based on the importance probability distribution of each robot node in the candidate set to extract... A set of key robot nodes is defined as a set of robot nodes. in, , The preset maximum critical node scaling factor. This represents the total number of robot nodes within the current sub-team. An adaptive enhancement factor is set based on the comprehensive importance score to amplify the local observation embedding of key nodes, resulting in enhanced local observation embedding, including: The adaptive enhancement factor is calculated based on the comprehensive importance score and is expressed as follows: in, As an adaptive enhancement factor, Basic enhancement factor, For the sigmoid function, A comprehensive importance score; The enhancement factor is multiplied element-wise with the local observation embedding to perform amplification, resulting in the enhanced local observation embedding, expressed as: in, To enhance local observation embedding, Embedded for local observation.

2. The multi-robot cooperative handling method according to claim 1, characterized in that, For each robot within a subteam, based on local observation embeddings and relational subgraphs, a team communication representation is obtained, including: The local observations of each robot within the subteam are embedded and feature-mapped using a multilayer perceptron to obtain the mapped features; The mapping features and relational subgraphs are input into the graph attention network to obtain the adaptive attention weights for each robot; Based on the mapping features and the adaptive attention weights of each robot, the intra-team communication representation of each robot is obtained.

3. The multi-robot cooperative handling method according to claim 1, characterized in that, For each sub-team, the information entropy metric, structure centrality metric, attention diversity metric, and robot intrinsic feature strength of each robot within the sub-team are calculated separately. These metrics are then weighted and fused to obtain a comprehensive importance score, including: Obtain the interaction matrix between robots; Based on the interaction matrix, the information entropy measure of each robot within the sub-team is calculated; Based on the interaction matrix, obtain the robot's in-degree, out-degree, and feature norm; The structural centrality measure is obtained by weighted summation of the in-degree, out-degree, and their characteristic norms. The attention diversity measure is calculated based on the information entropy measure. Obtain the inherent feature strength of the robot; The information entropy measure, structure centrality measure, attention diversity measure, and intrinsic feature strength are weighted and fused to obtain a comprehensive importance score.

4. The multi-robot cooperative handling method according to claim 1, characterized in that, The process involves normalizing and weighting the comprehensive importance score and the original structural strength score to obtain the final score for each robot. The robot with the highest final score is selected as the representative of the sub-team, including: Extract the number of robot edges and edge strength features from the multi-robot relationship graph; The original structural strength score is calculated based on the number of robot edge connections and the edge strength characteristics. The overall importance score and the original structural strength score are normalized within the team to obtain the team's overall importance score and the team's original structural strength score. The team's overall importance score and the team's original structural strength score are weighted and fused to obtain the final score for each robot. The robot with the highest final score is selected as the representative of the sub-team.

5. The multi-robot cooperative handling method according to claim 1, characterized in that, The enhanced local observation embedding, inter-team communication representation, and intra-team communication representation of each robot are concatenated to obtain the final decision representation of each robot, as follows: in, To indicate the final decision, To enhance local observation embedding, For inter-team communication, This refers to communication within the team.

6. A multi-robot cooperative handling system, characterized in that it comprises: The acquisition module is used to acquire input data of the multi-robot system. The input data includes local observation embeddings of each robot and a multi-robot relationship graph. The local observation embeddings are features generated by the robot's own position coordinates, current velocity vector, load capacity level, distance and direction of obstacles in the field of view, position and weight level of the target to be transported, and relative orientation of other robots. The multi-robot relationship graph is the initial communication topology graph between robots. The team division module is used to divide all robots into teams based on the multi-robot relationship graph, resulting in several sub-teams and the corresponding relationship sub-graphs of the sub-teams. The internal communication representation acquisition module is used to obtain the internal communication representation of each robot in each sub-team based on local observation embedding and relational subgraph; Team communication is represented as: in, For team communication, Embedded for localized observation of the robot, It is a multilayer perceptron. For mapping features, For the team The corresponding relationship subgraph, For graph attention networks, Indicates the first One robot; The calculation module is used to calculate the information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength of each robot in each sub-team. The information entropy measure, structure centrality measure, attention diversity measure, and robot intrinsic feature strength are weighted and fused to obtain a comprehensive importance score. The comprehensive importance score is normalized and mapped through the softmax function to obtain the importance probability distribution. The screening module is used to calculate a dynamic threshold based on the statistical characteristics of the importance probability distribution, and to select a set of key robot nodes from the sub-team by combining a hybrid selection strategy of probability sampling and deterministic selection, wherein one robot is considered as a node. The enhancement module is used to set an adaptive enhancement factor for each node in the selected set of key robot nodes based on the comprehensive importance score, and to amplify the local observation embedding of the key nodes to obtain the enhanced local observation embedding. The weighting module is used to normalize and then weight and fuse the comprehensive importance score and the original structural strength score to obtain the final score of each robot. The robot with the highest final score is selected as the representative of the sub-team. The original structural strength score is calculated from the static topological features extracted from the multi-robot relationship graph. The team representation calculation module is used to perform weighted aggregation of the intra-team communication representations of each key robot node set based on the comprehensive importance score of the key robot node set if the set of key robot nodes selected from the sub-team is a non-empty set, so as to obtain the team representation of the sub-team; if the set of key robot nodes is an empty set, the intra-team communication representation represented by the sub-team is directly used as the team representation of the sub-team. The interaction module is used to input all team representations into the multi-head attention module, generate a global interaction message for each team, and broadcast the global interaction message to all members within the team as a representation for inter-team communication. The fusion module is used to splice together the enhanced local observation embedding, inter-team communication representation and intra-team communication representation of each robot to obtain the final decision representation of each robot. The final decision representation is input into the policy network and the robot's handling result is output. The handling result includes the robot's movement direction, movement position and handling cargo level. The dynamic threshold is calculated based on the statistical characteristics of the importance probability distribution. A hybrid selection strategy combining probability sampling and deterministic selection is used to select a set of key robot nodes from within the sub-team, including: The mean and standard deviation are calculated based on the importance probability distribution of each robot node within the subteam and the number of robots. The importance probability distribution is composed of the importance probabilities of several robot nodes. The dynamic threshold is calculated based on the mean and the standard deviation. Robot nodes whose importance probability is greater than or equal to the dynamic threshold are selected as candidate nodes. Set the probability of a deterministic choice; Each time a key node is filtered, a list is generated. A random number that is uniformly distributed within an interval; If the random number is greater than the deterministic selection probability, then a deterministic selection method is used to select the candidate with the highest importance probability from the candidate set. A set of key robot nodes is defined as a set of robot nodes. If the random number is less than the deterministic selection probability, then a random sampling method is used, and multinomial sampling is performed based on the importance probability distribution of each robot node in the candidate set to extract... A set of key robot nodes is defined as a set of robot nodes. in, , The preset maximum critical node scaling factor. This represents the total number of robot nodes within the current sub-team. An adaptive enhancement factor is set based on the comprehensive importance score to amplify the local observation embedding of key nodes, resulting in enhanced local observation embedding, including: The adaptive enhancement factor is calculated based on the comprehensive importance score and is expressed as follows: in, As an adaptive enhancement factor, Basic enhancement factor, For the sigmoid function, A comprehensive importance score; The enhancement factor is multiplied element-wise with the local observation embedding to perform amplification, resulting in the enhanced local observation embedding, expressed as: in, To enhance local observation embedding, Embedded for local observation.

7. A terminal device, comprising a memory and a processor, characterized in that, The memory stores a computer program that can run on a processor, and when the processor loads and executes the computer program, it employs the method described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the method described in any one of claims 1 to 5.

Images