Multi-robot communication and computational resource allocation methods for embodied collaborative tasks
By combining Markov decision processes and graph convolutional networks with a proximal policy optimization algorithm, a heterogeneous graph model is constructed, which solves the problem of improper resource allocation in multi-robot systems and achieves dynamic resource matching and improved stability of collaborative tasks.
Patent Information
- Application Number
- CN202610862720.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-15
- Publication Date
- 2026-11-17
AI Technical Summary
Existing resource allocation methods fail to effectively combine robot embodied interaction states and task semantics, leading to communication congestion and collaborative task failures, especially in the case of improper resource allocation in complex dynamic environments.
By combining Markov decision processes and graph convolutional networks (GCNs) with the proximal policy optimization (PPO) algorithm, a heterogeneous graph model is constructed to represent the robot's embodied interaction state and task relationships in real time, thereby achieving dynamic matching of resource allocation and optimizing communication bandwidth and edge computing power allocation.
It improves resource utilization efficiency and collaborative task stability in complex and dynamic environments, and can dynamically adjust resource allocation according to the embodied interaction state, prioritizing the resource needs of highly interactive and collaborative tasks.
Smart Images

Figure CN123123348A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of embodied intelligence and communication computing resource management technology, specifically relating to a multi-robot resource allocation method driven by task semantics and embodied collaborative perception. Background Technology
[0002] With the rapid development of artificial intelligence technology, multi-robot systems are evolving from automated operations in fixed scenarios to autonomous collaboration in complex dynamic environments, finding widespread application in warehousing and logistics, industrial manufacturing, disaster relief, and other fields. Embossed intelligent robots not only need to perform physical actions but also continuously perceive the environment, coordinate actions, and interact with control during tasks to maintain stable operation. In multi-robot collaborative scenarios, robots need to exchange sensory information, control commands, and task status data in real time, placing high demands on wireless communication and edge computing resources. However, both wireless spectrum resources and edge server computing power are limited. When multiple robots simultaneously upload high-dimensional sensory data or perform collaborative control tasks, communication congestion and increased task queuing delays can easily occur, thus affecting the stability of collaborative task execution.
[0003] Existing resource allocation methods still have two limitations. First, they typically focus only on network status and link quality, lacking awareness of the physical interaction characteristics of embodied intelligent tasks and ignoring the differences in action synchronization, environmental interaction, and collaborative control requirements among different tasks. Second, they neglect the behavioral coupling relationships between robots. In highly collaborative tasks such as collaborative handling and precision assembly, multiple robots need to continuously maintain action synchronization and state consistency. If a node experiences communication or insufficient computing power, the entire collaborative task can easily fail. Therefore, how to comprehensively consider the embodied interaction state of robots, the semantic value of the task, and behavioral coupling relationships during resource allocation to achieve dynamic resource optimization for embodied collaborative tasks is a pressing problem that needs to be solved. Summary of the Invention
[0004] The technical problem this invention aims to solve is to provide a method for multi-robot communication and computing resource allocation for embodied collaborative tasks, addressing the shortcomings of prior art. This method constructs a heterogeneous graph of multi-robot clusters and edge servers, representing in real-time the embodied interaction states, task interaction relationships, communication link states, and edge computing load of each robot. This invention not only optimizes resources based on traditional network states but also combines the embodied interaction behaviors and collaborative relationships of robots in the physical environment to achieve dynamic matching between resource allocation and embodied task requirements. Furthermore, it models the system task utility maximization problem as a Markov Decision Process (MDP) and proposes a Graph Convolutional Network-Proximal Policy Optimization (GCN-PPO) algorithm. GCN is used to extract multi-robot collaborative relationships and embodied interaction state features, while PPO is used to achieve adaptive optimization allocation of communication bandwidth and edge computing resources.
[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A multi-robot resource allocation method driven by task semantics and embodied collaborative perception includes the following steps: S1: The multi-robot collaborative operation system is modeled as a Markov decision process, and a graph convolutional network is used as a feature extraction module. By recursively aggregating the node and edge features in the heterogeneous graph, the resource sensitivity and collaborative priority of each robot task to environmental interaction feedback are dynamically evaluated, providing an embodied collaborative state representation for reinforcement learning decision-making. The heterogeneous graph has robots and edge servers as nodes and communication links and computation offloading links as edges. The state representation includes robot spatial position, motion speed, motion state, task type, and environmental interaction state. S2: The graph-level features extracted by the graph convolutional network are input into the Actor-Critic architecture of the near-end policy optimization algorithm. The near-end policy optimization algorithm makes resource decisions and outputs resource allocation actions to achieve a tilted allocation of communication bandwidth and edge computing resources towards tasks with high interaction sensitivity and strong cooperative coupling. The dynamic priority of resource allocation is obtained by the graph convolutional network based on the robot's embodied interaction state, task action features, and cooperative behavior relationships. S3: Obtain immediate rewards based on the resource allocation actions, store the interaction samples in the same policy experience cache, calculate the advantage function using generalized advantage estimation, update the parameters of the Actor network and Critic network through the truncated objective function of the near-end policy optimization, and iterate training until the policy converges and stabilizes.
[0006] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied collaborative perception of the present invention, the state space of the Markov decision model in step S1 is defined as the real-time features of the nodes and edges in the heterogeneous graph. The state characteristics of the robot node include: three-dimensional position, current motion speed, current motion state, current task type, and remaining battery power; The status characteristics of edge server nodes include: available computing power and task queue length; The state characteristics of the communication edges include: the channel quality between robots and the degree of behavioral coupling; The state characteristics of the offloading edge include: uplink channel quality and offloading edge task data volume; The action space of the Markov decision model is defined as the communication bandwidth allocation ratio and edge computing power allocation ratio of each robot, and the sum of the bandwidth allocation ratio and the sum of the computing power allocation ratio of all robots are constrained to the range of [0,1]. The reward function of the Markov decision model It consists of task completion utility, behavior coupling reward, and system energy consumption, in order to enhance the system's resource guarantee capability for highly collaborative tasks.
[0007] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied collaborative perception of the present invention, in step S2, the dynamic resource priority is generated in the following way: after message passing through a multi-layer graph convolutional network, the final feature vector of each robot node is input into a fully connected mapping layer and processed by a softmax normalization function to obtain the dynamic resource priority of the robot in the current time slot. This priority represents the robot's sensitivity to resources and the priority of resource allocation.
[0008] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied cooperative perception in this invention, in step S2, the graph convolutional network serves as the front-end feature extraction module of the near-end policy optimization algorithm, embedded in the decision-making process of the Actor-Critic network; the graph convolutional network models the cooperative relationship between nodes and edges through a message passing mechanism, and its single-layer graph convolution calculation process includes: Edge feature-assisted update: Update the edge features based on the current layer node features and edge features connecting two nodes; Neighborhood edge feature aggregation: Aggregate the updated features of all adjacent edges of each node to obtain a neighborhood information representation vector; Node state update: Based on the node's current layer features, the aggregated neighborhood information representation vector, and the learnable weight matrix, a new feature vector of the node is obtained after processing by an activation function.
[0009] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied cooperative perception of the present invention, in step S1, the resource sensitivity and cooperative priority on which the embodied cooperative state representation is based are realized by predefining embodied cooperative semantic values for different task types in the system; wherein, key tasks are assigned higher embodied cooperative semantic values, and regular tasks are assigned lower embodied cooperative semantic values; the key tasks include cooperative handling and precision assembly, and the regular tasks include environmental patrol and standby.
[0010] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied collaborative perception of the present invention, the specific method for calculating the advantage function using the generalized advantage estimation method in step S3 is as follows: First, calculate the timing difference error for each time slot; Then, based on the timing difference error and the attenuation factor estimated by the generalized dominance, the dominance function value of the current time slot is obtained by summing them up. The truncation objective function for near-end policy optimization constrains the policy update magnitude by limiting the probability ratio between the old and new policies. The loss function of the Critic network is constructed based on the mean square error between the cumulative discount reward target value and the state value estimate output by the Critic network.
[0011] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied collaborative perception of the present invention, in step S1, the robot movement in the multi-robot collaborative operation system follows a Gaussian-Markov model, and the updates of its speed and direction depend on the average value of past states, the memory degree coefficient, and Gaussian random variables; the channel gain between robots and between robots and edge servers is modeled using a path loss model.
[0012] As a further preferred embodiment of the multi-robot resource allocation method driven by task semantics and embodied collaborative perception of the present invention, in step S3, feature extraction, action generation, reward feedback and network parameter update are uniformly incorporated into the reinforcement learning closed-loop training process to achieve dynamic optimization of communication and computing resources of the embodied intelligent multi-robot system.
[0013] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects: 1. This invention proposes a multi-robot communication and computing resource allocation method for embodied intelligent collaborative tasks; the method integrates the graph structure modeling capability of GCN and the reinforcement learning decision-making capability of PPO, aiming to maximize the system task utility under the constraints of limited bandwidth and computing power; 2. This invention can combine the robot's embodied interaction state and collaborative behavior relationship in the physical environment to achieve dynamic matching between resource allocation and task requirements; by introducing an embodied collaborative semantic value and behavior coupling mechanism, the system can identify the sensitivity of different tasks to resources based on the robot's real-time embodied interaction state and prioritize the resource requirements of high-interaction and high-collaboration tasks. 3. This invention introduces a heterogeneous graph modeling approach; relying on the features of GCN recursively aggregated nodes and edges, it achieves unified modeling of multi-robot collaborative relationships, communication link status, and edge computing power status, which can adapt to different robot network structures.
[0014] 4. This invention introduces the GCN-PPO joint optimization framework, which combines graph structure feature extraction capability with the sequential decision-making capability of reinforcement learning. It continuously optimizes resource allocation strategy through closed-loop iterative training. With the help of GCN, the robot's embodied interaction state, behavioral coupling degree and communication state characteristics are transmitted. Based on this, PPO makes dynamic resource decisions, thereby improving resource utilization efficiency and collaborative task stability in complex dynamic scenarios. Attached Figure Description
[0015] Figure 1 For the application scenario of this invention, it consists of a robot execution layer and an edge server layer; Figure 2 Here is the flowchart for the GCN-PPO algorithm; Figure 3 This is a flowchart of the present invention; Figure 4 This paper compares the convergence performance of the GCN-PPO algorithm used in this invention with that of the SAC algorithm, the PPO-MLP algorithm, and the average allocation algorithm under the system model of this invention. Detailed Implementation
[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings: The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention. Detailed Implementation
[0017] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. The preferred embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.
[0018] This invention proposes a multi-robot resource allocation method driven by task semantics and embodied collaborative perception. In this multi-robot collaborative operation scenario, a heterogeneous graph is constructed with robots and edge servers as nodes and communication links and computation offloading links as edges.
[0019] In this scenario, each robot is equipped with a communication module, a perception module, and local computing capabilities to perform physical tasks and interact with the environment. Different tasks have different embodied collaborative semantic values. Among them, tasks such as collaborative handling and precision assembly are more sensitive to communication latency and computing power continuity due to continuous motion synchronization and high-frequency environmental interaction; independent tasks such as environmental patrol and standby have a higher tolerance for resource fluctuations. The task cycle is divided into multiple time slots of equal size, within which the robot executes the corresponding task. A single task can be completed by one or more robots, with tasks executed collaboratively by multiple robots exhibiting a higher degree of behavioral coupling.
[0020] The method specifically includes the following steps: (1) To address the dual limitations of communication bandwidth and edge computing power, and the dynamic changes in embodied collaborative relationships, the resource allocation problem is modeled as a Markov decision process. To enable resource scheduling to perceive the robot's embodied interaction state, this invention introduces embodied feature information such as robot spatial location, movement speed, action motion state, task type, and environmental interaction state on the basis of traditional network state. Based on this, GCN is introduced as a feature extraction module. By recursively aggregating node and edge features, the resource sensitivity and collaborative priority of the robot task to environmental interaction feedback are dynamically evaluated, providing an embodied collaborative state representation for reinforcement learning decision-making.
[0021] (2) Based on the graph-level features extracted by GCN in step (1), the graph features extracted by GCN are input into the Actor-Critic architecture, and the PPO algorithm is used to make resource decisions to achieve the tilting allocation of resources to tasks with high interaction sensitivity and strong collaborative coupling. The dynamic resource priority is jointly learned by GCN based on the robot's embodied interaction state, task action features and collaborative behavior relationship. The higher the value, the higher the resource tilt given by the system in the resource allocation process.
[0022] (3) Based on the resource allocation action output in step (2) and obtain the immediate reward, store the interaction sample in the same policy experience cache, use generalized advantage estimation (GAE) to calculate the advantage function, update the parameters of the Actor network and Critic network by truncating the objective function through PPO, and iterate until the policy converges and stabilizes.
[0023] The state space of the Markov decision model in step (1) Defined as real-time features of heterogeneous graph nodes and edges, where robot state includes position, velocity, action, task type, and remaining battery power; edge server state includes available computing power and task queue length; communication edge state includes link quality and behavioral coupling; and unloading edge state includes unloaded data volume and uplink status. Action Space Defined as the ratio of communication bandwidth to edge computing power for the robot (the sum of bandwidth ratios and computing power ratios are both constrained within the range of [0,1]); reward function It consists of task completion utility, behavior coupling reward, and system energy consumption. By introducing embodied collaborative reward items, the system's resource guarantee capability for highly collaborative tasks is enhanced.
[0024] In step (2), GCN serves as the front-end feature extraction module of PPO, embedding the decision-making process of the Actor-Critic network. It models the collaborative relationship between nodes and edges through a message passing mechanism. The network weight parameters of GCN, Actor network and Critic network are randomly initialized, and the network parameters are continuously updated through multiple rounds of environmental interaction sampling and gradient backpropagation. Under the constraints of communication bandwidth and edge computing power, the strategy gradually converges to the optimal.
[0025] The feature extraction, action generation, reward feedback and network parameter update in step (3) are uniformly incorporated into the reinforcement learning closed-loop training process to achieve dynamic optimization of communication and computing resources of the embodied intelligent multi-robot system.
[0026] Example: 1. System Model: like Figure 1 As shown, the multi-robot collaborative operation system consists of A robot and It consists of several edge servers. Each robot executes a corresponding physical task in each time slot and completes real-time interaction of sensing information, control commands, and collaborative status through a wireless network. For ease of calculation, the entire task duration is... Divided into equal intervals There are 1 time slot, and the length of each time slot is 1. The positions of the robot and the edge server are modeled using a three-dimensional Cartesian coordinate system. In the... In the first time slot, the first The position of each robot is represented as ,in For a fixed height; since the edge server location is fixed and independent of time, the first... The location of each edge server is represented as follows: The robot's movement follows a Gauss-Markov model, and its speed... and direction The update formula is:
[0027]
[0028]
[0029] in, and They represent robots Average speed and direction, and This is the memory index, reflecting the degree of dependence on past states in the current state. , Let be a random variable that follows a Gaussian distribution.
[0030] Based on the above velocity and direction update equations, the robot's position update formula is:
[0031]
[0032] The channel gain between robots and between robots and edge servers uses a path loss model:
[0033]
[0034] in, For the first Time-slot robot With robots The distance between them For the first Time-slot robot and edge servers The distance between them The channel gain at the reference distance. This is the path loss index.
[0035] 2. Embodied Collaborative Semantic Value and Behavioral Coupling Model: To allocate resources more favorably to tasks with high sensitivity to embodied interactions and strong collaborative behaviors, the system performs an embodied collaborative semantic value assessment on each robot's current task. Assume the system has... For each task type, the first... The embodied collaborative semantic value of a task class is defined as:
[0036]
[0037] Among them, critical tasks (such as handling fragile items and precision assembly) are given higher embodied collaborative semantic value, while routine tasks (such as environmental patrol and standby) are given lower embodied collaborative semantic value.
[0038] To further characterize the embodied cooperative behavior relationships between robots, this invention introduces dynamic behavioral coupling degree:
[0039] in, robot With robots Spatial distance, This is a normalization factor for the frequency of robot action interactions per unit time. The closer the robots are, the more frequent their interactions, and the stronger the need for action synchronization, the higher the degree of behavioral coupling. The task consistency factor is defined as follows:
[0040]
[0041] This dynamic behavior coupling mechanism can continuously characterize the collaborative relationships in embodied intelligence scenarios and provide support for subsequent message propagation and priority learning in graph convolutional networks.
[0042] 0. Markov Decision Process: To solve the joint resource optimization problem in a dynamic environment, the system is modeled as a Markov decision process, using a quadruple. Composition. Because channel states, task generation, and robot movement are all random, the state transition probability... Since it is difficult to model accurately, a model-free reinforcement learning method is used to solve the problem.
[0043] state space In time slots ,state Defined as real-time features of nodes and edges in a heterogeneous graph. Includes the 3D positions of each robot. Current motion speed status Current motion state Current task type and remaining battery power The characteristics of an edge server node include available computing power. With task queue length The characteristics of the communication edge include the channel quality between robots. and behavioral coupling The characteristics of the offloading side include uplink channel quality. The amount of data related to the unloading side task.
[0044] Action space :action Defined as the ratio of communication bandwidth allocated to each robot to the ratio of edge computing power. It satisfies the following constraints: , , ,
[0045] reward function The goal is to maximize task semantic energy efficiency and ensure the physical stability of embodied collaboration. The reward function is defined as the overall utility of the system task, minus the system communication and computational energy consumption, and additionally introduces an embodied collaboration reward term based on behavioral coupling degree.
[0046]
[0047] in, The dynamic resource priority, learned in real time by GCN, is used to characterize the robot's resource sensitivity and resource allocation priority in the current embodied collaborative task. The task completion quality coefficient (1 for complete, discounted value for partial completion, 0 for failure). This is the energy consumption penalty coefficient. For time slots Communication energy consumption For time slots Computational energy consumption, This is used for collaborative reward weighting. When robots have a higher behavioral coupling relationship and the corresponding tasks have a higher dynamic resource priority, the system encourages resources to be tilted towards collaborative tasks with high embodied value through collaborative reward items, thereby improving the resource guarantee capability of collaborative tasks.
[0048] This reward function not only optimizes the efficiency of communication and computing resource utilization, but also enhances the system's resource guarantee capability for highly embodied collaborative tasks.
[0049] 4. State representation based on heterogeneous graphs: A graph convolutional network is used to perform a structured representation of robot nodes, edge server nodes, and link relationships.
[0050] In this model, the constructed heterogeneous graph uses nodes to represent individual robots and edge servers, while edges represent communication links and computation offloading links. The required policy input for the model is... It is a state The graph-structured representation consists of the following features: robot node features Edge server node characteristics Communication edge characteristics and unloading edge features Therefore, the state can be defined as:
[0051]
[0052] Node features and edge feature vectors are further defined as follows:
[0053] The aforementioned heterogeneous graph can simultaneously characterize the robot's embodied interaction state, cooperative behavior relationships, and edge server resource status.
[0054] 5. GCN-PPO Joint Optimization Architecture: To achieve dynamic resource optimization in embodied intelligence scenarios, this invention proposes a GCN-PPO joint optimization framework.
[0055] 1) GCN Feature Extraction Mechanism: Based on traditional GCN, an edge feature-assisted update mechanism is introduced. Its computation process includes three stages: edge feature-assisted update, neighborhood aggregation, and node state update. In the... During layer graph convolution, the formula for updating edge features is:
[0056]
[0057] in, For the connection nodes before the update With nodes The edge feature vectors, and They are nodes and nodes In the The feature vector of the layer, For edge update function, This is the updated edge feature vector.
[0058] Subsequently, the domain edge features are aggregated:
[0059] in, For nodes The set of neighboring nodes, The number of neighboring nodes. For nodes The neighborhood information representation vector after aggregating all neighborhood edge features.
[0060]
[0061] in, For the first The learnable weight matrix of a layered graph convolutional network. For activation function, For nodes After the first The new feature vectors are updated after each layer. After multiple layers of message passing, GCN is able to learn the behavioral coupling relationships, spatial interaction relationships, differences in action synchronization, and differences in resource sensitivity between robots.
[0062] Finally, after After the GCN layer, the first The dynamic resource priority of each robot node can be defined using a fully connected mapping layer and a softmax normalization function:
[0063] in, This indicates the robot's dynamic resource priority.
[0064] Subsequently, global average pooling is performed on the features of all robot nodes to obtain global graph-level features. :
[0065] in, Represents the set of robot nodes. This indicates the number of robot nodes.
[0066] This graph-level feature integrates information on the overall system topology and collaborative relationships, and serves as a shared input for the Actor network and the Critic network.
[0067] 2) Proximal strategy optimization decision: Graph-level features extracted by GCN Input the PPO's Actor-Critic architecture. The Actor network receives... After processing by the fully connected layer, the mean vector of the output action distribution is obtained. and standard deviation vector For each robot By sampling from its corresponding Gaussian distribution, the robot resource allocation action is obtained:
[0068]
[0069] in, This indicates the proportion of bandwidth allocated for robot communication. This indicates the proportion of computing power allocated to the edge of the computer.
[0070] If the sampling result exceeds the [0,1] interval, it is truncated. If the sum of the segmentation ratios of all robots exceeds 1, it is scaled proportionally to meet the constraints.
[0071] Critic networks receive the same graph-level features After processing by the fully connected layer, the value estimate of the current state is output. This is used for the subsequent calculation of the advantage function.
[0072] 3) Strategy Update and Iterative Optimization: After the system executes resource allocation actions, it will receive immediate reward feedback. The generalized advantage estimation method is used to calculate the advantage function.
[0073] in, For time slots The timing difference error, This is a discount factor used to balance the weights of current rewards and future long-term rewards. These are the parameters of the Critic network.
[0074] The advantage function is defined as:
[0075] in, This is the GAE attenuation factor, used to adjust the bias and variance.
[0076] PPO updates the policy magnitude by truncating the objective function. Its Actor network's optimization objective is defined as:
[0077]
[0078] in, The probability ratio between the old and new strategies. For Actor network parameters, The Actor network parameters before the update. This is for truncating parameters.
[0079] In the Critic network, the loss function is defined as:
[0080] in, To from time slot The initial cumulative discount return target value is defined as follows:
[0081] in, This represents an immediate reward for future time slots. The cumulative discounted reward is used to measure the overall utility of the current state in terms of task completion benefits, embodied synergy benefits, and resource consumption under long-term policy execution, and serves as a supervision objective for the Critic network to improve the accuracy of state value estimation.
[0082] The Actor network and Critic network are jointly optimized, and the network parameters are iteratively updated until the policy converges and stabilizes.
[0083] The GCN-PPO algorithm proposed in this invention realizes dynamic closed-loop linkage between the robot's embodied interaction state and the resource scheduling process. For example... Figure 4 As shown, compared with the average allocation strategy and the PPO method based on fully connected networks, this invention exhibits superior performance in terms of strategy convergence speed, task utility, and resource utilization efficiency. This method can dynamically adjust the resource allocation strategy based on changes in robot motion state, behavioral coupling relationships, and environmental feedback during task execution, prioritizing resource requirements for tasks with high embodied collaborative semantic value, thereby improving the execution stability and resource utilization efficiency of embodied intelligent multi-robot collaborative tasks in complex dynamic environments.
[0084] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention. All technical features in this embodiment can be freely combined according to actual needs.
[0085] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A multi-robot resource allocation method driven by task semantics and embodied collaborative perception, characterized in that, Includes the following steps: S1: The multi-robot collaborative operation system is modeled as a Markov decision process, and a graph convolutional network is used as a feature extraction module. By recursively aggregating the node and edge features in the heterogeneous graph, the resource sensitivity and collaborative priority of each robot task to environmental interaction feedback are dynamically evaluated, providing an embodied collaborative state representation for reinforcement learning decision-making. The heterogeneous graph has robots and edge servers as nodes and communication links and computation offloading links as edges. The state representation includes robot spatial position, motion speed, motion state, task type, and environmental interaction state. S2: The graph-level features extracted by the graph convolutional network are input into the Actor-Critic architecture of the near-end policy optimization algorithm. The near-end policy optimization algorithm makes resource decisions and outputs resource allocation actions to achieve a tilted allocation of communication bandwidth and edge computing resources towards tasks with high interaction sensitivity and strong cooperative coupling. The dynamic priority of resource allocation is obtained by the graph convolutional network based on the robot's embodied interaction state, task action features, and cooperative behavior relationships. S3: Obtain immediate rewards based on the resource allocation actions, store the interaction samples in the same policy experience cache, calculate the advantage function using generalized advantage estimation, update the parameters of the Actor network and Critic network through the truncated objective function of the near-end policy optimization, and iterate training until the policy converges and stabilizes.
2. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 1, characterized in that, The state space of the Markov decision model in step S1 is defined as the real-time features of the nodes and edges in the heterogeneous graph. The state characteristics of the robot node include: three-dimensional position, current motion speed, current motion state, current task type, and remaining battery power; The status characteristics of edge server nodes include: available computing power and task queue length; The state characteristics of the communication edges include: the channel quality between robots and the degree of behavioral coupling; The state characteristics of the offloading edge include: uplink channel quality and offloading edge task data volume; The action space of the Markov decision model is defined as the communication bandwidth allocation ratio and edge computing power allocation ratio of each robot, and the sum of the bandwidth allocation ratio and the sum of the computing power allocation ratio of all robots are constrained to the range of [0,1]. The reward function of the Markov decision model It consists of task completion utility, behavior coupling reward, and system energy consumption, in order to enhance the system's resource guarantee capability for highly collaborative tasks.
3. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 2, characterized in that, In step S2, the dynamic resource priority is generated in the following way: after message passing through a multi-layer graph convolutional network, the final feature vector of each robot node is input into a fully connected mapping layer and processed by a softmax normalization function to obtain the dynamic resource priority of the robot in the current time slot. This priority represents the robot's sensitivity to resources and the priority of resource allocation.
4. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 1, characterized in that, In step S2, the graph convolutional network serves as the front-end feature extraction module of the near-end policy optimization algorithm, embedded in the decision-making process of the Actor-Critic network. The graph convolutional network models the collaborative relationships between nodes and edges through a message passing mechanism, and its single-layer graph convolution calculation process includes: Edge feature-assisted update: Update the edge features based on the current layer node features and edge features connecting two nodes; Neighborhood edge feature aggregation: Aggregate the updated features of all adjacent edges of each node to obtain a neighborhood information representation vector; Node state update: Based on the node's current layer features, the aggregated neighborhood information representation vector, and the learnable weight matrix, a new feature vector of the node is obtained after processing by an activation function.
5. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 1, characterized in that, In step S1, the resource sensitivity and collaboration priority on which the embodied collaboration state representation is based are achieved by predefining embodied collaboration semantic values for different task types in the system; wherein, key tasks are assigned higher embodied collaboration semantic values, and routine tasks are assigned lower embodied collaboration semantic values; the key tasks include collaborative handling and precision assembly, and the routine tasks include environmental patrol and standby.
6. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 1, characterized in that, In step S3, the specific method for calculating the advantage function using the generalized advantage estimation method is as follows: First, calculate the timing difference error for each time slot; Then, based on the timing difference error and the attenuation factor estimated by the generalized dominance, the dominance function value of the current time slot is obtained by summing them up. The truncation objective function for near-end policy optimization constrains the policy update magnitude by limiting the probability ratio between the old and new policies. The loss function of the Critic network is constructed based on the mean square error between the cumulative discount reward target value and the state value estimate output by the Critic network.
7. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 1, characterized in that, In step S1, the robot movement in the multi-robot collaborative operation system follows a Gaussian-Markov model, and the updates of its speed and direction depend on the average value of past states, the memory coefficient, and Gaussian random variables; the channel gain between robots and between robots and edge servers is modeled using a path loss model.
8. The multi-robot resource allocation method driven by task semantics and embodied collaborative perception according to claim 1, characterized in that, In step S3, feature extraction, action generation, reward feedback, and network parameter updates are uniformly incorporated into the reinforcement learning closed-loop training process to achieve dynamic optimization of communication and computing resources of the embodied intelligent multi-robot system.
