Multi-robot optimal command assignment and affine formation overall obstacle avoidance navigation method based on 5g communication

Abstract

The application discloses a kind of multi-robot optimal instruction distribution and affine formation overall obstacle avoidance navigation method based on 5G communication, method is adapted to the three-layer hardware architecture of cloud edge, including cloud server end, edge server end, robot end, and by four algorithm modules respectively deployed on it, including senior task instruction issuing module deployed on cloud server, multi-robot optimal formation configuration distribution module and distributed reinforcement learning navigation control module etc. deployed on edge server;The application thus efficiently completes the transmission of user senior formation instruction from cloud to edge to robot, and then enables multi-robot system to efficiently complete the formation task required by high-end instruction without commander intervention;Can avoid the cumbersome of user cloud scheduling instruction in the whole process, reduce the tension degree of user command, solve the dimension disaster and poor scalability in centralized scheme and other problems.

Description

Technical Field

[0001] This invention relates to the field of multi-robot overall obstacle avoidance formation navigation, specifically, a multi-robot optimal command dispatch and affine formation overall obstacle avoidance navigation method based on 5G communication. Background Technology

[0002] In recent years, multi-robot collaboration and interaction technologies have seen tremendous development, becoming a crucial area of ​​robotics research. Compared to single robots, multi-robot systems are equipped with more sensing devices, possess a wider sensing range, and offer flexible and varied topologies, all of which enhance their task execution capabilities. Furthermore, multi-robot systems exhibit a degree of robustness; if a robot malfunctions due to unforeseen circumstances during normal operation, the system can internally coordinate to temporarily replace the faulty robot and continue the task. To fully realize the potential of multi-robot systems, a reliable communication architecture and an efficient unified scheduling algorithm framework are essential.

[0003] The 5G-based cloud-edge-device architecture perfectly aligns with the communication architecture requirements of multi-robot systems. Users issue commands in the cloud, the edge provides algorithmic computing power, and the robots execute the underlying commands. The key challenge lies in designing an efficient, unified scheduling algorithm framework for this architecture. The core issue is how to rationally distribute task responsibilities across the three platforms to leverage their respective strengths and mitigate their weaknesses. Specifically, for the cloud server, which directly interacts with the user, the key challenge is avoiding cumbersome cloud commands and reducing user workload, allowing them to focus on high-level decision-making. For the middle-layer edge server, the key challenge is using algorithms to optimally distribute macro-level task commands from the upper layers to specific tasks on the robots based on their capabilities. For the bottom-layer robot, the key challenge is perfectly executing the specific tasks assigned from the edge server locally.

[0004] On the other hand, formation navigation is an essential foundational task for multi-robot systems. Considering the overall obstacle avoidance problem within a multi-robot formation is crucial for scenarios such as collaborative transport, search and rescue, and collaborative observation and mutual localization. The primary constraint is that obstacles are not allowed to cross the overall formation topology. Since the robots are not rigidly connected, overall obstacle avoidance can be achieved by adjusting the formation topology. Therefore, this task is essentially a multi-robot formation navigation planning task that achieves overall obstacle avoidance through adaptive formation adjustment. The challenges of this problem are: 1) The entire formation must avoid obstacles as a whole, rather than individually. A centralized planning method is more suitable, turning it into an individual obstacle avoidance problem rather than a distributed one. This facilitates the aggregation of local perception information using 5G communication and naturally avoids collisions between robots. 2) Considering that if the design of the policy action space directly determines the actions of each robot node, the dimension of the action space will be at least 2*n (where n is the number of robot nodes), resulting in a huge solution space. Therefore, we consider using the concept of affine formation to model the problem, transforming the original individual robot planning into an affine transformation parameter planning problem for the entire formation relative to a nominal formation template, reducing the degrees of freedom to 6. 3) Affine transformation parameters are only meaningful based on a specified nominal formation template, so we must consider the algorithm's scalability to different numbers of nominal templates and its generalization to different formation configurations. We consider using a deep reinforcement learning-based approach, which allows for interactive training in a simulation environment and the learning of better formation representations through neural networks. Summary of the Invention

[0005] This invention addresses the challenges of optimal command dispatching and affine formation obstacle avoidance for multiple robots in a 5G-based cloud-edge architecture. It provides a method for optimal command dispatching and affine formation obstacle avoidance navigation for multiple robots based on 5G communication. In the multi-robot command dispatching stage, based on the advanced formation task commands issued by the user on the cloud server, the optimal command dispatching module on the edge server dispatches commands to multiple robots, and then distributes them to each robot for execution via a distributed reinforcement learning navigation control module. In the optimal command decision-making stage, the overall obstacle avoidance navigation problem is modeled as an affine formation planning problem. A centralized affine formation planning module deployed on the edge server completes global and local decision-making planning at the formation level, and further distributes the commands to each robot for closed-loop execution via a distributed reinforcement learning navigation control module. This process completes the overall transmission and execution of the user's advanced formation commands from the cloud to the edge and then to the robot.

[0006] This invention is achieved through the following technical solution:

[0007] This invention discloses a multi-robot optimal command dispatch and affine formation overall obstacle avoidance navigation method based on 5G communication. The method is adapted to the cloud-edge-device three-layer hardware architecture of 5G communication, including a cloud server, an edge server, and a robot, and consists of four algorithm modules deployed on each of them. These include an advanced task command issuance module deployed on the cloud server, a multi-robot optimal formation configuration and dispatch module and a distributed reinforcement learning navigation control module deployed on the edge server, and a centralized affine formation planning module deployed on each robot.

[0008] The algorithmic approach comprises two phases: a multi-machine instruction dispatch phase and an optimal instruction decision phase.

[0009] Multi-machine instruction dispatch stage:

[0010] The advanced task instruction issuing module deployed on the cloud service receives high-end formation task instructions from users, including multi-robot affine formation target formation templates, initial formation parameters, target formation parameters, and other instructions, and issues them to the edge server.

[0011] The multi-robot optimal formation configuration assignment module deployed on the edge server calculates the optimal initial formation configuration assignment for each robot based on the multi-robot formation nominal formation template and initial formation parameters issued by the advanced task instruction issuing module, and then sends it to each robot.

[0012] The distributed reinforcement learning navigation control module deployed on the robot takes the optimal initial formation configuration assigned by the edge server as the navigation target and drives each robot to reach the initial formation configuration.

[0013] Optimal instruction decision-making phase:

[0014] The centralized affine formation planning module deployed on the edge server integrates the target formation template, target formation parameter information issued by the advanced task instruction issuing module deployed on the cloud server, and the actual formation configuration of each robot uploaded and summarized by each robot terminal, to plan the real-time navigation sub-targets of each robot and issue them to each robot terminal.

[0015] The distributed reinforcement learning navigation control module deployed on each robot calculates the control commands for each robot in real time based on the real-time navigation sub-targets planned by the centralized affine formation planning module. This drives each robot to move towards the overall formation target and uploads the updated robot status information to the edge server. This forms a closed-loop feedback operation with the centralized affine formation planning module until all robots reach the formation target.

[0016] As a further improvement, the multi-robot optimal formation configuration assignment module of the present invention specifically comprises:

[0017] First, the initial formation configuration matrix is ​​obtained by affine transformation of the information such as the multi-robot formation nominal formation template and initial formation parameters issued by the advanced task instruction issuance module through the initial formation configuration calculation module.

[0018] Secondly, the task assignment efficiency matrix calculation module calculates the target assignment efficiency matrix based on the initial formation configuration matrix and the list of robots to be assigned, by comprehensively considering the efficiency function of each robot's target navigation efficiency and task execution efficiency. The target navigation cost of each robot is estimated by the inference of the critic evaluation network in the distributed reinforcement learning navigation control module.

[0019] Finally, the optimal assignment algorithm module solves the maximum weight allocation problem of the weighted bipartite graph based on the efficiency matrix, thereby calculating the optimal initial formation configuration of each robot and sending it to each robot for the distributed reinforcement learning navigation control module to use as the target point for forming the initial formation.

[0020] As a further improvement, the centralized affine formation planning module of the present invention is specifically as follows:

[0021] It consists of two parts: a global affine formation planner based on the reinforcement learning-assisted radiation parameter space RRT* algorithm and a local affine formation planner based on reinforcement learning.

[0022] First, the global affine formation planner based on the reinforcement learning-assisted affine parameter space RRT* algorithm plans a generalized path at the global affine formation parameter level based on the target formation template, target formation parameters, and pre-acquired global map information describing the current environment from the high-level task instruction module.

[0023] Secondly, the reinforcement learning-based local affine formation planner calculates the desired local formation configuration plan for each robot based on the global path sub-objective and information such as the actual formation configuration and local observations of the multi-robots collected from each robot end, and sends it to each robot end for execution through the reinforcement learning policy network based on TD3.

[0024] Finally, the global affine formation planner based on the reinforcement learning-assisted affine parameter space RRT* algorithm and the local affine formation planner based on reinforcement learning are also closely related and interdependent: On the one hand, the global affine formation planner based on the reinforcement learning-assisted affine parameter space RRT* algorithm can divide the complex global affine formation planning problem into a series of local affine formation planning problems through a sampling-based path search algorithm, and provide the planned global path points to the local affine formation planner based on reinforcement learning as sub-objectives to guide its local planning solution; on the other hand, the reinforcement learning policy network part in the local affine formation planner based on reinforcement learning can provide the Steer function that conforms to formation kinematics, obstacle avoidance constraints, and energy-time optimality for the search tree expansion process of the RRT* algorithm in the global planner. At the same time, the reinforcement learning evaluation network can provide the corresponding Steer cost evaluation, replacing the process of planning first and then collision detection, improving planning efficiency, and assisting in the expansion and rewiring process of the search tree.

[0025] The beneficial effects of this invention are as follows:

[0026] The multi-robot optimal command dispatch and affine formation overall obstacle avoidance navigation method based on 5G communication proposed in this invention solves two major problems in multi-robot formation navigation tasks under cloud-edge-device communication architecture:

[0027] Firstly, it enables efficient transmission of advanced user formation commands: This method constructs an algorithmic bridge between the upper-layer cloud server and the lower-layer robot end through a multi-robot optimal formation configuration and dispatch module and a centralized affine formation planning module deployed on the edge server, respectively, in the multi-robot command dispatch and optimal command decision stages. This efficiently completes the transmission of advanced user formation commands from the cloud to the edge and then to the robot end, enabling the multi-robot system to efficiently complete formation tasks requiring high-end commands without the need for commander intervention. It avoids the cumbersome cloud-based scheduling commands throughout the process and reduces the stress on the user's command.

[0028] Secondly, this method addresses the technical challenges in overall obstacle avoidance planning for formation: if a distributed algorithm is used for overall obstacle avoidance planning, the efficiency will be low and it will be difficult to obtain the optimal solution. This method adopts a centralized approach while introducing the concept of affine formation to model the problem. That is, the original planning of each robot is transformed into a planning problem of affine transformation parameters of the formation as a whole relative to a nominal formation template. This reduces the planning degree of freedom to 6 degrees of freedom (and does not change with the number of robots participating in the formation), solving the problems of dimensionality curse and poor scalability in the centralized solution.

[0029] The above describes the beneficial effects of this invention on multi-robot formation navigation tasks under a cloud-edge-device communication architecture. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the overall framework of this method;

[0031] Figure 2 This is a framework diagram of the multi-robot optimal formation configuration and assignment module;

[0032] Figure 3 This is a network framework diagram of the reinforcement learning policy in the distributed reinforcement learning navigation control module.

[0033] Figure 4 This is the overall framework diagram of the centralized affine formation planning module;

[0034] Figure 5 This is a network framework diagram of the reinforcement learning policy in the reinforcement learning local affine formation planner within the centralized affine formation planning module. Detailed Implementation

[0035] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0036] The purpose of this invention is to address the needs of efficient transmission of advanced user formation commands, overall obstacle avoidance, and solutions to the curse of dimensionality and poor scalability in multi-robot formation navigation under a cloud-edge-device communication architecture. This invention proposes an optimal multi-robot command dispatch and affine formation obstacle avoidance navigation method based on 5G communication. This method efficiently transmits advanced user formation commands from the cloud to the edge and then to the robot, enabling multi-robot systems to complete formation tasks requiring high-level commands without commander intervention. Figure 1 This is a data flow diagram of the present invention;

[0037] The specific implementation method of the present invention is as follows:

[0038] Step 1: In the multi-robot instruction dispatch phase, the advanced task instruction delivery module deployed on the cloud service receives high-end formation task instructions from the user, including multi-robot affine formation target formation templates, initial formation parameters, target formation parameters, etc., and sends them to the edge server.

[0039] The advanced task instruction issuance module is specifically a human-computer interaction interface deployed on a cloud server, capable of accepting high-level formation task instructions from users or commanders and distributing them to edge servers; wherein the nominal formation template, initial formation parameters, and target formation parameters of the multi-robot affine formation are defined according to the affine formation problem as follows:

[0040] In the affine formation problem, the basis for representing the formation is a nominal formation template:

[0041]

[0042] The formation template is defined as the position vector matrix of each node relative to the formation center. Real-time formation configuration is obtained from this nominal formation template through a set of time-varying affine transformations, and the formation planning result at any given time will belong to the affine transformation subspace of the nominal formation.

[0043] r i (t)=A(t)r 0i +B(t), i∈{1,…,n}

[0044] in

[0045] As shown in the above equation, the affine transformation process includes a two-dimensional linear transformation matrix and a translation matrix, which can be specifically decomposed into six-dimensional affine transformation parameters such as rotation, shearing, scaling, and translation:

[0046] P aff (t)=[θ(t),m(t),s x (t),s y (t),b x (t),b y (t)]

[0047] Both the initial formation parameters and the target formation parameters can be represented as the above 6-dimensional vector form;

[0048] Step 2: In the multi-robot instruction dispatch phase, the multi-robot optimal formation configuration dispatch module deployed on the edge server calculates the optimal initial formation configuration for each robot based on the multi-robot formation nominal formation template and initial formation parameters issued by the advanced task instruction issuing module, and then sends it to each robot. The multi-robot optimal formation configuration dispatch module consists of three parts: an initial formation configuration calculation module, a task dispatch efficiency matrix calculation module, and an optimal dispatch algorithm module. Figure 2 ):

[0049] The initial formation configuration calculation module specifically converts the initial formation parameters issued by the advanced task instruction issuing module into an affine transformation corresponding to the initial formation, and applies the affine transformation to the received multi-robot formation target formation template to calculate the initial formation configuration matrix.

[0050] The task assignment efficiency matrix calculation module specifically works as follows: After obtaining the initial formation configuration matrix and the list of robots to be scheduled (ensuring that the total number of robots to be scheduled is greater than the number of robots required for formation), it is necessary to select a subset of robots suitable for completing this formation task from the candidate robot set, and match each robot with the most suitable position in the initial formation as the navigation target to form the initial formation; due to the optimal allocation characteristics of the problem, the problem is considered to be modeled as a maximum weight allocation problem of a weighted bipartite graph: where the weighted bipartite graph G is defined as n,n =(X,Y):X={x1,x2,...x n Let Y = {y1, y2, ..., y} be the set of candidate robots. n} represents the initial set of formation positions (because the original number of initial formation positions is less than the number of candidate robots, virtual useless nodes need to be introduced to fill the original incomplete bipartite graph into a complete bipartite graph), with edge x i y j The weight is w ij (0 indicates no matching, and the weight of any edge connected to a useless node is set to 0), find the maximum weight complete matching of G;

[0051] The weights of edges in a bipartite graph are key to defining this problem. To maximize the overall efficiency and minimize the overall cost of multiple robots forming an initial overall formation while maximizing their overall ability to complete subsequent tasks, this scheme defines a task assignment efficiency matrix to measure the suitability weights between any candidate robot and any candidate formation position. Each element in the task assignment efficiency matrix defines the corresponding efficiency value for assigning a specific formation position to a specific robot. Its calculation considers the weighted sum of the following efficiency factors: 1. Target navigation efficiency: Considering environmental information from the robot's current position, the potential cost of navigating to the target point (estimated by the critic evaluation network in the distributed reinforcement learning navigation control module, which can efficiently predict the difficulty of reaching the target point); 2. Task execution efficiency: A weighted combination of the robot's remaining endurance, sensor observation capability level (number and accuracy of sensors), motion capability (robot's degrees of freedom, upper limit of velocity and acceleration), and payload capacity.

[0052] The optimal assignment algorithm module specifically uses the KM (Kuhn-Munkres) algorithm to solve the maximum weight allocation problem of the weighted bipartite graph based on the efficiency matrix obtained by the task assignment efficiency matrix calculation module. This calculates the optimal initial formation configuration for each robot and sends it to each robot for the distributed reinforcement learning navigation control module to use as the target point for forming the initial formation.

[0053] Step 3: In the multi-machine instruction dispatch stage, the distributed reinforcement learning navigation control module deployed on the robot side ( Figure 3Using the optimal initial formation configuration assigned by the edge server as the navigation target, the speed control commands for each robot are calculated through a reinforcement learning policy network to drive each robot to reach the initial formation configuration.

[0054] The distributed reinforcement learning navigation control module specifically employs the following Markov decision process modeling for the local navigation control problem of a single robot:

[0055] The state space includes environmental perception information and navigation target position information. Environmental perception information is the robot's observation of the surrounding environment, especially obstacles, using a 2D LiDAR point cloud, expressed as an occupied grid map in its local coordinate system. The navigation target position information is issued by the edge server and needs to be transformed into the robot's coordinate system based on its actual position information (obtained from GPS or other positioning schemes). The action space consists of velocity commands v and w (linear velocity along the x-axis and angular velocity along the z-axis in the robot's coordinate system). Because it is a continuous action space, the reinforcement learning algorithm uses the TD3 (Dual Delay Deep Deterministic Policy Gradient) framework, which is an algorithm... The actor-critic off-policy reinforcement learning framework consists of an actor policy network that maps environmental perception information and navigation goals of each robot into velocity commands, and a critic evaluation network that evaluates the value of the actor network's decision based on the current state. The critic evaluation network is used only during training to guide the training of the actor network and updates by minimizing the MSE error between the evaluation value and the target value. During training, the actor network updates by maximizing the cumulative expected reward of the critic evaluation (deterministic policy gradient). In TD3, all target networks are updated using soft updates (EMA).

[0056] The reinforcement learning actor policy network structure in the distributed reinforcement learning navigation control module is as follows: An obstacle feature vector is obtained by encoding the occupied grid map describing obstacle information through a CNN (Convolutional Neural Network); the navigation target position, transformed to the ontological coordinate system, is encoded into a target feature vector through an MLP (Multilayer Perceptron Network); the above two feature vectors are concatenated into a total feature vector and mapped to the final speed command output through an MLP. The reinforcement learning critic evaluation network structure is largely the same as the actor network, except that at the input end, it additionally includes encoding the actor action decision through an MLP and concatenating it into the total feature vector, and at the output end, it outputs a one-dimensional joint value evaluation of the current state and actions.

[0057] Step 4: In the optimal instruction decision-making stage, the centralized affine formation planning module deployed on the edge server, based on the target formation template, target formation parameter information issued by the advanced task instruction issuing module deployed on the cloud server, and the actual formation configuration of each robot uploaded and summarized by each robot terminal, plans the real-time navigation sub-targets for each robot and issues them to each robot terminal; the centralized affine formation planning module specifically consists of two parts: a global affine formation planner based on the reinforcement learning-assisted radial parameter space RRT* algorithm and a local affine formation planner based on reinforcement learning. Figure 4 ):

[0058] The reinforcement learning-based local affine formation planner is specifically modeled as follows: The Markov decision process for the affine formation planning problem is modeled as follows:

[0059] The state space is designed as follows: obstacle information, real-time affine formation representation, and target affine formation representation. The obstacle information is represented by the coordinate transformation of each robot's 2D LiDAR point cloud observations of obstacles into an occupied grid map centered on the formation center. The real-time affine formation representation specifically includes the nominal formation template r0 and the real-time affine transformation parameters P. aff (t); the target affine formation can be directly calculated using the target affine transformation parameter P. target_aff The action space is designed as an adjustment of the affine transformation parameters, characterized by the rate of change of the affine transformation parameters relative to time.

[0060]

[0061] Since the two scale parameters are not uniform in addition but uniform in multiplication, their logarithms are taken first.

[0062] The reward function design includes: a sparse reward for the affine transformation parameters upon reaching the target and a dense reward designed using RewardShaping; a sparse penalty for collisions with obstacles in the formation topology and a dense penalty designed using RewardShaping; velocity and acceleration penalties for each node considering the motion capability constraints of each robot; and a motion capability penalty introduced to prevent formation changes from becoming too frequent and unstable. Because it is a continuous action space, the reinforcement learning algorithm uses the TD3 (Dual Delay Deep Deterministic Policy Gradient) framework.

[0063] The overall framework of the algorithm is as follows Figure 5As shown: The actor policy network in the TD3 algorithm framework is used to map obstacle information (obstacle information from multiple robots' local observations, assuming each robot has a limited perception range, and the obstacle information within the range is summarized into an occupied grid map with the formation center as the midpoint), nominal formation template, target affine formation parameters, and actual affine formation parameters into adjustment amounts for the affine formation parameters. These are then superimposed on the current actual affine transformation parameters to generate the desired affine transformation parameters. The desired position of each robot can be obtained by performing the corresponding affine transformation on the nominal template.

[0064] For feedback loops, if it is assumed that the underlying controller of the multi-robot system can perfectly track the trajectory of the desired formation configuration, the actual affine parameters input to the network at the next time step are the desired affine transformation parameters planned in this case. However, if the tracking is not perfect, it involves real-time identification of the affine transformation parameters of the actual formation relative to the nominal template. This means projecting the actual formation configuration onto the affine transformation subspace of the nominal template to obtain the closest configuration. Essentially, this involves finding the most matching affine transformation matrix through the least squares process and then solving the corresponding affine transformation parameters inversely.

[0065] The actor policy network structure and decision-making process in the TD3 algorithm framework are as follows: A CNN (Convolutional Neural Network) is used to encode the obstacle-occupied grid map to obtain obstacle feature vectors. Considering the need for scalable representation of changes in the number of robots in the formation, the variable-scale nominal formation template matrix is ​​first expressed in graph form. The column vectors of the nominal formation template matrix, representing the positions of each robot in the template formation, serve as the node features of the graph. The adjacency matrix of the graph is then encoded using a GNN (Graph Neural Network) to obtain the nominal formation template feature vector. The target and actual affine transformation parameters are also encoded using a multilayer perceptron network to obtain two corresponding feature vectors. These four feature vectors are concatenated into a fixed-length total feature vector, which is then mapped through a multilayer perceptron network to the rate of change of the affine transformation parameters defined in the reinforcement learning action space. The critic evaluation network structure is largely the same as the actor network, except that at the input end, it additionally encodes the actor action decisions through a fully connected layer and concatenates them into the total feature vector. The output at the output end is a one-dimensional joint value evaluation of the current state actions.

[0066] The global affine formation planner based on reinforcement learning-assisted affine parameter space RRT* is as follows: For long-distance affine formation planning, since the target position is outside the perception range, the affine formation planning policy network trained by reinforcement learning is easily trapped in local optima due to its limited local perception. Therefore, a framework combining the classic global planner and local planner is introduced to plan a generalized path at the global affine formation parameter level based on the target formation template, target formation parameters and pre-acquired global map information describing the current environment from the high-level task instruction module.

[0067] The reinforcement learning-assisted affine parameter space RRT* algorithm constructs a generalized path tree at the affine formation level within the six-dimensional affine transformation formation parameter space using random sampling. During the generation of new nodes in the path search tree, the actor policy network in the reinforcement learning-based local affine formation planner is used as a steer function to generate local trajectories between formation nodes that conform to formation kinematics, obstacle avoidance constraints, and energy-time optimality. The critic evaluation network in the reinforcement learning-based local affine formation planner provides the corresponding trajectory cost evaluation, replacing the classic process of Euclidean distance plus collision detection. This avoids dense collision detection calculations at the formation level, improving the efficiency of RRT* random search tree expansion and rewiring processes. The affine transformation parameter trajectories obtained through this framework can be used to perform affine transformations on the initial formation template to obtain the trajectory of each robot, which is then fed into the underlying trajectory tracking controller of each robot for trajectory tracking.

[0068] Step 5: In the optimal instruction decision-making stage, the distributed reinforcement learning navigation control module deployed on each robot (as described in Step 3) calculates the control instructions for each robot in real time based on the real-time navigation sub-targets planned by the centralized affine formation planning module. This drives each robot to move towards the overall formation target and uploads the updated robot status information to the edge server. This forms a closed-loop feedback loop with the centralized affine formation planning module until the entire robot formation reaches the affine formation target (without loss of generality, the distance threshold to the target is set to 1 meter in this example), at which point the entire process ends.

[0069] Obviously, the above is not a limitation on the implementation method. For those skilled in the art, different variations can be made based on the above description, and these variations are still within the protection scope of this invention.

Claims

1. A method for optimal instruction dispatching and affine formation obstacle avoidance navigation for multiple robots based on 5G communication, characterized in that, include: The method described is adapted to the three-layer hardware architecture of cloud, edge, and terminal in 5G communication, including cloud server, edge server, and robot, and consists of four algorithm modules deployed on each of them. These include an advanced task instruction issuing module deployed on the cloud server, a multi-robot optimal formation configuration and assignment module and a distributed reinforcement learning navigation control module deployed on the edge server, and a centralized affine formation planning module deployed on each robot. The algorithmic approach described herein comprises two phases: a multi-machine instruction dispatch phase and an optimal instruction decision phase. Multi-machine instruction dispatch stage: The advanced task instruction issuing module deployed on the cloud service receives high-end formation task instructions from users, including multi-robot affine formation target formation template, initial formation parameters, and target formation parameter instructions, and issues them to the edge server. The multi-robot optimal formation configuration assignment module deployed on the edge server calculates the optimal initial formation configuration assignment for each robot based on the multi-robot formation nominal formation template and initial formation parameters issued by the advanced task instruction issuing module, and then sends it to each robot. The distributed reinforcement learning navigation control module deployed on the robot takes the optimal initial formation configuration assigned by the edge server as the navigation target and drives each robot to reach the initial formation configuration. Optimal instruction decision-making phase: The centralized affine formation planning module deployed on the edge server integrates the target formation template, target formation parameter information issued by the advanced task instruction issuing module deployed on the cloud server, and the actual formation configuration information of each robot uploaded and summarized by each robot terminal to plan the real-time navigation sub-targets of each robot and issue them to each robot terminal. The distributed reinforcement learning navigation control module deployed on each robot calculates the control commands for each robot in real time based on the real-time navigation sub-targets planned by the centralized affine formation planning module. This drives each robot to move towards the overall formation target and uploads the updated robot status information to the edge server. This forms a closed-loop feedback operation with the centralized affine formation planning module until all robots reach the formation target.

2. The multi-robot optimal command dispatch and affine formation obstacle avoidance navigation method based on 5G communication according to claim 1, characterized in that, The multi-robot optimal formation configuration assignment module is specifically as follows: First, the initial formation configuration matrix is ​​obtained by affine transformation of the multi-robot formation nominal formation template and initial formation parameter information issued by the advanced task instruction issuance module through the initial formation configuration calculation module. Secondly, the task assignment efficiency matrix calculation module calculates the target assignment efficiency matrix based on the initial formation configuration matrix and the list of robots to be assigned, by comprehensively considering the efficiency function of each robot's target navigation efficiency and task execution efficiency. The target navigation cost of each robot is estimated by the inference of the critic evaluation network in the distributed reinforcement learning navigation control module. Finally, the optimal assignment algorithm module solves the maximum weight allocation problem of the weighted bipartite graph based on the efficiency matrix, thereby calculating the optimal initial formation configuration of each robot and sending it to each robot for the distributed reinforcement learning navigation control module to use as the target point for forming the initial formation.

3. The multi-robot optimal command dispatch and affine formation obstacle avoidance navigation method based on 5G communication according to claim 1, characterized in that, The centralized affine formation planning module is specifically as follows: Including reinforcement learning-assisted affine parameter space RRT The algorithm consists of two parts: a global affine formation planner and a local affine formation planner based on reinforcement learning. Firstly, based on the reinforcement learning-assisted affine parameter space RRT The algorithm's global affine formation planner plans a generalized path at the global affine formation parameter level based on the target formation template, target formation parameters, and pre-acquired global map information describing the current environment from the high-level task instruction module. Secondly, the reinforcement learning-based local affine formation planner calculates the desired local formation configuration plan for each robot based on the global path sub-objective and the actual formation configuration and local observation information of multiple robots collected from each robot end, and sends it to each robot end for execution. Finally, based on the reinforcement learning-assisted affine parameter space RRT... The global affine formation planner and the reinforcement learning-based local affine formation planner are also closely related and interdependent: on the one hand, the reinforcement learning-assisted affine parameter space RRT... The algorithm's global affine formation planner uses a sampling-based path search algorithm to divide the complex global affine formation planning problem into local affine formation planning problems. The planned global pathpoints are then provided to the reinforcement learning-based local affine formation planner as sub-objectives to guide its local planning solution. Furthermore, the reinforcement learning policy network in the reinforcement learning-based local affine formation planner is the RRT (Reinforcement Learning Principle) of the global planner. The algorithm's search tree expansion process provides a Steer function that conforms to formation kinematics, obstacle avoidance constraints, and energy-time optimality. At the same time, the reinforcement learning evaluation network provides a corresponding Steer cost evaluation, replacing the process of planning first and then collision detection, and assisting in the expansion and rewiring of the search tree.

Images