Distributed Multi-Robot Navigation Method, System, Storage Medium and Device

Abstract

The present invention relates to a distributed multi-robot navigation method, system, storage medium and device, including the following steps: obtaining the RGB image, lidar data and motion vector data of the robot at a specified moment, extracting features from the RGB image and lidar data based on a visual encoder, and converting the features of the RGB image into a visual feature hidden vector; fusing the visual feature hidden vector and lidar features through a sensor fusion network to obtain a sensor-level feature vector; based on a graph neural network, aggregating the sensor-level feature vectors of all neighboring robots within the communication range of the robot to obtain an agent-level neighborhood perception vector; using the connected sensor-level feature vector and agent-level neighborhood perception vector as the inputs of an actor network and a critic network, setting a reward function with respect to target arrival, collision avoidance and time efficiency to obtain the optimal navigation strategy of the robot, and realizing the cooperative navigation of the robot team.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-robot collaborative navigation, and specifically to a distributed multi-robot navigation method, system, storage medium and device. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] In a multi-robot navigation scenario, due to the presence of more moving obstacles and spatial conflicts, the limitation effect of navigation performance by the incompleteness of robot perception is amplified. Existing technologies use multiple sensors to improve the perception ability. However, simply splicing the inputs of multiple sensors does not well help the robot fully perceive the surrounding environment. In a complex real-world environment, the learned strategies cannot meet the requirements of robot team collaborative navigation.

[0004] On the one hand, different types of sensors can provide rich environmental information. Therefore, existing technologies use deep reinforcement learning as a policy tool to fuse the information obtained by multiple sensors, in order to learn a robust autonomous driving policy based on multi-sensor inputs and obtain an end-to-end driving policy network. However, the images in real scenarios usually contain complex textures and task-independent visual interferences such as reflections and shadows. Relying solely on the reinforcement learning signal to optimize the control strategy of the model is very inefficient.

[0005] On the other hand, existing technologies achieve the cooperation of multi-robot systems through the way of multi-robot information aggregation. The key point of such methods lies in how to aggregate information between neighboring robots. For example, the vector splicing method can aggregate the states of all other agents. However, this neural network structure depends on the number of robots, resulting in poor scalability for group systems of different scales. Some existing technologies adopt the mean embedding method, taking the mean of the high-dimensional features of neighboring robots as the fusion representation to achieve the permutation invariance of robots. In addition, in order to distinguish neighboring robots of different importance, some methods use long short-term memory (LSTM) for agent-level state fusion to process sequences of uncertain length into fixed-size hidden state vectors. However, when the density of robots reaches a sufficiently high level, because the contributions of different robots cannot be distinguished, the effects of the above methods are not ideal. Summary of the Invention

[0006] To solve the technical problems existing in the above-mentioned background art, the present invention provides a distributed multi-robot navigation method, system, storage medium and device. In the first stage, the method fuses observations at the sensor level to obtain better environmental perception. In the second stage, the method aggregates information at the agent level to achieve effective coordination. By learning the strategy of multi-robot collaborative navigation through deep reinforcement learning, it is possible to achieve collaborative navigation and collision avoidance in a complex real-world environment.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] The first aspect of the present invention provides a distributed multi-robot navigation method, including the following steps:

[0009] Obtain the RGB image, lidar data and motion vector data of the robot at a specified moment, extract features from the RGB image and lidar data based on a visual encoder, and convert the features of the RGB image into a visual feature hidden vector;

[0010] Fuse the visual feature hidden vector and lidar features through a sensor fusion network to obtain a feature vector at the sensor level;

[0011] Based on a graph neural network, aggregate the sensor-level feature vectors of all neighboring robots within the communication range of the robot to obtain a neighborhood perception vector at the agent level;

[0012] Use the connected sensor-level feature vector and the agent-level neighborhood perception vector as the input of the actor network and the critic network, set a reward function with respect to goal reaching, collision avoidance and time efficiency to obtain the optimal navigation strategy of the robot, and achieve collaborative navigation of the robot team.

[0013] The visual encoder projects the latent representation in the RGB image onto a semantic segmentation map and a depth estimation map to obtain a visual feature hidden vector representing pixel-level prediction; the latent representation contains the understanding of semantic and geometric information.

[0014] The sensor fusion network has H attention fusion units. In the h-th attention fusion unit, the outputs of all attention fusion units are concatenated together through weighted summation and projected onto the fusion features, and the fusion features are the feature vectors at the sensor level of the robot.

[0015] In the graph neural network, the communication topology of the robot team is converted into a bidirectional graph, where each node represents a robot. If the Euclidean distance between two nodes is less than the communication radius r comm , then there is a bidirectional edge between the two nodes, and they receive messages from each other's nodes. The message consists of the sensor-level representation of robot i and motion measurement obtained by concatenation of

[0016] The graph neural network has M independent attention heads. In the m-th head, the message of robot i is projected into queries, keys, and values through three linear transformations with matrices and respectively. The importance scores of robot i and its neighbor robot j are as follows:

[0017]

[0018] where d K represents the dimension of the key and is used as a scaling factor. After receiving the messages from all neighbors, robot i calculates the normalized attention scores for each neighbor:

[0019]

[0020] where N i represents the set of neighbors of robot i; the neighborhood message aggregated by robot i in the first round is generated by projecting the concatenated vector output by all M attention heads:

[0021]

[0022] where σ is the activation function and f proj is a fully connected layer with a set number of activation function units; the receptive field of the robot is expanded by collecting information from multi-hop neighbors, and the multi-hop message is aggregated and regarded as the functional representation at the agent level, that is, the neighborhood perception vector at the agent level is obtained.

[0023] The actor network has at least two sequential fully connected layers and an output layer with at least two branches, where one outputs the translational velocity and the other outputs the rotational velocity The critic network generates the state value V i t .

[0024] The second aspect of the present invention provides a system for implementing the above method, including:

[0025] A sensor encoding module, configured to: acquire the RGB image, lidar data, and motion vector data of the robot at a specified moment, extract features from the RGB image and lidar data based on a visual encoder, and convert the features of the RGB image into a visual feature hidden vector;

[0026] The hierarchical perception fusion module is configured to: fuse the visual feature latent vector and the lidar feature through a sensor fusion network to obtain a feature vector at the sensor level; based on a graph neural network, aggregate the feature vectors at the sensor level of all neighboring robots within the communication range of the robot to obtain a neighborhood perception vector at the agent level;

[0027] The policy learning module is configured to: use the connected feature vectors at the sensor level and the neighborhood perception vectors at the agent level as the inputs of the actor network and the critic network, set a reward function with respect to goal reaching, collision avoidance, and time efficiency to obtain the optimal navigation policy of the robot, and achieve cooperative navigation of the robot team.

[0028] The third aspect of the present invention provides a computer-readable storage medium.

[0029] A computer-readable storage medium has a computer program stored thereon, and when the program is executed by a processor, the steps in the distributed multi-robot navigation method as described above are implemented.

[0030] The fourth aspect of the present invention provides a computer device.

[0031] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in the distributed multi-robot navigation method as described above are implemented.

[0032] Compared with the prior art, the above one or more technical solutions have the following beneficial effects:

[0033] 1. In the first stage, the observation data at the sensor level is fused to obtain better environmental perception. In the second stage, the information at the agent level is aggregated to achieve effective coordination. Through the multi-robot cooperative navigation strategy, it is possible to achieve cooperative navigation and collision avoidance in a complex real-world environment.

[0034] 2. For each robot, a RGB camera and a lidar are used for data acquisition, and compact features of the two modes are extracted through corresponding encoders. The sensor-level and agent-level information is used to improve perception for cooperative navigation in complex scenarios.

[0035] 3. It can effectively fuse the sensor-level information and the agent-level interaction information to obtain an efficient perception representation.

[0036] 4. Through the composite reward strategy based on reinforcement learning, a final steering instruction is generated for each robot. Description of the Drawings

[0037] The accompanying drawings forming a part of this invention are used to provide a further understanding of the invention. The schematic embodiments and descriptions thereof of the invention are used to explain the invention and do not constitute an improper limitation of the invention.

[0038] Figure 1 It is a schematic diagram of the model structure of the distributed multi-robot navigation method based on hierarchical perception fusion provided by one or more embodiments of the present invention;

[0039] Figure 2 It is a schematic diagram of training a visual encoder using semantic segmentation and depth estimation tasks provided by one or more embodiments of the present invention;

[0040] Figure 3 It is a schematic diagram of the sensor fusion network provided by one or more embodiments of the present invention;

[0041] Figure 4 It is a sliding average reward graph of the training process provided by one or more embodiments of the present invention;

[0042] Figures 5(a)-(d) are graphs of various episode termination conditions of the training process provided by one or more embodiments of the present invention;

[0043] Figures 6(a)-(c) are evaluation results of various metrics in the basic test process provided by one or more embodiments of the present invention;

[0044] Figures 7(a)-(c) are evaluation results of various metrics in the scalability experiment process provided by one or more embodiments of the present invention. Detailed implementation manners

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0047] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] As described in the background art, in the complex environment of the real world, the learned strategies cannot meet the requirements of collaborative navigation of a robot team. Therefore, the following embodiments provide a distributed multi-robot navigation method, system, storage medium, and device, and propose a distributed multi-robot navigation method based on hierarchical perception fusion. In the first stage, this method fuses observations at the sensor level to obtain better environmental perception. In the second stage, this method aggregates information at the agent level to achieve effective coordination. By learning the strategy of multi-robot collaborative navigation through deep reinforcement learning, it is possible to achieve collaborative navigation and collision avoidance in a complex real-world environment.

[0049] Embodiment 1:

[0050] The model architecture of the distributed multi-robot navigation method based on hierarchical perception fusion proposed in this embodiment is as Figure 1 shown.

[0051] (1) First, make relevant explanations for the problem form:

[0052] 1) Decentralized partially observable Markov decision process:

[0053] In this embodiment, the multi-robot motion planning problem is formulated as each robot in the team planning to reach its own designated target point within a specified time without colliding with other robots and obstacles. This embodiment assumes that all robots are homogeneous, non-holonomic differential robots with neighborhood communicability. This task can be modeled as a decentralized partially observable Markov decision process (Dec-POMDP) with neighborhood communication, which is a sequential decision-making problem where each agent makes a decision based on local observations and neighborhood information sharing. The generation of the new global state depends on the joint actions of all agents, which affects their private observations, indicating that the environment of each robot is non-stationary.

[0054] At each discrete time step t, the i-th robot generates its action and communication radius r comm through the policy based on its own observation and the set of messages received within To find the optimal policy, the goal of this embodiment is to maximize the expected discounted return of each robot:

[0055]

[0056] where π -i is the joint policy of all robots except robot i. In the method of this embodiment, due to the homogeneity of the robots, the parameters of the policy π i are shared among all other robots.

[0057] 2) Observation and Action:

[0058] At each discrete time step t, the observation of the i-th robot consists of three parts: where the visual observation is an RGB color image with 128*128 pixels taken by the front RGB camera with a 90-degree field of view of the robot, and the 2D laser measurement is obtained by the 2D lidar installed on top of each robot. In addition, the motion measurement vector where and represent the current translational speed and rotational speed of the i-th robot respectively, and represent the distance and angle of the target g i (the target point that cannot be detected by the camera or lidar) of the i-th robot respectively, while represents the orientation of the i-th robot in the world coordinate system. It should be noted that the method does not directly depend on the position in the world coordinate system to avoid overfitting of the position information during training, so it is robust to coordinates when deployed in different environments.

[0059] (2) Sensor Feature Encoding Part:

[0060] The details of two encoders are described here, and these two encoders are designed to extract the features of the visual and lidar inputs.

[0061] 1) Visual Feature Encoder:

[0062] The visual feature encoder is to convert the high-dimensional RGB image into an effective latent representation which should be robust to the complex textures of the scene for facilitating policy learning. Learning from pixels does follow an elegant end-to-end parameter optimization paradigm, but it may not be applicable in complex environments. The most important reason is that it is difficult to optimize the whole framework based on only weak reinforcement learning signals, and due to the complexity of the samples, the learning process requires a large amount of data. In addition, in terms of generality, when the visual features of the scene change slightly, this policy may perform poorly. Here, the learning process of the visual feature encoder is decoupled from the RL learning of the rest of the framework to improve its scene understanding ability.

[0063] Rich prior knowledge can enable the model to have good scene understanding ability, thus having satisfactory generalization ability and robustness to unknown environments. Here, multiple auxiliary tasks are selected to train the visual feature encoder. Using the encoder-decoder architecture, the shared encoder is trained under the supervision of semantic segmentation and depth estimation tasks, such asFigure 2 As shown. In this embodiment, a VGG 16-layer network is adopted, and the last classification layer is removed as the visual feature encoder, which encodes RGB images into latent representations. Two task-specific decoders with the same structure of five deconvolution layers are designed to project the latent representations into pixel-level predictions of semantic segmentation maps and depth estimation maps respectively. The ReLU activation function and batch normalization layer are applied to each deconvolution layer. The number of filters in the deconvolution layers are 512, 256, 128, 64, and 32 respectively. The kernel size, stride, and padding of each deconvolution layer are set to 3, 2, and 1 respectively. In this embodiment, the standard cross-entropy loss is adopted for semantic segmentation, and the L1 loss is adopted for depth estimation, and the losses of these two branches are added as the total loss to optimize the parameters. Among them, there are four types of semantic segmentation labels for the semantic segmentation task, namely background (i.e., walls and ceilings), passage areas, robots, and obstacles, which encourage the robot to distinguish collaborators and static objects during navigation.

[0064] Through dual-task supervised learning, the latent representations generated by the shared encoder contain the understanding of semantic and geometric information. In the reinforcement learning training stage, the weights of the visual feature encoder are frozen, and a task-oriented extractor is designed to further extract features related to the navigation task. In the task-oriented extractor, first, the latent representation is convolved into a feature map of size 128×4×4 through a 1×1 convolutional layer containing 128 ReLU filters, and then through a batch normalization layer. Then, the feature map is flattened and passed through two consecutive fully connected layers, which have 256 and 128 rectifier units respectively. Finally, the feature vector is obtained and regarded as the task-related visual representation.

[0065] 2) Radar feature encoder:

[0066] Since the laser measurements obtained by lidar are not affected by different textures, the reinforcement learning signal is directly used to optimize the radar feature encoder. First, the radar data is normalized, and the input features are convolved using two convolutional layers with one-dimensional convolutional (Conv1D) kernels. The kernel sizes of each convolutional layer are set to 5 and 3 respectively, and the stride of both is 2. Two fully connected layers with 512 and 128 rectifier units respectively are added to generate the radar feature vector with the same size as the visual feature vector for attention pattern fusion size.

[0067] (3) Hierarchical perception fusion part

[0068] This section first presents the designed sensor fusion network, which uses an attention mechanism to fuse the encoded visual and lidar features into a sensor-level information representation. Then, graph convolutional neighborhood fusion with an attention kernel is used to aggregate agent-level information from an uncertain number of neighbors to obtain the final perception vector.

[0069] 1) Sensor fusion network:

[0070] A fusion method with an attention mechanism is adopted, which can adaptively learn the relative importance of the two modalities, as Figure 3 shown. Specifically, in this embodiment, H attention fusion units are introduced to enhance the stability of the training process. In the h-th attention fusion unit, the result is generated through a weighted summation operation:

[0071]

[0072] where and are coefficients calculated by concatenating two feature vectors and applying two consecutive FC layers with 128 and 2 units respectively, and their activation functions are the LeakyReLU and Softmax functions respectively. Finally, the outputs of all H attention fusion units are concatenated together and then projected onto the fused feature which is considered as the compact sensor-level representation of robot i.

[0073] 2) Interaction based on graph convolution:

[0074] The communication topology of the robot team is formalized as a bidirectional graph, where each node represents a robot, and if the Euclidean distance between two nodes is less than the communication radius r comm , there is a bidirectional edge between the two nodes, which means they can receive messages from each other's nodes. The message is obtained by concatenating the sensor-level representation of robot i and the motion measurement . In addition, each node has a self-loop if there are no neighbors. This embodiment adopts graph convolution with multi-head attention, allowing each robot to selectively determine the relative importance of different neighbors and aggregate their messages accordingly.

[0075] Specifically, this embodiment implements M independent attention heads. In the m-th head, the message of robot i is projected into query, key, and value by respectively performing three linear transformations with matrices and . Therefore, the importance score between robot i and its neighbor robot j is calculated as follows:

[0076]

[0077] Among them, d K represents the dimension of the key and is used for the scaling factor. After receiving messages from all neighbors, robot i calculates the normalized attention score for each neighbor:

[0078]

[0079] Among them, N i represents the set of neighbors of robot i. Then, the aggregated neighborhood message in the first round for robot i is generated by concatenating the output vectors projected by all M attention heads:

[0080]

[0081] Among them, σ is the LeakyReLU activation function, and f proj is a fully connected layer with 133 LeakyReLU units.

[0082] Through multi-hop interaction (the method in this embodiment is three rounds), the receptive field of the robot is expanded by collecting information from multi-hop neighbors. The final multi-hop message is aggregated and regarded as the functional representation at the agent level.

[0083] (4) Policy learning part:

[0084] 1) Actor and critic networks:

[0085] This embodiment adopts a policy-based actor-critic algorithm, namely approximate policy optimization, to optimize the network parameters. In particular, this embodiment notes that the information of the robot will decrease as the number of aggregations increases. Therefore, skip connections are introduced to enhance individual features, and then the connected features pass through a fully connected layer with 128 LeakyReLU units and are respectively transmitted to the actor network and the critic network. Specifically, the actor network is implemented by two sequential fully connected layers, with 128 and 32 LeakyReLU units respectively, and then an output layer with two branches, where one outputs the translational velocity with a Sigmoid non-linearity, and the other outputs the rotational velocity with a Tanh non-linearity. The first two layers of the critic network are the same as those of the actor network, and an output layer with one neuron is used to generate the state value V i t .

[0086] 2) Composite reward function:

[0087] The multi-robot motion planning task in this embodiment consists of three sub-goals, namely goal reaching, collision avoidance, and time efficiency. Therefore, a composite reward function is designed to feedback signals considering multiple sub-goals, avoiding the problem of sparse rewards during training. Specifically, at a specified time t, the reward of robot i is as follows:

[0088]

[0089] Among them, and are respectively designed for goal reaching, collision avoidance, and motion optimization.

[0090] First of all, the goal reaching reward is calculated by the following formula:

[0091]

[0092] Among them, d g is the Euclidean distance from robot i to the target g i when robot i reaches the target, λ1 is a large positive reward for reaching the target, and λ2 is a small positive number used to encourage the robot to move towards the target during navigation.

[0093] Secondly, the collision avoidance reward is obtained by the following formula:

[0094]

[0095] Among them, λ3 is a large negative penalty for collision. And both λ4 and λ5 are positive numbers used to make the robot aware of the danger of collision in advance. is the minimum distance in the laser observation while d ca is the predefined danger distance.

[0096] Thirdly, in order to avoid sharp turns and accelerate navigation, the motion refinement reward is calculated by the formula:

[0097]

[0098] Among them, both λ6 and λ7 are small negative numbers. The former is used to punish a large rotational speed, while the latter is a small time penalty to encourage the robot to complete the task as soon as possible.

[0099] Except for the time penalty λ7, all reward signals are included in the robot observation, which helps the framework focus on the information most relevant to the task, thus facilitating policy and value learning.

[0100] (4) Experimental results

[0101] In this embodiment, a multi-robot collaborative navigation experiment is carried out in a simulation environment to prove the superiority of the method of this embodiment over the baseline method.

[0102] 1) Model implementation:

[0103] In this embodiment, the experiment is carried out on a workstation equipped with an Intel I7-9800X CPU (3.80GHz) and an NVIDIA GTX 2080Ti GPU, and a training and testing environment is constructed in the 3D simulator of PyBullet. This embodiment selects Turtlebot3 as the robot model. Before the reinforcement learning stage, supervised learning is first carried out to train the semantic segmentation and depth estimation tasks of the visual encoder. The supervised learning dataset contains 5000 RGB images collected from the simulated scene, and the size of each sample is 3×128×128. The Adam optimizer is used for supervised learning, and the model is trained for 1000 epochs with a batch size of 64. Then, the weights of the visual encoder are frozen, and the proximal policy optimization algorithm and the Adam optimizer are used to optimize the rest of the framework. The learning rate of reinforcement learning is set to 5×10 -5 , and the trainable parameters will be updated every 128 time steps. For each update of the model, the batch size is set to 32.

[0104] 2) Training scenario:

[0105] The training scenario is a 5m×5m room in the simulation environment, with N irregular obstacles, whose positions and types vary randomly every N u times during model updates to ensure the randomness of the environment. Six robots operate independently in their respective scenarios to complete the navigation task. At the initialization of each episode, the robots are placed at fixed positions at one end of the room and need to reach the target point at the other end of the room within the maximum movement time step N m . This is to increase the possibility of path conflicts between robots because each robot needs to pass through the central area of the room. In addition, each robot's episode has four switching conditions: timeout, collision with an obstacle, collision with a collaborator, and success (i.e., reaching the target). The training process parameters are shown in Table 1.

[0106] Table 1: Training process parameter settings

[0107]

[0108]

[0109] 3) Baseline method and evaluation metrics:

[0110] ① Baseline methods: In the experiments of this embodiment, the proposed method in this embodiment is named SAPI and compared with the following five baselines, including two state-of-the-art methods and three variants of the method in this embodiment.

[0111] MRV-A: This is a vision-based motion method that operates in an end-to-end manner, using omnidirectional RGB images from the first-person perspective for observation without any pre-training process. In this embodiment, its original discrete action policy is changed to a continuous action policy, and the same reward function as SAPI is used to achieve fairness.

[0112] SelComm: This is an advanced lidar-based method. Each robot needs to perform global communication and then select the K most relevant neighbors to share information. Here, according to the original method, K is set to 3. In this embodiment, the model trained in the original setting is used and compared with SAPI for evaluation.

[0113] SAPI-Seg: This is an ablation version of the method in this embodiment, where the visual encoder is only pre-trained on the semantic segmentation task, that is, it can only extract semantic information from images. The remaining settings are the same as SAPI.

[0114] SAPI-Dep: In this ablation method, the visual encoder is only pre-trained on the depth estimation task to obtain the ability to extract geometric information, and other settings are the same as SAPI.

[0115] SAPI-S: To show the effect of agent-level information aggregation, in this ablation version, all robots do not communicate with each other but have the same sensor-level perception ability as SAPI.

[0116] ② Evaluation metrics: In the experiments of this embodiment, the following three metrics are used to comprehensively evaluate the performance of each method.

[0117] Success rate: It represents the percentage of the number of successful cases in the total number of evaluated cases. If the robot reaches the target without collision within the maximum time step N m it is considered successful.

[0118] Extra distance ratio: It represents the percentage of the redundant length in the entire trajectory of successful cases. A larger extra distance ratio means that the robot detours a longer distance.

[0119] Average speed: Measures the average speed of successful cases.

[0120] 4) Evaluation in the simulation environment:

[0121] ① Training convergence analysis:

[0122] AsFigure 4 As shown, during the training process of each method, this embodiment records the cumulative reward for each episode and plots the rolling reward curve, where the rolling reward represents the average cumulative reward over the past 2000 episodes. In addition, to clearly illustrate how the model learns navigation skills, this embodiment also records the switching conditions for each episode and derives the rolling rates of all switching conditions over the past 2000 episodes, namely the success rate, obstacle collision rate, collaborator collision rate, and timeout rate, as shown in Figs. 5(a)-(d). It can be observed that the proposed method SAPI outperforms all baseline models because it converges to the highest reward and success rate. Through the analysis of the curves, the following further conclusions can be drawn:

[0123] Compared with the pure vision method, sensor-level information fusion and visual encoder preprocessing help the robot navigate in complex environments. This embodiment finds that although MRV-A claims to be able to work in simple environments, it almost fails in complex situations. Specifically, at the end of training, MRV-A only learns the preliminary skill of avoiding collisions with static obstacles, as shown in Fig. 5(b). This also shows that due to the low data efficiency caused by complex visual information, it is difficult to learn the navigation strategy end-to-end from pixels.

[0124] Compared with SAPI-Seg and SAPI-Dep, SAPI with richer visual input prior knowledge has better navigation performance due to its stronger scene understanding ability. Compared with the embedding with one type of prior knowledge, the visual embedding that combines semantic and geometric information does not increase the computational complexity. In addition, this embodiment notes that compared with geometric information, the semantic information of visual input is more helpful in the training scenario. A possible reason is that laser measurements can already provide some geometric features of the environment.

[0125] By comparing SAPI and SAPI-S, it can be demonstrated that the performance of collaborative navigation can indeed be improved by introducing agent-level interaction. The lack of communication with collaborators increases the unpredictability and non-stationarity of the environment. In particular, the rolling reward and success rate of SAPI-S fluctuate at the same level after 50000 episodes, mainly due to the relatively high collaborator conflict rate, as shown in Fig. 5(c).

[0126] An overview of skill learning can be obtained through Figure 4 and Figure 5(a) - Figure 5(d)To summarize, in this embodiment, SAPI is taken as an example. In the initial stage of training, due to the randomness of model parameters, the actions are almost random. Conversely, the rolling reward curve shows large oscillations or even drops. After that, as the robot begins to master the initial collision avoidance skills, the obstacle collision rate drops rapidly, as shown in Fig. 5(b), while the collaborator collision rate begins to increase because collaborative navigation has not been learned yet, as shown in Fig. 5(c). At the same time, the rise and fall of the timeout rate indicate that the robot is learning the balance between goal navigation and collision avoidance through a trial-and-error process, as shown in Fig. 5(d). In the later stage of training, the upper limits of the capabilities of different models start to emerge. Among them, the timeout rate of MRV-A only begins to rise slowly because low data efficiency leads to a large exploration requirement.

[0127] ② Comparison with the baseline method in multiple scenarios:

[0128] In this subsection, this embodiment conducts a large number of experiments in various scenarios without any fine-tuning procedures to evaluate the performance of all models, except for MRV-a, which has been proven ineffective in the training scenarios of this embodiment. There are 200 test cases in each evaluation.

[0129] Performance in the training scenario: This embodiment evaluates the model in the training scenario and measures its performance according to three metrics, namely the success rate, the extra distance rate, and the average speed, as shown in Figs. 6(a)-(c). It can be observed that SAPI has the highest success rate and the lowest extra distance rate, as well as a relatively high average speed. This indicates that SAPI allows each robot to reach the destination at a relatively fast speed while following a shorter path. At the same time, this embodiment finds that SelComm performs poorly in the environment of this embodiment due to the lack of perception of complex obstacles. The lowest success rate and the highest extra distance rate of SAPI-S indicate that effective agent-level interaction is crucial for collision avoidance and motion coordination among robots.

[0130] Generalization to different scenarios: To evaluate the generalization ability of the method, this embodiment sets up three scenarios, namely the crowded scenario, the corridor scenario, and the dynamic scenario, where all robots need to complete the position exchange task. The crowded scenario has ten randomly placed obstacles, which is twice that of the training scenario, while other configurations are the same as the training scenario. The corridor scenario is 3×12 meters in size, with five randomly placed obstacles and eight robots, and its narrow passage area increases the possibility of robot path conflicts. In addition, the dynamic scenario is the same size as the training scenario but contains two dynamic obstacles, set at 0.2 m / s and 0.3 m / s respectively. Table 2 shows the quantitative performance of all methods in each generalization scenario.

[0131] Table 2: Performance in generalization scenarios

[0132]

[0133]

[0134] It can be seen that since the SAPI of this embodiment integrates powerful perception capabilities and efficient interaction capabilities, the success rate of SAPI is the highest in all general scenarios. In particular, this embodiment notes that in crowded scenarios, the semantic information of visual observation is more important than the geometric information because it is necessary to effectively identify and avoid densely displayed obstacles. In crowded scenarios, due to the increase in the number of complex obstacles, the performance of Selcomm drops sharply. In addition, in corridor scenarios with long-distance navigation requirements and few obstacles, geometric information is more helpful for collaborative navigation. This embodiment also notes that due to the high requirements for motion coordination in narrow areas, SAPI-S performs the worst in corridor scenarios. In addition, the performance and trajectories in dynamic scenarios show that the strategy guided by the designed composite reward function can enable the robot to avoid obstacles in advance to a certain extent.

[0135] Scalability of large robot teams: Here, this embodiment tests scalability, that is, whether the strategy for a small number of robots is still effective for a large robot team. This embodiment constructs scene 1 (scene1) with a size of 8×8 meters and scene 2 (scene2) with a size of 10×10 meters, which contain 12 obstacles and 20 obstacles respectively. Then, this embodiment designs six robot teams of different sizes for the position exchange task: there are 12, 16, and 20 robots in scene 1, and 24, 30, and 36 robots in scene 2. It is worth noting that the more robots there are in the same scene, the greater the possibility of path conflicts, and the increase in the scene area means that the robots need to navigate a longer distance. The model performance under different team sizes is shown in Figures 7(a)-(c). The method of this embodiment is still effective for large robot teams, even in a system where the number of robots is six times that of the training scenario.

[0136] First, the expansion of the robot team does indeed affect the success rate of the method because it means more complex interactions and less accessible areas. In particular, as the number of robots increases, the performance of SAPI-S drops sharply due to the lack of motion coordination ability. Second, as the robot density increases, the extra distance rate will increase because a robot needs to detour a longer distance to avoid moving conflicts with its collaborators. Finally, due to the crowded environment, robots tend to make cautious decisions and navigate at a lower speed to ensure safety.

[0137] In summary, this example designs a framework to integrate sensor-level and agent-level information to enhance robot perception, thereby facilitating collaborative navigation in complex scenarios. The pre-training of visual assistance tasks significantly improves the sample efficiency of the model, and the combination of semantic prior and geometric prior is proven to be effective for navigation tasks. In addition, an attention-based sensor fusion network is designed to effectively integrate sensor-level features. The graph convolutional interaction with attention kernels is proven to be highly beneficial for the motion coordination of robots. In addition, a composite reward function with multiple sub-goals is designed to guide the learning of navigation strategies. The superiority of the method in this embodiment, its generalization ability for unknown scenarios, and its scalability for large robot teams are demonstrated through extensive experiments.

[0138] Embodiment 2:

[0139] A system for implementing the above method, comprising:

[0140] A sensor encoding module, configured to: obtain the RGB image, lidar data, and motion vector data of the robot at a specified moment, extract features from the RGB image and lidar data based on a visual encoder, and convert the features of the RGB image into a visual feature hidden vector;

[0141] A hierarchical perception fusion module, configured to: fuse the visual feature hidden vector and lidar features through a sensor fusion network to obtain a sensor-level feature vector; based on a graph neural network, aggregate the sensor-level feature vectors of all neighboring robots within the communication range of the robot to obtain an agent-level neighborhood perception vector;

[0142] A policy learning module, configured to: use the connected sensor-level feature vector and agent-level neighborhood perception vector as the input of the actor network and the critic network, set a reward function with target arrival, collision avoidance, and time efficiency to obtain the optimal navigation policy of the robot, and realize the collaborative navigation of the robot team.

[0143] In the first stage, fuse the observation data at the sensor level to obtain better environmental perception. In the second stage, aggregate the information at the agent level to achieve effective coordination. Through the strategy of multi-robot collaborative navigation, it is possible to achieve collaborative navigation and collision avoidance in a complex real-world environment.

[0144] Embodiment 3:

[0145] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps in the distributed multi-robot navigation method described in Embodiment 1 above.

[0146] In the first stage of the distributed multi-robot navigation method, the observation data at the sensor level is fused to obtain better environmental perception. In the second stage, the information at the agent level is aggregated to achieve effective coordination. Through the strategy of multi-robot cooperative navigation, it is possible to achieve cooperative navigation and collision avoidance in a complex real-world environment.

[0147] Embodiment 4:

[0148] This embodiment provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the distributed multi-robot navigation method described in Embodiment 1 above.

[0149] In the first stage of the distributed multi-robot navigation method, the observation data at the sensor level is fused to obtain better environmental perception. In the second stage, the information at the agent level is aggregated to achieve effective coordination. Through the strategy of multi-robot cooperative navigation, it is possible to achieve cooperative navigation and collision avoidance in a complex real-world environment.

[0150] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For the specific implementation manners, reference can be made to the relevant description part of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0151] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A distributed multi-robot navigation method, characterized in that: It includes the following steps: Obtain the RGB image, lidar data, and motion vector data of the robot at a specified moment, extract features from the RGB image and lidar data based on a visual encoder, and convert the features of the RGB image into a visual feature latent vector; Fuse the visual feature latent vector and lidar features through a sensor fusion network to obtain a sensor-level feature vector; based on a graph neural network, aggregate the sensor-level feature vectors of all neighboring robots within the communication range of the robot to obtain an agent-level neighborhood perception vector; Use the connected sensor-level feature vector and agent-level neighborhood perception vector as the inputs of the actor network and critic network, set a reward function based on target arrival, collision avoidance, and time efficiency to obtain the optimal navigation strategy of the robot, and achieve cooperative navigation of the robot team.

2. The distributed multi-robot navigation method according to claim 1, characterized in that: The visual encoder projects the latent representation in the RGB image onto a semantic segmentation map and a depth estimation map to obtain a visual feature latent vector representing pixel-level prediction; the latent representation contains the understanding of semantic and geometric information.

3. The distributed multi-robot navigation method according to claim 1, characterized in that: The sensor fusion network has H attention fusion units. In the h-th attention fusion unit, the outputs of all attention fusion units are concatenated together by weighted summation and projected onto a fusion feature, and the fusion feature is the sensor-level feature vector of the robot.

4. The distributed multi-robot navigation method according to claim 1, characterized in that: In the graph neural network, the communication topology of the robot team is converted into a bidirectional graph, where each node represents a robot. If the Euclidean distance between two nodes is less than the communication radius r comm , there is a bidirectional edge between the two nodes, and they receive messages from each other's nodes. The messages are obtained by concatenating the sensor-level representation and motion measurement of robot i.

5. The distributed multi-robot navigation method according to claim 4, wherein: The graph neural network has M independent attention heads. In the m-th head, the message of robot i is projected into query, key, and value through three linear transformations with matrices and respectively. The importance scores of robot i and its neighbor robot j are as follows: where d K represents the dimension of the key and is used as the scaling factor. After receiving messages from all neighbors, robot i calculates the normalized attention scores for each neighbor: Among them, N i represents the neighbor set of robot i.

6. The distributed multi-robot navigation method according to claim 4, wherein: Domain messages aggregated by the first-round robot i Generated by projecting the concatenated vectors output by all M attention heads: where σ is the activation function, and f proj is a fully connected layer with a set number of activation function units; the receptive field of the robot is expanded by collecting information from multi-hop neighbors, and multi-hop messages are aggregated and regarded as the functional representation at the agent level, that is, the neighborhood perception vector at the agent level is obtained.

7. The distributed multi-robot navigation method according to claim 1, characterized in that: The actor network has at least two sequential fully connected layers and an output layer with at least two branches, where one output is the translational velocity and the other output is the rotational velocity The critic network generates a state value based on the output layer of the neurons 8. Distributed multi-robot navigation system, characterized in that: It includes: A sensor encoding module, configured to: obtain the RGB image, lidar data, and motion vector data of the robot at a specified moment, extract features from the RGB image and lidar data based on a visual encoder, and convert the features of the RGB image into a visual feature latent vector; A hierarchical perception fusion module, configured to: fuse the visual feature latent vector and lidar features through a sensor fusion network to obtain a sensor-level feature vector; based on a graph neural network, aggregate the sensor-level feature vectors of all neighboring robots within the communication range of the robot to obtain an agent-level neighborhood perception vector; A policy learning module, configured to: use the connected sensor-level feature vector and agent-level neighborhood perception vector as the inputs of the actor network and critic network, set a reward function based on target arrival, collision avoidance, and time efficiency to obtain the optimal navigation strategy of the robot, and achieve cooperative navigation of the robot team.

9. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the distributed multi-robot navigation method according to any one of claims 1-7 above.

10. A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the distributed multi-robot navigation method according to any one of claims 1-7.

Images