Lightweight map prediction and fusion system for indoor multi-robot exploration

Abstract

The invention relates to a lightweight map prediction and fusion system for indoor multi-robot exploration, and the system comprises a plurality of robots which are used for exploring a target environment and collecting local observation data meeting preset conditions in the exploration process; the prediction module is used for processing the local observation data of each robot by using a pre-constructed map prediction model to obtain a corresponding local prediction map, and the map prediction model takes a teacher-student model as a framework and is obtained by combining the framework with a generative adversarial network and knowledge distillation loss training; and the fusion module is used for fusing the local prediction maps of the plurality of robots by using uncertainty so as to integrate the local observation and prediction information of the plurality of robots and obtain a global map of the target environment under the preset space-time constraint. Therefore, the technical problem that the accuracy of the global map is affected due to the fact that the calculation loss is large, the multi-robot collaborative fusion efficiency is low and the fusion failure or false fusion phenomenon exists in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a lightweight map prediction and fusion system for indoor multi-robot exploration. Background Art

[0002] In the field of autonomous robotic exploration, multi-robot collaborative operations in unknown indoor environments have long faced the dual challenges of map-building efficiency and system resource constraints. Traditional incremental exploration methods adopt a "gradual map expansion" strategy, such as the Frontier-Based Exploration (FBEX) algorithm, in which robots gradually explore unknown areas and update local maps. This method works well in environments with simple structures, but in complex indoor scenes (such as multi-story buildings and multi-compartment layouts), robots are often blocked by obstacles, resulting in "blind spots" in their field of view, due to the detection range of lidar or visual sensors (typically 2-10 meters). This leads to problems such as repeated path planning and invalid return trips. For example, when a robot detects the entrance to a room in a corridor, it needs to make multiple round trips to fully build a map of the room's interior, which increases exploration time by more than 30%.

[0003] In related technologies, multi-robot collaboration can effectively speed up exploration efficiency. However, in multi-robot collaboration scenarios, related technologies face more complex challenges.

[0004] Computing resource bottlenecks limit model deployment, and traditional global map prediction models have large parameter magnitudes.

[0005] When multiple robots first establish communication, their local maps typically overlap less than 20% due to their different exploration areas. Directly applying traditional registration algorithms like ICP (Iterative Closest Point) can lead to fusion failure due to insufficient feature matching. For example, if two robots explore from opposite ends of a corridor, their initial maps only overlap slightly at the entrance. In this scenario, the traditional ORB feature matching algorithm can produce fewer than 10 correct matches, failing to meet the robustness requirements of the RANSAC algorithm.

[0006] In addition, the fusion methods of related technologies are prone to "false fusion" when robots collaborate based on predicted maps - that is, areas with large prediction errors are mistakenly merged as real environments, resulting in topological errors in the global map.

[0007] In summary, the relevant technologies have high computational consumption, low efficiency of multi-robot collaborative fusion, and the occurrence of fusion failure or false fusion, which in turn affects the accuracy of the global map and needs to be improved urgently. Summary of the Invention

[0008] The present application provides a lightweight map prediction and fusion system for indoor multi-robot exploration to solve the technical problems in related technologies, such as high computational consumption, low efficiency of multi-robot collaborative fusion, fusion failure or false fusion, which in turn affects the accuracy of the global map.

[0009] The first aspect of the present application provides a lightweight map prediction and fusion system for indoor multi-robot exploration, including: multiple robots, used to explore the target environment and collect local observation data that meets preset conditions during the exploration process; a prediction module, used to use a pre-built map prediction model to process the local observation data of each robot to obtain a corresponding local prediction map, wherein the map prediction model is based on a teacher-student model framework, and is obtained by combining the framework with a generative adversarial network and knowledge distillation loss training; a fusion module, used to perform uncertainty modeling on the local observation data of each robot to obtain a modeling result, and fuse the local prediction maps of the multiple robots based on the modeling result to integrate the local observation and prediction information of the multiple robots to obtain a global map of the target environment under preset time and space constraints.

[0010] Optionally, in one embodiment of the present application, each robot includes: a positioning unit for obtaining the robot's first positioning information and exploration path; a collection unit for collecting corresponding environmental data with the robot as the center during the exploration process; a communication unit for bidirectionally communicating with other robots within a preset communication range to adjust the exploration path based on the second positioning information of the other robots; a judgment unit for judging whether the robot meets the preset data output conditions in combination with the first positioning information, the exploration path and the environmental data; and an output unit for outputting the local observation data based on the environmental data, the first positioning information and the exploration path when the preset data output conditions are met.

[0011] Optionally, in one embodiment of the present application, the map prediction model includes: a teacher model, used to extract environmental features from the local observation data, so as to use the environmental features to generate a teacher prediction map that meets a preset richness condition; a student model, used to receive the projected environmental features output by the intermediate layer of the teacher model, and generate a corresponding student prediction map based on the projected environmental features; a discriminator, used to distinguish between the real map of the target environment and the student prediction map, and capture the differences in unknown areas in the target environment, and construct a loss function based on the differences to train the student model using the loss function until the trained student model meets the preset accuracy condition; a loss trainer, which calculates the difference between the corresponding layer features of the teacher model and the trained student model, and guides the trained student model to learn the feature extraction capability of the teacher model to output the local prediction map.

[0012] Optionally, in one embodiment of the present application, the fusion module includes: a modeling unit for calculating the pixel-level uncertainty of the local observation data to obtain a corresponding pixel-level uncertainty map; an extraction unit for obtaining feature points of each robot that meet preset reliability conditions based on the pixel-level uncertainty map and the real map of the target environment; a matching unit for matching feature points from different robots, and calculating corresponding matching scores by introducing uncertainty weighting to obtain corresponding matching pairs and weighted matching information; a spatial transformation unit for estimating the corresponding spatial transformation matrix based on the matching pairs and the weighted matching information; a temporal transformation unit for obtaining a corresponding temporal transformation matrix based on the spatial transformation matrix; and a fusion unit for combining the spatial transformation matrix, the temporal transformation matrix and the feature points to obtain the global map.

[0013] Optionally, in one embodiment of the present application, the calculation expression of the pixel-level uncertainty is:

[0014]

[0015] Among them, U(x,y) represents the pixel-level uncertainty, α and β represent the corresponding weighting coefficients, represents the predicted probability variance, and S(x,y) represents the edge strength.

[0016] Optionally, in one embodiment of the present application, the time transformation unit includes: a constraint unit, configured to constrain the spatial transformation matrix through a sliding window to obtain the time transformation matrix;

[0017] Among them, the expression of the sliding window constraint is:

[0018]

[0019] Where H(t) represents the total transformation at the current time t, the subscript F represents the Frobenius norm, γ represents the weighting coefficient, and M t represents the map at the current time t, M consistent represents the desired consistent map state, and ∈ represents the threshold parameter.

[0020] The second aspect of the present application provides a lightweight map prediction and fusion method for indoor multi-robot exploration, including: obtaining local observation data collected by multiple robots during the exploration process that meets preset conditions; using a pre-built map prediction model to process the local observation data of each robot to obtain a corresponding local prediction map, wherein the map prediction model is based on a teacher-student model framework, and is obtained by combining the framework with a generative adversarial network and knowledge distillation loss training; uncertainty modeling is performed on the local observation data of each robot to obtain a modeling result, and the local prediction maps of the multiple robots are fused based on the modeling result to integrate the local observation and prediction information of the multiple robots to obtain a global map of the target environment under preset time and space constraints.

[0021] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the lightweight map prediction and fusion method for indoor multi-robot exploration as described in the above embodiment.

[0022] A fourth aspect of the present application provides a computer-readable storage medium storing computer instructions for enabling the computer to execute the lightweight map prediction and fusion method for indoor multi-robot exploration as described in the above embodiments.

[0023] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned lightweight map prediction and fusion method for indoor multi-robot exploration.

[0024] The embodiments of the present application can utilize multiple robots for collaborative exploration and perform local map prediction based on the observation data collected by each robot during exploration. During prediction, a lightweight map prediction model is used to achieve efficient and accurate data output under resource-constrained conditions, enabling efficient deployment of multiple robots. Furthermore, when fusing multi-robot data, robust map merging is achieved through uncertainty modeling and spatiotemporal constraints, thereby improving the exploration efficiency and map quality of multiple robots in complex indoor environments. This solves the technical problems in related technologies, such as high computational cost, low efficiency of multi-robot collaborative fusion, and the occurrence of fusion failures or false fusion, which in turn affect the accuracy of the global map.

[0025] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 Schematic diagram of the structure of a lightweight map prediction and fusion system for indoor multi-robot exploration provided in accordance with an embodiment of the present application;

[0028] Figure 2 This is a schematic diagram of the principles of a lightweight map prediction and fusion system for indoor multi-robot exploration provided according to one embodiment of the present application;

[0029] Figure 3 A schematic diagram of the principles of a map prediction model provided according to one embodiment of the present application;

[0030] FIG4( a ) is a schematic diagram showing comparison results of coverage and map quality using different methods according to an embodiment of the present application;

[0031] FIG4( b ) is a schematic diagram showing a comparison of exploration efficiency of different methods provided according to an embodiment of the present application;

[0032] FIG4( c ) is a schematic diagram showing a comparison of exploration efficiency of different methods provided according to another embodiment of the present application;

[0033] Figure 5 This is a flowchart of a lightweight map prediction and fusion method for indoor multi-robot exploration provided in accordance with an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0036] The following describes a lightweight map prediction and fusion system for indoor multi-robot exploration according to an embodiment of the present application with reference to the accompanying drawings. In response to the technical problems mentioned in the above background technology, such as high computational cost, low efficiency of multi-robot collaborative fusion, fusion failure or false fusion, which in turn affect the accuracy of the global map, the present application provides a lightweight map prediction and fusion system for indoor multi-robot exploration. In this system, multiple robots can be used for collaborative exploration, and local map prediction is performed based on the observation data collected by each robot during exploration. During the prediction, a lightweight map prediction model is used to achieve efficient and accurate data output under resource-constrained conditions, thereby achieving efficient deployment of multiple robots. When performing multi-robot data fusion, robust map merging is achieved through uncertainty modeling and spatiotemporal constraints, thereby improving the exploration efficiency and map quality of multiple robots in complex indoor environments. As a result, the technical problems of high computational cost, low efficiency of multi-robot collaborative fusion, fusion failure or false fusion, which in turn affect the accuracy of the global map in the related art are solved.

[0037] Understandably, deep learning-based map prediction technology has become a research hotspot in recent years. Typical solutions, such as the MapEx algorithm, employ a LaMa (Large Mask Inpainting) network architecture, combining U-Net with Fourier convolution to achieve global map prediction. Katyal et al. utilize U-Net to complete the occupancy grid for unobserved areas. These methods, by learning from environmental structural patterns in historical exploration data, can predict the distribution of obstacles outside the sensor's field of view in advance. However, existing prediction models commonly suffer from "model bloat": the LaMa-Fourier network has 27 million parameters, requiring 66,000 MFLOPs of computation for a single inference, far exceeding the computing power limit of small robot embedded processors (such as the NVIDIA Jetson Nano, approximately 100 GFLOPs). This results in significant inference latency in practical deployments (single-frame predictions take over 500ms).

[0038] The technical solutions for multi-robot collaborative operations in related technologies have core flaws:

[0039] First, computing resource bottlenecks limit model deployment. Traditional global map prediction models cannot run on edge devices with less than 4GB of memory due to their large number of parameters (such as U-Net, which has 31 million parameters). This leads to a serious mismatch between "algorithm performance" and "hardware capabilities." Second, there is a lack of multi-robot collaboration mechanisms. Existing methods are mostly designed for single-robot scenarios, without considering the geographical constraints of multi-agent collaboration. Figure 1First, there are problems with consistency maintenance, such as the lack of modeling of the uncertainty of the robot's relative posture, which leads to "cracks" or "overlapping dislocations" in the fused map; third, there is insufficient adaptability to dynamic environments. Traditional fusion algorithms (such as map merging based on particle filtering) cannot handle the real-time iterative updates of the predicted map. When the robot moves and causes the predicted map to change dynamically, spatiotemporal continuity is prone to breakage. For example, at a 10Hz update frequency, the fusion delay of traditional methods can reach 200ms, resulting in delayed robot decision-making.

[0040] The root cause of these defects is that the relevant technologies have not achieved an organic combination of "model lightweight" and "multi-agent collaboration" - on the one hand, the global prediction model has not been optimized for the characteristics of the robot hardware; on the other hand, the map fusion process does not fully utilize the uncertainty information of the predicted map, making it difficult to achieve robust collaboration in indoor environments without GPS or global positioning.

[0041] Based on the above-mentioned defects, the lightweight map prediction and fusion system for indoor multi-robot exploration proposed in the embodiment of the present application can provide a solution for the fusion of lightweight map prediction and uncertainty perception to address the problems existing in indoor multi-robot exploration technology, such as high computing resource consumption, low efficiency of multi-robot collaborative fusion, and poor adaptability to dynamic environments.

[0042] Specifically, the embodiments of the present application can solve the following technical problems: First, to overcome the defects of large parameter scale and high computational complexity of traditional global map prediction models, the embodiments of the present application can achieve efficient deployment of resource-constrained robot platforms through lightweight design; Second, to solve the problem of few map overlapping areas and difficult global alignment when multiple robots collaborate, the embodiments of the present application can construct a distributed map fusion mechanism that does not require a predefined coordinate system; Third, to cope with the challenges of spatiotemporal consistency in dynamic updates of predicted maps, the embodiments of the present application can achieve robust map merging through uncertainty modeling and spatiotemporal constraints, thereby improving the exploration efficiency and map quality of multiple robots in complex indoor environments.

[0043] Specifically, Figure 1 A schematic diagram of the structure of a lightweight map prediction and fusion system for indoor multi-robot exploration provided in an embodiment of the present application.

[0044] like Figure 1 As shown, the lightweight map prediction and fusion system 10 for indoor multi-robot exploration includes: multiple robots 100, a prediction module 200 and a fusion module 300.

[0045] Specifically, multiple robots 100 are used to explore the target environment and collect local observation data that meets preset conditions during the exploration process.

[0046] The embodiment of the present application can improve the exploration efficiency by collaborative exploration through multiple robots 100, wherein each robot 100 will collect peripheral data with itself as the center during the exploration process to obtain corresponding local observation data, wherein the preset conditions will be explained below.

[0047] Optionally, in one embodiment of the present application, each robot 100 includes: a positioning unit, a collection unit, a communication unit, a judgment unit and an output unit.

[0048] The positioning unit is used to obtain the first positioning information and exploration path of the robot 100.

[0049] The collection unit is used to collect corresponding environmental data with the robot 100 as the center during the exploration process.

[0050] The communication unit is used for bidirectional communication with other robots 100 within a preset communication range to adjust the exploration path based on the second positioning information of the other robots 100.

[0051] The judgment unit is used to judge whether the robot 100 meets the preset data output conditions based on the first positioning information, the exploration path and the environmental data.

[0052] The output unit is used to output local observation data based on the environmental data, the first positioning information and the exploration path when the preset data output conditions are met.

[0053] In the actual implementation process, in the data collection phase, each robot 100 in an unknown 2D environment models the environment as a 2D occupancy grid with a matrix E∈{0,1} W×H At each time step t, the robot 100 senses obstacles within the local perception radius l and encodes the information into a local occupancy grid m t , and incrementally fuse these local observations to build and update its own map M i , and know that you are in M i The robot 100 can also communicate bidirectionally with other robots 100 within the communication range to share information. When constructing the dataset, the HouseExpo simulator and the KTH dataset were used. Regions were randomly selected and the positions of the two robots 100 were initialized. When their shortest path distance and local observation maps met specific conditions, the maps were encoded into three channels: obstacle, uncertain, and free. The local observation map, true map, and coordinates of each robot 100 at each time step were stored, representing the local observation data at each time step.

[0054] The prediction module 200 is used to process the local observation data of each robot 100 using a pre-built map prediction model to obtain a corresponding local prediction map, wherein the map prediction model is based on a teacher-student model framework and is obtained by combining the framework with a generative adversarial network and knowledge distillation loss training.

[0055] The embodiments of this application utilize a pre-built lightweight map prediction model to process local observation data. This model employs a teacher-student framework based on generative adversarial networks and knowledge distillation to guide the student model to match the feature extraction capabilities of the teacher model, thereby generating a local prediction map. This lightweight design enables efficient deployment on resource-constrained robotic platforms, overcoming the limitations of traditional global map prediction models, which suffer from large parameter sizes and high computational complexity.

[0056] Optionally, in one embodiment of the present application, the map prediction model includes: a teacher model, a student model discriminator and a loss trainer.

[0057] Among them, the teacher model is used to extract environmental features from local observation data to use the environmental features to generate a teacher prediction map that meets the preset richness conditions.

[0058] The student model is used to receive the projected environmental features output by the intermediate layer of the teacher model and generate the corresponding student prediction map based on the projected environmental features.

[0059] The discriminator is used to distinguish between the real map of the target environment and the student's predicted map, and capture the differences in the unknown areas of the target environment. A loss function is constructed based on the differences to train the student model using the loss function until the trained student model meets the preset accuracy conditions.

[0060] The loss trainer calculates the difference between the corresponding layer features of the teacher model and the trained student model, and guides the trained student model to learn the feature extraction ability of the teacher model to output a local prediction map.

[0061] As a possible implementation method, in the map prediction model of the embodiment of the present application, the teacher model includes additional convolution layers and fast Fourier convolution modules for more accurately extracting features from the local observation map. These features are projected to the corresponding layers of the student model through 1×1 convolution, so that the student model can approximate the extraction performance of the teacher model while greatly reducing the number of parameters and computational cost. When training the teacher model, within the framework of the generative adversarial network, the discriminator improves the ability to distinguish between the real map and the predicted map by enhancing the convolution and fast Fourier convolution modules. The model only focuses on the uncertain areas (represented by masks) in the predicted observation map, and the adversarial loss is only calculated in these areas. At the same time, the loss function also incorporates the feature reconstruction loss based on the high receptive field ResNet to encourage the model to match the structural features of the real map, and the L2 (mean square error loss) loss to penalize the pixel differences between the prediction and the real map in the uncertain area. In addition to using the same loss function as the teacher, the student model also applies a four-layer distillation loss to guide it to match the feature extraction ability of the teacher model to generate a local prediction map.

[0062] The fusion module 300 is used to perform uncertainty modeling on the local observation data of each robot 100 to obtain a modeling result, and to fuse the local prediction maps of multiple robots based on the modeling result to integrate the local observation and prediction information of multiple robots 100 to obtain a global map of the target environment under preset time and space constraints.

[0063] After obtaining the local prediction maps of multiple robots 100, the embodiment of the present application can use uncertainty and spatiotemporal constraints to achieve robust map merging to address the spatiotemporal consistency challenges in the dynamic update of the prediction map, thereby improving the exploration efficiency and map quality of multiple robots in complex indoor environments.

[0064] Optionally, in one embodiment of the present application, the fusion module 300 includes: a modeling unit, an extraction unit, a matching unit, a spatial transformation unit, a temporal transformation unit and a fusion unit.

[0065] Among them, the modeling unit is used to calculate the pixel-level uncertainty of local observation data to obtain the corresponding pixel-level uncertainty map.

[0066] The extraction unit is used to obtain feature points of each robot 100 that meet preset reliability conditions based on the pixel-level uncertainty map and the real map of the target environment.

[0067] The matching unit is used to match feature points from different robots 100 and calculate corresponding matching scores by introducing uncertainty weighting to obtain corresponding matching pairs and weighted matching information.

[0068] The spatial transformation unit is used to estimate the corresponding spatial transformation matrix based on the matching pairs and the weighted matching information.

[0069] The time transformation unit is used to obtain a corresponding time transformation matrix based on the spatial transformation matrix.

[0070] The fusion unit is used to combine the spatial transformation matrix, the temporal transformation matrix and the feature points to obtain a global map. The calculation expression of the pixel-level uncertainty is:

[0071]

[0072] Among them, U(x,y) represents the pixel-level uncertainty, α and β represent the corresponding weighting coefficients, It represents the predicted probability variance, S(x,y) represents the edge strength extracted by operators such as Canny, and reflects the significance of the structural features of the pixel position. The higher the edge strength, the more important the structural information at that position.

[0073] The fusion module 300 can perform uncertainty modeling on the local map of each robot 100 and define pixel-level uncertainty in is the predicted probability variance, S(x,y) is the Canny edge strength, and the probability and structural clues are balanced by weighting coefficients α=0.7 and β=0.3. In this embodiment of the present application, the fusion module 300 can define the uncertainty of each pixel (x,y) as the predicted occupancy variance Weighted sum of the structural edge responses S(x,y), with coefficients α=0.7 and β=0.3.

[0074] In the feature extraction stage, ORB key points, edge features and morphological features are integrated, and only reliable features with uncertainty less than 0.8 times the mean are retained for matching, excluding or reducing the influence of unreliable regional features, and weighted matching scores are calculated based on uncertainty. Enhanced noise resistance, uncertainty calculation based on matching feature points, using a relaxed Lowe ratio (0.9) to retain more matching pairs, using only ω m >0.5, and incorporate the matching weights into the RANSAC selection to improve the robustness to noisy correspondences. Decompose the transformation into spatial and temporal parts. That is, combine the weighted RANSAC algorithm to estimate the spatial transformation matrix H space In order to deal with the spatiotemporal discontinuity caused by dynamic updates, the time transformation matrix H is introduced time =exp(λΔt)I, constrained by the sliding window Avoid prediction drift. Finally, lightweight fusion is achieved through hierarchical multi-resolution feature partitioning (coarse granularity 0.25x, medium granularity 0.5x, full resolution), reducing the computational complexity from O(N 2 M) is reduced to O(NMlogM), supporting multi-robot distributed parallel computing.

[0075] A lightweight fusion method based on hierarchical multi-resolution feature partitioning and distributed parallel computing is adopted to extract features at coarse, medium and full resolutions. Coarse features are used for fast initial alignment, intermediate features are refined and transformed, and full-resolution matching is limited to overlapping areas to reduce computational complexity. Finally, the aligned maps are merged into a global map.

[0076] Optionally, in one embodiment of the present application, the time transformation unit includes: a constraint unit.

[0077] The constraint unit is used to constrain the spatial transformation matrix through a sliding window to obtain a temporal transformation matrix.

[0078] Among them, the expression of the sliding window constraint is:

[0079]

[0080] Among them, H(t) represents the total transformation at the current time t, the subscript F represents the Frobenius norm (a kind of matrix norm), which is used to measure the difference between the two transformation matrices H(t) and H(t-1) to ensure the smoothness of the transformation, and γ represents a weighting coefficient used to balance the transformation smoothness constraint and the ground Figure 1 The weight of the consistency constraint, M t represents the map at the current time t, M t ⊙M t-1 It represents the element-by-element product of the current map and the previous map, and is used to measure the consistency of the overlapping area between the two. The overlapping area refers to the part of the map constructed by the robot at two consecutive moments in the multi-robot exploration process where the spatial position overlaps. consistent Represents the expected consistent map state, which is used to measure the consistency of the current map and the previous map in the overlapping area to ensure the continuity of the fused map. ∈ represents a threshold parameter used to determine whether the constraint is satisfied. If the expression result is less than ∈, the current transformation is accepted; otherwise, it falls back to the previous reliable state to avoid error accumulation.

[0081] In order to deal with the spatiotemporal discontinuity caused by dynamic updating, the embodiment of the present application introduces the time transformation matrix H time =exp(λΔt)I, through the sliding window constraint Avoid forecast drift.

[0082] Combine Figures 2 to 4(c)As shown, the working principle of the lightweight map prediction and fusion system for indoor multi-robot exploration of the embodiment of the present application is described in detail using an embodiment.

[0083] like Figure 2 As shown, the embodiment of the present application is composed of a lightweight local map prediction model and an uncertainty perception map fusion module 300.

[0084] In terms of lightweight map prediction models, embodiments of the present application can adopt a teacher-student framework combined with generative adversarial networks and knowledge distillation technology. The input data of the framework comes from the local observation maps collected by the robot during the exploration process. These maps are encoded by the sensor of the robot 100 to the perception information of the surrounding environment and are divided into three channels: obstacles, free areas, and uncertain areas. When the dataset is constructed, the HouseExpo simulator and the KTH dataset are used to randomly select environmental areas and initialize the robot position. When the robot path distance and the proportion of obstacles and free areas in the local map meet specific conditions, the corresponding map data and coordinates are stored to form the input samples required for training.

[0085] When the model framework is running, the teacher model receives the local observation map as input, extracts features with the help of additional convolutional layers and fast Fourier convolution modules, and generates a high-fidelity prediction map; the student model receives the features projected by the teacher model through 1×1 convolution, which significantly reduces the number of parameters while approximating the feature extraction ability of the teacher model. During training, the discriminator of the generative adversarial network distinguishes between the real map and the predicted map. The adversarial loss is only calculated for the uncertain areas in the observation map (marked by the mask), prompting the model to focus on the prediction of unknown areas. At the same time, through the feature reconstruction loss based on the high receptive field ResNet, the model is forced to match the structural features of the real map, and the L2 loss is used to penalize the pixel differences between the prediction and the real map in the uncertain areas to ensure prediction accuracy.

[0086] During training, the student model uses the same adversarial loss, feature reconstruction loss, and L2 loss as the teacher model. In addition, it introduces a four-layer distillation loss. By calculating the differences in features of the corresponding layers between the teacher and student models, the student model is guided to learn the feature extraction capabilities of the teacher model. Ultimately, it outputs a lightweight local prediction map, achieving a balance between high accuracy and low computational overhead.

[0087] like Figure 3 As shown in the figure, the teacher model extracts local observation map features through a convolutional network containing a fast Fourier convolution module, and uses the discriminator f in the generative adversarial network framework to generate D (·) Distinguish between the real map and the predicted map, and only calculate the adversarial loss for the uncertainty area (Uncertain) At the same time, feature reconstruction loss based on high receptive field ResNet is introduced and L2 loss To ensure the structural fidelity of the predicted map, the student model projects the feature layer of the teacher model through 1×1 convolution and combines it with distillation loss. Optimizations have compressed the model parameters to 342KB, significantly reducing computational overhead (MACs to only 336MFLOPs), making it suitable for resource-constrained robotic platforms. The model uses three input channels (obstacles, free space, and uncertainty region) to predict unobserved areas, focusing on local maps rather than global inference, improving prediction robustness in complex environments.

[0088] In the uncertainty perception map fusion module 300, the embodiment of the present application designs a distributed fusion framework including probability modeling, feature alignment and spatiotemporal constraints. In the fusion module 300, the input data mainly comes from the local map set {M i (t)}, each local map contains the corresponding pose p i , Occupancy probability variance And the structural characteristics S i These local maps are generated by the robots using their sensors to perceive the environment and combining them with local predictions from a map prediction model. They record the distribution of obstacles, free space, and uncertainty areas in their respective primary observation areas. Input data also includes the communication status between robots 100. When a robot enters communication range, a fusion process is triggered, transmitting its respective local maps and associated uncertainty parameters.

[0089] The fusion module 300 can perform uncertainty modeling on the local map of each robot 100 and define pixel-level uncertainty in is the predicted probability variance, S(x,y) is the Canny edge strength, and the weighting coefficients α = 0.7 and β = 0.3 are used to balance the probability and structural clues. In the feature extraction stage, ORB key points, edge features and morphological features are integrated, and only reliable features with uncertainty less than 0.8 times the mean are retained for matching. The matching score is weighted by uncertainty. Enhance noise resistance and combine weighted RANSAC algorithm to estimate the spatial transformation matrix H space In order to deal with the spatiotemporal discontinuity caused by dynamic updates, the time transformation matrix H is introduced time =exp(λΔt)I, constrained by the sliding window Avoid prediction drift. Finally, lightweight fusion is achieved through hierarchical multi-resolution feature partitioning (coarse granularity 0.25x, medium granularity 0.5x, full resolution), reducing the computational complexity from O(N 2 N) is reduced to O(NMlogM), supporting multi-robot distributed parallel computing.

[0090] Finally, a hierarchical multi-resolution feature alignment strategy is adopted to gradually refine the transformation matrix H at coarse, medium and full resolutions. fine , lightweight fusion is performed based on overlapping area features, and each local map is merged into a unified global map M through weighted mixing (weight range 0.3-0.7) global The output global map integrates the local observation and prediction information of multiple robots 100, eliminates map discontinuities, and forms a global environment representation with a consistent coordinate system, spatiotemporal coherence, and uncertainty quantification, providing a unified map foundation for collaborative exploration by multiple robots 100.

[0091] In the multi-robot exploration protocol, each robot 100 independently selects an exploration target based on a local predicted map, plans a path through frontier detection and information gain maximization strategy, and triggers map fusion only within the communication range. During fusion, low-resolution features and uncertainty masks are exchanged to reduce the amount of data transmitted. Map merging without global positioning is achieved through a decentralized protocol, and a unified global occupancy grid map is finally generated to guide collective exploration. The fusion process uses hierarchical multi-resolution feature partitioning to achieve map merging without global positioning, adapting to real-time collaboration in dynamic indoor environments. The embodiment of the present application reduces the computational load through local prediction, improves fusion robustness through uncertainty modeling, and achieves multi-robot collaboration without a predefined coordinate system, effectively solving the resource bottleneck and collaboration efficiency problems of the existing technology.

[0092] In practical applications, the model's lightweightness and prediction accuracy are shown in Table 1, which compares the map prediction model. The map prediction model has only 342K parameters, a reduction of over 98% compared to U-Net (31.0M) and LaMa-Fourier (27.0M). Its computational complexity (MACs) is only 336MFLOPs, far lower than the comparison method (up to 66,000MFLOPs). Furthermore, the prediction quality metric, LPIPS, is 0.216, lower than U-Net's 0.238 and LaMa-Fourier's 0.303. The KID value of 0.054 is significantly better than MI-GAN's 0.307, demonstrating that the model maintains structural prediction accuracy while being lightweight. This performance stems from the combination of the teacher-student framework and knowledge distillation, which compresses the model size through feature transfer, while the adversarial training of the generative adversarial network ensures the authenticity of the prediction details.

[0093] Table 1

[0094]

[0095] In terms of map fusion robustness, as shown in Table 2, Table 2 is a comparison table of map fusion of predicted maps and observed maps. In the predicted map fusion scenario, the ORB improved algorithm of MSO achieved a mean_rmse of 0.80 and a mean_dice of 0.59, which is significantly improved compared with the traditional ORB algorithm (mean_rmse0.85, mean_dice0.51). In a dynamic environment, the spatiotemporal constraint mechanism controls the standard deviation of the fusion error (success_dist_std) to 79.89, which is better than the 77.14 of the traditional method, proving that uncertainty modeling and spatiotemporal transformation estimation effectively solve the problem of map misalignment. This effect is due to the pixel-level uncertainty weighted matching (ω m ) combined with the sliding window constraint to avoid prediction drift and false fusion.

[0096] Table 2

[0097]

[0098] In terms of multi-robot exploration efficiency, Figure 4(a)-Figure 4(c) As shown, MSO achieved 98% map coverage in single-robot scenarios on the KTH dataset, a 22% improvement over MapEx. When working with multiple robots, lightweight fusion reduced exploration time by 40%, and achieved superior map quality (FID value of 62.51) compared to baseline methods such as UPEN and IG-Hector. This performance is attributed to local predictions that reduce communication overhead (transmitting only low-resolution features), as well as the synergy between frontier detection and information gain strategies, enabling efficient multi-robot division of labor without global alignment.

[0099] According to the lightweight map prediction and fusion system for indoor multi-robot exploration proposed in the embodiments of this application, multiple robots can be used for collaborative exploration, and local map predictions can be made based on the observation data collected by each robot during exploration. During prediction, a lightweight map prediction model is used to achieve efficient and accurate data output under resource-constrained conditions, enabling efficient deployment of multiple robots. When performing multi-robot data fusion, robust map merging is achieved through uncertainty modeling and spatiotemporal constraints, thereby improving the exploration efficiency and map quality of multiple robots in complex indoor environments. This solves the technical problems in related technologies such as high computational cost, low efficiency of multi-robot collaborative fusion, fusion failure or false fusion, and thus affecting the accuracy of the global map.

[0100] Next, a lightweight map prediction and fusion method for indoor multi-robot exploration proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0101] Figure 5This is a flowchart of a lightweight map prediction and fusion method for indoor multi-robot exploration according to an embodiment of the present application.

[0102] like Figure 5 As shown in FIG, the lightweight map prediction and fusion method for indoor multi-robot exploration includes the following steps:

[0103] In step S501, local observation data meeting preset conditions collected by multiple robots during the exploration process is obtained.

[0104] In step S502, the local observation data of each robot is processed using a pre-built map prediction model to obtain a corresponding local prediction map, wherein the map prediction model is based on a teacher-student model framework, which is obtained by combining the framework with a generative adversarial network and knowledge distillation loss training.

[0105] In step S503, uncertainty modeling is performed on the local observation data of each robot to obtain a modeling result. Based on the modeling result, the local prediction maps of multiple robots are fused to integrate the local observation and prediction information of multiple robots to obtain a global map of the target environment under preset time and space constraints.

[0106] It should be noted that the aforementioned explanation of the embodiment of the lightweight map prediction and fusion system for indoor multi-robot exploration is also applicable to the lightweight map prediction and fusion method for indoor multi-robot exploration in this embodiment, and will not be repeated here.

[0107] According to the lightweight map prediction and fusion method for indoor multi-robot exploration proposed in the embodiments of this application, multiple robots can be used for collaborative exploration, and local map prediction can be performed based on the observation data collected by each robot during exploration. During prediction, a lightweight map prediction model is used to achieve efficient and accurate data output under resource-constrained conditions, enabling efficient deployment of multiple robots. When performing multi-robot data fusion, robust map merging is achieved through uncertainty modeling and spatiotemporal constraints, thereby improving the exploration efficiency and map quality of multiple robots in complex indoor environments. This solves the technical problems in related technologies such as high computational cost, low efficiency of multi-robot collaborative fusion, fusion failure or false fusion, and thus affecting the accuracy of the global map.

[0108] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0109] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0110] When the processor 602 executes the program, the lightweight map prediction and fusion method for indoor multi-robot exploration provided in the above embodiment is implemented.

[0111] Furthermore, the electronic device further includes:

[0112] The communication interface 603 is used for communication between the memory 601 and the processor 602 .

[0113] The memory 601 is used to store computer programs that can be run on the processor 602 .

[0114] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0115] If the memory 601, processor 602, and communication interface 603 are implemented independently, the communication interface 603, memory 601, and processor 602 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0116] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0117] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0118] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned lightweight map prediction and fusion method for indoor multi-robot exploration.

[0119] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the lightweight map prediction and fusion method for indoor multi-robot exploration provided by an embodiment of the present invention.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0121] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0122] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0123] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0124] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0125] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0126] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0127] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A lightweight map prediction and fusion system for indoor multi-robot exploration, characterized by: include: Multiple robots are used to explore the target environment and collect local observation data that meets preset conditions during the exploration process; A prediction module, which processes each robot's local observation data using a pre-built map prediction model to obtain a corresponding local predicted map. The map prediction model is based on a teacher-student model framework, which is trained using a generative adversarial network and knowledge distillation loss. A fusion module is used to perform uncertainty modeling on the local observation data of each robot to obtain a modeling result, and to fuse the local prediction maps of the multiple robots based on the modeling result to integrate the local observation and prediction information of the multiple robots to obtain a global map of the target environment under preset spatiotemporal constraints.

2. The system according to claim 1, wherein: Each of the robots comprises: A positioning unit, used to obtain the robot's first positioning information and exploration path; A collection unit, used to collect corresponding environmental data with the robot as the center during the exploration process; a communication unit, configured to bidirectionally communicate with other robots within a preset communication range to adjust the exploration path based on the second positioning information of the other robots; a judgment unit, configured to judge whether the robot meets a preset data output condition based on the first positioning information, the exploration path, and the environmental data; An output unit is used to output the local observation data based on the environmental data, the first positioning information and the exploration path when the preset data output condition is met.

3. The system according to claim 1, wherein: The map prediction model includes: a teacher model for extracting environmental features from the local observation data to generate a teacher prediction map that meets a preset richness condition using the environmental features; The student model is configured to receive the projected environmental features output by the intermediate layer of the teacher model and generate a corresponding student prediction map based on the projected environmental features; a discriminator, configured to distinguish between the real map of the target environment and the predicted map of the student, and to capture the differences in the unknown areas of the target environment, and to construct a loss function based on the differences, so as to train the student model using the loss function until the trained student model meets a preset accuracy condition; A loss trainer calculates the difference between the features of the corresponding layers of the teacher model and the trained student model, and guides the trained student model to learn the feature extraction capability of the teacher model to output the local prediction map.

4. The system according to claim 1, wherein: The fusion module includes: a modeling unit, configured to calculate pixel-level uncertainty of the local observation data to obtain a corresponding pixel-level uncertainty map; an extraction unit, configured to obtain feature points of each robot that meet preset reliability conditions based on the pixel-level uncertainty map and a real map of the target environment; The matching unit is used to match feature points from different robots and calculate the corresponding matching scores by introducing uncertainty weighting to obtain corresponding matching pairs and weighted matching information; a spatial transformation unit, configured to estimate a corresponding spatial transformation matrix based on the matching pairs and the weighted matching information; A time transformation unit, configured to obtain a corresponding time transformation matrix based on the spatial transformation matrix; A fusion unit is used to combine the spatial transformation matrix, the temporal transformation matrix and the feature points to obtain the global map.

5. The system according to claim 4, characterized in that The calculation expression of the pixel-level uncertainty is: Among them, U(x,y) represents the pixel-level uncertainty, α and β represent the corresponding weighting coefficients, represents the predicted probability variance, and S(x,y) represents the edge strength.

6. The system according to claim 4, characterized in that The time conversion unit includes: a constraint unit, configured to constrain the spatial transformation matrix through a sliding window to obtain the temporal transformation matrix; Among them, the expression of the sliding window constraint is: Where H(t) represents the total transformation at the current time t, the subscript F represents the Frobenius norm, γ represents the weighting coefficient, and M t represents the map at the current time t, M consistent represents the desired consistent map state, and ∈ represents the threshold parameter.

7. A lightweight map prediction and fusion method for indoor multi-robot exploration, characterized in that: The method is applied to the lightweight map prediction and fusion system for indoor multi-robot exploration according to any one of claims 1 to 6, wherein the method comprises the following steps: Obtain local observation data that meets preset conditions collected by multiple robots during the exploration process; Using a pre-built map prediction model to process each robot's local observation data to obtain a corresponding local predicted map, wherein the map prediction model is based on a teacher-student model framework, which is trained by combining a generative adversarial network with a knowledge distillation loss. Uncertainty modeling is performed on the local observation data of each robot to obtain a modeling result. Based on the modeling result, the local prediction maps of the multiple robots are fused to integrate the local observation and prediction information of the multiple robots to obtain a global map of the target environment under preset spatiotemporal constraints.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the lightweight map prediction and fusion method for indoor multi-robot exploration as claimed in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the lightweight map prediction and fusion method for indoor multi-robot exploration as claimed in claim 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed, it is used to implement the lightweight map prediction and fusion method for indoor multi-robot exploration as claimed in claim 7.

Images