Edge-assisted cooperative mapping system and method suitable for communication limited environment

Abstract

In order to solve the mapping precision problem of collaborative mapping in a bandwidth-limited environment, the invention provides an edge-assisted collaborative mapping system and method specially designed for a communication-limited environment, the importance of key frames and landmarks can be quantified through a bandwidth-aware map distillation technology, and the mapping precision of the key frames and the landmarks is improved. And the sparsification degree of the data is dynamically adjusted according to real-time bandwidth evaluation, so that only the most critical map information is transmitted. Besides, a bandwidth coordinator is also introduced to optimize a data transmission strategy, and a credibility evaluator is also introduced to process communication congestion, so that accurate map construction and positioning can be realized under the condition that network resources are limited. Through the design, the reliability and the accuracy of cooperative work of the multi-robot system in a large-scale scene are improved while the communication efficiency is kept. The invention discloses an edge-assisted cooperative mapping system and method suitable for a communication limited environment.

Description

Technical Field

[0001] The present invention relates to the field of visual scene mapping technology, and more specifically, to an edge-assisted collaborative mapping system and method suitable for communication-restricted environments. Background Art

[0002] With the rapid development of autonomous systems, simultaneous localization and mapping (SLAM) technology has become a hot topic of research. This technology is essential for robots to build maps and self-localize in unknown environments. Traditional SLAM methods mainly focus on single agents, but with the increasing task requirements and the continuous expansion of robot working space, researchers have begun to turn to multi-agent systems (MAS), hoping to achieve efficient resource sharing and task optimization through collaboration between agents. In actual MAS, cameras have become the mainstream choice of robot sensors due to their high cost-effectiveness, low energy consumption, and ease of large-scale deployment, thus promoting significant progress in visual SLAM technology.

[0003] Recently, the COVINS-G framework fuses the backend with the frontend by introducing an innovative frontend wrapper that reduces the amount of data transmission by re-extracting landmark points (LM) in key frames (KF). However, the landmark points abstracted from the original sensor data may not fully capture the environmental characteristics, thus limiting the further improvement of the system performance.

[0004] The SwarmMap edge intelligence framework proposed by Tsinghua University effectively expands the scale of MAS by proposing data and task offloading strategies and map analysis methods. Although this method has improved accuracy and robustness compared to single SLAM, it is not optimized for the actual application scenario of bandwidth compression of a large number of new features newly observed in exploration tasks, and the mapping accuracy is not high in communication-restricted environments. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention optimizes data transmission to adapt to the current network conditions, aims to improve the efficiency of robot map construction in communication-constrained environments and reduce the system uncertainty caused thereby, thereby achieving more efficient SLAM performance in actual scenarios with limited resources.

[0006] In order to solve the above technical problems, the first aspect of the present invention provides an edge-assisted collaborative mapping system suitable for communication-constrained environments, including: multiple independent agents, an edge server, and a communication pipeline;

[0007] The agent is used to perceive the environment and collect visual information through the sensors it carries, and independently run the visual SLAM algorithm to generate keyframes and landmarks. It selects keyframes using a method based on information entropy, and then dynamically adjusts the sparsity of keyframes and landmarks according to the currently available network bandwidth to finally generate a submap.

[0008] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then fuse the sub-maps according to the fusion priority to obtain the global map;

[0009] The communication pipe is used to connect the intelligent agent with the edge server.

[0010] In one embodiment, the agent includes a tracking and mapping module and a map distillation module;

[0011] The tracking and mapping module includes a pose estimator and a first key frame selector, wherein the pose estimator is used to perceive the environment and collect visual information through the onboard sensor, and perform pose estimation based on the collected visual information, and the first key frame selector is used to select key frames using scene complexity based on information entropy;

[0012] The map distillation module includes an information gain quantizer, a landmark selector, a second key frame selector and a bandwidth coordinator. The gain quantizer is used to quantify the importance of each landmark through landmark stability and information gain. The landmark selector is used to perform sparsification operations on landmarks. The second key frame selector is used to select key frames by considering the spatial distribution between key frames and the importance of the landmarks they contain. The bandwidth coordinator is used to dynamically adjust the sparsification degree of key frames and landmarks according to the current network bandwidth conditions.

[0013] In one implementation, the first key frame selector is specifically used to:

[0014] Maintain a first-in-first-out queue to store recent frames and calculate the information entropy of new frames;

[0015] If the ratio of the information entropy of the new frame to the average information entropy of all frames in the queue exceeds a predefined threshold, the insertion of a new keyframe is triggered.

[0016] In one implementation, the gain quantizer is specifically configured to:

[0017] The stability of the landmark is determined by how many key frames the landmark is observed in, and the information gain is quantified by the impact of the landmark on the uncertainty of the pose estimation. The formula for quantifying the information gain is:

[0018] q a =f det (Q)

[0019] where qa represents the information gain of the landmark, f det (*) is the logarithm of the determination function, where: Q = W T W, P l and P kl is the vector from the optical center of the camera to the lth landmark point and the vector of the feature point corresponding to the landmark point in the kth key frame, is the latest rotation matrix estimated by the odometry, and the landmark stability score is:

[0020] q s =n

[0021] q s represents the stability score of the landmark observed by n key frames;

[0022] The importance of each landmark is quantified based on landmark stability and information gain.

[0023] In one implementation, the landmark selector is specifically used to:

[0024] The landmark sparsification task is formulated as a mixed integer programming problem. For a given landmark set {LM i}, combined with the stability score q calculated by the landmark information gain quantization module s , information gain q a Given the following constraints:

[0025]

[0026] stAx+ξ≥KI;Bx+φ≥I

[0027] where α and β represent weighted parameters that balance the importance of landmark stability and information gain, x represents a binary vector whose i-th element indicates whether landmark i is selected; λ1 and λ2 are weight vectors to be optimized, and A is a binary matrix representing LM i Yuan A ij In the key frame KF j is visible or invisible in the map, K represents the map compression factor, which indicates the maximum number of landmark points that can be observed in each keyframe, and B is a binary matrix representing LM i Yuan B ij Whether to be projected onto the grid g j In it, ξ and φ are soft constraints to ensure that the problem is solvable, and I is the identity matrix.

[0028] In one implementation, the second key frame selector is specifically used to:

[0029] For a given keyframe set {KF j} and entropy factor The , the keyframe trimming task is formulated as a mixed integer programming problem with the following constraints:

[0030]

[0031] s.tEy+ε≥The e

[0032] Where d represents the vector of spatial distribution of the jth keyframe, y is a binary vector indicating whether the jth element selects keyframe j, λ is the weight vector to be optimized, ε is a soft constraint to ensure that the problem is solvable, E represents the observation matrix of the indices of the landmarks observed by the keyframes, and Th e The pruning threshold for keyframes.

[0033] In one implementation, the bandwidth coordinator is specifically configured to:

[0034] Monitor the real-time bandwidth and dynamically adjust the sparsification factor K and key frame pruning threshold Th according to the available bandwidth e , dynamically adjust the sparsity of key frames and landmarks, and give the constraints as follows:

[0035]

[0036] Among them, K is the map compression factor, N LMs is the average number of landmarks observed by a keyframe in the unextracted sub-image, g(*) represents the keyframe sparsification ratio, and D LM and D KF represents the data size of the original landmark and the data size of the keyframe, B t is the available bandwidth at time Δt, where Δt is the data transmission interval.

[0037] In one embodiment, the edge server includes an agent management module, a map fusion module and a map management module;

[0038] The agent management module is used to manage the sub-maps of all agents, perform bookkeeping for all agents, and quantify the credibility of each sub-map based on the transmission stability of the channel corresponding to each sub-map;

[0039] The map fusion module is used to perform place recognition based on descriptor similarity and calculate the transformation between the newly received keyframe and the candidate keyframe, identifying and integrating map data from different agents to build a complete environment model;

[0040] The map management module is used to merge the agent sub-maps transformed by the map fusion module into the global map, where the merged maps are stored in descending order of credibility.

[0041] In one embodiment, the communication pipeline includes a Socket interface module and a sub-end management component module, wherein the Socket interface module is arranged at the intelligent body end, and includes a map buffer and a bandwidth monitor, the map buffer is used to alleviate the incompleteness of mapping caused by communication interruption, the bandwidth monitor regularly monitors the available bandwidth, and sets bandwidth limits for landmark and key frame filtering, and the sub-end management component module is arranged at the edge server end, and is used to feed back the update information of the global map to the intelligent body end in real time.

[0042] Based on the same inventive concept, the second aspect of the present invention provides an edge-assisted collaborative mapping method suitable for a communication-constrained environment, which is implemented based on the edge-assisted collaborative mapping system suitable for a communication-constrained environment described in the first aspect, including:

[0043] The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries, and independently runs the visual SLAM algorithm to generate keyframes and landmarks. The keyframes are selected using a method based on information entropy, and then the sparsity of keyframes and landmarks is dynamically adjusted according to the currently available network bandwidth. Finally, a submap is generated and sent to the edge server through a communication pipeline.

[0044] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then the sub-maps are fused according to the fusion priority to obtain the global map, which is then sent to the intelligent agent through the communication pipeline.

[0045] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0046] The present invention provides an edge-assisted collaborative mapping system and method suitable for communication-constrained environments. Through bandwidth-aware map distillation technology, it can quantify the importance of key frames and landmarks, and dynamically adjust the sparsity of data based on real-time bandwidth evaluation to ensure that only the most critical map information is transmitted. In addition, the present invention also introduces a bandwidth coordinator to optimize data transmission strategies, and a credibility evaluator to handle communication congestion to ensure accurate map construction and positioning even when network resources are limited. These designs improve the reliability and accuracy of collaborative work of multi-robot systems in large-scale scenarios while maintaining communication efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 This is a framework diagram of an edge-assisted collaborative mapping system suitable for communication-restricted environments disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Embodiment 1

[0051] This embodiment discloses an edge-assisted collaborative mapping system suitable for communication-constrained environments. Figure 1 ,The system includes: a plurality of independent agents, an edge server, and a communication pipeline;

[0052] The agent is used to perceive the environment and collect visual information through the sensors it carries, and independently run the visual SLAM algorithm to generate keyframes and landmarks. It selects keyframes using a method based on information entropy, and then dynamically adjusts the sparsity of keyframes and landmarks according to the currently available network bandwidth to finally generate a submap.

[0053] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then fuse the sub-maps according to the fusion priority to obtain the global map;

[0054] The communication pipe is used to connect the intelligent agent with the edge server.

[0055] Specifically, the present invention relates to three parts: an intelligent agent, an edge server, an edge server end, and a communication pipeline connecting the intelligent agent end and the server end.

[0056] 1. Intelligent terminal design

[0057] In order to balance communication bandwidth and map quality, the design idea of ​​this invention is to collect as much environmental information as possible on the agent side, compress this data before transmission to reduce transmission bandwidth, and reconstruct this information on the edge server side. Each agent side consists of two major component modules: tracking and mapping module, map distillation module.

[0058] 1. Tracking and mapping module: This module specifically includes two functional components: pose estimation and keyframe selection.

[0059] (1) Pose Estimator (Unit)

[0060] Pose estimation is used to determine the position and orientation of a robot or sensor in an environment. In the present invention, each agent runs an independent visual odometer front end to estimate the position and posture of the agent in the world coordinate system by tracking key feature points in consecutive frames.

[0061] (2) First keyframe selector

[0062] The selection of keyframes is critical to the quality of the map and the efficiency of the system. Keyframes usually contain multiple landmarks that serve as identifiable reference points in the map. The present invention uses scene complexity based on information entropy to select keyframes, rather than simply based on the association of landmarks. This method captures environmental changes more accurately by evaluating the information entropy of the scene, thereby achieving accurate and comprehensive map construction in environments with different textures. Specifically, the present invention maintains a first-in-first-out queue to store the most recent frames and calculates the information entropy of the new frame. If the ratio of the information entropy of the new frame to the average information entropy of all frames in the queue exceeds a predefined threshold, the insertion of a new keyframe is triggered.

[0063] 2. Map distillation module: After completing the keyframe selection, the data flow will go to the map distillation component module, which will further extract and compress the submap according to the bandwidth. This module specifically contains four functional components: information gain quantizer, landmark selector, keyframe selector, and bandwidth coordinator.

[0064] (1) Information Gain Quantizer

[0065] In visual SLAM, a single keyframe contains hundreds of landmarks, but not all landmarks have the same meaning. Landmark quantification is to determine the importance of each landmark in pose estimation, evaluate their contribution to the map, and retain the most valuable landmarks. The present invention quantifies the importance of each landmark through landmark stability and information gain. Landmark stability is defined by how many keyframes a landmark is observed by, while information gain is quantified by the impact of the landmark on the uncertainty of pose estimation. This quantification method helps to retain the most important landmarks when bandwidth is limited. With respect to information gain, the present invention defines the following calculation formula to quantify information gain:

[0066] q a =f det (Q)

[0067] where q a represents the information gain of the landmark, f det (*) is the logarithm of the deterministic function, where:

[0068] Q=W T ·W

[0069]

[0070] P l and P kl is from the optical center of the camera to the lth landmark point LM l And the feature point KP corresponding to the landmark point in the kth key frame kl The vector of is the latest rotation matrix estimated by the odometry. For landmark stability, for the LM observed by n keyframes Pi , its stability score q s Defined as:

[0071] q s =n

[0072] (2) Landmark Selector

[0073] Landmark selection is to perform sparse operations on landmarks. This functional component aims to reduce the number of transmitted landmarks in order to reduce data transmission and bandwidth consumption. The intuitive approach is to minimize the amount of data transmission by using a greedy algorithm to select high-value landmarks. However, since the gains of all remaining features need to be calculated at each iteration, this method significantly increases the computational complexity. The bandwidth-aware landmark selection problem is similar to the NP knapsack problem, where landmarks are "items", their bandwidth cost is "weight", their contribution to SLAM is "value", and dynamic bandwidth resources are "backpacks" with changing capacity. The goal is to dynamically select feature points that optimize SLAM accuracy based on real-time bandwidth. The present invention formulates the landmark sparseness task as a mixed integer programming problem. For a given landmark set {LM i}, combined with the stability score vector q calculated by the landmark information gain quantization module s , information gain vector q a Given the following constraints:

[0074]

[0075] stAx+ξ≥KI;Bx+φ≥I

[0076] Where α and β are weighted parameters that balance the importance of landmark stability and information gain. The present invention selects landmarks with higher gains. When the agent rotates violently, such as a drone platform, α should be set to a higher value to select landmarks that can be tracked for a longer time in the future. x is a binary vector whose i-th element indicates whether landmark i is selected (if x i =1, then delete LM i );λ1 and λ2 are weight vectors to be optimized. A is a binary matrix, which represents LM i Yuan A ij In the key frame KF j. K is the map compression factor, which indicates the maximum number of landmark points that can be observed in each keyframe. K directly determines the sparsification level and should be adjusted according to the current available bandwidth to achieve bandwidth-aware sparsification. Therefore, K will be adjusted by the bandwidth coordinator introduced later. B is a binary matrix, where LM i Yuan B ij is projected onto the grid g j or not projected onto the grid g i where ξ and φ are soft constraints to ensure that the problem is solvable. I is the identity matrix.

[0077] (3) Second keyframe selector

[0078] The second keyframe selector prunes the keyframes. This functional component is designed to further reduce the number of keyframes before transmission to optimize bandwidth usage. Specifically, this functional component does this by considering the spatial distribution between keyframes and the importance of the landmarks they contain. The system will prioritize those keyframes that contain multiple high-value landmarks and are widely distributed in space. Through keyframe selection, the system can reduce the amount of data transmitted while maintaining map quality. To ensure efficient data processing and accurate map construction, keyframe selection is guided by two key principles: First, keyframes must be sparsely distributed in space, which requires keyframes to be either spatially far apart or have significantly different perspectives to reduce redundancy. Second, the remaining landmarks after sparseness must be correlated, requiring each keyframe to capture many high-value landmarks to ensure reliable inter-frame correlation. For a given keyframe set {KF j} and entropy factor Th e , thus formulating the key frame repair task as a mixed integer programming problem, giving the following constraints:

[0079]

[0080] s.tEy+ε≥Th e

[0081] Where d is a vector representing the spatial distribution of each keyframe, and a larger d j This means that the j-th key frame KF j far from adjacent keyframes or its orientation is different from other keyframes. y is a binary vector indicating whether the jth element selects keyframe j (if y j =1, then delete KF j ). λ is the weight vector to be optimized. ε is the soft constraint to ensure the problem is solvable. E is the KF i The observation matrix of the indices of the observed landmarks. The degree of keyframe sparsification is given by Th eIt is determined that it will also be guided by the bandwidth coordinator introduced later to achieve adjustment to adapt to the available bandwidth.

[0082] (4) Bandwidth Coordinator

[0083] This functional component is responsible for dynamically adjusting the sparsification and pruning degree of map data according to the current network bandwidth conditions. It ensures that data transmission and resource utilization efficiency are optimized under limited bandwidth. Specifically, the bandwidth coordinator monitors the real-time bandwidth conditions and dynamically adjusts the sparsification factor (K) and key frame pruning threshold (Th) according to the available bandwidth. e ). In this way, the system can reduce the amount of data transmitted when bandwidth is limited, and transmit more detailed data when bandwidth is sufficient to maintain the accuracy and integrity of the map, given the following constraints:

[0084]

[0085] Among them, N LMs is the average number of landmarks observed by a keyframe in the unextracted sub-image; g(*) represents the keyframe sparsification ratio, where D LM and D KF Indicates the data size of the original landmark or keyframe; B t is the available bandwidth at time Δt. Δt is the data transmission interval, which is set to 10 seconds in the present invention, which means that a sub-map is extracted every 10 seconds and transmitted in the following 10 seconds.

[0086] 2. Edge Server

[0087] The edge server merges all sub-maps into the global map. For each received keyframe, a lightweight scene descriptor is extracted to detect loop closures and complete map fusion. Specifically, it contains three functional component modules: agent management, map fusion, and map management.

[0088] Agent Management Module: This component manages the sub-maps of all agents and performs bookkeeping for all agents. In the credibility evaluation component, the credibility of each sub-map is quantified according to the transmission stability of the channel corresponding to each sub-map.

[0089] (1) Bookkeeping Component

[0090] The bookkeeping component manages all agents. Once an agent is initialized, it tries to connect to the edge server via the pre-stored IP address and port. The edge server evaluates the submap quality of each agent and once the backend receives at least 30 keyframes for each agent, the most stable agent's local coordinates are selected as the global coordinates of the system.

[0091] (2) Credibility Assessment Component

[0092] This component quantifies the trustworthiness of each submap based on the transmission stability of its corresponding channel, and evaluates the submap quality based on transmission stability (defined as the product of successful transmission rate and the used communication bandwidth) and map connectivity (defined as the average observation time of all landmarks in the submap). Higher values ​​indicate more stable data association and higher submap quality. If a cycle is detected in a submap, its connectivity score will be set to 1 because it provides a strong connection to other submaps. In an environment with dynamically changing bandwidth, submaps with continuity and consistency are considered more reliable and are prioritized for merging. Two standard normalized weighted sums will be calculated for each incoming submap fragment, and the priority queue on the server will determine the map merging order.

[0093] Map fusion module: The map fusion component performs location recognition based on descriptor similarity, calculates the transformation between the newly received keyframe and the candidate keyframe, identifies and integrates map data from different agents to build a complete environment model. Specifically, it consists of two functional components: loop detection and pose optimization:

[0094] Loop detection component

[0095] Loop detection is performed using the Bag of Words (BoW) model to calculate a binary descriptor for each feature point. At the same time, the system preloads a constructed visual dictionary, which is learned from a large number of descriptors through a hierarchical clustering method. Each "word" in the visual dictionary represents a set of similar descriptors. When a new keyframe is added to the system, the system calculates the similarity score between it and all historical keyframes. In the present invention, the similarity score is calculated using the Hamming distance.

[0096] Pose Optimization Component

[0097] This component combines the inter-frame constraints constructed by the odometer and the long-term constraints constructed by the loop closure into a graph optimization problem. Specifically, the invention aims to minimize a cost function that measures the difference between all observed values ​​and predicted values. This problem is actually a least squares problem: min∑ m,n ρ(e mn (p m ,p n ;z mn ))

[0098] where e mn (p m ,p n ;z mn ) is the residual between the mth and nth keyframes, indicating the difference between the keyframe p m and p nThe predicted relative pose and the actual observed z mn The difference between them; ρ is a robust kernel function used to reduce the impact of outliers on the optimization results.

[0099] The present invention uses the Levenberg-Marquardt algorithm (an optimization method for solving nonlinear least squares problems) to solve the least squares problem, thereby completing posture optimization.

[0100] Map management module: merges the agent sub-maps transformed by the map fusion module into the global map, where the map merges are stored in descending order of credibility.

[0101] Communication pipeline design

[0102] In order to solve the dynamic communication channel conditions, the present invention optimizes the communication pipeline between the intelligent end and the server end, which is composed of a Socket interface module set at the intelligent end and a sub-end management component module set at the server end.

[0103] The socket interface module is set at the intelligent agent end and includes a map buffer and a bandwidth monitor. The map buffer is used to alleviate the incompleteness of mapping caused by communication interruption. The bandwidth monitor regularly monitors the available bandwidth and sets bandwidth limits for landmark and keyframe filtering. The sub-end management component module is set at the edge server end to feed back the update information of the global map to the intelligent agent end in real time.

[0104] Embodiment 2

[0105] Based on the same inventive concept, this embodiment discloses an edge-assisted collaborative mapping method suitable for a communication-constrained environment, which is implemented based on the edge-assisted collaborative mapping system suitable for a communication-constrained environment described in Embodiment 1, including:

[0106] The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries, and independently runs the visual SLAM algorithm to generate keyframes and landmarks. The keyframes are selected using a method based on information entropy, and then the sparsity of keyframes and landmarks is dynamically adjusted according to the currently available network bandwidth. Finally, a submap is generated and sent to the edge server through a communication pipeline.

[0107] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then the sub-maps are fused according to the fusion priority to obtain the global map, which is then sent to the intelligent agent through the communication pipeline.

[0108] During the specific implementation process, the specific working process of the mapping method of the present invention is as follows:

[0109] In recent years, with the rise of autonomous systems, there has been a growing demand for robotics that can perform simultaneous localization and mapping (SLAM) in unknown environments. Especially in large-scale scenarios, multi-robot systems can improve mapping efficiency and reduce the uncertainty caused by the limitations of individual robot sensors through collaboration. However, in these scenarios, the communication resources between robots are often limited and dynamically changing, which poses a challenge to the actual deployment of robot groups. Existing collaborative SLAM (C-SLAM) systems often find it difficult to maintain map accuracy and robot positioning reliability when faced with bandwidth limitations and unstable communications. The technology of the present invention implements an edge-assisted collaborative mapping method designed specifically for communication-constrained environments. Through bandwidth-aware map distillation technology, it ensures that only the most critical map information is transmitted to the edge server in a communication-constrained environment, thereby optimizing bandwidth usage while ensuring map accuracy. At the same time, by quantifying the importance of key frames and landmarks, dynamically adjusting data transmission strategies, and performing efficient map fusion and optimization on edge servers, the collaborative SLAM performance of multi-robot systems under dynamic and constrained network conditions can be improved. According to the modules implemented above, the specific process is as follows:

[0110] First, after multiple intelligent agents (robots) are deployed in the environment, they perceive the environment and collect visual information through the sensors they carry. At the same time, each intelligent agent independently runs the visual SLAM algorithm to generate key frames and landmarks, and uses the information entropy-based method to select key frames to adapt to environments with different textures and accurately capture environmental changes.

[0111] The agent then dynamically adjusts the sparsification of keyframes and landmarks according to the currently available network bandwidth to optimize data transmission, and uses Map Buffer to temporarily store data to reduce data loss when the network is unstable or the bandwidth is limited. On the edge server side, the received map data is managed by the Handler component, and the Credibility Assessor evaluates the credibility of the sub-map and determines the fusion priority. Subsequently, the Map Fusion component is used to fuse the local maps into a global map to handle closed-loop detection and pose optimization problems. The Map Manager component is responsible for maintaining and optimizing the global map to ensure its consistency and accuracy.

[0112] Finally, the updated information of the global map is fed back to the intelligent agents in real time to assist them in more accurate positioning and map construction, forming a dynamically iterative and continuously updated global map to support efficient collaborative SLAM of multi-robot systems in bandwidth-constrained environments.

[0113] In general, the beneficial effects of the present invention include:

[0114] ① Optimize transmission efficiency: The present invention dynamically adjusts the data transmission volume through the bandwidth coordinator, optimizes the sparsity of key frames and landmarks according to the currently available bandwidth conditions, retains the landmarks that contribute most to pose estimation, prunes key frames, and reduces unnecessary data transmission, thereby ensuring maximum map information transmission efficiency under limited bandwidth.

[0115] ② Improve the real-time performance of the system: The update information of the global map on the server side is fed back to the intelligent body in real time, assisting them in more accurate positioning and map construction, forming a dynamically iterated and continuously updated global map.

[0116] ③ Improve map construction accuracy: This invention introduces a credibility evaluator to deal with communication congestion. By evaluating the transmission stability and map connectivity of sub-maps, the most reliable data is processed first, thereby improving the accuracy of the global map.

[0117] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic creative concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An edge-assisted collaborative mapping system suitable for communication-constrained environments, characterized in that: include: Multiple independent agents, an edge server, and communication pipelines; The agent is used to perceive the environment and collect visual information through the sensors it carries, and independently run the visual SLAM algorithm to generate keyframes and landmarks. It selects keyframes using a method based on information entropy, and then dynamically adjusts the sparsity of keyframes and landmarks according to the currently available network bandwidth to finally generate a submap. The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then fuse the sub-maps according to the fusion priority to obtain the global map; The communication pipe is used to connect the intelligent agent with the edge server.

2. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, characterized in that: The agent includes a tracking and mapping module and a map distillation module; The tracking and mapping module includes a pose estimator and a first key frame selector, wherein the pose estimator is used to perceive the environment and collect visual information through the onboard sensor, and perform pose estimation based on the collected visual information, and the first key frame selector is used to select key frames using scene complexity based on information entropy; The map distillation module includes an information gain quantizer, a landmark selector, a second key frame selector and a bandwidth coordinator. The gain quantizer is used to quantify the importance of each landmark through landmark stability and information gain. The landmark selector is used to perform sparsification operations on landmarks. The second key frame selector is used to select key frames by considering the spatial distribution between key frames and the importance of the landmarks they contain. The bandwidth coordinator is used to dynamically adjust the sparsification degree of key frames and landmarks according to the current network bandwidth conditions.

3. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, characterized in that: The first keyframe selector is specifically used for: Maintain a first-in-first-out queue to store recent frames and calculate the information entropy of new frames; If the ratio of the information entropy of the new frame to the average information entropy of all frames in the queue exceeds a predefined threshold, the insertion of a new keyframe is triggered.

4. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, characterized in that: The gain quantizer is specifically used for: The stability of the landmark is determined by how many key frames the landmark is observed in, and the information gain is quantified by the impact of the landmark on the uncertainty of the pose estimation. The formula for the quantified information gain matrix Q is: q a =f det (Q) where q a represents the information gain of the landmark, f det (*) is the logarithm of the determinant function, and the information gain matrix Q is decomposed into the transpose of the matrix W and the product of W itself, that is, Q = W T ·W, operator· represents dot product; The superscript symbol ∧ represents the antisymmetric matrix operation; P l and P kl is the vector from the optical center of the camera to the lth landmark point and the vector of the feature point corresponding to the landmark point in the kth key frame, is the latest rotation matrix estimated by the odometry, and the landmark stability score is: q s =n q s represents the stability score of the landmark observed by n key frames; The importance of each landmark is quantified based on landmark stability and information gain.

5. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 4, characterized in that: The landmark selector is specifically used for: The landmark sparsification task is formulated as a mixed integer programming problem. For a given landmark set {LM i }, combined with the stability score q calculated by the landmark information gain quantization module s , information gain q a Given the following constraints: stAx+ξ≥KI;Bx+φ≥I where α and β represent weighted parameters that balance the importance of landmark stability and information gain, x represents a binary vector whose i-th element indicates whether landmark i is selected; λ1 and λ2 are weight vectors to be optimized, and A is a binary matrix representing LM i Yuan A ij In the key frame KF j is visible or invisible in the map, K represents the map compression factor, which indicates the maximum number of landmark points that can be observed in each keyframe, and B is a binary matrix representing LM i Yuan B ij Whether to be projected onto the grid g j In it, ξ and φ are soft constraints to ensure that the problem is solvable, and I is the identity matrix.

6. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, characterized in that: The second keyframe selector is specifically used for: For a given keyframe set {KF j } and entropy factor Th e , the keyframe trimming task is formulated as a mixed integer programming problem with the following constraints: s.tEy+ε≥Th e Where d represents the vector of spatial distribution of the jth keyframe, y is a binary vector indicating whether the jth element selects keyframe j, λ is the weight vector to be optimized, ε is a soft constraint to ensure that the problem is solvable, E represents the observation matrix of the indices of the landmarks observed by the keyframes, and Th e The pruning threshold for keyframes.

7. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 6, characterized in that: The bandwidth coordinator is specifically used for: Monitor the real-time bandwidth and dynamically adjust the sparsification factor K and key frame pruning threshold Th according to the available bandwidth e , dynamically adjust the sparsification degree of key frames and landmarks, and give the constraints as follows: Among them, K is the map compression factor, N LMs is the average number of landmarks observed by a keyframe in the unextracted sub-image, g(*) represents the keyframe sparsification ratio, and D LM and D KF represents the data size of the original landmark and the data size of the keyframe, B t is the available bandwidth at time Δt, where Δt is the data transmission interval.

8. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, characterized in that: The edge server includes an agent management module, a map fusion module, and map management; The agent management module is used to manage the sub-maps of all agents, perform bookkeeping for all agents, and quantify the credibility of each sub-map based on the transmission stability of the channel corresponding to each sub-map; The map fusion module is used to perform place recognition based on descriptor similarity and calculate the transformation between the newly received keyframe and the candidate keyframe, identifying and integrating map data from different agents to build a complete environment model; The map management module is used to merge the agent sub-maps transformed by the map fusion module into the global map, where the merged maps are stored in descending order of credibility.

9. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, characterized in that: The communication pipeline includes a Socket interface module and a sub-end management component module, wherein the Socket interface module is set at the intelligent body end and includes a map buffer and a bandwidth monitor. The map buffer is used to alleviate the incompleteness of mapping caused by communication interruption. The bandwidth monitor regularly monitors the available bandwidth and sets bandwidth limits for landmark and key frame filtering. The sub-end management component module is set at the edge server end and is used to feed back the update information of the global map to the intelligent body end in real time.

10. An edge-assisted collaborative mapping method suitable for communication-constrained environments, characterized in that: The edge-assisted collaborative mapping system applicable to a communication-constrained environment according to any one of claims 1 to 9 is implemented, comprising: The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries, and independently runs the visual SLAM algorithm to generate keyframes and landmarks. The keyframes are selected using a method based on information entropy, and then the sparsity of keyframes and landmarks is dynamically adjusted according to the currently available network bandwidth. Finally, a submap is generated and sent to the edge server through a communication pipeline. The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then the sub-maps are fused according to the fusion priority to obtain the global map, which is then sent to the intelligent agent through the communication pipeline.

Images