An edge assisted cooperative mapping system and method suitable for a communication-restricted environment

Abstract

In order to solve the problem of mapping accuracy of collaborative mapping in bandwidth limited environment, the application proposes an edge assisted collaborative mapping system and method specially designed for communication limited environment, which can quantify the importance of key frames and landmarks through bandwidth aware map distillation technology, and dynamically adjust the degree of data sparsification according to real-time bandwidth evaluation, so as to ensure that only the most critical map information is transmitted. In addition, the application also introduces a bandwidth coordinator to optimize the data transmission strategy, and a credibility evaluator to handle communication congestion, so as to ensure that accurate map construction and positioning can be realized under the condition of limited network resources. These designs make it maintain communication efficiency while improving the reliability and accuracy of multi-robot system working cooperatively in large-scale scenarios. The application discloses an edge assisted collaborative mapping system and method suitable for communication limited environment.

Description

Technical Field

[0001] The present invention relates to the field of visual scene mapping technology, and more specifically, to an edge-assisted collaborative mapping system and method suitable for communication-constrained environments. Background Art

[0002] With the rapid development of autonomous systems, simultaneous localization and mapping (SLAM) technology has become a research hotspot. This technology is crucial for robots to build maps and self-localize in unknown environments. Traditional SLAM methods mainly focus on single agents. However, with the increasing task requirements and the continuous expansion of robot workspaces, researchers have begun to turn to multi-agent systems (MAS), hoping to achieve efficient resource sharing and task optimization through collaboration between agents. In actual MAS, cameras have become the mainstream choice of robot sensors due to their high cost-effectiveness, low energy consumption, and ease of large-scale deployment, thus driving significant progress in visual SLAM technology.

[0003] Recently, the COVINS-G framework has been developed to fuse the backend with the frontend by introducing an innovative frontend wrapper that reduces the amount of data transmission by re-extracting landmark points (LM) from keyframes (KF). However, landmark points abstracted from raw sensor data may not fully capture the characteristics of the environment, thus limiting further improvements in system performance.

[0004] Tsinghua University's SwarmMap edge intelligence framework effectively scales up MAS by proposing data and task offloading strategies and map analysis methods. While this approach improves accuracy and robustness compared to single-machine SLAM, it lacks optimization for the practical application of bandwidth compression for the large number of newly observed features during exploration missions, resulting in low mapping accuracy in communication-restricted environments. Summary of the Invention

[0005] In response to the problems existing in the prior art, the present invention aims to improve the efficiency of robot map construction in communication-constrained environments by optimizing data transmission to adapt to the current network conditions, and reduce the system uncertainty caused by this, thereby achieving more efficient SLAM performance in actual scenarios with limited resources.

[0006] In order to solve the above technical problems, the first aspect of the present invention provides an edge-assisted collaborative mapping system suitable for communication-constrained environments, comprising: multiple independent agents, an edge server, and a communication pipeline;

[0007] The intelligent agent is configured to perceive the environment and collect visual information through the mounted sensors, and independently run a visual SLAM algorithm to generate key frames and landmarks, select the key frames based on an information entropy-based method, and dynamically adjust the sparsification degree of the key frames and landmarks according to the current available network bandwidth, and finally generate a submap;

[0008] The edge server is configured to evaluate the credibility of the submap and determine a fusion priority, and then fuse the submap according to the fusion priority to obtain a global map;

[0009] The communication pipeline is configured to communicatively connect the intelligent agent and the edge server.

[0010] In an embodiment, the intelligent agent comprises a tracking and mapping module and a map distillation module;

[0011] The tracking and mapping module comprises a pose estimator and a first key frame selector, wherein the pose estimator is configured to perceive the environment and collect visual information through the mounted sensors, and perform pose estimation according to the collected visual information, and the first key frame selector is configured to select key frames using a scene complexity based on information entropy;

[0012] The map distillation module comprises an information gain quantifier, a landmark selector, a second key frame selector, and a bandwidth coordinator, wherein the gain quantifier is configured to quantify the importance of each landmark through landmark stability and information gain, the landmark selector is configured to perform a sparsification operation on the landmarks, the second key frame selector is configured to consider the spatial distribution between key frames and the importance of the landmarks contained therein to select key frames, and the bandwidth coordinator is configured to dynamically adjust the sparsification degree of the key frames and landmarks according to the current network bandwidth condition.

[0013] In an embodiment, the first key frame selector is specifically configured to:

[0014] maintain a first-in-first-out queue to store the most recent frames, and calculate the information entropy of the new frame;

[0015] if the ratio of the information entropy of the new frame to the average information entropy of all frames in the queue exceeds a predefined threshold, a new key frame is inserted.

[0016] In an embodiment, the gain quantifier is specifically configured to:

[0017] determine the landmark stability by how many key frames the landmark is observed, and quantify the information gain by the influence of the landmark on the pose estimation uncertainty, wherein the formula for quantifying the information gain is:

[0018] q a = f det (Q)

[0019] where qa represents the information gain of the landmark, f det (*) is the logarithm of the determination function, where: Q = W T W, P l and P kl is the vector from the optical center of the camera to the lth landmark point and the vector of the feature point corresponding to the landmark point in the kth key frame, is the latest rotation matrix estimated by the odometry, and the landmark stability score is:

[0020] q s =n

[0021] q s represents the stability score of the landmark observed by n keyframes;

[0022] The importance of each landmark is quantified based on landmark stability and information gain.

[0023] In one embodiment, the landmark selector is specifically used to:

[0024] The landmark sparsification task is formulated as a mixed integer programming problem. For a given landmark set {LM i}, combined with the stability score q calculated by the landmark information gain quantification module s , information gain q a Given the following constraints:

[0025]

[0026] stAx+ξ≥KI;Bx+φ≥I

[0027] Where α and β represent weighted parameters that balance the importance of landmark stability and information gain, x represents a binary vector whose i-th element indicates whether landmark i is selected; λ1 and λ2 are weight vectors to be optimized, and A is a binary matrix representing LM i Yuan A ij In the key frame KF j is visible or invisible, K represents the map compression factor, which indicates the maximum number of landmark points that can be observed in each keyframe, and B is a binary matrix representing LM i Yuan B ij Whether it is projected onto the grid g j where ξ and φ are soft constraints to ensure that the problem is solvable, and I is the identity matrix.

[0028] In one embodiment, the second key frame selector is specifically configured to:

[0029] For a given keyframe set {KF j} and entropy factor The , the keyframe retouching task is formulated as a mixed integer programming problem with the following constraints:

[0030]

[0031] s.tEy+ε≥The e

[0032] Where d represents the vector of spatial distribution of the jth keyframe, y is a binary vector indicating whether the jth element selects keyframe j, λ is the weight vector to be optimized, ε is a soft constraint to ensure that the problem is solvable, E represents the observation matrix of the indices of the landmarks observed by the keyframe, and Th e The threshold for keyframe pruning.

[0033] In one embodiment, the bandwidth coordinator is specifically configured to:

[0034] Monitor the real-time bandwidth situation and dynamically adjust the sparsification factor K and key frame pruning threshold Th according to the available bandwidth e , the sparsification degree of keyframes and landmarks is dynamically adjusted, and the constraints given are:

[0035]

[0036] Among them, K is the map compression factor, N LMs is the average number of landmarks observed by the keyframe in the unextracted sub-image, g(*) represents the keyframe sparsification ratio, and D LM and D KF Indicates the data size of the original landmark and the data size of the keyframe, B t is the available bandwidth at time Δt, where Δt is the data transmission interval.

[0037] In one embodiment, the edge server includes an agent management module, a map fusion module, and a map management module;

[0038] The agent management module is used to manage the submaps of all agents, perform bookkeeping for all agents, and quantify the credibility of each submap based on the transmission stability of the channel corresponding to each submap;

[0039] The map fusion module is used to perform place recognition based on descriptor similarity and calculate the transformation between the newly received keyframe and the candidate keyframe, identifying and integrating map data from different agents to build a complete environment model;

[0040] The map management module is used to merge the agent submaps transformed by the map fusion module into the global map, where the merged maps are stored in descending order of credibility.

[0041] In one embodiment, the communication pipeline includes a socket interface module and a sub-end management component module, wherein the socket interface module is set at the intelligent body end and includes a map buffer and a bandwidth monitor. The map buffer is used to alleviate the incompleteness of mapping caused by communication interruption. The bandwidth monitor regularly monitors the available bandwidth and sets bandwidth limits for landmark and keyframe filtering. The sub-end management component module is set at the edge server end and is used to feed back the update information of the global map to the intelligent body end in real time.

[0042] Based on the same inventive concept, a second aspect of the present invention provides an edge-assisted collaborative mapping method suitable for communication-constrained environments, which is implemented based on the edge-assisted collaborative mapping system suitable for communication-constrained environments described in the first aspect, including:

[0043] The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries, and independently runs the visual SLAM algorithm to generate keyframes and landmarks. The keyframes are selected using a method based on information entropy, and the sparsity of keyframes and landmarks is dynamically adjusted according to the currently available network bandwidth. Finally, a submap is generated and sent to the edge server through the communication pipeline;

[0044] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority. The sub-maps are then fused according to the fusion priority to obtain the global map, which is then sent to the intelligent agent through the communication pipeline.

[0045] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0046] The present invention provides an edge-assisted collaborative mapping system and method suitable for communication-constrained environments. Through bandwidth-aware map distillation technology, it can quantify the importance of keyframes and landmarks, and dynamically adjust the sparsity of data based on real-time bandwidth evaluation to ensure that only the most critical map information is transmitted. In addition, the present invention also introduces a bandwidth coordinator to optimize data transmission strategies and a credibility evaluator to handle communication congestion, ensuring accurate map construction and positioning even when network resources are limited. These designs improve the reliability and accuracy of multi-robot systems working together in large-scale scenarios while maintaining communication efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 This is a framework diagram of an edge-assisted collaborative mapping system suitable for communication-restricted environments disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0050] Example 1

[0051] This embodiment discloses an edge-assisted collaborative mapping system suitable for communication-constrained environments. Figure 1 ,The system includes: multiple independent agents, an edge server, and communication pipelines;

[0052] The agent is used to perceive the environment and collect visual information through the sensors it carries, and independently run the visual SLAM algorithm to generate keyframes and landmarks. It uses an information entropy-based method to select keyframes, and then dynamically adjusts the sparsity of keyframes and landmarks based on the currently available network bandwidth to finally generate a submap.

[0053] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then fuse the sub-maps according to the fusion priority to obtain the global map;

[0054] The communication pipeline is used to connect the agent to the edge server for communication.

[0055] Specifically, the present invention relates to three parts: an intelligent agent, an edge server, an edge server end, and a communication pipeline connecting the intelligent agent end and the server end.

[0056] 1. Intelligent Body Design

[0057] To balance communication bandwidth and map quality, the design of this invention collects as much environmental information as possible on the agent side, compresses this data before transmission to reduce bandwidth, and reconstructs this information on the edge server side. Each agent side consists of two major components: a tracking and mapping module and a map distillation module.

[0058] 1. Tracking and mapping module: This module specifically includes two functional components: pose estimation and keyframe selection.

[0059] (1) Pose Estimator (Unit)

[0060] Pose estimation is used to determine the position and orientation of a robot or sensor in an environment. In this paper, each agent runs a separate visual odometry front-end that estimates the agent's position and pose in a world coordinate system by tracking key feature points in successive frames.

[0061] (2) First keyframe selector

[0062] The selection of keyframes is critical to the quality of the map and the efficiency of the system. Keyframes usually contain multiple landmarks that serve as identifiable reference points in the map. The present invention uses scene complexity based on information entropy to select keyframes, rather than simply based on the association of landmarks. This method captures environmental changes more accurately by evaluating the information entropy of the scene, thereby achieving accurate and comprehensive map construction in environments with different textures. Specifically, the present invention maintains a first-in-first-out queue to store the most recent frames and calculates the information entropy of the new frame. If the ratio of the information entropy of the new frame to the average information entropy of all frames in the queue exceeds a predefined threshold, it triggers the insertion of a new keyframe.

[0063] 2. Map Distillation Module: After keyframe selection, the data flow is transferred to the Map Distillation Component Module, which further extracts and compresses submaps based on bandwidth. This module specifically consists of four functional components: Information Gain Quantizer, Landmark Selector, Keyframe Selector, and Bandwidth Coordinator.

[0064] (1) Information Gain Quantizer

[0065] In visual SLAM, a single keyframe contains hundreds of landmarks, but not all landmarks have the same meaning. Landmark quantification is to determine the importance of each landmark in pose estimation, evaluate their contribution to the map, and retain the most valuable landmarks. The present invention quantifies the importance of each landmark through landmark stability and information gain. Landmark stability is defined by how many keyframes a landmark is observed by, while information gain is quantified by the impact of the landmark on the uncertainty of pose estimation. This quantification method helps to retain the most important landmarks when bandwidth is limited. With respect to information gain, the present invention defines the following calculation formula to quantify information gain:

[0066] q a =f det (Q)

[0067] where q a represents the information gain of the landmark, f det (*) is the logarithm of the deterministic function, where:

[0068] Q=W T W

[0069]

[0070] P l and P kl is the vector from the camera's optical center to the l-th landmark point LM l and the corresponding feature point KP kl in the k-th keyframe, is the latest rotation matrix estimated by the odometry. For landmark stability, for LM Pi observed by n keyframes, its stability score q s is defined as:

[0071] q s = n

[0072] (2) Landmark selector

[0073] Landmark selection, i.e. the sparsification of landmarks, is the function component that aims to reduce the number of transmitted landmarks to reduce data transmission and bandwidth consumption. The intuitive approach is to select high-value landmarks to minimize data transmission using a greedy algorithm. However, this method significantly increases the computational complexity as it needs to calculate the gain of all remaining features at each iteration. The bandwidth-aware landmark selection problem is similar to the NP knapsack problem, where landmarks are "items", their bandwidth cost is "weight", their contribution to SLAM is "value", and dynamic bandwidth resources are "knapsacks" with varying capacities. The goal is to dynamically select features that optimize SLAM accuracy based on real-time bandwidth. The present invention formulates the landmark sparsification task as a mixed integer programming problem, for a given landmark set {LM i}, combining the stability score vector q s calculated by the landmark information gain quantification module, the information gain vector q a is given as follows:

[0074]

[0075] s.t.Ax+ξ≥KI;Bx+φ≥I

[0076] where α and β are weighting parameters that balance the importance of landmark stability and information gain. The present invention selects landmarks with higher gain. When the agent rotates dramatically, such as a UAV platform, α should be set to a higher value to select landmarks that can be tracked for a longer time in the future. x is a binary vector whose i-th element indicates whether to select landmark i (if x i = 1, delete LM i ); λ1 and λ2 are the weight vectors to be optimized. A is a binary matrix where A i indicates that LM ij is in keyframe KF jis visible or invisible in the map. K is the map compression factor, which indicates the maximum number of landmark points that can be observed in each keyframe. K directly determines the sparsification level and should be adjusted according to the current available bandwidth to achieve bandwidth-aware sparsification. Therefore, K will be adjusted by the bandwidth coordinator introduced later. B is a binary matrix, which represents LM i Yuan B ij is projected onto the grid g j or not projected onto the grid g i where ξ and φ are soft constraints to ensure that the problem is solvable. I is the identity matrix.

[0077] (3) Second keyframe selector

[0078] The second keyframe selector prunes the keyframes. This functional component is designed to further reduce the number of keyframes before transmission to optimize bandwidth usage. Specifically, this functional component does this by considering the spatial distribution between keyframes and the importance of the landmarks they contain. The system will prioritize those keyframes that contain multiple high-value landmarks and are widely distributed in space. Through keyframe selection, the system can reduce the amount of data transmitted while maintaining map quality. To ensure efficient data processing and accurate map construction, keyframe selection is guided by two key principles: First, keyframes must be sparsely distributed in space, which requires keyframes to be either spatially far apart or have significantly different perspectives to reduce redundancy. Second, the remaining landmarks after sparseness must be correlated, requiring each keyframe to capture many high-value landmarks to ensure reliable inter-frame correlation. For a given keyframe set {KF j} and entropy factor Th e , thus formulating the keyframe retouching task as a mixed integer programming problem with the following constraints:

[0079]

[0080] s.tEy+ε≥Th e

[0081] Where d is a vector representing the spatial distribution of each keyframe, and a larger d j This means that the j-th key frame KF j It is far away from the adjacent keyframe or its orientation is different from other keyframes. y is a binary vector indicating whether the jth element selects keyframe j (if y j =1, then delete KF j ). λ is the weight vector to be optimized. ε is the soft constraint to ensure the problem is solvable. E is the KF i The observation matrix of the indices of the observed landmarks. The degree of keyframe sparsification is determined by Th eIt will also be guided by the bandwidth coordinator introduced later to achieve adjustments that adapt to the available bandwidth.

[0082] (4) Bandwidth Coordinator

[0083] This functional component is responsible for dynamically adjusting the sparsification and pruning degree of map data according to the current network bandwidth conditions. It ensures that data transmission and resource utilization efficiency are optimized under limited bandwidth. Specifically, the bandwidth coordinator monitors the real-time bandwidth conditions and dynamically adjusts the sparsification factor (K) and keyframe pruning threshold (Th) according to the available bandwidth. e In this way, the system can reduce the amount of data transmitted when bandwidth is limited, and transmit more detailed data when bandwidth is sufficient to maintain the accuracy and integrity of the map, given the following constraints:

[0084]

[0085] Among them, N LMs is the average number of landmarks observed by a keyframe in the unextracted sub-image; g(*) represents the keyframe sparsification ratio, where D LM and D KF Indicates the data size of the original landmark or keyframe; B t is the available bandwidth at time Δt. Δt is the data transmission interval, which is set to 10 seconds in this invention, which means that a submap is extracted every 10 seconds and transmitted within the following 10 seconds.

[0086] 2. Edge Server

[0087] The edge server merges all submaps into the global map. For each received keyframe, it extracts a lightweight scene descriptor used to detect loop closures and complete map fusion. Specifically, it consists of three functional component modules: agent management, map fusion, and map management.

[0088] Agent Management Module: This component manages all agent submaps and performs bookkeeping for all agents. In the credibility evaluation component, the credibility of each submap is quantified based on the transmission stability of the corresponding channel of each submap.

[0089] (1) Bookkeeping component

[0090] The bookkeeping component manages all agents. Once an agent is initialized, it attempts to connect to the edge server via a pre-stored IP address and port. The edge server evaluates the quality of each agent's submap and, once the backend has received at least 30 keyframes for each agent, selects the most stable agent's local coordinates as the system's global coordinates.

[0091] (2) Credibility Assessment Component

[0092] The component quantifies the reliability of each submap according to the transmission stability of the channel each submap corresponds to, evaluating submap quality according to transmission stability (defined as the product of successful transmission rate and used communication bandwidth) and map connectivity (defined as the average observation time of all landmarks in the submap). Higher values indicate more stable data association and higher submap quality. If a loop is detected in a submap, its connectivity score will be set to 1 as it provides strong connections to other submaps. In environments where bandwidth varies dynamically, submaps with continuity and consistency are considered more reliable and are prioritized for merging. A priority queue on the server will determine the map merging order, normalizing and weighting the two criteria computed for each incoming submap segment.

[0093] Map fusion module: The map fusion functional component performs place recognition based on descriptor similarity and computes the transformation between newly received keyframes and candidate keyframes, identifying and integrating map data from different agents to construct a complete environment model. Specifically, it consists of two functional components: loop closure detection and pose optimization:

[0094] Loop closure detection component

[0095] Loop closure detection is completed using the Bag of Words (BoW) model, which computes a binary descriptor for each feature point. At the same time, the system will pre-load a constructed visual dictionary, which is learned from a large number of descriptors through hierarchical clustering. Each "word" in the visual dictionary represents a group of similar descriptors. When a new keyframe is added to the system, the system calculates the similarity score between it and all historical keyframes. In this invention, the similarity score is calculated using the Hamming distance.

[0096] Pose optimization component

[0097] This component fuses the inter-frame constraints constructed by the odometry and the long-term constraints constructed by the loop into a graph optimization problem. Specifically, the invention aims to minimize a cost function that measures the difference between all observations and predictions. This problem is actually a least squares problem: min∑ m,n ρ(e mn (p m ,p n );z mn ))

[0098] where e mn (p m ,p n ; z mn ) is the residual between the mth and nth keyframes, representing the difference between the predicted and observed values based on keyframes p m and p nThe predicted relative pose and the actual observation z mn The difference between them; ρ is a robust kernel function used to reduce the impact of outliers on the optimization results.

[0099] The present invention uses the Levenberg-Marquardt algorithm (an optimization method for solving nonlinear least squares problems) to solve the above least squares problem, thereby completing posture optimization.

[0100] Map management module: merges the agent submaps transformed by the map fusion module into the global map, where the merged maps are stored in descending order of credibility.

[0101] Communication pipeline design

[0102] In order to solve the dynamic communication channel conditions, the present invention optimizes the communication pipelines of the intelligent end and the server end, which are respectively arranged in the Socket interface module of the intelligent end and the sub-end management component module of the server end.

[0103] The socket interface module is set on the intelligent agent side and includes a map buffer and a bandwidth monitor. The map buffer is used to alleviate the incompleteness of mapping caused by communication interruptions. The bandwidth monitor regularly monitors the available bandwidth and sets bandwidth limits for landmark and keyframe filtering. The sub-end management component module is set on the edge server side to feed back the updated information of the global map to the intelligent agent side in real time.

[0104] Example 2

[0105] Based on the same inventive concept, this embodiment discloses an edge-assisted collaborative mapping method suitable for communication-constrained environments, which is implemented based on the edge-assisted collaborative mapping system suitable for communication-constrained environments described in Example 1, including:

[0106] The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries, and independently runs the visual SLAM algorithm to generate keyframes and landmarks. The keyframes are selected using a method based on information entropy, and the sparsity of keyframes and landmarks is dynamically adjusted according to the currently available network bandwidth. Finally, a submap is generated and sent to the edge server through the communication pipeline;

[0107] The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority. The sub-maps are then fused according to the fusion priority to obtain the global map, which is then sent to the intelligent agent through the communication pipeline.

[0108] During the specific implementation process, the mapping method of the present invention works as follows:

[0109] In recent years, with the rise of autonomous systems, there is a growing demand for robotic technology that can perform Simultaneous Localization and Mapping (SLAM) in unknown environments. Especially in large-scale scenarios, multiple robot systems can improve mapping efficiency and reduce uncertainty caused by single robot sensor limitations through collaboration. However, in these scenarios, communication resources between robots are often limited and dynamic, which poses challenges to the practical deployment of robot groups. Existing collaborative SLAM (C-SLAM) systems often struggle to maintain map accuracy and robot localization reliability when faced with bandwidth limitations and communication instability. The present technology implements an edge-assisted collaborative mapping method designed specifically for communication-limited environments. Through bandwidth-aware map distillation techniques, it ensures that only the most critical map information is transmitted to the edge server in a communication-limited environment, thereby optimizing bandwidth usage while ensuring map accuracy. At the same time, by quantifying the importance of keyframes and landmarks, it dynamically adjusts data transmission strategies and performs efficient map fusion and optimization on the edge server to improve the collaborative SLAM performance of multi-robot systems in dynamic and limited network conditions. According to the above implemented modules, the specific process is as follows:

[0110] First, after multiple agents (robots) are deployed in the environment, they perceive the environment through the sensors mounted and collect visual information. At the same time, each agent independently runs a visual SLAM algorithm to generate keyframes and landmarks, and uses an information entropy-based method to select keyframes to adapt to different textured environments and accurately capture environmental changes.

[0111] Then, the agent side dynamically adjusts the sparseness of keyframes and landmarks according to the current available network bandwidth to optimize data transmission, while using Map Buffer (map cache) to temporarily store data to reduce data loss when the network is unstable or bandwidth is limited. On the edge server side, the received map data is managed by the Handler component, the Credibility Assessor (credibility assessment component) evaluates the credibility of the sub-map and determines the fusion priority. Subsequently, the Map Fusion (map fusion) component is used to fuse local maps into a global map, handling loop closure detection and pose optimization problems. The Map Manager (map management) component is responsible for maintaining and optimizing the global map to ensure its consistency and accuracy.

[0112] Finally, the updated information of the global map is fed back to the agent side in real time to assist them in more accurate localization and map construction, forming a dynamic and iterative global map that is constantly updated to support efficient collaborative SLAM of multi-robot systems in bandwidth-limited environments.

[0113] Overall, the benefits of the present invention include:

[0114] ①Optimize transmission efficiency: the application dynamically adjusts the transmission amount of data through the bandwidth coordinator, optimizes the sparsification degree of key frames and landmarks according to the current available bandwidth condition, retains the landmarks with the largest contribution to pose estimation, prunes the key frames, and reduces unnecessary data transmission, thereby ensuring the maximum map information transmission efficiency under limited bandwidth.

[0115] ②Improve system real-time performance: the update information of the server-side global map is fed back to the agent side in real time, assisting them in more accurate positioning and map construction, forming a dynamic iteration and constantly updated global map.

[0116] ③Improve mapping accuracy: the application introduces a credibility evaluator to handle communication congestion, processes the most reliable data by evaluating the transmission stability and map connectivity of the sub-map, and improves the accuracy of the global map.

[0117] Those skilled in the art will understand that embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0119] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications to the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. An edge-assisted collaborative mapping system suitable for communication-constrained environments, characterized by: include: Multiple independent agents, an edge server, and communication channels; The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries. It then independently runs the visual SLAM algorithm to generate keyframes and landmarks. It uses an information entropy-based method to select keyframes and then dynamically adjusts the sparsity of keyframes and landmarks based on the currently available network bandwidth to ultimately generate a submap. The intelligent agent includes a tracking and mapping module and a map distillation module. The tracking and mapping module includes a pose estimator and a first keyframe selector. The pose estimator is used to perceive the environment and collect visual information through the onboard sensors, and perform pose estimation based on the collected visual information. The first keyframe selector is used to select keyframes using scene complexity based on information entropy. The map distillation module includes an information gain quantizer, a landmark selector, a second keyframe selector, and a bandwidth coordinator. The gain quantizer is used to quantify the importance of each landmark through landmark stability and information gain. The landmark selector is used to perform a sparse operation on the landmarks. The second keyframe selector is used to consider the spatial distribution between keyframes and the importance of the landmarks they contain to select keyframes. The bandwidth coordinator is used to dynamically adjust the sparsity of keyframes and landmarks according to the current network bandwidth conditions. The gain quantizer is specifically used for: The stability of the landmark is determined by how many key frames the landmark is observed in, and the information gain is quantified by the impact of the landmark on the uncertainty of the pose estimation. The formula for the quantified information gain matrix Q is: in Information gain of landmarks, is the logarithm of the determinant function, and the information gain matrix Q is decomposed into the product of the transpose of the matrix W and W itself, that is, , operator represents the dot product; , where the superscript symbol Indicates the antisymmetric matrix operation; and From the optical center of the camera to the The vector of the landmark point and the landmark point in The vector of the feature point corresponding to each key frame, is the latest rotation matrix estimated by the odometry, and the landmark stability score is: represents the stability score of the landmark observed by n keyframes; Quantify the importance of each landmark based on landmark stability and information gain; The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority, and then fuse the sub-maps according to the fusion priority to obtain the global map; The communication pipeline is used to connect the agent to the edge server for communication.

2. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, wherein: The first keyframe selector is specifically used for: Maintain a first-in-first-out queue to store the most recent frames and calculate the information entropy of new frames; If the ratio of the information entropy of the new frame to the average information entropy of all frames in the queue exceeds a predefined threshold, the insertion of a new keyframe is triggered.

3. The edge-assisted collaborative mapping system for communication-restricted environments according to claim 1, wherein: Landmark selectors are specifically used for: The landmark sparsification task is formulated as a mixed integer programming problem. For a given landmark set { }, combined with the stability score calculated by the landmark information gain quantification module , information gain Given the following constraints: where α and β represent weighting parameters that balance the importance of landmark stability and information gain, and x represents a binary vector whose i-th element indicates whether landmark i is selected; and is the weight vector to be optimized, A is a binary matrix, representing yuan At keyframe K represents the map compression factor, which indicates the maximum number of landmark points observed in each keyframe, and B is a binary matrix representing yuan Whether it is projected onto the grid middle, and are soft constraints to ensure that the problem is solvable. is the identity matrix.

4. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, wherein: The second keyframe selector is specifically used for: For a given set of keyframes { } and entropy factor , the keyframe retouching task is formulated as a mixed integer programming problem with the following constraints: Among them, d represents the vector of the spatial distribution of the jth key frame, y is a binary vector, indicating whether the jth element selects key frame j, is the weight vector to be optimized, To ensure the solvability of the problem, E represents the observation matrix of the indices of the landmarks observed by the keyframes, The threshold for keyframe pruning.

5. The edge-assisted collaborative mapping system for communication-restricted environments according to claim 4, characterized in that: The bandwidth coordinator is specifically used to: Monitor real-time bandwidth conditions and dynamically adjust the map compression factor K and keyframe pruning threshold based on available bandwidth , the sparsification degree of keyframes and landmarks is dynamically adjusted, and the constraints given are: Among them, K is the map compression factor, is the average number of landmarks observed by a keyframe in the unextracted sub-image, g(*) represents the keyframe sparsification ratio, and Indicates the data size of the original landmark and the data size of the keyframe, For time The available bandwidth at The data transmission interval.

6. The edge-assisted collaborative mapping system for communication-constrained environments according to claim 1, wherein: The edge server includes an intelligent agent management module, a map fusion module, and map management; The agent management module is used to manage the submaps of all agents, perform bookkeeping for all agents, and quantify the credibility of each submap based on the transmission stability of the channel corresponding to each submap; The map fusion module is used to perform place recognition based on descriptor similarity and calculate the transformation between the newly received keyframe and the candidate keyframe, identifying and integrating map data from different agents to build a complete environment model; The map management module is used to merge the agent submaps transformed by the map fusion module into the global map, where the merged maps are stored in descending order of credibility.

7. The edge-assisted collaborative mapping system for communication-restricted environments according to claim 1, wherein: The communication pipeline includes a socket interface module and a sub-end management component module. The socket interface module is set at the intelligent agent end and contains a map buffer and a bandwidth monitor. The map buffer is used to alleviate the incompleteness of mapping caused by communication interruption. The bandwidth monitor regularly monitors the available bandwidth and sets bandwidth limits for landmark and keyframe filtering. The sub-end management component module is set at the edge server end and is used to feed back the update information of the global map to the intelligent agent end in real time.

8. An edge-assisted collaborative mapping method suitable for communication-constrained environments, characterized in that: The edge-assisted collaborative mapping system for communication-constrained environments according to any one of claims 1 to 7 is implemented, comprising: The intelligent agent is used to perceive the environment and collect visual information through the sensors it carries, and independently runs the visual SLAM algorithm to generate keyframes and landmarks. It uses an information entropy-based method to select keyframes, and then dynamically adjusts the sparsity of keyframes and landmarks based on the currently available network bandwidth. Finally, a submap is generated and sent to the edge server through a communication pipeline. The intelligent agent includes a tracking and mapping module and a map distillation module. The tracking and mapping module includes a pose estimator and a first keyframe selector. The pose estimator is used to perceive the environment and collect visual information through the onboard sensors, and perform pose estimation based on the collected visual information. The first keyframe selector is used to select keyframes using scene complexity based on information entropy. The map distillation module includes an information gain quantizer, a landmark selector, a second keyframe selector, and a bandwidth coordinator. The gain quantizer is used to quantify the importance of each landmark through landmark stability and information gain. The landmark selector is used to perform a sparse operation on the landmarks. The second keyframe selector is used to consider the spatial distribution between keyframes and the importance of the landmarks they contain to select keyframes. The bandwidth coordinator is used to dynamically adjust the sparsity of keyframes and landmarks according to the current network bandwidth conditions. The gain quantizer is specifically used for: The stability of the landmark is determined by how many key frames the landmark is observed in, and the information gain is quantified by the impact of the landmark on the uncertainty of the pose estimation. The formula for the quantified information gain matrix Q is: in Information gain of landmarks, is the logarithm of the determinant function, and the information gain matrix Q is decomposed into the product of the transpose of the matrix W and W itself, that is, , operator represents the dot product; , where the superscript symbol Indicates the antisymmetric matrix operation; and From the optical center of the camera to the The vector of the landmark point and the landmark point in The vector of the feature point corresponding to each key frame, is the latest rotation matrix estimated by the odometry, and the landmark stability score is: represents the stability score of the landmark observed by n keyframes; Quantify the importance of each landmark based on landmark stability and information gain; The edge server is used to evaluate the credibility of the sub-maps and determine the fusion priority. The sub-maps are then fused according to the fusion priority to obtain the global map, which is then sent to the intelligent agent through the communication pipeline.

Images