Mobile robot-based environment mapping method and terminal

Abstract

The application provides a mobile robot-based environment mapping method and a terminal. The method comprises: acquiring a current image collected by a monocular camera of a monitoring robot and corresponding odometer pose information; performing visual feature extraction on the current image to generate current global visual description information; uploading the current image to a cloud server and instructing the cloud server to perform monocular depth estimation on the current image to generate a metric depth map; acquiring the metric depth map in the cloud server; and constructing a two-dimensional grid map according to the metric depth map, the global visual description information and the odometer pose information. The method can realize real-time construction of a map with accurate scales on a low-cost monitoring robot equipped with only a monocular camera and an embedded processor.

Description

Technical Field

[0001] This invention relates to the field of monitoring technology, and in particular to an environmental mapping method and terminal based on a mobile robot. Background Technology

[0002] In related technologies, when mobile robots map their surroundings, mapping is typically completed based on environmental information acquired through cameras. One example is mapping schemes based on monocular visual SLAM. These schemes use only a single camera, simultaneously estimating camera pose and constructing a map through feature extraction, matching, and motion reconstruction. However, this approach cannot recover absolute scale from monocular images, leading to inaccurate absolute distances in the map. Furthermore, traditional monocular SLAM algorithms are computationally intensive, making them difficult to run in real-time on embedded devices. Summary of the Invention

[0003] The technical problem to be solved by this invention is: how to build a map with accurate scale in real time on a low-cost monitoring robot equipped only with a monocular camera and an embedded processor.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A mobile robot-based environment mapping method includes: Acquire the current image captured by the monocular camera of the monitoring robot and the corresponding odometry pose information; Visual features are extracted from the current image to generate current global visual description information; The current image is uploaded to the cloud server, and the cloud server is instructed to perform monocular depth estimation on the current image to generate a metric depth map; Obtain the metric depth map within the cloud server; A two-dimensional grid map is constructed based on the metric depth map, global visual description information, and odometry pose information.

[0005] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A mobile robot-based environment mapping terminal includes a memory, a processor, and storage components. A computer program on the memory and executable on a processor, wherein the processor executes the computer program to implement an environment mapping method based on a mobile robot as described above.

[0006] The beneficial effects of this invention are as follows: by constructing an asynchronous parallel processing architecture that integrates edge and cloud, the monitoring robot collects the current image and odometry pose information through a monocular camera, extracts visual features to generate current global visual description information, and uploads the current image to a cloud server. The cloud server is responsible for monocular depth estimation to generate a metric depth map. Then, the monitoring robot uses the metric depth map, global visual description information, and odometry pose information to realize the real-time construction of a map with accurate scale on a low-cost monitoring robot equipped only with a monocular camera and an embedded processor. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating the steps of an environment mapping method based on a mobile robot according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the system structure of an environment mapping method based on a mobile robot according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an environment mapping terminal based on a mobile robot according to an embodiment of the present invention. Detailed Implementation

[0008] Definitions:

[0009] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0010] In existing technologies, environmental mapping technology for mobile robots mainly relies on the following solutions: (1) Depth camera-based mapping scheme: The 3D point cloud information of the scene is directly acquired by the depth camera and the environment map is constructed by combining the SLAM algorithm. This scheme has high accuracy, but the depth camera is expensive, consumes a lot of power and is bulky, making it unsuitable for cost-sensitive small monitoring robot products.

[0011] (2) LiDAR-based mapping scheme: 2D or 3D LiDAR sensors are used to acquire distance information of the environment, and two-dimensional or three-dimensional maps are constructed using algorithms such as GMAPping and Cartographer. This scheme has high accuracy and good robustness, but the cost of LiDAR is much higher than that of cameras, and it cannot acquire visual semantic information.

[0012] (3) Mapping scheme based on binocular / multi-view vision: Two or more cameras are used to obtain disparity maps through stereo matching, and then depth information is calculated. This scheme avoids the cost problem of depth cameras, but requires accurate binocular positioning, computationally intensive stereo matching algorithms, and performs poorly in areas with missing textures.

[0013] (4) Mapping scheme based on monocular vision SLAM: This scheme uses only a single camera to simultaneously estimate camera pose and build a map through feature extraction, matching, and motion reconstruction. This scheme has the lowest sensor cost, but it has an inherent scale ambiguity problem, that is, it cannot recover the absolute scale from monocular images, resulting in inaccurate absolute distances in the map. In addition, traditional monocular SLAM algorithms have a large computational load and are difficult to run in real time on embedded devices.

[0014] To address the shortcomings of existing technologies, this solution aims to solve the following technical problem: how to utilize cloud-based monocular depth estimation services to construct a real-time, accurately scaled 2D grid map of the robot's operating environment using a low-cost monitoring robot equipped only with a monocular camera and an embedded processor, with low computational and communication costs. This enables obstacle detection and traversable area labeling within the robot's operating environment. Specifically, the following sub-problems need to be addressed: 1) Depth estimation problem of monocular images: Monocular cameras cannot directly obtain the depth information of the scene, and need to use deep learning models to estimate the pixel-by-pixel depth value from a single RGB image.

[0015] 2) Limited computing resources on the edge: Monitoring robots typically use low-power embedded processors, which cannot directly run large-scale deep estimation neural network models. Computationally intensive tasks need to be offloaded to the cloud.

[0016] 3) The problem of converting depth maps to 2D maps: How to fuse pixel-by-pixel depth maps with robot pose information, and generate a 2D occupied grid map that can be used for navigation through 3D point cloud projection, ground / obstacle classification and rasterization.

[0017] 4) Unreliable depth map edge region problem: Monocular depth estimation often produces inaccurate depth values ​​at object edges and occluded areas. Direct use of these values ​​can lead to incorrect obstacles or false passable areas appearing on the map.

[0018] To at least address the aforementioned issues, this invention employs an edge-cloud collaborative asynchronous parallel processing architecture: the monitoring robot is responsible for image acquisition, visual feature extraction, and map construction, while the cloud server handles monocular depth estimation. The two exchange data asynchronously via cloud storage to achieve depth estimation and feature extraction. In this way, a map with accurate scale can be constructed in real time on a low-cost monitoring robot equipped only with a monocular camera and an embedded processor.

[0019] The following details an environment mapping method based on a mobile robot according to the present invention. Please refer to [link / reference]. Figure 1 The method 100 includes steps 101 to 105; the method is applied to an edge-cloud collaborative system architecture, such as... Figure 2As shown, the device (robot) includes a camera module, an NPU feature extraction module, a frame data management module, and a map building module; the cloud includes an object storage service, a depth estimation server (running the depth estimation model), and a Redis task queue. The specific interaction process is as follows: Step 101: Acquire the current image and corresponding odometry pose information captured by the monocular camera of the monitoring robot. For example, during the robot's movement, the navigation system periodically acquires RGB image frames from the monocular camera and simultaneously obtains the robot's current odometry pose information (two-dimensional plane coordinates x, y and heading angle θ). The system generates a unique timestamp identifier for each frame, saves the RGB image to the rgb subdirectory of the temporary storage directory, and saves the pose information to the pose subdirectory, forming a frame data record to be processed. Each frame data record includes: map number, timestamp, pose triplet (x, y, θ), and confirmation status flag. The system maintains three frame data queues: a queue to be uploaded, a queue to be processed, and a queue to be confirmed, corresponding to the three stages of the processing pipeline.

[0020] Step 102: Extract visual features from the current image to generate current global visual description information; for example, the system starts another independent background thread (asynchronous thread two) to run a lightweight visual location recognition model on the device's neural network processor (NPU / IPU) to extract global visual descriptors.

[0021] Step 103: Upload the current image to the cloud server, instructing the cloud server to perform monocular depth estimation on the current image and generate a metric depth map; for example, the system starts an independent background thread (asynchronous thread one) to upload the RGB image to the cloud depth estimation server and obtain the depth map result. When uploading data: take a frame of data from the queue to be uploaded and check the network connection status; then upload the RGB image file to the cloud object storage service via the HTTP interface; retry if the upload fails, up to 30 times, with a 1-second interval between each retry.

[0022] This step is executed in parallel with step 102 without blocking each other. Since the feature extraction speed on the NPU is usually faster than the network transmission and computing time of cloud depth estimation, it is necessary to wait for the depth results to be returned after feature extraction is completed. The overall latency depends on the slower cloud processing link, thus maximizing the use of computing resources on the device and in the cloud.

[0023] Step 104: Obtain the depth map measurement from the cloud server; for example, when the system finds that the depth estimation result has been generated in the cloud, after obtaining the download URL of the depth map, download the depth map (in millimeters) in 16-bit unsigned integer format to the depth subdirectory of the local temporary storage directory.

[0024] Step 105: Construct a two-dimensional grid map based on the measured depth map, global visual description information, and odometry pose information. That is, once the depth map and descriptor are ready, move the frame data to be processed from the processing queue to the confirmation queue for processing; Once the user or the upper-level navigation system confirms that a batch of frame data is valid, the system performs the following data entry operation on the frames in the confirmation queue: (1) Move the RGB image, depth map, and visual description subfile from the temporary storage directory to the final storage directory; (2) The timestamp and pose information of the frame data are appended to the pose log file (path_{map number}.txt) of the corresponding map number, in the format: {timestamp} {x} {y} {θ}; (3) Load the visual descriptors into the descriptor database in memory for subsequent visual relocalization; (4) Call the depth map to map processing function to fuse the depth information of the frame data into the two-dimensional raster map.

[0025] As can be seen from the above description, the beneficial effects of the present invention are as follows: by constructing an asynchronous parallel processing architecture with end-to-cloud collaboration, the monitoring robot collects the current image and odometry pose information through a monocular camera, extracts visual features to generate current global visual description information, and uploads the current image to the cloud server. The cloud server is responsible for monocular depth estimation to generate a metric depth map. Then, the monitoring robot uses the metric depth map, global visual description information, and odometry pose information to realize the real-time construction of a map with accurate scale on a low-cost monitoring robot equipped only with a monocular camera and an embedded processor.

[0026] In one embodiment of the present invention, step 102, which extracts visual features from the current image and generates current global visual description information, includes: scaling the current image to a preset size and inputting it into a trained visual location recognition model, and outputting the current global visual description information; and saving the current global visual description information in the form of a binary file to a temporary storage directory.

[0027] For example, after scaling the RGB image to 320x320 pixels, it is input into the MixVPR visual location recognition model, which runs on the device's NPU and outputs a 4096-dimensional floating-point global visual descriptor. The descriptor is then saved as a binary file to the desc subdirectory of the temporary storage directory. While waiting for cloud-based depth estimation to complete, once both the depth map and descriptor are ready, the frame is moved from the processing queue to the confirmation queue.

[0028] As described above, by scaling the current image to a uniform preset size and inputting it into the lightweight visual location recognition model, global visual descriptors can be quickly extracted on the device's NPU / IPU. At the same time, the descriptors are saved in binary file form to a temporary storage directory, which facilitates quick loading during subsequent map import and database retrieval, avoids redundant calculations, and improves the overall system efficiency.

[0029] In one embodiment of the present invention, step 104, obtaining the depth measurement map from the cloud server, includes: polling the cloud server to check whether the depth measurement map has been generated; if so, downloading the depth measurement map from the cloud server. For example, when a monitoring robot uploads the current image to the cloud, the monitoring robot's system will poll the cloud to check whether a depth estimation result has been generated, with a polling interval of 1 second and a timeout of 60 seconds.

[0030] As described above, obtaining the cloud depth estimation results through polling queries achieves asynchronous decoupling between the device and the cloud. While waiting for the cloud depth map to be generated, the device can perform local tasks such as visual feature extraction in parallel without blocking, maximizing the utilization of the device's computing resources. At the same time, the polling mechanism is simple, reliable, and easy to implement, meeting the lightweight communication needs of embedded monitoring robots.

[0031] In one embodiment of the present invention, uploading the current image to the cloud server in step 103 includes: Step 301: Upload the current image to the cloud server, instructing the cloud server to process the current image using a trained monocular depth estimation model to obtain an initial metric depth map. For example, the cloud depth estimation server uses a monocular depth estimation model based on the DINOv2 ViT-L / 14 visual Transformer backbone network and a multi-scale decoder, taking an RGB image as input and outputting a pixel-by-pixel metric depth map (unit: meters). The depth values ​​are converted from meters to millimeters, cropped to the range [0, 20000], and saved as a 16-bit PNG image. Subsequently, the cloud server performs special encoding processing on edge regions with discontinuous depth.

[0032] Step 302: Instruct the cloud server to traverse the edge pixels in the initial metric depth map. For each traversed target edge pixel, calculate the difference between the maximum and minimum local depth values ​​of the target edge pixel in a preset neighborhood. If the ratio of the difference to the depth value of the target edge pixel exceeds a threshold, the target edge pixel is determined to be a depth edge pixel. For example, in a 3x3 neighborhood, calculate the difference between the maximum and minimum local depth values. When the ratio of the difference to the depth value exceeds a threshold (e.g., 3%), the pixel is determined to be a depth edge pixel, thus realizing edge detection of the depth map.

[0033] Step 303: Instruct the cloud server to add a preset offset to the depth value of the depth edge pixels to generate a metric depth map; for example, add an offset of 20,000 mm to the depth value of the depth edge pixels for encoding; if the actual depth of a certain depth edge pixel is 1500 mm, then the encoded storage value is 21500. This allows the device to distinguish between edge pixels and normal pixels by determining whether the depth value is greater than 20,000 after receiving the depth map. Subtracting 20,000 from the depth value of the edge pixels restores the true depth value.

[0034] As described above, by adding edge pixel offset encoding to the cloud-based depth estimation model, the pixel depth values ​​of depth discontinuous regions (object edges) are added with a preset offset, enabling the device to accurately identify and distinguish depth edge pixels from ordinary pixels during subsequent map construction. Since the monocular depth estimation model has low prediction accuracy at object edges and is prone to depth jumps, by specially marking edge pixels, the device can exclude these unreliable edge pixels when determining obstacles, thereby effectively avoiding the generation of false obstacles and misjudgment of passable areas, significantly improving the accuracy of grid maps and the safety of robot navigation.

[0035] In one embodiment of the present invention, step 105, which constructs a two-dimensional grid map based on the metric depth map, global visual description information, and odometry pose information, includes steps 501 to 506: Step 501: Based on the odometry pose information and the camera's physical installation parameters, construct a transformation matrix from the camera coordinate system to the world coordinate system. For example, construct a transformation matrix T from the robot body to the world coordinate system based on the robot's odometry pose information (x, y, θ). wb Based on the camera's physical installation parameters (e.g., the camera is installed 10cm above the body and tilted downwards at a 22-degree angle), construct the camera-to-body transformation matrix T. bc Finally, the transformation matrix T from the camera to the world coordinate system is obtained. wc =T wb *T bc .

[0036] Step 502: Perform depth value unit conversion and edge region masking on the measured depth map; for example, after reading the 16-bit depth map, convert the depth value from millimeters to meters, and perform masking on the image edge region (50 pixels on each side and 10 pixels on each side) to eliminate unreliable edge depth values; perform scaling correction on the depth value (multiply by a correction factor of 0.75) to compensate for the systematic bias of the depth estimation model.

[0037] Step 503: Based on the camera intrinsic parameters of the monocular camera, convert the two-dimensional pixel coordinates and depth values ​​in the preprocessed depth map into three-dimensional point coordinates in the camera coordinate system; for example, for each sampled pixel in the depth map (stride=8), use the camera intrinsic parameters (focal length f) x f y Main point c x c y Perform back projection to convert the 2D pixel coordinates and depth values ​​into 3D point coordinates in the camera coordinate system: X c =(uc x )*d / f x Y c =(vc y )*d / f y Z c =d. Where d is the depth value corresponding to the two-dimensional pixel coordinates.

[0038] Step 504: Transform the 3D point coordinates from the camera coordinate system to the world coordinate system according to the transformation matrix; that is, using the transformation matrix T wc Transform 3D points in the camera coordinate system to the world coordinate system.

[0039] Step 505: In the world coordinate system, classify the points in the three-dimensional point coordinates into ground points and obstacle points; for example, the points can be classified according to their height, into ground points or obstacle points.

[0040] Step 506: Project the ground points and obstacle points onto a two-dimensional grid map, and assign them passable and obstacle attributes respectively. For example, project the classified points onto a two-dimensional grid map (resolution: 5 cm per pixel); assign a grid pixel value of 200 to the ground points (passable area), assign a grid pixel value of 50 to the obstacle points (obstacle area), and keep the uncovered area as 0 (unknown area); when the map area is insufficient to cover the newly projected points, the map automatically expands in all directions.

[0041] As described above, a precise coordinate transformation matrix is ​​constructed based on the odometry pose and camera installation parameters, ensuring accurate mapping of the 3D point cloud from the camera coordinate system to the world coordinate system. Unit conversion and edge region masking of the depth map eliminate the influence of inconsistent depth value units and unreliable pixels at image edges. 3D point clouds are generated through camera intrinsic parameter backprojection, providing accurate geometric information for ground / obstacle classification. Finally, rasterization processing generates a clearly labeled 2D raster map of traversable and obstacle areas, providing a directly usable environmental representation for the robot's autonomous navigation.

[0042] In one embodiment of the present invention, step 505, classifying points in the three-dimensional point coordinates into ground points and obstacle points in the world coordinate system, includes: classifying points whose absolute value of the Z-axis coordinate in the world coordinate system is less than or equal to a first height threshold as ground points; classifying points whose Z-axis coordinate in the world coordinate system falls within the range of a second height threshold to a third height threshold as obstacle points; wherein the second height threshold is greater than the first height threshold, and the third height threshold is greater than the second height threshold. For example: 1) Points with an absolute height of less than or equal to 0.1 meters are considered ground points (passable horizontal ground).

[0043] 2) Points with a height between 0.25 meters and 0.8 meters are identified as obstacle points.

[0044] Meanwhile, depth pixels that have undergone edge encoding (values ​​greater than 20000) are excluded from being used as the basis for obstacle determination, but are allowed to participate in the determination of ground points.

[0045] As described above, by setting a first height threshold (e.g., 0.1 meters) to classify points near the ground as ground points, the problem of misjudgment caused by ground undulations and measurement noise is solved; by setting a second and third height threshold (e.g., 0.25 meters to 0.8 meters) to classify points at medium heights as obstacle points, the height range of common obstacles is effectively covered, thereby realizing the classification of ground points and obstacle points.

[0046] In one embodiment of the present invention, after step 506 projects the ground points and obstacle points onto a two-dimensional grid map, the method further includes: traversing all the ground points, performing neighborhood filtering on the traversed target ground point, determining whether there are obstacle points within a preset radius around the target ground point, and if so, excluding the target ground point. For example, in a grid with a step size of 4 pixels, checking whether there are obstacle points within a radius of 4 pixels around each ground point, and excluding the ground point if they exist.

[0047] As described above, by performing neighborhood filtering on ground points and checking whether there are obstacle points around each ground point, a safe buffer zone can be formed between the passable area and the obstacle, avoiding the ground points at the edge of the obstacle being misjudged as passable areas due to depth estimation errors.

[0048] In one embodiment of the present invention, the method further includes: when the robot needs to determine its own position in an existing map, performing the following relocation process, including steps 106 to 109: Step 106: Determine whether a relocation instruction has been received. If so, extract the current global visual description information corresponding to the current image; for example, run the MixVPR model on the current RGB image to extract a 4096-dimensional global visual descriptor.

[0049] Step 107: Obtain the historical frame data that is most similar to the current global visual description information in the historical dataset; the historical frame data includes historical frame images, historical frame depth maps, and historical world poses; for example, the cosine similarity calculation is performed between the above 4096-dimensional global visual descriptor and all stored descriptors in the memory database to find the most similar historical frame data, and obtain the corresponding historical frame images, historical frame depth maps, and historical world poses.

[0050] Step 108: Solve the relative pose based on the current image, historical frame images, and historical frame depth maps; Step 109: Obtain the world pose corresponding to the current image based on the relative pose and the historical world pose. That is, combine the relative pose with the world pose of the historical frames to obtain the current world pose, thus completing the relocalization.

[0051] As described above, the relocation function based on visual location recognition is realized by utilizing the stored historical frame data. When the robot loses its pose due to accumulated odometry error or accidental movement during the patrol process after mapping is completed, it can be relocated by matching the similarity between the current frame image and the historical database, thereby quickly finding its own position.

[0052] In one embodiment of the present invention, step 108, which calculates the relative pose based on the current image, the historical frame image, and the historical frame depth map, includes steps 801 to 804: Step 801: Extract and match feature points between the current image and the historical frame image to generate matching feature point pairs; for example, run the LightGlue feature matching model to extract and match 960 feature point pairs between the current image and the historical frame image.

[0053] Step 802: Filter the matching feature point pairs by confidence level to obtain valid feature point pairs; for example, set the key point confidence level to be greater than 0.1 and the matching confidence level to be greater than 1.8 to perform quality filtering on the feature point pairs to obtain valid feature point pairs.

[0054] Step 803: Solve the relative pose of the effective feature point pairs and the historical frame depth map by combining essential matrix factorization with scale estimation algorithm to obtain the initial relative pose; that is, solve the relative pose by combining essential matrix factorization with RANSAC scale estimation, and use the 3D-2D PnP method to solve it during degradation.

[0055] Step 804: Perform a rationality check on the initial relative pose. If the check is successful, the relative pose is obtained. For example, set rotation angle limits including: pitch angle < 7 degrees, yaw angle < 60 degrees, roll angle < 12 degrees; set translation distance limits including: x < 1.5 meters, y < 0.4 meters, z < 2.0 meters; after the initial relative pose passes the checks of rotation angle limits and translation distance limits, confirm that the relative pose is valid and output it.

[0056] As described above, in the pose estimation process of relocalization, high-precision inter-frame registration is achieved by extracting and matching feature points from the current image and historical frame images to generate sparse feature point pairs; low-quality feature point pairs are filtered out by confidence level, improving the robustness of pose estimation; the scale ambiguity problem of monocular vision pose estimation is solved by using the essential matrix factorization combined with the scale estimation algorithm to recover the absolute scale using the depth map of historical frames; the PnP solution scheme is used in the case of degradation to ensure the robustness of the algorithm; and obvious erroneous pose estimation results are excluded by rationality verification to ensure that the pose output by relocalization is always within a reasonable physical range.

[0057] This invention provides specific application scenarios to illustrate the above method: Application Scenario 1: Initial Mapping of Home Security Patrol Robot When a home monitoring robot equipped with a monocular camera is used for the first time, the user can remotely control the robot to patrol various rooms in the home via a mobile app.

[0058] During its movement, the robot continuously acquires images and sends them to the cloud for depth estimation. Simultaneously, visual feature descriptors are extracted on the on-device NPU, and the two processes are asynchronous and parallel. This involves executing steps 101 to 103.

[0059] Once the user confirms the mapping is complete, the system generates a two-dimensional grid map covering the entire home environment, marking obstacles such as walls and furniture, as well as passable ground areas. This leads to steps 104 and 105.

[0060] Afterward, the robot can autonomously plan patrol routes based on the map, automatically patrol the home, and conduct security monitoring.

[0061] Application Scenario 2: Regular Inspections of Commercial Premises In commercial settings such as shopping malls and office buildings, maintenance personnel remotely control robots to navigate along predetermined routes and create environmental maps. During the mapping process, RGB images are uploaded to the cloud for depth estimation, while visual descriptors are extracted on the device itself; both processes are asynchronous and parallel. Once the map is complete, the robot, based on the stored map and visual descriptor database, achieves autonomous navigation and real-time relocalization during subsequent patrols without human intervention.

[0062] Application Scenario 3: Incremental Map Updates for Industrial Environments In industrial environments such as factory workshops, monitoring robots perform regular inspections. When a robot discovers new obstacles or changes in passageways while running on an existing map, it can add the new frame data to the mapping process, updating a local map area without having to rebuild the entire map.

[0063] Please refer to Figure 3 The present invention also provides an environment mapping terminal based on a mobile robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the environment mapping method based on a mobile robot as described above.

[0064] In summary, this invention discloses an environmental mapping method and terminal based on a mobile robot. It employs an asynchronous parallel processing architecture with edge-cloud collaboration. The device is responsible for image acquisition, visual feature extraction, and map construction, while the cloud server is responsible for monocular depth estimation and generating a metric depth map. The two are asynchronously decoupled through a polling mechanism, maximizing the utilization of edge and cloud computing resources to achieve near real-time mapping with low latency. Simultaneously, the cloud performs offset encoding on the edge pixels of the depth map, enabling the device to accurately identify depth discontinuities and effectively avoid misjudgments of false obstacles or passable areas due to edge estimation errors, significantly improving the reliability of the grid map. The device generates a two-dimensional grid map annotating passable and obstacle areas through coordinate transformation, depth value correction, ground and obstacle height classification, and neighborhood filtering. Combined with a visual descriptor-based relocalization function, when odometry cumulative errors or pose loss occur, it utilizes historical frame data, feature matching, essential matrix factorization, scale estimation, and rationality verification to quickly recover the absolute scale world pose, ensuring continuous navigation stability. It enables the automated construction of accurate-scale 2D raster maps from monocular RGB images, applicable to scenarios such as home security patrols, commercial site monitoring, and industrial inspections, providing reliable environmental perception and autonomous navigation capabilities for low-cost embedded monitoring robots.

[0065] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for environment mapping based on mobile robots, characterized in that, include: Acquire the current image captured by the monocular camera of the monitoring robot and the corresponding odometry pose information; Visual features are extracted from the current image to generate current global visual description information; The current image is uploaded to the cloud server, and the cloud server is instructed to perform monocular depth estimation on the current image to generate a metric depth map; Obtain the metric depth map within the cloud server; A two-dimensional grid map is constructed based on the metric depth map, global visual description information, and odometry pose information.

2. The environment mapping method based on a mobile robot according to claim 1, characterized in that, The step of extracting visual features from the current image to generate current global visual description information includes: The current image is scaled to a preset size and input into the trained visual location recognition model, and the current global visual description information is output. The current global visual description information is saved as a binary file to a temporary storage directory.

3. The environment mapping method based on a mobile robot according to claim 1, characterized in that, The process of obtaining the metric depth map within the cloud server includes: The system polls the cloud server to check if the metric depth map has been generated. If it is found, the system downloads the metric depth map from the cloud server.

4. The environment mapping method based on a mobile robot according to claim 1, characterized in that, Uploading the current image to the cloud server includes: The current image is uploaded to the cloud server, and the cloud server is instructed to process the current image using a trained monocular depth estimation model to obtain an initial metric depth map. The cloud server is instructed to traverse the edge pixels in the initial depth measurement map. For each target edge pixel traversed, the difference between the maximum and minimum values ​​of the local depth of the target edge pixel in a preset neighborhood is calculated. If the ratio of the difference to the depth value of the target edge pixel exceeds a threshold, the target edge pixel is determined to be a depth edge pixel. The cloud server is instructed to add a preset offset to the depth value of the depth edge pixel to generate a depth measurement map.

5. The environment mapping method based on a mobile robot according to claim 1, Its characteristics are The step of constructing a two-dimensional grid map based on the metric depth map, global visual description information, and odometry pose information includes: Based on the odometer pose information and the camera's physical installation parameters, a transformation matrix from the camera coordinate system to the world coordinate system is constructed. The depth map is subjected to depth value unit conversion and edge region masking. Based on the camera intrinsic parameters of the monocular camera, the two-dimensional pixel coordinates and depth values ​​in the preprocessed depth measurement map are converted into three-dimensional point coordinates in the camera coordinate system. The 3D point coordinates are transformed from the camera coordinate system to the world coordinate system according to the transformation matrix. In the world coordinate system, the points in the three-dimensional point coordinates are classified into ground points and obstacle points; The ground points and obstacle points are projected onto a two-dimensional grid map and assigned passable and obstacle attributes respectively.

6. The environment mapping method based on a mobile robot according to claim 5, characterized in that, The classification of points in the three-dimensional point coordinates into ground points and obstacle points in the world coordinate system includes: Points whose absolute value of the Z-axis coordinate in the world coordinate system is less than or equal to the first height threshold are identified as ground points; Points whose Z-axis coordinates in the world coordinate system are within the range of the second height threshold to the third height threshold are identified as obstacle points; The second height threshold is greater than the first height threshold, and the third height threshold is greater than the second height threshold.

7. The environment mapping method based on a mobile robot according to claim 5, characterized in that, After projecting the ground points and obstacle points onto the two-dimensional grid map, the process further includes: Traverse all the ground points, and for each target ground point, perform neighborhood filtering to determine whether there are any obstacle points within a preset radius around the target ground point. If there are, exclude the target ground point.

8. The environment mapping method based on a mobile robot according to claim 1, characterized in that, The method further includes: Determine whether a relocation command has been received; if so, extract the current global visual description information corresponding to the current image. Obtain historical frame data from the historical dataset that is most similar to the current global visual description information; the historical frame data includes historical frame images, historical frame depth maps, and historical world poses. The relative pose is determined based on the current image, historical frame images, and historical frame depth maps. The world pose corresponding to the current image is obtained based on the relative pose and the historical world pose.

9. The environment mapping method based on a mobile robot according to claim 8, characterized in that, The step of solving the relative pose based on the current image, historical frame images, and historical frame depth maps includes: Feature points are extracted and matched between the current image and the historical frame image to generate matching feature point pairs; The matching feature point pairs are filtered by confidence to obtain valid feature point pairs; The relative pose of the effective feature point pairs and the historical frame depth map is solved by combining essential matrix factorization with a scale estimation algorithm to obtain the initial relative pose. The initial relative pose is validated for reasonableness. If the validation is successful, the relative pose is obtained.

10. A mobile robot-based environment mapping terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the environment mapping method based on a mobile robot as described in any one of claims 1 to 9.

Images