A real-time fine mapping method suitable for underwater robots

Abstract

The application discloses a real-time fine mapping method suitable for an underwater robot, which comprises the following steps: firstly, a 3D Gaussian map is constructed by using a first frame; then, key frame determination is performed on the following image frames; and finally, an incremental map is constructed by using the key frames; the process effectively overcomes the optical interference specific to the underwater environment, filters out the geometric noise caused by suspended particulate matters and the like, and significantly improves the geometric consistency and color authenticity of the map; all frames are tracked in real time, accurate poses are obtained and output, the process effectively overcomes the interference of underwater optical degradation on pose estimation, and ensures the robustness and accuracy of the tracking process.

Description

A Real-Time Fine Mapping Method for Underwater Robots Technical Field

[0001] This invention relates to a mapping method for underwater robots, and more particularly to a real-time, high-resolution mapping method suitable for underwater robots. Background Technology

[0002] With the increasing demand for marine resource exploration, underwater infrastructure maintenance, and ecological monitoring, the autonomous operation capability of underwater robots has become crucial. The core technology for achieving fully autonomous navigation is Simultaneous Localization and Mapping (SLAM), which involves the robot estimating its own pose and building an environmental map in real time in an unknown environment. Because the underwater environment is a complex optical propagation medium, light undergoes attenuation and scattering as it travels through water: attenuation, especially the strong absorption of long wavelengths (such as red light), results in images with a severe blue-green tint and a sharp decrease in contrast; scattering includes image blurring caused by forward scattering and underwater haze effects caused by backscattering, making the image appear as if veiled, with significant loss of detail. Therefore, underwater robot SLAM faces severe challenges not present in terrestrial environments.

[0003] Existing underwater robot SLAM methods fall into the following categories, but each has its own corresponding technical problems:

[0004] 1. Traditional geometry-based SLAM: These methods typically process degraded images directly, performing tracking and mapping through feature extraction or direct methods. While efficient, the reconstructed maps are usually sparse or semi-dense point clouds, failing to provide the dense 3D models with consistent light perception required for underwater robots to perform fine tasks. Furthermore, they lack inherent modeling and compensation capabilities for optical distortions in water, resulting in limited robustness.

[0005] 2. Neural Radiation Field-Based SLAM: In recent years, neural radiation field technology has been introduced into the field of underwater robot SLAM, such as SeaThru-NeRF. By integrating an underwater image formation model into NeRF, true-color scene restoration and novel view synthesis have been successfully achieved. This method generates high-quality models and effectively overcomes water degradation effects. However, NeRF's volumetric rendering method based on ray-mapping and neural networks is extremely time-consuming. The training and rendering speeds are far from meeting the stringent real-time requirements of underwater robots, severely limiting its application in underwater robot SLAM.

[0006] 3. 3DGS-based SLAM system: 3D Gaussian sputtering (3DGS) is a groundbreaking technology in the field of 3D reconstruction. It uses an explicit 3D Gaussian ellipsoid to represent the scene and performs image synthesis through efficient differential sputtering rendering technology, achieving real-time, lighting-level novel view generation with rendering speeds reaching hundreds of frames per second. However, because 3DGS optimizes and fits the observed image, directly applying native 3DGS to underwater environments can incorrectly learn color distortion and optical noise caused by water as attributes of the scene itself. This results in a large number of non-physical planktonic artifacts in the map, destroying the geometric consistency of the scene and reducing reliability.

[0007] In summary, existing real-time methods cannot provide detailed and physically accurate reconstruction results, while methods that can restore true color cannot meet real-time requirements. That is, they cannot obtain physically accurate and detailed true-color environment models while ensuring real-time pose tracking and map building efficiency. Summary of the Invention

[0008] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a real-time fine mapping method suitable for underwater robots.

[0009] The technical solution of this invention is: a real-time fine mapping method suitable for underwater robots, which is carried out according to the following steps:

[0010] Step 1. Acquire RGB-D image data;

[0011] Step 2. Image data preprocessing;

[0012] Step 3. Determine if this is the first frame image. If yes, set it as a keyframe and pre-build a 3D Gaussian map. If no, proceed to step 4.

[0013] Step 4. Determine if it is a keyframe. If yes, proceed to both steps 5 and 6. If no, proceed to step 6.

[0014] Step 5. Perform incremental map construction according to the following steps:

[0015] Step 5.1 Based on the 3D Gaussian map constructed in the previous keyframe, perform intelligent point cloud supplementation;

[0016] Step 5.2 Determine whether the 3D Gaussian point cloud of the current keyframe is a floating object. If yes, perform floating object suppression; otherwise, retain the 3D Gaussian point cloud.

[0017] Step 5.3 Perform local bundle adjustment and joint optimization of water body physical parameters on the obtained image;

[0018] Step 5.4 Obtain and output the current 3D Gaussian map and water body physical parameters;

[0019] Step 6. Follow the steps below to perform real-time tracking:

[0020] Step 6.1 Pose initialization;

[0021] Step 6.2 Preliminary optimization of camera pose;

[0022] Step 6.3 Based on the 3D Gaussian map constructed from the preliminary optimized camera pose and the previous keyframe, reliable 3D Gaussian point clouds are selected.

[0023] Step 6.4 Use a reliable 3D Gaussian point cloud to refine the pose, obtain and output the accurate pose.

[0024] The preferred method for pre-constructing the 3D Gaussian map in step 3 is to create an initial 3D Gaussian point cloud through back projection, initialize the geometric parameters of the 3D Gaussian point cloud according to the spatial point density, calculate the covariance based on the distance between points, set the opacity to an empirical initial value, and assign the color directly from the RGB image.

[0025] The preferred method for determining a keyframe in step 4 is as follows: if the displacement between the current frame and the most recent keyframe exceeds a preset threshold or the area of ​​the new region observed in the current frame exceeds a preset proportion of the total field of view, then the current frame is a keyframe.

[0026] The preferred step 5.1 is as follows: Based on the observation data of the current keyframe, identify the image region that was not fully explained by the 3D Gaussian map constructed by the previous keyframe, i.e., the severely sparse point cloud region; perform back projection processing on the severely sparse point cloud region to generate new 3D Gaussian points; and fuse the new 3D Gaussian points into the original point cloud data of the current keyframe to complete the intelligent point cloud supplementation.

[0027] The preferred step 5.2 is as follows: Calculate the projection depth of each 3D Gaussian point cloud. Compared with depth map observations absolute difference ,like Furthermore, its color coefficients are consistent with the currently estimated global backscattered color. If the Euclidean distance meets the preset condition, then the 3D Gaussian point cloud is determined to be a floating object, and suppression is performed according to the following formula:

[0028] ;

[0029] in This represents the opacity of the 3D Gaussian point cloud at the current moment. This represents the new opacity obtained after suppressing the update. It is a geometric inconsistency indicator function. It is a preset distance threshold, when The function value is 1 when the condition is met, and 0 otherwise. It is the color similarity weight. It is the color of Gaussian point clouds. With global backscatter color The Euclidean distance between them , These are preset coefficients.

[0030] The preferred step 5.3 is as follows: every 5 new keyframes received, within a sliding window containing 10 keyframes, the camera pose, 3D Gaussian parameters, and water physical parameters are jointly optimized.

[0031] The preferred step 6.2 is as follows: First, generate the prediction image Î using the media rendering mode, and then follow the formula... Synthetic underwater degradation images ,in It's rendering depth. It is the global backscattered color, β D β B These are the scattering / absorption attenuation coefficient and the background light attenuation coefficient, respectively; then calculate... Compared with the actual observed image I c The photometric error between them was analyzed using T. c The next iteration of optimization yields the preliminary optimized camera pose P. c .

[0032] The preferred step 6.3 is as follows: For the 3D Gaussian points visible from the current viewpoint, the camera pose P is initially optimized. c Calculate its projection depth d i Then calculate the projection depth d. i The corresponding pixel depth value in the observed depth map D' absolute differences between The observation depth map D' is obtained based on a 3D Gaussian map constructed from the previous keyframe. If ε is a preset distance threshold, then the 3D Gaussian point cloud is a reliable 3D Gaussian point cloud.

[0033] The preferred step 6.4 is as follows: use the selected reliable 3D Gaussian point cloud set Gs to render the image, calculate the rendering loss and perform Ts iterations of optimization to obtain and output the accurate pose P.

[0034] This invention first constructs a 3D Gaussian map using the first frame, then performs keyframe determination on subsequent image frames, using keyframes for incremental map construction. This involves sequentially performing intelligent point cloud supplementation, planktonic suppression, local bundle adjustment, and joint optimization of water physical parameters to obtain and output the current 3D Gaussian map and water physical parameters. This process effectively overcomes the unique optical interference of the underwater environment by dynamically managing the 3D Gaussian point cloud and introducing physical constraints, filtering out geometric noise caused by suspended particles, and significantly improving the geometric consistency and color accuracy of the map. Real-time tracking is performed on all frames, sequentially performing pose initialization, preliminary optimization of camera pose, selection of reliable 3D Gaussian points, and refinement of pose using reliable 3D Gaussian points to obtain and output the accurate pose. This process effectively overcomes the interference of underwater optical degradation on pose estimation, ensuring the robustness and accuracy of the tracking process. This results in underwater robot mapping achieving a balance between real-time performance, geometric accuracy, and color accuracy, ensuring efficient real-time pose tracking and map construction while obtaining a physically correct, detailed, and true-color environment model. Attached Figure Description

[0035] Figure 1 is a flowchart of an embodiment of the present invention. Detailed Implementation

[0036] The embodiments of the present invention are implemented based on the following systems and tasks:

[0037] System hardware and software configuration

[0038] Underwater robot platform: It adopts a small observation-grade ROV (remotely operated vehicle) as a carrier and has basic motion control and depth and attitude perception capabilities.

[0039] Core processing unit: Equipped with an embedded AI computing module, running Ubuntu 20.04 LTS.

[0040] Visual perception sensor: An Intel RealSense D455 depth camera is used. The active infrared projection of this camera may be severely attenuated underwater. Therefore, this embodiment of the invention can utilize its high-quality RGB images and passive depth maps calculated by stereo vision algorithms. The camera frame rate is set to 15 Hz and the resolution is set to 848x480 to balance processing speed and accuracy.

[0041] Software environment: The main programming languages ​​are Python 3.8 and C++. The core computing libraries include the PyTorch deep learning framework (for model optimization and gradient calculation) and CUDA (for accelerating 3D Gaussian sputtering rendering and optimization processes on GPUs).

[0042] 2. Task Scenario Description

[0043] The mission is to conduct an autonomous inspection of a nearshore coral reef ecosystem. The underwater robot will navigate along a pre-defined path, and the system must complete the following tasks in real time:

[0044] 2.1 Real-time estimation of the robot's precise six-DOF pose relative to the starting point;

[0045] 2.2 Build and update a true-color, high-detail 3D map of coral reefs online;

[0046] 2.3 Real-time output of estimated water body optical parameters to provide data support for environmental monitoring.

[0047] The specific method is shown in Figure 1, and the following steps should be followed:

[0048] Step 1. Acquire RGB-D image data, i.e., the underwater robot hovers at the starting point, and the system continuously acquires RGB-D images after startup;

[0049] Step 2. Image data preprocessing, i.e., filtering and hole filling of the depth map of the frame image;

[0050] Step 3. Determine if this is the first frame image. If yes, set it as a keyframe and pre-build a 3D Gaussian map. If no, proceed to step 4.

[0051] The pre-built 3D Gaussian map is created by back projection to create an initial 3D Gaussian point cloud, the geometric parameters of the 3D Gaussian point cloud are initialized according to the spatial point density, the covariance is calculated based on the distance between points, the opacity is set to an empirical initial value, and the color is directly assigned from the RGB image.

[0052] Step 4. Determine if it is a keyframe. If yes, proceed to both steps 5 and 6. If no, proceed to step 6.

[0053] The criteria for determining a keyframe are: if the displacement between the current frame and the most recent keyframe exceeds a preset threshold (30cm) or the area of ​​the new region observed in the current frame exceeds a preset proportion (40%) of the total field of view, then the current frame is a keyframe.

[0054] Step 5. Perform incremental map construction according to the following steps:

[0055] Step 5.1 Based on the 3D Gaussian map constructed in the previous keyframe, perform intelligent point cloud supplementation; that is, based on the observation data of the current keyframe, identify image regions that are not fully explained by the 3D Gaussian map constructed in the previous keyframe, i.e., severely sparse point cloud regions, such as complex gaps or structural edges of coral reefs; perform back projection processing on the severely sparse point cloud regions to generate new 3D Gaussian points; and fuse the new 3D Gaussian points into the original point cloud data of the current keyframe to complete intelligent point cloud supplementation.

[0056] Step 5.2 Determine whether the 3D Gaussian point cloud of the current keyframe is a floating object. If yes, perform floating object suppression; otherwise, retain the 3D Gaussian point cloud.

[0057] Specifically, the projection depth of each 3D Gaussian point cloud is calculated. Compared with depth map observations absolute difference ,like Furthermore, its color coefficients are consistent with the currently estimated global backscattered color. If the objects are similar, the 3D Gaussian point cloud is determined to be floating objects, and suppression is performed according to the following formula:

[0058] ;

[0059] in This represents the opacity of the 3D Gaussian point cloud at the current moment. This represents the new opacity obtained after suppressing the update. It is a geometric inconsistency indicator function. It is a preset distance threshold (0.1m), when The function value is 1 if the condition is met, otherwise the function value is 0. It is the color similarity weight. It is the color of Gaussian point clouds. With global backscatter color The Euclidean distance between them , These are preset coefficients;

[0060] Step 5.3 Perform local bundle adjustment and joint optimization of water body physical parameters on the obtained image;

[0061] Specifically, optimization is performed every 5 new keyframes received. Within a sliding window containing 10 keyframes, the camera pose, all visible 3D Gaussian parameters, and water physics parameters, such as global backscatter color, are jointly optimized. Scattering / absorption attenuation coefficient β D Background light attenuation coefficient β B The optimization is divided into two stages: first, the map is optimized with the pose fixed; then, all variables are optimized simultaneously, and the gradient descent method is used to minimize the rendering loss.

[0062] Step 5.4 Obtain and output the current 3D Gaussian map and water body physical parameters. β D and β B ;

[0063] Step 6. Follow the steps below to perform real-time tracking:

[0064] Step 6.1 Pose initialization: Based on the uniform velocity model, the pose of the current frame is predicted using the pose changes of the previous two frames.

[0065] Step 6.2 Initial optimization of camera pose;

[0066] First, generate the predicted image Î using the media rendering mode, then follow the formula Synthetic underwater degradation images ,in It's rendering depth. It is the global backscattered color, β D β B These are the scattering / absorption attenuation coefficient and the background light attenuation coefficient, respectively; then calculate... Compared with the actual observed image I c The photometric error between the two positions is optimized through Tc (Tc=5) iterations to obtain the preliminary optimized camera pose P. c ;

[0067] The image generated by this mode simulates the degradation effect observed by actual underwater cameras, so that the rendered image and the real image acquired by the sensor are in the same physical domain during pose optimization, thereby achieving physical consistency matching. This effectively smooths high-frequency noise in the image and geometric artifacts in the early stage of the map. Combined with the inherent constraints of the physical model, the optimization process can prioritize global convergence and robustness, avoid getting stuck in erroneous local optima due to local detail interference, and effectively prevent tracking failure.

[0068] Step 6.3 Based on the 3D Gaussian map constructed from the preliminary optimized camera pose and the previous keyframe, select reliable 3D Gaussian point clouds;

[0069] For the 3D Gaussian point visible from the current viewpoint, the camera pose P is initially optimized. c Calculate its projection depth d i Then calculate the projection depth d. i The corresponding pixel depth value in the observed depth map D' absolute differences between ,like ε is a preset distance threshold, ε=0.1m, then the 3D Gaussian point cloud is a reliable 3D Gaussian point cloud; the observation depth map D' is obtained based on the 3D Gaussian map constructed from the previous keyframe. Since the 3D Gaussian map naturally contains depth information, only the spatial position attributes of the 3D Gaussian point cloud are retained, and other attributes are removed, so that the depth observation map D' of the corresponding observation pose can be obtained.

[0070] Step 6.4 Refine the pose using reliable 3D Gaussian point clouds, that is, use the selected reliable 3D Gaussian point cloud set Gs to render the image, calculate the rendering loss and perform Ts=5 iterations of optimization to obtain and output the accurate pose P.

[0071] Implementation Results: This embodiment verifies that the method described in this invention can run stably on the specified hardware platform. When processing RGB-D data streams with a resolution of 848×480 and an input frequency of 15Hz, the average processing frame rate is stable at 10-12 FPS. This indicates that the system takes approximately 83-100 milliseconds on average from receiving one frame of image to completing pose calculation and map update, which matches the sensor data input cycle (approximately 96 milliseconds), ensuring timely data processing without continuous cumulative delay. When the frame rate fluctuation range is controlled within ±1 FPS, it exhibits good stability, successfully achieving simultaneous real-time pose tracking and true-color detailed map construction. The generated 3D Gaussian map effectively eliminates color distortion and haze effects in the water, with a clear geometric structure, providing a high-quality data foundation for assessing the health status of coral reefs.

Claims

1. A real-time fine mapping method suitable for underwater robots, characterized in that... The following steps are followed: Step 1. Acquire RGB-D image data; Step 2. Image data preprocessing; Step 3. Determine if it is the first frame image. If yes, set it as a keyframe and pre-build a 3D Gaussian map. If no, proceed to Step 4; Step 4. Determine if it is a keyframe. If yes, proceed to Steps 5 and 6 simultaneously. If no, proceed to Step 6; Step 5. Perform incremental map construction sequentially according to the following steps: Step 5.1 Based on the 3D Gaussian map constructed from the previous keyframe, intelligent point cloud supplementation is performed; Step 5.2 Determine whether the 3D Gaussian point cloud of the current keyframe is planktonic. If yes, planktonic suppression is performed; otherwise, the 3D Gaussian point cloud is retained; Step 5.3 Perform local bundle adjustment and joint optimization of water body physical parameters on the obtained image; Step 5.4 Obtain and output the current 3D Gaussian map and water body physical parameters; Step 6. Perform real-time tracking in sequence according to the following steps: Step 6.1 Pose initialization; Step 6.2 Preliminary optimization of camera pose; Step 6.3 Based on the preliminary optimized camera pose and the 3D Gaussian map constructed from the previous keyframe, select reliable 3D Gaussian point clouds; Step 6.4 Use reliable 3D Gaussian point clouds to refine the pose, obtain and output the accurate pose.

2. The real-time fine mapping method for underwater robots according to claim 1, characterized in that... The pre-built 3D Gaussian map in step 3 is created by back projection to create an initial 3D Gaussian point cloud, initializes the geometric parameters of the 3D Gaussian point cloud according to the spatial point density, calculates the covariance based on the distance between points, sets the opacity to an empirical initial value, and assigns the color directly from the RGB image.

3. The real-time fine mapping method for underwater robots according to claim 2, characterized in that... The criteria for determining a keyframe in step 4 are as follows: if the displacement between the current frame and the most recent keyframe exceeds a preset threshold or the area of ​​the new region observed in the current frame exceeds a preset proportion of the total field of view, then the current frame is a keyframe.

4. The real-time fine mapping method for underwater robots according to claim 3, characterized in that... Step 5.1 is as follows: Based on the observation data of the current keyframe, identify the image regions that are not fully explained by the 3D Gaussian map constructed by the previous keyframe, i.e., severely sparse point cloud regions; perform back projection processing on the severely sparse point cloud regions to generate new 3D Gaussian point clouds; and fuse the new 3D Gaussian point clouds into the original point cloud data of the current keyframe to complete intelligent point cloud supplementation.

5. The real-time fine mapping method for underwater robots according to claim 4, characterized in that... Step 5.2 is as follows: Calculate the projection depth of each 3D Gaussian point cloud. Compared with depth map observations absolute difference ,like Furthermore, its color coefficients are consistent with the currently estimated global backscattered color. If the Euclidean distance meets the preset condition, then the 3D Gaussian point cloud is determined to be a floating object, and suppression is performed according to the following formula: ;in This represents the opacity of the 3D Gaussian point cloud at the current moment. This represents the new opacity obtained after suppressing the update. It is a geometric inconsistency indicator function. It is a preset distance threshold, when The function value is 1 when the condition is met, and 0 otherwise. It is the color similarity weight. It is the color of Gaussian point clouds. With global backscatter color The Euclidean distance between them 、 These are preset coefficients.

6. The real-time fine mapping method for underwater robots according to claim 5, characterized in that... Step 5.3 is as follows: For every 5 new keyframes received, within a sliding window containing 10 keyframes, the camera pose, 3D Gaussian parameters, and water physical parameters are jointly optimized.

7. The real-time fine mapping method for underwater robots according to claim 6, characterized in that... Step 6.2 is as follows: First, generate the prediction image using the media rendering mode. Then follow the formula Synthetic underwater degradation images ,in It's rendering depth. It is the global backscattered color, β D β B These are the scattering / absorption attenuation coefficient and the background light attenuation coefficient, respectively; then calculate... Compared with the actual observed image I c The photometric error between them was analyzed using T. c The next iteration of optimization yields the preliminary optimized camera pose P. c .

8. The real-time fine mapping method for underwater robots according to claim 7, characterized in that... Step 6.3 is as follows: For the 3D Gaussian points visible from the current viewpoint, the camera pose P is initially optimized. c Calculate its projection depth d i Then calculate the projection depth d. i The corresponding pixel depth value in the observed depth map D' absolute differences between The observation depth map D' is obtained based on a 3D Gaussian map constructed from the previous keyframe. If ε is a preset distance threshold, then the 3D Gaussian point cloud is a reliable 3D Gaussian point cloud.

9. The real-time fine mapping method for underwater robots according to claim 8, characterized in that... Step 6.4 is as follows: Use the selected reliable 3D Gaussian point cloud set Gs to render the image, calculate the rendering loss and perform Ts iterations of optimization to obtain and output the accurate pose P.

Images