Complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data

Abstract

The invention provides a complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data. The algorithm comprises the steps of 1, obtaining initial key frame data through an SLAM system; 2, processing the key frame data and generating a three-dimensional point cloud; and 3, performing three-dimensional reconstruction on the point cloud data to generate a three-dimensional model. A point cloud registration algorithm is improved, and more accurate point cloud data is obtained. According to the technology provided by the invention, through dynamic object identification and filtering, the system can cope with activity factors such as pedestrians and vehicles, so that high-quality reconstruction is still kept in a crowded or non-static environment, especially in a scene with large noise, illumination variation or shielding, the algorithm can significantly improve the effect of three-dimensional reconstruction, and the method is suitable for large-scale popularization and application. And the SLAM system can maintain high precision and stability in a complex environment.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data. Background Art

[0002] Glossary:

[0003] Point cloud: A data set consisting of a large number of discrete points in space, usually used to describe the three-dimensional shape and structure of an object or environment. Each point contains location information and possible additional attributes (such as color or intensity).

[0004] Point cloud registration: Point cloud registration is the process of aligning point cloud data in different coordinate systems to the same coordinate system through rotation and translation to restore the complete scene or object structure.

[0005] Sensor: A device used to collect environmental information or motion data, providing the necessary location information and environmental characteristics for the algorithm.

[0006] ORB (Oriented FAST and Rotated BRIEF) feature: is an algorithm used to detect and describe key points in images. It is a combination of Oriented FAST and Rotated BRIEF, so it is called ORB.

[0007] As industrial manufacturing develops towards intelligence and automation, mobile robots in large and complex environments have become an important research hotspot. Such robots need to perform precise positioning and environmental modeling in unknown or changing complex environments to achieve efficient navigation and operation capabilities.

[0008] To obtain point cloud data in a complex environment, it is necessary to rely on the multi-sensor fusion and precise positioning capabilities of the SLAM (Simultaneous Localization and Mapping) system. The SLAM system uses sensors to collect three-dimensional information in the environment and records the point cloud data of the scene during movement. Since some scenes have many objects and the environment is complex, SLAM sensors need to have a higher resolution and remove interference caused by noise, dynamic objects or environmental occlusion through real-time data processing to ensure the integrity and accuracy of the point cloud data. In addition, the SLAM system needs to optimize the registration algorithm during the data collection process to maintain accurate pose estimation between each frame of the point cloud, so as to obtain an accurate overall point cloud structure.

[0009] After the point cloud data is collected, 3D reconstruction is required. The point cloud data will be precisely registered and spliced ​​to gradually generate a continuous 3D model. 3D reconstruction in complex environments faces challenges such as environmental occlusion, sparse features, and dynamic factors. An incremental update method is used to enable the model to adapt to environmental changes and provide continuous 3D reconstruction effects.

[0010] This 3D reconstruction algorithm based on SLAM point cloud data can be applied to scenarios such as indoor environments and engineering tests that require 3D modeling of complex environments.

[0011] SLAM technology is a key technology that enables robots to simultaneously complete their own positioning and environmental map construction in unknown environments. It is mainly composed of two core parts: the front end and the back end. Figure 2 As shown, its functions are achieved through the close cooperation of multiple modules.

[0012] In front-end processing, such as Figure 1 As shown in the figure, the system first collects environmental data through sensors. For visual SLAM, feature extraction and matching are performed, and these features need to have good invariance and distinguishability. Through feature matching, the camera motion can be estimated, and then a preliminary pose estimate can be obtained. At the same time, the front end is also responsible for screening external points and handling occlusion issues to provide reliable data association. The back-end optimization mainly processes the pose and landmark information provided by the front end, and realizes the global optimization of the robot trajectory and map by constructing a graph optimization problem.

[0013] In terms of system architecture, Figure 2 As shown in the figure, traditional SLAM includes important modules such as loop detection, mapping and relocalization. Loop detection is used to identify whether the robot returns to a previously visited location, which is crucial to eliminating cumulative errors. The map building module is responsible for organizing environmental information into a structured form. The relocalization module enables the system to quickly determine its own position in a known map, which is important for the robustness and practicality of the system.

[0014] However, traditional SLAM systems have the following defects:

[0015] (1) Traditional SLAM systems are highly complex, and feature extraction, matching, and optimization processes require a large amount of computing resources, which is often difficult to meet in application scenarios with high real-time requirements.

[0016] (2) The system has poor adaptability to the environment and is prone to failure in dynamic scenes. It also relies too much on feature points and feature lines in the environment, which significantly reduces the positioning accuracy in environments with sparse features.

[0017] (3) Traditional SLAM’s map expression and scene understanding capabilities are relatively limited. Sparse maps are difficult to meet navigation needs. The system lacks semantic understanding capabilities and is difficult to effectively identify and process dynamic objects. In complex environments, traditional SLAM’s map building capabilities are limited, making it difficult to generate comprehensive and accurate maps.

[0018] Generating 3D modeling from point cloud is a complex process from raw point cloud data to the final 3D model. The acquired raw point cloud usually contains noise, outliers and overlapping areas, and needs to be preprocessed before it can be used for subsequent modeling. However, existing technologies often have the following defects:

[0019] (1) During the data collection stage, point cloud acquisition is often limited by equipment and environment, resulting in blind spots, occlusions, and uneven density in the collected data. In particular, it is difficult to obtain complete geometric information for complex scenes.

[0020] (2) In the process of point cloud processing and reconstruction, traditional methods are sensitive to noise and missing data. Although denoising and filtering can remove some interference, they may also lose useful detail information.

[0021] (3) Traditional modeling methods have high computational overhead when processing large-scale point clouds, especially when performing mesh optimization and texture mapping, which often consumes a lot of computing resources and time. For real-time application scenarios, the processing speed is difficult to meet the requirements. Summary of the invention

[0022] In view of this, the present application provides a complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data to overcome the above-mentioned problems existing in the prior art.

[0023] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0024] Complex environment 3D reconstruction algorithm based on SLAM point cloud data, including:

[0025] Step 1: Obtain initial key frame data through the SLAM system;

[0026] Step 2: Process key frame data and generate 3D point cloud;

[0027] Step 3: Reconstruct the point cloud data into three dimensions to generate a three-dimensional model.

[0028] Furthermore, the step 1 of obtaining the initial key frame data through the SLAM system specifically includes:

[0029] Step 1.1: Get the initial frame data of the sensor and initialize it;

[0030] Step 1.2: Use the target detection model YOLOv5 to detect dynamic objects in the scene, and mark the detected dynamic object area as a potential dynamic area. The target detection model YOLOv5 introduces the GhostNet module and the ECA attention mechanism;

[0031] Step 1.3: Use ORB feature points for inter-frame matching based on the ORB-SLAM3 system;

[0032] Step 1.4: Perform initial pose estimation to determine the robot's initial position and orientation in the environment, providing a reference for subsequent map construction;

[0033] Step 1.5: Track the local map and determine whether a new keyframe needs to be created. When the number of feature point matches in the current frame is insufficient, keyframe insertion is triggered.

[0034] Step 1.6: Local map optimization. After the keyframes are inserted, the map structure is optimized in the local area, including adding new 3D map points, removing map points with large errors, and performing local BA optimization to improve the accuracy of the local map.

[0035] Step 1.7: Perform closed loop detection and global optimization. When a closed loop is detected between the current frame and the historical frame, that is, the current frame and the historical frame correspond to the same position, the global BA optimization is triggered to correct the accumulated error and optimize the accuracy of the entire map.

[0036] Step 1.8: Determine whether the current frame needs to be generated and inserted as a new key frame, obtain the initial point cloud data, and pass the robot's posture information and the sensor's depth information to the point cloud construction thread.

[0037] Furthermore, the step 2 of processing the key frame data and generating a three-dimensional point cloud specifically includes:

[0038] Step 2.1: The constructor initializes the global point cloud map and voxel filter, and initializes the leaf parameters;

[0039] Step 2.2: Generate the corresponding 3D point cloud based on the RGB image and depth image of the key frame;

[0040] Step 2.3: Construct a multi-scale point cloud pyramid, take the original point cloud as the high-resolution layer, then set the grid size to generate the medium-resolution layer and the low-resolution layer;

[0041] Step 2.4: Point cloud registration, first start with the low-resolution layer to register, find the nearest neighbor point with the smallest Euclidean distance for each point in the source point cloud in the target point cloud, then adjust the weight value according to the size of the Euclidean distance, calculate the registration error between the source point cloud and the nearest neighbor point according to the adjusted weight value, use the least squares method to optimize the rigid body transformation, obtain the optimization result, and finally apply the optimization result to the source point cloud; the medium-resolution layer uses the optimization result of the low-resolution layer as the initial rigid body transformation result to perform the above-mentioned point cloud registration operation, and similarly, the high-resolution layer uses the optimization result of the medium-resolution layer as the initial rigid body transformation result to perform the point cloud registration operation;

[0042] Step 2.5: Update the global point cloud map and perform real-time visualization through the point cloud visualization module of PCL to display the point cloud map and show the three-dimensional structure of the environment.

[0043] Furthermore, in step 3, the three-dimensional reconstruction of the point cloud data to generate the three-dimensional model specifically includes:

[0044] Step 3.1: Import point cloud data and ensure that the data contains the three-dimensional coordinates (x, y, z) of the points. When loading the point cloud, check whether the point cloud data type is legal so as to carry out subsequent processing steps;

[0045] Step 3.2: After loading, preprocess the point cloud to improve data quality and computational efficiency. The preprocessing includes downsampling, denoising, and normal vector estimation.

[0046] Step 3.3: Reconstruct the mesh;

[0047] Step 3.4: After the mesh reconstruction is completed, post-processing is performed to improve the quality of the mesh model, and the post-processing includes removing duplicate triangles and removing unreferenced vertices;

[0048] Step 3.5: Finally, the processed mesh model is saved as a 3D file format PLY file.

[0049] Furthermore, the step 3.3 of performing mesh reconstruction includes:

[0050] Step 3.31: Initialize the octree structure, input the point cloud data and set the maximum depth parameter to control the resolution; recursively divide the space, map the point cloud data to each node of the octree, and adjust the data density of the node to strike a balance between resolution and computational complexity;

[0051] Step 3.32: The normal information of the point cloud is converted into a divergence field by using the Poisson reconstruction algorithm, and the divergence field is solved by using the Poisson reconstruction algorithm to generate an implicit function;

[0052] Step 3.33: Setting reconstruction parameters, including solver type and precision, performing solving to generate voxel data of implicit function for subsequent surface extraction;

[0053] Step 3.34: Use the Marching Cubes algorithm to extract the isosurface from the implicit function and convert it into a triangular mesh, making it suitable for a variety of 3D model formats.

[0054] Furthermore, the post-processing process in step 3.4 includes:

[0055] Step 3.41: Initialize the hash table;

[0056] Step 3.42: sort the vertices of each triangle by index and store them in a hash table, wherein the triangle is a basic unit constituting a triangular mesh;

[0057] Step 3.43: Determine whether there are duplicate triangles. If so, remove the duplicates. Otherwise, execute step 3.43.

[0058] Step 3.44: Check whether each vertex of the triangle is referenced by at least one triangle, mark the referenced vertices, and remove the unreferenced vertices;

[0059] Step 3.45: Update the vertex indices of the triangle.

[0060] Compared with the prior art, the beneficial effects of this application are:

[0061] 1. SLAM usually faces obstacles and interference in dynamic scenes. The technology of this application enables the system to cope with active factors such as pedestrians and vehicles by identifying and filtering dynamic objects, so as to maintain high-quality reconstruction in crowded or non-static environments. Especially in scenes with high noise, changing lighting or occlusion, the algorithm can significantly improve the effect of three-dimensional reconstruction, so that the SLAM system can maintain high accuracy and stability in complex environments;

[0062] 2. By optimizing the target detection model used for feature extraction, the model size and calculation delay are reduced, the real-time performance and calculation efficiency are improved, and the application requirements of robots with strong real-time performance are met;

[0063] 3. Through high-density point cloud processing, multi-scale modeling and texture fusion, the detail performance in complex geometric structures is enhanced, more details are obtained and the 3D reconstruction quality of complex scenes is improved;

[0064] 4. Improve the point cloud registration algorithm to obtain more accurate point cloud data. Based on the ICP (Iterative Closest Point) algorithm of point cloud registration, multi-scale ICP registration and weighted ICP based on multi-scale point cloud pyramid are introduced to improve the registration accuracy and reduce the number of iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 This is a demonstration diagram of the traditional SLAM operation in the background technology;

[0067] Figure 2 It is a schematic diagram of the SLAM workflow in the background technology;

[0068] Figure 3 This is a flowchart of the algorithm for 3D reconstruction of complex environments based on SLAM point cloud data for this application;

[0069] Figure 4 Flowchart for obtaining keyframe data for this application;

[0070] Figure 5 Flowchart for dynamic detection of the target detection model for this application;

[0071] Figure 6 Schematic diagram of removing dynamic features (taking people as an example) for this application;

[0072] Figure 7 Generate point cloud flow chart for key frame data of this application;

[0073] Figure 8 The initial point cloud infographic generated for this application;

[0074] Fig. 9 A schematic diagram of a three-dimensional point cloud map generated from an initial point cloud in this application;

[0075] Fig.10 Flow chart of 3D reconstruction for this application;

[0076] Fig.11 This is a schematic diagram of the three-dimensional model after three-dimensional reconstruction in this application. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0078] like Figure 3 As shown in the figure, the complex environment 3D reconstruction algorithm based on SLAM point cloud data includes:

[0079] Step 1: Obtain initial key frame data through the SLAM system;

[0080] Step 2: Process key frame data and generate 3D point cloud;

[0081] Step 3: Reconstruct the point cloud data into three dimensions to generate a three-dimensional model.

[0082] Further, such as Figure 4 As shown, the step 1 of obtaining the initial key frame data through the SLAM system specifically includes:

[0083] Step 1.1: Get the initial frame data of the sensor and initialize it;

[0084] Step 1.2: Use the target detection model YOLOv5 to detect dynamic objects in the scene, and mark the detected dynamic object area as a potential dynamic area. The target detection model YOLOv5 introduces the GhostNet module and the ECA attention mechanism;

[0085] The dynamic object may be a pedestrian or a vehicle, etc.

[0086] In order to improve the real-time performance, GhostNet is added on the basis of YOLOV5, such as Figure 5 As shown in the figure, GhostNet introduces a "ghost module" to generate "ghost features" and reduce redundant calculations, thereby significantly reducing the amount of calculations and parameters. In embedded devices and edge computing environments, deep learning models can be run with fewer hardware resources. However, GhostNet has caused the loss of image feature extraction to a certain extent. In order to improve the accuracy of target detection, the ECA attention mechanism (efficient channel attention mechanism) is introduced to improve the network representation ability by adaptively adjusting the channel weights without additional parameters and computational overhead, thereby improving the performance of the algorithm while maintaining computational efficiency, which can be better applied to complex environments.

[0087] Step 1.3: Use ORB feature points for inter-frame matching based on the ORB-SLAM3 system;

[0088] Extract ORB feature points from each frame, use LK optical flow method to track these feature points between frames, calculate their motion trajectory, check whether the feature points are located in the potential dynamic area marked by the target detection model, calculate the optical flow vector of the feature points, compare it with the background optical flow field, identify the feature points of dynamic objects, and remove the feature points marked as dynamic. Figure 6 In addition, the ORB-SLAM3 system uses ORB feature points for inter-frame matching and calculates the motion between point pairs to determine whether there is rotation and translation between adjacent frames.

[0089] Step 1.4: Perform initial pose estimation to determine the robot's initial position and orientation in the environment, providing a reference for subsequent map construction;

[0090] Step 1.5: Track the local map and determine whether a new keyframe needs to be created. When the number of feature point matches in the current frame is insufficient, keyframe insertion is triggered.

[0091] Tracking the local map, the system matches the current frame with the previous keyframe or local map, estimates the sensor camera's pose, and determines whether a new keyframe needs to be created. When the number of feature point matches in the current frame is insufficient, keyframe insertion is triggered.

[0092] Step 1.6: Local map optimization. After the keyframes are inserted, the map structure is optimized in the local area, including adding new 3D map points, removing map points with large errors, and performing local BA optimization to improve the accuracy of the local map.

[0093] The local BA optimization is a local Bundle Adjustment optimization.

[0094] Step 1.7: Perform closed loop detection and global optimization. When a closed loop is detected between the current frame and the historical frame, that is, the current frame and the historical frame correspond to the same position, the global BA optimization is triggered to correct the accumulated error and optimize the accuracy of the entire map.

[0095] Step 1.8: Determine whether the current frame needs to be generated and inserted as a new key frame, obtain the initial point cloud data, and pass the robot's posture information and the sensor's depth information to the point cloud construction thread.

[0096] The generated initial point cloud information is shown in the figure Figure 8 shown.

[0097] Further, such as Figure 7 As shown, the processing of key frame data and generating a three-dimensional point cloud in step 2 specifically includes:

[0098] Step 2.1: The constructor initializes the global point cloud map and voxel filter, and initializes the leaf parameters;

[0099] First, the constructor initializes the global point cloud map and voxel filter. In this implementation, in order to control the point cloud density, the leaf parameter size is initialized to 0.03 meters. The leaf is a basic unit in the voxel grid filter. In actual operation, the leaf size can be reduced to improve accuracy or increased to reduce the computational burden as required.

[0100] Step 2.2: Generate the corresponding 3D point cloud based on the RGB image and depth image of the key frame;

[0101] By traversing the pixels of the RGB image and the depth image, the robot's 3D coordinates in the camera coordinate system are calculated using the depth information. Then, combined with the camera parameters of the key frame, the robot's 3D coordinates in the camera coordinate system are converted to 3D coordinate positions in the real world and color information is assigned to each point. The camera parameters include internal and external parameters and depth information. When generating the point cloud, the code removes pixels in the dynamic area to ensure that the generated map only contains static scenes. The 3D point cloud map generated by the initial point cloud is shown in Figure 1. Fig. 9 shown.

[0102] Step 2.3: Construct a multi-scale point cloud pyramid, take the original point cloud as the high-resolution layer, then set the grid size to generate the medium-resolution layer and the low-resolution layer;

[0103] Specifically, a multi-scale point cloud pyramid is constructed, and the original point cloud is used as a high-resolution layer, and its grid parameter size remains unchanged. At the same time, a medium-resolution layer whose grid parameter size is twice the grid parameter of the high-resolution layer and a low-resolution layer whose grid parameter size is four times the grid parameter of the high-resolution layer are generated. In this embodiment, the grid size is set to 0.01m, all points are completely retained, and then the grid sizes are set to 0.02m and 0.04m respectively to generate a medium-resolution layer and a low-resolution layer. The grid is a parameter in point cloud processing. The smaller the grid, the higher the accuracy, and the larger the grid, the lower the accuracy.

[0104] Step 2.4: Point cloud registration, first start with the low-resolution layer to register, find the nearest neighbor point with the smallest Euclidean distance for each point in the source point cloud in the target point cloud, then adjust the weight value according to the size of the Euclidean distance, calculate the registration error between the source point cloud and the nearest neighbor point according to the adjusted weight value, use the least squares method to optimize the rigid body transformation, obtain the optimization result, and finally apply the optimization result to the source point cloud; the medium-resolution layer uses the optimization result of the low-resolution layer as the initial rigid body transformation result to perform the above-mentioned point cloud registration operation, and similarly, the high-resolution layer uses the optimization result of the medium-resolution layer as the initial rigid body transformation result to perform the point cloud registration operation;

[0105] The Euclidean distance is the straight-line distance between two points. The weight value is adjusted according to the size of the Euclidean distance: the closer the distance, the higher the weight, and the farther the distance, the lower the weight. The obtained weight is introduced into the registration error calculation formula to calculate the registration error between the source point cloud and the nearest neighbor point; the least squares method is used to optimize the rigid body transformation to obtain the optimization result, that is, the rotation matrix and translation vector are calculated by matching point pairs to minimize the registration error; the rigid body transformation includes rotation operations and translation operations.

[0106] Step 2.5: Update the global point cloud map and perform real-time visualization through the point cloud visualization module of PCL to display the point cloud map and show the three-dimensional structure of the environment.

[0107] Further, such as Fig.10 As shown, in step 3, the point cloud data is 3D reconstructed to generate a 3D model, which specifically includes:

[0108] Step 3.1: Import point cloud data and ensure that the data contains the three-dimensional coordinates (x, y, z) of the points. When loading the point cloud, check whether the point cloud data type is legal so as to carry out subsequent processing steps;

[0109] Step 3.2: After loading, preprocess the point cloud to improve data quality and computational efficiency. The preprocessing includes downsampling, denoising, and normal vector estimation.

[0110] Downsampling downsamples voxels to reduce the number of points and reduce the computational burden; denoising uses statistical filtering to remove isolated or deviated noise points and enhance data consistency; normal vector estimation calculates the direction of the surface normal vector for each point, which is important for Poisson reconstruction because it affects the orientation and smoothness of the surface.

[0111] Step 3.3: Reconstruct the mesh;

[0112] Mesh is a simplified form of expression of point cloud, which represents the collection of vertices and polygons of polyhedral shapes in 3D reconstruction. These meshes are usually composed of triangles, quadrilaterals or other simple convex polygons. Mesh reconstruction is to connect the "points" in the point cloud data to form "small patches (meshes)". These patches (meshes) are combined like the shell of a room or object, making the room or object look more three-dimensional.

[0113] Furthermore, the step 3.3 of performing mesh reconstruction includes:

[0114] Step 3.31: Initialize the octree structure, input the point cloud data and set the maximum depth parameter to control the resolution; recursively divide the space, map the point cloud data to each node of the octree, and adjust the data density of the node to strike a balance between resolution and computational complexity;

[0115] Step 3.32: The normal information of the point cloud is converted into a divergence field by using the Poisson reconstruction algorithm, and the divergence field is solved by using the Poisson reconstruction algorithm to generate an implicit function;

[0116] Step 3.33: Setting reconstruction parameters, including solver type and precision, performing solving to generate voxel data of implicit function for subsequent surface extraction;

[0117] Step 3.34: Use the Marching Cubes algorithm to extract the isosurface from the implicit function and convert it into a triangular mesh, making it suitable for a variety of 3D model formats.

[0118] Step 3.4: After the mesh reconstruction is completed, post-processing is performed to improve the quality of the mesh model, and the post-processing includes removing duplicate triangles and removing unreferenced vertices;

[0119] Furthermore, the post-processing process in step 3.4 includes:

[0120] Step 3.41: Initialize the hash table;

[0121] Step 3.42: sort the vertices of each triangle by index and store them in a hash table, wherein the triangle is a basic unit constituting a triangular mesh;

[0122] Step 3.43: Determine whether there are duplicate triangles. If so, remove the duplicates. Otherwise, execute step 3.43.

[0123] Step 3.44: Check whether each vertex of the triangle is referenced by at least one triangle, mark the referenced vertices, and remove the unreferenced vertices;

[0124] Step 3.45: Update the vertex indices of the triangle.

[0125] Step 3.5: Finally, the processed mesh model is saved as a 3D file format PLY file.

[0126] Post-processing operations can reduce redundant data and reduce file size.

[0127] The schematic diagram of the 3D model after 3D reconstruction is as follows Fig.11 shown.

[0128] This application can improve the accuracy and robustness of three-dimensional reconstruction. The algorithm can maintain high precision and stability of the SLAM system in complex environments by improving data fusion and feature extraction. Especially in scenes with high noise, changing illumination or occlusion, these algorithms can significantly improve the effect of three-dimensional reconstruction. SLAM usually faces obstacles and interference in dynamic scenes. The patented technology enables the system to cope with active factors such as pedestrians and vehicles through dynamic object recognition and filtering, thereby maintaining high-quality reconstruction in crowded or non-static environments. In addition, more accurate point cloud data can be obtained by improving the point cloud registration algorithm. On the basis of the ICP (Iterative Closest Point) algorithm of point cloud registration, multi-scale ICP registration and weighted ICP based on multi-scale point cloud pyramid are introduced to improve the registration accuracy and reduce the number of iterations.

[0129] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A complex environment 3D reconstruction algorithm based on SLAM point cloud data, characterized by: include: Step 1: Obtain initial key frame data through the SLAM system; Step 2: Process key frame data and generate 3D point cloud; Step 3: Reconstruct the point cloud data into three dimensions to generate a three-dimensional model.

2. The complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data as claimed in claim 1, characterized in that: The step 1 of obtaining the initial key frame data through the SLAM system specifically includes: Step 1.1: Get the initial frame data of the sensor and initialize it; Step 1.2: Use the target detection model YOLOv5 to detect dynamic objects in the scene, and mark the detected dynamic object area as a potential dynamic area. The target detection model YOLOv5 introduces the GhostNet module and the ECA attention mechanism; Step 1.3: Use ORB feature points for inter-frame matching based on the ORB-SLAM3 system; Step 1.4: Perform initial pose estimation to determine the initial position and orientation of the robot in the environment, providing a reference for subsequent map construction; Step 1.5: Track the local map and determine whether a new keyframe needs to be created. When the number of feature point matches in the current frame is insufficient, keyframe insertion is triggered. Step 1.6: Local map optimization. After the keyframes are inserted, the map structure is optimized in the local area, including adding new 3D map points, removing map points with large errors, and performing local BA optimization to improve the accuracy of the local map. Step 1.7: Perform closed loop detection and global optimization. When a closed loop is detected between the current frame and the historical frame, that is, the current frame and the historical frame correspond to the same position, trigger global BA optimization to correct the accumulated error and optimize the accuracy of the entire map. Step 1.8: Determine whether the current frame needs to be generated and inserted as a new key frame, obtain the initial point cloud data, and pass the robot's posture information and the sensor's depth information to the point cloud construction thread.

3. The complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data as claimed in claim 1 or 2, characterized in that: Processing the key frame data and generating a three-dimensional point cloud in step 2 specifically includes: Step 2.1: The constructor initializes the global point cloud map and voxel filter, and initializes the leaf parameters; Step 2.2: Generate the corresponding 3D point cloud based on the RGB image and depth image of the key frame; Step 2.3: Construct a multi-scale point cloud pyramid, take the original point cloud as the high-resolution layer, then set the grid size to generate the medium-resolution layer and the low-resolution layer; Step 2.4: Point cloud registration, first start with the low-resolution layer to register, find the nearest neighbor point with the smallest Euclidean distance for each point in the source point cloud in the target point cloud, then adjust the weight value according to the size of the Euclidean distance, calculate the registration error between the source point cloud and the nearest neighbor point according to the adjusted weight value, use the least squares method to optimize the rigid body transformation, obtain the optimization result, and finally apply the optimization result to the source point cloud; the medium-resolution layer uses the optimization result of the low-resolution layer as the initial rigid body transformation result to perform the above-mentioned point cloud registration operation, and similarly, the high-resolution layer uses the optimization result of the medium-resolution layer as the initial rigid body transformation result to perform the point cloud registration operation; Step 2.5: Update the global point cloud map and perform real-time visualization through the point cloud visualization module of PCL to display the point cloud map and show the three-dimensional structure of the environment.

4. The complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data as claimed in claim 3, characterized in that: In step 3, the point cloud data is 3D reconstructed to generate a 3D model, which specifically includes: Step 3.1: Import point cloud data and ensure that the data contains the three-dimensional coordinates (x, y, z) of the points. When loading the point cloud, check whether the point cloud data type is legal so as to carry out subsequent processing steps; Step 3.2: After loading, preprocess the point cloud to improve data quality and computational efficiency. The preprocessing includes downsampling, denoising, and normal vector estimation. Step 3.3: Reconstruct the mesh; Step 3.4: After the mesh reconstruction is completed, post-processing is performed to improve the quality of the mesh model, and the post-processing includes removing duplicate triangles and removing unreferenced vertices; Step 3.5: Finally, the processed mesh model is saved as a 3D file format PLY file.

5. The complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data as claimed in claim 4, characterized in that: The step 3.3 of mesh reconstruction includes: Step 3.31: Initialize the octree structure, input the point cloud data and set the maximum depth parameter to control the resolution; recursively divide the space, map the point cloud data to each node of the octree, and adjust the data density of the node to strike a balance between resolution and computational complexity; Step 3.32: The normal information of the point cloud is converted into a divergence field by using the Poisson reconstruction algorithm, and the divergence field is solved by using the Poisson reconstruction algorithm to generate an implicit function; Step 3.33: Setting reconstruction parameters, including solver type and precision, performing solving to generate voxel data of implicit function for subsequent surface extraction; Step 3.34: Use the Marching Cubes algorithm to extract the isosurface from the implicit function and convert it into a triangular mesh, making it suitable for a variety of 3D model formats.

6. The complex environment three-dimensional reconstruction algorithm based on SLAM point cloud data as claimed in claim 5, characterized in that: The post-processing process in step 3.4 includes: Step 3.41: Initialize the hash table; Step 3.42: sort the vertices of each triangle by index and store them in a hash table, wherein the triangle is a basic unit constituting a triangular mesh; Step 3.43: Determine whether there are duplicate triangles. If so, remove the duplicates. Otherwise, execute step 3.

43. Step 3.44: Check whether each vertex of the triangle is referenced by at least one triangle, mark the referenced vertices, and remove the unreferenced vertices; Step 3.45: Update the vertex indices of the triangle.

Images