Indoor three-dimensional reconstruction method based on SLAM and 3DGS technology

Abstract

The invention relates to an indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies, and the method comprises the following steps: S1, obtaining indoor high-definition image information through a binocular camera, and transmitting the indoor high-definition image information to an ORB-SLAM3 module in real time; s2, performing indoor image downsampling, extracting and matching ORB feature points, performing camera pose estimation and other processing on the ORB-SLAM3; s3, seamlessly transmitting three-dimensional sparse data generated by the ORB-SLAN3 module into a 3DGS module, initializing SFM point cloud into a three-dimensional Gaussian ellipsoid set, visually projecting the three-dimensional Gaussian ellipsoid set to a rasterization plane under a world coordinate system, and performing rapid micro-rasterization rendering and the like; and S4, displaying a reconstruction result of the rendered three-dimensional model, and optimizing the model. According to the invention, portable room structure information, accurate space dimension measurement and flexible furniture layout adjustment can be provided for a user, and the convenience and practicability of home design are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of indoor three-dimensional reconstruction, and specifically to an indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies. Background Art

[0002] Indoor scene reconstruction plays an important role in the fields of computer graphics and vision, and has had a significant impact on fields such as robotics and augmented reality. In recent years, the progress of neural radiance fields (NeRF) and 3D Gaussian splatting (3DGS) technologies has significantly improved the scene rendering efficiency, accelerated the generation of new perspective images, and simplified tasks such as dynamic reconstruction, geometric editing, and physical simulation. In addition, the widespread application of consumer-grade RGB-D cameras has promoted the progress of indoor scene reconstruction technology. The rapid development of intelligent indoor design has raised higher requirements for people's understanding and utilization of indoor spaces. In traditional indoor design, conventional three-dimensional reconstruction combines colmap and 3DGS (3D Gaussian Splatting) to achieve sparse reconstruction and dense reconstruction, but there are deficiencies in modeling speed and accuracy, and the reconstruction effect is average. Summary of the Invention

[0003] In view of the above problems, the present invention provides an indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies.

[0004] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0005] An indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies, comprising the following steps:

[0006] S1. Use a binocular camera to obtain indoor high-definition image information and transmit it into the ORB-SLAM3 module in real time;

[0007] S2. ORB-SLAM3 performs downsampling of indoor images, extracts and matches ORB feature points, and performs camera pose estimation and other processes;

[0008] S3. The three-dimensional sparse data generated by the ORB-SLAM3 module is seamlessly transmitted into the 3DGS module for operations such as initializing SFM point clouds into a three-dimensional Gaussian ellipsoid set, visually projecting onto a rasterized plane in the world coordinate system, and fast differentiable rasterization rendering;

[0009] S4. Display the reconstruction results of the rendered three-dimensional model and optimize the model.

[0010] Further, in step S1, the binocular camera calibrates the internal parameters and distortion parameters of the left and right cameras respectively, and at the same time calibrates the external parameter matrix of the right camera in the coordinate system of the left camera; by capturing the calibration board image and using the calibration algorithm, the camera internal parameter matrix is calculated to ensure that the image can be mapped to the three-dimensional space without distortion.

[0011] Further, in step S2, when collecting the left and right high-definition images, the left high-definition images are saved in the way of a buffer sequence. When orb-slam3 determines that the left downsampled image before a certain moment is a key frame, the system will save the left high-definition image associated with the left downsampled image, and the associated left high-definition images of non-key frames before the current moment will be deleted.

[0012] Further, in step S3, in the 3DGS module, a standard pinhole camera model is configured.

[0013] Further, in step S4, the generated 3DGS model is regionally cropped to retain the region of interest to exclude irrelevant data.

[0014] The beneficial effects of the present invention are as follows:

[0015] The present invention can provide users with portable room structure information, accurate space size measurement and flexible furniture layout adjustment, significantly improving the convenience and practicality of home design. Compared with the traditional panoramic camera technology, the 3D reconstruction solution of the present invention not only provides a more three-dimensional spatial perception experience, but also has the ability of distance calculation and environmental layout modification, greatly enhancing the user's interactive experience and satisfaction. In addition, large furniture companies have models of many furniture under their own brands in order to provide better service experiences. Users can upload their own indoor models and then use the furniture model library provided by large furniture companies for subsequent work. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the main flow chart of the present invention;

[0017] Figure 2 is the overall flow chart of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0018] To better understand the present invention, the following combines Figure 1 and Figure 2 to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] The overall method of the present invention is asFigure 1 As shown in the figure: First, use a binocular camera to obtain high-definition indoor image information, and transmit it to the ORB-SLAM3 module in real time. Perform downsampling on the indoor image, extract and match ORB feature points, and perform camera pose estimation and other processing. The three-dimensional sparse data generated by the ORB-SLAM3 module can be seamlessly transmitted into the 3DGS module, and operations such as initializing SFM point clouds into a three-dimensional Gaussian ellipsoid set, visually projecting to a rasterized plane in the world coordinate system, and fast differentiable rasterization rendering are performed. Finally, the rendered three-dimensional model is displayed for the reconstruction result and the model is optimized. At the same time, the distance between models of the indoor three-dimensional model can be measured, and a home furnishing model can be imported for interior design.

[0020] The specific steps are as follows:

[0021] S1. Use a binocular camera to obtain high-definition indoor image information and transmit it to the ORB-SLAM3 module in real time. Calibrate the internal parameters and distortion parameters of the left and right cameras respectively, and at the same time calibrate the external parameter matrix of the right camera in the coordinate system of the left camera. By taking calibration board images and using a calibration algorithm to calculate the camera internal parameter matrix, ensure that the image can be mapped to the three-dimensional space without distortion. This step is crucial, which can reduce the error caused by camera distortion and provide guarantee for the accuracy of SLAM. The matrix generated by the external parameter calibration provides stereo parallax information for the binocular vision system and can provide an accurate relative positioning basis for SLAM depth estimation. Obtain the image information of the room through the binocular camera and transmit it to the visual SLAM system in real time to generate continuous image frames as the input data source of the SLAM system. The advantage of the binocular camera is that it can capture a stereo view, enabling the direct use of the depth information of the image in subsequent processing steps, which helps to accurately reconstruct the three-dimensional model. In order to improve the real-time processing ability of the SLAM system, perform downsampling processing on the original high-definition image. The downsampled image is used as the input for SLAM calculation, reducing the processing burden and improving the real-time performance of the SLAM algorithm. However, for subsequent high-quality 3DGS rendering, cache the high-definition left-eye image sequence in the SLAM system to ensure the best effect for subsequent key-frame reconstruction and rendering.

[0022] S2. ORB-SLAM3 performs downsampling on the indoor image, extracts and matches ORB feature points, and performs camera pose estimation and other processing; during the operation of SLAM, retain the images with the original resolution of the left-eye key frames, the key frame poses, and the 3D coordinates of the feature points and directly convert them into the.bin file format accepted by Colmap, which is the input standard accepted by 3DGS to ensure data compatibility. Through this format conversion, the sparse three-dimensional data generated by SLAM can be seamlessly input into the 3DGS system, preparing for high-quality dense three-dimensional reconstruction.

[0023] S3. Seamlessly transfer the three-dimensional sparse data generated by the ORB-SLAM3 module into the 3DGS module for operations such as initializing SFM point clouds into a three-dimensional Gaussian ellipsoid set, visually projecting them onto a rasterized plane in the world coordinate system, and performing fast differentiable rasterization rendering. In the 3DGS rendering system, configure a standard pinhole camera model. This model conforms to the optical characteristics of an actual camera, can effectively reduce light projection errors, and improve the accuracy of reconstruction and rendering effects. The addition of the pinhole model precisely maps parameters such as the camera field of view and focal length, further ensuring the consistency of the 3DGS model with the actual scene. Use the.bin file exported by SLAM as input for 3DGS training. Through the differentiable rasterization and adaptive density control technologies in the 3DGS system, the system can reduce unnecessary computational workload while maintaining high precision, making the reconstruction process efficient and delicate. 3DGS learns the geometric and texture features of the scene from key frames through multi-view geometry and deep learning techniques to generate a realistic three-dimensional model. Use the.bin file as input for training and save the rendered.ply file of 3dgs. This format supports subsequent visualization and measurement analysis. The rendered model has high resolution and high fidelity, facilitating further three-dimensional space perception analysis.

[0024] S4. Display the reconstruction results and optimize the rendered three-dimensional model. Perform regional cropping on the generated 3DGS model to retain the region of interest (such as a specific area of a room) to exclude irrelevant data, improving the clarity and storage efficiency of the model. Regional cropping can also further optimize the model details according to user needs, making the visualization experience more focused and accurate. Rotate and adjust the floor of the room in the model to align the floor with the horizontal plane. This step is to provide a stable benchmark during measurement and analysis, ensuring the positioning of elements in space is consistent with the actual room structure, facilitating subsequent space analysis and layout optimization. On the optimized 3DGS model, spatial measurements can be performed at any point to achieve precise analysis of the indoor space. This is crucial for applications such as space design and furniture layout. At the same time, the model also supports adding other 3DGS models, such as virtual furniture models, to help users explore different layout schemes in three-dimensional space and support diverse needs in interior design.

[0025] such as Figure 2As shown, first, the system receives the left-eye image and the right-eye image as inputs. For the left-eye image, the system first performs image downsampling to reduce the image resolution and the computational load of subsequent processing. Then, the system extracts ORB (Oriented FAST and Rotated BRIEF) features, which are points with obvious contrast and good distribution found in the image and can be used for subsequent matching and tracking. After extracting the ORB features, the system finds the corresponding feature points in the left-eye and right-eye images through a feature matching algorithm, thereby calculating the preliminary pose information. Then, the system tracks the local map, using the known map information and new image data to update and optimize the pose. At the same time, the system also performs map creation work in the background. This includes generating key frames, creating map points, and continuously updating and improving the map through the local mapping thread. During this process, the system also performs loop detection and correction to eliminate the accumulated errors and maintain the consistency and accuracy of the map.

[0026] In addition to the binocular ORB-SLAM3 module, there is also a "3DGS module". This module mainly processes tasks related to 3D geometric structures, such as the processing of the original score of the left-eye key frame, the extraction of map points, the calculation of the pose of the left-eye key frame, etc. In addition, the 3DGS module can also extract regions of interest and perform more detailed analysis and processing on these regions.

[0027] Finally, there is an "editing module" which may be used to modify and optimize the finally generated 3D model or map. In this module, the user can perform operations such as individual damage adjustment, densification, or pruning to improve the quality and appearance of the model. In addition, these techniques may be used for further optimization and visualization of the generated 3D model.

[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies, characterized in that: It includes the following steps: S1. Use a binocular camera to obtain indoor high-definition image information and transmit it to the ORB-SLAM3 module in real time; S2. ORB-SLAM3 performs indoor image downsampling, extracts and matches ORB feature points, and performs camera pose estimation and other processes; S3. The three-dimensional sparse data generated by the ORB-SLAM3 module is seamlessly transmitted to the 3DGS module for operations such as initializing SFM point clouds into a three-dimensional Gaussian ellipsoid set, visually projecting to a rasterized plane in the world coordinate system, and fast differentiable rasterization rendering; S4. Display the reconstruction results and optimize the model for the rendered three-dimensional model.

2. The indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies according to claim 1, wherein: In step S1, the binocular camera calibrates the internal parameters and distortion parameters of the left and right cameras respectively, and at the same time calibrates the external parameter matrix of the right camera in the coordinate system of the left camera; by taking calibration board images, the camera internal parameter matrix is calculated using a calibration algorithm to ensure that the image can be mapped to three-dimensional space without distortion.

3. The indoor three-dimensional reconstruction method based on SLAM and 3DGS technology according to claim 1, wherein: In step S2, when collecting left and right high-definition images, the left high-definition images are saved in the way of a buffer sequence. When the orb-slam3 determines that the left downsampled image before a certain moment is a key frame, the system will save the left high-definition image associated with the left downsampled image, and the associated left high-definition images of non-key frames before the current moment will be deleted.

4. The indoor three-dimensional reconstruction method based on SLAM and 3DGS technologies according to claim 1, characterized in that: In step S3, in the 3DGS module, a standard pinhole camera model is configured.

5. The indoor three-dimensional reconstruction method based on SLAM and 3DGS technology according to claim 1, wherein: In step S4, the generated 3DGS model is regionally cropped to retain the region of interest to exclude irrelevant data.

Images