An underwater three-dimensional reconstruction method based on Gaussian topological disassimilation and cross-space dimension reduction mapping

Abstract

This invention discloses an underwater 3D reconstruction method based on Gaussian topological alienation and cross-spatial dimensionality reduction mapping. The method initializes point cloud generation and defines 3D Gaussian primitives based on RGB image data acquired by an underwater camera. It then performs center alienation processing on the 3D Gaussian primitives to divide all 3D Gaussian primitives in space into two sub-regions with different opacities from the central plane. A Gaussian cross-spatial dimensionality reduction model is established, transforming the topologically alienated 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system. The 3D Gaussian primitives in the camera coordinate system are then mapped to 2D Gaussian primitives in the imaging space using the Jacobian matrix. Color parameters from the RGB image data are mapped to a spherical harmonic function model. Finally, rendering based on 2D Gaussian primitives is performed, combined with the opacity of the 3D Gaussian primitives, thereby completing the 3D reconstruction of the underwater environment.

Description

Technical Field

[0001] This invention relates to the field of underwater 3D reconstruction and rendering technology, and more particularly to an underwater 3D reconstruction method based on Gaussian topological alienation and cross-space dimensionality reduction mapping. Background Technology

[0002] With the widespread application of underwater optical detection and imaging technologies in marine resource exploration and environmental monitoring, the demand for high-precision 3D reconstruction and rendering of underwater environments is becoming increasingly urgent. Existing underwater 3D reconstruction and rendering technologies mostly utilize single point clouds or images as input data, resulting in low data utilization, poor reconstruction detail, and high rendering distortion. Although subsequent studies have used mapping primitives to fuse point clouds and image data, when faced with complex geometric reconstruction areas, the scale control of these mapping primitives is weak, making it difficult to adapt to the detailed features of different structures, leading to significant deficiencies in the detailed representation of the reconstruction results. Summary of the Invention

[0003] To address the technical problems existing in the prior art, this invention employs an underwater 3D reconstruction technique based on Gaussian topological alienation and cross-spatial dimensionality reduction mapping. By performing topological alienation processing on 3D Gaussian primitives and reducing each 3D Gaussian primitive to a 2D Gaussian primitive on the imaging plane across space, adaptive control and optimization of the Gaussian topological morphology are achieved during the modeling process, thereby realizing high-quality underwater 3D reconstruction and efficient rendering. Therefore, the disclosed underwater 3D reconstruction method based on Gaussian topological alienation and cross-spatial dimensionality reduction mapping specifically includes the following steps:

[0004] S1. Input the RGB image acquired by the underwater camera into the Structure from Motion (SfM) algorithm to obtain a set of initial sparse point clouds. Define three-dimensional Gaussian elements and their geometric centers, three-dimensional covariance matrix, opacity and color parameters based on the points on the point cloud. S2. Establish a Gaussian topological alienation model and perform central alienation processing on the three-dimensional Gaussian primitives to divide all three-dimensional Gaussian primitives in space into two sub-regions with different opacities by the central plane. S3. Establish a Gaussian cross-space dimensionality reduction model, transform the topologically alienated 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system, and map the 3D Gaussian primitives in the camera coordinate system to the 2D Gaussian primitives in the imaging space through the Jacobian matrix. S4 maps the color parameters in the RGB image data to the spherical harmonic function model, and performs rendering based on the two-dimensional Gaussian primitive by combining the opacity of the three-dimensional Gaussian primitive.

[0005] Furthermore, the Structure from Motion (SfM) algorithm extracts and matches feature points from multi-view sequential image data of the seabed, generates projections of points in the underwater 3D scene from different viewpoints, and then performs pose calculation and initializes point cloud generation.

[0006] Furthermore, the three-dimensional covariance matrix represents the pose and scale information of the three-dimensional Gaussian elements and is composed of a rotation matrix and a scaling matrix.

[0007] Furthermore, the Gaussian topological alienation model performs central alienation processing on the three-dimensional Gaussian primitives. Based on the geometric center of the three-dimensional Gaussian primitives, a central plane is defined, dividing all three-dimensional Gaussian primitives in space into two sub-regions with different opacities by their central plane.

[0008] Furthermore, the Gaussian cross-space dimensionality reduction model transforms the topologically alienated 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system, and maps the 3D Gaussian primitives in the camera coordinate system to the 2D Gaussian primitives in the imaging space through the Jacobian matrix.

[0009] Furthermore, the Jacobian matrix maps the three-dimensional Gaussian elements in the camera coordinate system to the imaging space. The Jacobian matrix in the imaging space is then approximated in two dimensions to form a two-dimensional covariance matrix. This reduces the dimensionality of the original three-dimensional Gaussian element's three-dimensional covariance matrix to the two-dimensional covariance matrix of the imaging plane. Based on parameters such as the two-dimensional covariance matrix, the two-dimensional Gaussian element is defined, thereby achieving dimensionality reduction from three-dimensional Gaussian elements to two-dimensional Gaussian elements.

[0010] Furthermore, the spherical harmonic function model projects the color information in the RGB image data acquired by the underwater camera onto the spherical harmonic function, which is used to model the color changes of Gaussian elements under different imaging planes.

[0011] Furthermore, the rendering based on two-dimensional Gaussian elements incorporates the opacity of three-dimensional Gaussian elements and a spherical harmonic function model with color information as additional parameters, and combines them with the distribution of two-dimensional Gaussian elements on the imaging plane for fusion rendering.

[0012] Compared with the prior art, the present invention has the following advantages: 1. The underwater 3D reconstruction technology based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping provided by this invention initializes point cloud generation using RGB image data collected by an underwater camera, and performs parameterized definitions on the points in the point cloud regarding 3D Gaussian primitives and their geometric centers, 3D covariance matrix and opacity, etc., to achieve deep fusion of point cloud and image data. While using point cloud data to characterize the geometric structure of the underwater environment, it also reproduces the real color and texture of the underwater reconstruction target based on the image data, comprehensively and accurately reproducing the underwater scene. Compared with traditional single data source methods, it greatly improves the accuracy of 3D reconstruction of complex underwater geometric areas.

[0013] 2. The underwater 3D reconstruction technology based on Gaussian topological alienation and cross-space dimensionality reduction mapping provided by this invention establishes a Gaussian topological alienation model and performs central alienation processing on 3D Gaussian primitives to divide all 3D Gaussian primitives in space into two sub-regions with different opacities from the central plane, thereby achieving targeted enhanced characterization of the boundaries of complex underwater targets.

[0014] 3. The underwater 3D reconstruction technology based on Gaussian topological alienation and cross-space dimensionality reduction mapping provided by this invention transforms the 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system by establishing a Gaussian cross-space dimensionality reduction model. Then, it maps them to the imaging space through the Jacobian matrix and performs a two-dimensional approximation fitting on the Jacobian matrix to approximate it as a two-dimensional covariance matrix, thereby achieving dimensionality reduction from the 3D covariance matrix to the 2D imaging plane. While maintaining the key geometric features of the Gaussian primitives, it significantly reduces the amount of computation and avoids the inefficiency caused by complex calculations in traditional methods. It ensures real-time performance while taking into account reconstruction and rendering accuracy.

[0015] In summary, the technical solution of this invention improves upon existing underwater 3D reconstruction technologies in practical applications, addressing issues such as poor reconstruction quality of complex target areas and weak geometric adaptability to detailed features of different structures. It achieves both high efficiency and high quality in 3D reconstruction and rendering in complex underwater environments. Therefore, the technical solution of this invention solves the problem of poor 3D reconstruction and rendering effects in complex underwater environments in existing technologies.

[0016] Based on the above reasons, this invention can be widely promoted in the field of underwater three-dimensional reconstruction technology. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping of the present invention.

[0019] Figure 2 This is a flowchart illustrating the underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping of the present invention.

[0020] Figure 3 This is a comparison of rendered images of underwater environments based on the underwater 3D reconstruction technology of this invention, which is based on Gaussian topological heterogeneity and cross-spatial dimensionality reduction mapping. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] like Figure 1 As shown, this invention provides an underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-spatial dimensionality reduction mapping, specifically including the following steps: S1. Input the RGB image acquired by the underwater camera into the Structure from Motion (SfM) algorithm to obtain a set of initial sparse point clouds. Define three-dimensional Gaussian elements and their geometric centers, three-dimensional covariance matrix, opacity and color parameters based on the points on the point cloud. In a specific implementation, as a preferred embodiment of the present invention, the Structure from Motion (SfM) algorithm extracts and matches feature points from multi-view sequential image data of the seabed, and generates projections of points in the underwater three-dimensional scene under different viewpoints, thereby performing pose calculation and initializing point cloud generation. The three-dimensional covariance matrix represents the pose and scale information of the three-dimensional Gaussian elements and is composed of a rotation matrix and a scaling matrix.

[0024] S2. Establish a Gaussian topological alienation model and perform central alienation processing on the three-dimensional Gaussian primitives to divide all three-dimensional Gaussian primitives in space into two sub-regions with different opacities from the central plane. In a specific implementation, as a preferred embodiment of the present invention, the Gaussian topological alienation model performs central alienation processing on the three-dimensional Gaussian primitives. Based on the geometric center of the three-dimensional Gaussian primitives, a central plane is defined, and all three-dimensional Gaussian primitives in space are divided into two sub-regions with different opacities by their central plane.

[0025] S3. Establish a Gaussian cross-space dimensionality reduction model, transform the topologically alienated 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system, and map the 3D Gaussian primitives in the camera coordinate system to the 2D Gaussian primitives in the imaging space through the Jacobian matrix. In a specific implementation, as a preferred embodiment of the present invention, the Gaussian cross-space dimensionality reduction model transforms the topologically alienated three-dimensional Gaussian primitives from the Gaussian coordinate system to the camera coordinate system, and maps the three-dimensional Gaussian primitives in the camera coordinate system to the two-dimensional Gaussian primitives in the imaging space through the Jacobian matrix. The Jacobian matrix maps the three-dimensional Gaussian elements in the camera coordinate system to the imaging space. The Jacobian matrix in the imaging space is approximated in two dimensions to approximate it as a two-dimensional covariance matrix. This reduces the three-dimensional covariance matrix of the original three-dimensional Gaussian elements to a two-dimensional covariance matrix of the imaging plane. Based on parameters such as the two-dimensional covariance matrix, the two-dimensional Gaussian elements are defined, thereby achieving dimensionality reduction from three-dimensional Gaussian elements to two-dimensional Gaussian elements.

[0026] S4. Map the color parameters in the RGB image data to the spherical harmonic function model, and combine the opacity of the three-dimensional Gaussian primitive to perform rendering based on the two-dimensional Gaussian primitive. In a specific implementation, as a preferred embodiment of the present invention, the spherical harmonic function model projects the color information in the RGB image data acquired by the underwater camera onto the spherical harmonic function, which is used to model the color changes of Gaussian elements under different imaging planes; The rendering based on two-dimensional Gaussian elements uses the opacity of three-dimensional Gaussian elements and a spherical harmonic function model with color information as additional parameters, and combines them with the distribution of two-dimensional Gaussian elements on the imaging plane for fusion rendering.

[0027] Example First, a three-dimensional Gaussian primitive is used as the reconstruction primitive for the seabed scene. A set of three-dimensional Gaussian primitives is defined. Each three-dimensional Gaussian element Opacity Geometric parameterization of the 3D Gaussian is performed, and the geometric center of the 3D Gaussian is defined. Scaling matrix Rotation matrix 3D covariance matrix Parameterization The three-dimensional Gaussian elements follow a distribution in space:

[0028] For three-dimensional Gaussian elements Central alienation is performed by defining a plane passing through the geometric center of the three-dimensional Gaussian meta-geometry. This allows any three-dimensional Gaussian element in space to be... They are all divided into two opacity values. and 3D Gaussian elements .when "Time" represents a special case of a three-dimensional Gaussian primitive. A three-dimensional Gaussian primitive after central alienation. Described in the following manner:

[0029] in For the segmentation plane passing through the center of the three-dimensional Gaussian unit The normal vector, And plane The sub-Gaussians on both sides are obtained by simply changing the sign of the normal vector. For plane The sub-Gaussian space portion on one side, that is, the effective region on the side of the centrally alienated Gaussian.

[0030] Perform coordinate system transformation on the 3D Gaussian element to transform the 3D Gaussian element. Transformation from Gaussian coordinate system to camera coordinate system: ,in For the camera rotation matrix, Let be the camera translation vector. According to the linear transformation rule: That is, the covariance matrix can be transformed into .

[0031] Through the Jacobian matrix The three-dimensional Gaussian elements in the camera coordinate system Mapped to ray space: The affine approximation Jacobian matrix after projection transformation can be approximated as a two-dimensional covariance matrix. This enables the dimensionality reduction of three-dimensional covariance to a two-dimensional imaging plane, realizing the distribution from Gaussian elements to the corresponding imaging space. .

[0032] Use spherical harmonic functions to determine the color of the view. Perform mapping and modeling, and then render based on two-dimensional Gaussian primitives: ,in As the first The color of a Gaussian element is defined by the view color in conjunction with a spherical harmonic function. For the first An opacity of one Gaussian unit. An additional parameter defined as the opacity of the Gaussian element.

[0033] This invention provides an underwater 3D reconstruction method based on Gaussian topological alienation and cross-spatial dimensionality reduction mapping. It initializes point cloud generation and defines 3D Gaussian primitives based on RGB image data acquired by an underwater camera. The 3D Gaussian primitives undergo center alienation to divide all 3D Gaussian primitives in space into two sub-regions with different opacities from the central plane. A Gaussian cross-spatial dimensionality reduction model is established, transforming the topologically alienated 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system. The 3D Gaussian primitives in the camera coordinate system are then mapped to 2D Gaussian primitives in the imaging space using a Jacobian matrix. Color parameters from the RGB image data are mapped to a spherical harmonic function model. Finally, rendering based on 2D Gaussian primitives is performed, combined with the opacity of the 3D Gaussian primitives, thereby completing the 3D reconstruction of the underwater environment.

[0034] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-spatial dimensionality reduction mapping, characterized in that, include: The RGB images acquired by the underwater camera are input into the structure-of-motion algorithm to obtain a set of initial sparse point clouds. Based on the points on the sparse point cloud, the three-dimensional Gaussian elements and geometric centers, the three-dimensional covariance matrix, opacity and color parameters are defined. A Gaussian topological alienation model is established, and the three-dimensional Gaussian primitives are subjected to central alienation processing, thereby dividing all three-dimensional Gaussian primitives in space into two sub-regions with different opacities by the central plane. A Gaussian cross-space dimensionality reduction model is established to transform the topologically alienated 3D Gaussian primitives from the Gaussian coordinate system to the camera coordinate system, and the 3D Gaussian primitives in the camera coordinate system are mapped to the 2D Gaussian primitives in the imaging space through the Jacobian matrix. The color parameters in the RGB image data are mapped to the spherical harmonic function model, and rendering based on the two-dimensional Gaussian primitive is performed by combining the opacity of the three-dimensional Gaussian primitive.

2. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 1, characterized in that: The motion reconstruction structure algorithm is used to read multi-view serialized image data of the seabed, extract and match feature points, generate projections of points in the underwater 3D scene from different viewpoints, and perform pose calculation and initial point cloud generation.

3. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 1, characterized in that: The Gaussian topological alienation model performs central alienation processing on three-dimensional Gaussian primitives. Based on the geometric center of the three-dimensional Gaussian primitive, a central plane is defined, dividing all three-dimensional Gaussian primitives in space into two sub-regions with different opacities by their central plane.

4. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 1, characterized in that: Based on the Jacobian matrix, the three-dimensional Gaussian elements in the camera coordinate system are mapped to the imaging space. The Jacobian matrix in the imaging space is approximated in two dimensions to a two-dimensional covariance matrix. The three-dimensional covariance matrix of the original three-dimensional Gaussian elements is reduced to the two-dimensional covariance matrix of the imaging plane. The two-dimensional Gaussian elements are defined based on the parameters of the two-dimensional covariance matrix, thereby realizing the dimensionality reduction from three-dimensional Gaussian elements to two-dimensional Gaussian elements.

5. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 1, characterized in that: The color information in the RGB image data acquired by the underwater camera is projected onto the spherical harmonic function based on the spherical harmonic function model, which is used to model the color changes of Gaussian elements under different imaging planes.

6. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 5, characterized in that: Rendering based on 2D Gaussian primitives incorporates the opacity of 3D Gaussian primitives and a spherical harmonic function model with color information as additional parameters, and combines them with the distribution of 2D Gaussian primitives on the imaging plane for fusion rendering.

7. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 1, characterized in that: Define a set of three-dimensional Gaussian elements Each three-dimensional Gaussian element Opacity Perform geometric parameterization on the 3D Gaussian and define the geometric center of the 3D Gaussian. Scaling matrix Rotation matrix 3D covariance matrix Parameterization The three-dimensional Gaussian elements follow a distribution in space: For three-dimensional Gaussian elements Central alienation is performed by defining a plane passing through the geometric center of the three-dimensional Gaussian meta-geometry. Control any three-dimensional Gaussian element within the control space It is divided into two opacity values. and 3D Gaussian elements ; when "At" represents a special case of a three-dimensional Gaussian primitive, specifically a three-dimensional Gaussian primitive after central alienation. It can be represented in the following way: in For the segmentation plane passing through the center of the three-dimensional Gaussian unit The normal vector, And plane The sub-Gaussians on both sides are obtained by changing the sign of the normal vector. For plane The sub-Gaussian space portion on one side is the effective region on the side of the centrally alienated Gaussian.

8. The underwater 3D reconstruction method based on Gaussian topological heterogeneity and cross-space dimensionality reduction mapping according to claim 1, characterized in that: Using Jacobi matrix The three-dimensional Gaussian elements in the camera coordinate system Mapped to ray space: ; The affine approximation Jacobian matrix after projection transformation is approximately a two-dimensional covariance matrix. By reducing the dimensionality of the three-dimensional covariance to the two-dimensional imaging plane, the distribution from Gaussian elements to the corresponding imaging space is realized. .

Images