Surface reconstruction method based on cross-dimensional Gaussian multi-scale feature combination

Abstract

The invention discloses a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature combination, and the method comprises the steps: obtaining a multi-view image of a reconstruction target region, obtaining the initialization point cloud information of the multi-view image through employing a global motion structure recovery algorithm, and building a radiation field parameter model based on a three-dimensional Gaussian mapping primitive; coordinate system transformation and Gaussian primitive dimensionality reduction are carried out on points on radiation field light, a bounding box is defined for each Gaussian primitive by utilizing spatial distribution, scale characteristics and opacity value parameters of three-dimensional Gaussian primitives, and tetrahedral mesh units are established; evaluating the relative positions of the vertexes of the tetrahedral mesh units and the contour surface by adopting the normal of the one-dimensional Gaussian primitives, establishing a combined evaluation model of the one-dimensional Gaussian normal based on the final vertex coordinates, and performing mesh point connection and surface extraction on the combined evaluation model, therefore, complex structure surface reconstruction based on one-dimensional Gaussian and three-dimensional Gaussian primitive multi-scale geometric feature combination is realized.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and more particularly to a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature combination. Background Technology

[0002] As an important development direction in the field of optical sensing, 3D reconstruction technology provides technical support for digital simulation and visual interaction of the environment.

[0003] Existing 3D reconstruction technologies mostly employ methods such as multi-view matching and Poisson reconstruction. Among them, multi-view matching only supports reflecting the general outline of the reconstructed area and cannot concretely represent the surface features of complex structures. Poisson reconstruction, on the other hand, transforms discretized point cloud information into a continuous surface function to achieve implicit expression. However, when faced with non-closed reconstruction environments, it will produce irregular pseudo-surfaces, which will affect the surface reconstruction accuracy for complex structures. At the same time, surface reconstruction methods relying on such implicit expressions are not suitable for subsequent physical modeling and simulation tasks.

[0004] To address this issue, a surface reconstruction technique based on cross-dimensional Gaussian multi-scale features is employed to define and reduce the dimensionality of Gaussian elements. Furthermore, a joint evaluation and optimization is performed based on the bounding box and tetrahedral mesh elements combined with the multi-scale features of Gaussian elements, thereby achieving surface reconstruction for complex structures. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention discloses a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature integration. This method uses three-dimensional Gaussian mapping primitives to explicitly characterize the radiation field of the reconstructed region, extracts multi-scale features through coordinate system transformation and Gaussian primitive dimensionality reduction, and establishes Gaussian bounding boxes and tetrahedral mesh units. Furthermore, it performs mesh extraction and evaluation optimization based on a joint evaluation model based on the final vertex coordinates and one-dimensional Gaussian normals, thereby achieving surface reconstruction for complex structures.

[0006] A surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint method specifically includes the following steps: Acquire multi-view images of the target region for reconstruction, use a global motion structure recovery algorithm to obtain the initial point cloud information of the multi-view images, and establish a radiation field parameter model based on three-dimensional Gaussian mapping primitives; The coordinate system is transformed and the Gaussian element is reduced to a one-dimensional Gaussian element by performing coordinate system transformation and Gaussian element dimensionality reduction on the points on the radiation field ray, and the three-dimensional Gaussian element is transformed into a one-dimensional Gaussian element and a one-dimensional Gaussian normal is defined. The spatial distribution, scale characteristics, and opacity parameters of the three-dimensional Gaussian elements are used to define the bounding box for each Gaussian element and to establish tetrahedral mesh elements, wherein the opacity of the tetrahedral mesh elements decreases from the center of the element to the vertex of the bounding box. The relative positions of vertices and isosurfaces of tetrahedral mesh elements are evaluated using the normals of one-dimensional Gaussian elements. The tetrahedral mesh element is divided into two parts by taking the intersection plane of the normal and the one-dimensional Gaussian element as the zero plane. Points in the positive direction region of the normal are defined as external points, points in the negative direction region of the normal are defined as internal points, and points on the zero plane are defined as isosurface points. Thus, the state index of the tetrahedral mesh element is obtained and the final vertex coordinates are calculated. A joint evaluation model of one-dimensional Gaussian normals is established based on the final vertex coordinates. Mesh points are connected and surfaces are extracted from this joint evaluation model, thereby realizing the reconstruction of complex structural surfaces based on the joint multi-scale geometric features of one-dimensional Gaussian and three-dimensional Gaussian elements.

[0007] Furthermore, when using a global motion structure recovery algorithm to obtain the initial point cloud information of the multi-view image: global feature point extraction and registration are performed on the multi-view image of the reconstructed target area, the view pose is analyzed, and the spatial points are located and the initial point cloud is generated according to the triangulation measurement algorithm.

[0008] Furthermore, when establishing the radiation field parameter model based on 3D Gaussian mapping primitives: a 3D Gaussian distribution defined by the covariance matrix and the mean is established based on the initial point cloud. This distribution is used as the mapping primitive, and the color and opacity parameter information of the 3D Gaussian mapping primitive is obtained from multi-view images. The spatial set where the mapping primitive is located is defined as the radiation field. The covariance matrix includes a rotation matrix and a scaling matrix, which are used to control the rotation angle and scale characteristics of the Gaussian primitives. The mean is generated from the coordinate information in the point cloud and is used to define the spatial distribution of the Gaussian primitives.

[0009] Furthermore, coordinate transformation is performed on the points on the radiation field rays, transforming the world coordinate system in which the imaging process takes place into the Gaussian coordinate system in which the mapping primitives reside.

[0010] Furthermore, when performing Gaussian dimensionality reduction on points on the radiation field rays, the three-dimensional Gaussian numerator is converted into a one-dimensional Gaussian numerator, and the normal is defined based on the one-dimensional Gaussian numerator, thereby achieving dimensionality reduction of the computation space and simplification of normal generation.

[0011] Furthermore, each vertex of the tetrahedral mesh cell is taken from a three-dimensional Gaussian bounding box, wherein the center opacity of the tetrahedral mesh cell is defined by the opacity of the source three-dimensional Gaussian cell.

[0012] Furthermore, the three-dimensional Gaussian bounding box is generated based on the spatial distribution and scale characteristic parameters of the three-dimensional Gaussian elements.

[0013] Furthermore, the zero plane, as an iso-plane tangent to the surface of the mapping primitive, is used to evaluate the generation and optimization of tetrahedral mesh elements based on the intersection position of the plane and the three-dimensional Gaussian bounding box; the tetrahedral mesh element state index stores the tetrahedral mesh intersection position in binary encoding for fast lookup when calculating the final vertex coordinates.

[0014] Furthermore, based on the final vertex coordinates and the joint evaluation model, after connecting the final vertex coordinates to form a mesh region, the internal and external positional relationship between the mesh and the reconstructed surface is evaluated according to the one-dimensional Gaussian normal direction and optimization is guided, thereby realizing the reconstruction of complex structure surfaces based on the joint multi-scale geometric features of one-dimensional Gaussian and three-dimensional Gaussian elements.

[0015] By employing the aforementioned technical solutions, this invention discloses a surface reconstruction method based on the joint use of multi-dimensional Gaussian multi-scale features. This method explicitly characterizes the radiation field of the reconstructed region by establishing three-dimensional Gaussian mapping primitives, extracts multi-scale features using coordinate system transformation and Gaussian primitive dimensionality reduction, and establishes Gaussian bounding boxes and tetrahedral mesh elements. Mesh extraction and evaluation optimization are then performed based on a joint evaluation model using the final vertex coordinates and one-dimensional Gaussian normals, achieving surface reconstruction for complex structures. Furthermore, this method expands from coordinate points to three-dimensional Gaussian primitives and then to Gaussian bounding boxes, and uses tetrahedral mesh elements for Gaussian bounding box evaluation to optimize the fit of the extracted surface to the real surface of the reconstructed region, innovatively changing the surface reconstruction method in three-dimensional reconstruction technology. This method also simplifies the normal definition process by reducing the three-dimensional Gaussian primitives to one-dimensional Gaussian primitives and directly uses the one-dimensional Gaussian normal as the evaluation basis for the relative position of mesh vertices and isosurfaces, thereby simplifying computation in multiple stages.

[0016] In summary, the technical solution of this invention improves upon existing 3D reconstruction technologies, which, in practical applications, rely on discrete and implicit representations that only support the approximate outline of the reconstructed region and cannot perform subsequent physical simulation and modeling tasks. It achieves both high efficiency and high quality in surface reconstruction based on multi-scale geometric features. Therefore, the technical solution of this invention solves the problem of surface reconstruction for complex structures in existing technologies. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature combination according to the present invention.

[0019] Figure 2 This is a schematic diagram of the network structure in the surface reconstruction method based on cross-dimensional Gaussian multi-scale feature combination of the present invention.

[0020] Figure 3 To establish the spatial geometric relationship between the Gaussian bounding box and the tetrahedral mesh element for the three-dimensional Gaussian element and the one-dimensional Gaussian element in the embodiments of the present invention.

[0021] Figure 4 This is a schematic diagram of mesh point connection and surface extraction based on the joint evaluation model of the final vertex coordinates and one-dimensional Gaussian normal in an embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram of surface reconstruction of the original scene in an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] like Figure 1 As shown, this invention provides a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint, comprising: S1. Obtain multi-view images of the target area to be reconstructed, use a global motion structure recovery algorithm to obtain the initial point cloud information of the multi-view images, and establish a radiation field parameter model based on three-dimensional Gaussian mapping primitives. In a specific implementation, as a preferred embodiment of the present invention, the global motion structure recovery algorithm extracts and registers global feature points in the multi-view images of the reconstructed target area, and while analyzing the view pose, it locates points in space and generates an initial point cloud based on the triangulation measurement algorithm. The radiation field parameter model based on the three-dimensional Gaussian mapping primitives establishes a three-dimensional Gaussian distribution defined by the covariance matrix and mean based on the initial point cloud, uses this distribution as the mapping primitive, and obtains parameter information such as color and opacity of the three-dimensional Gaussian mapping primitives from multi-view images, defining the spatial set of the mapping primitives as the radiation field.

[0026] The covariance matrix is ​​composed of a rotation matrix and a scaling matrix, which are used to control the rotation angle and scale characteristics of Gaussian elements; the mean is generated from the coordinate information in the point cloud and is used to define the spatial distribution of Gaussian elements.

[0027] S2. Perform coordinate system transformation and Gaussian element dimensionality reduction on the points on the radiation field rays, transforming the three-dimensional Gaussian elements into one-dimensional Gaussian elements and defining one-dimensional Gaussian normals. In a specific implementation, as a preferred embodiment of the present invention, the coordinate system transformation of the points on the radiation field rays is performed to transform the world coordinate system in which the imaging process takes place into the Gaussian coordinate system in which the mapping primitives are located. The Gaussian element dimensionality reduction transforms a three-dimensional Gaussian element into a one-dimensional Gaussian element, and defines normals based on the one-dimensional Gaussian element, thereby achieving dimensionality reduction of the computation space and simplification of normal generation.

[0028] S3. Define a bounding box for each Gaussian element and establish tetrahedral mesh elements using the spatial distribution, scale characteristics and opacity parameters of the three-dimensional Gaussian elements. The opacity of the tetrahedral mesh elements decreases from the center of the element to the vertex of the bounding box. In a specific implementation, as a preferred embodiment of the present invention, the three-dimensional Gaussian bounding box is generated by defining parameters such as the spatial distribution and scale characteristics of the three-dimensional Gaussian elements; Each vertex of the tetrahedral mesh cell is taken from a three-dimensional Gaussian bounding box, and the opacity of the mesh cell decreases from the cell center to the vertices on the three-dimensional Gaussian bounding box, wherein the opacity of the cell center is defined by the opacity of the source three-dimensional Gaussian mesh cell.

[0029] S4. The relative positions of the vertices and isosurfaces of the tetrahedral mesh element are evaluated using the normals of the one-dimensional Gaussian element. The tetrahedral mesh element is divided into two parts by taking the intersection plane of the normal and the one-dimensional Gaussian element as the zero plane. Points in the positive direction region of the normal are defined as external points, points in the negative direction region of the normal are defined as internal points, and points on the zero plane are defined as isosurface points. Then, the state index of the tetrahedral mesh element is obtained and the final vertex coordinates are calculated.

[0030] In a specific implementation, as a preferred embodiment of the present invention, the normal of a one-dimensional Gaussian element is used to quickly evaluate the relative position of the mesh vertices and the isosurface. The intersection plane of the normal and the one-dimensional Gaussian element is used as the zero plane to divide the space where the tetrahedral mesh is located into two parts. Points in the region along the positive direction of the normal are defined as external points, points in the region along the negative direction of the normal are defined as internal points, and points on the zero plane are defined as isosurface points. Thus, the tetrahedral state index is obtained and the final vertex coordinates are calculated. The zero plane, as an iso-plane tangent to the surface of the mapping primitive, is used to evaluate the generation and optimization of tetrahedral meshes based on the location of the intersection point between the plane and the 3D Gaussian bounding box.

[0031] The tetrahedral mesh cell state index stores the positions of the tetrahedral mesh intersections in binary encoding, which is used for quick lookup when calculating the final vertex coordinates.

[0032] S5. Based on the final vertex coordinates, establish a joint evaluation model of one-dimensional Gaussian normals, connect grid points and extract surfaces from the joint evaluation model, thereby realizing the reconstruction of complex structural surfaces based on the joint multi-scale geometric features of one-dimensional Gaussian and three-dimensional Gaussian elements.

[0033] In a specific implementation, as a preferred embodiment of the present invention, the joint evaluation model based on the final vertex coordinates and the one-dimensional Gaussian normal, after connecting the final vertex coordinates to form a mesh region, evaluates the internal and external positional relationship between the mesh and the reconstructed surface according to the direction of the one-dimensional Gaussian normal and guides optimization, thereby realizing the reconstruction of complex structural surfaces based on the joint multi-scale geometric features of one-dimensional Gaussian and three-dimensional Gaussian elements.

[0034] Example like Figure 1 As shown, this invention provides a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint; as Figure 2 As shown, this invention provides a schematic diagram of the network structure for a surface reconstruction method based on cross-dimensional Gaussian multi-scale feature integration.

[0035] like Figure 3 As shown, this embodiment provides a method for establishing the spatial geometric relationship between a Gaussian bounding box and tetrahedral mesh elements using 3D Gaussian elements and 1D Gaussian elements. The Gaussian bounding box and tetrahedral mesh elements are established based on the 3D Gaussian elements. Then, the internal points, external points, and iso-points of the iso-surface segmented region are determined based on the 1D Gaussian definition of the normal within the Gaussian bounding box. This process completes the evaluation of the zero plane (i.e., the iso-surface) and the final vertex calculation. The final vertex coordinates and 1D Gaussian normals are used to jointly evaluate mesh point connections and surface extraction, thereby achieving complex structural surface reconstruction based on the joint multi-scale geometric features of 1D Gaussian and 3D Gaussian elements.

[0036] like Figure 4As shown, this is a schematic diagram illustrating mesh point connection and surface extraction based on a joint evaluation model using the final vertex coordinates and one-dimensional Gaussian normals in the embodiment; as... Figure 5 As shown, this embodiment provides a schematic diagram of surface reconstruction for the original scene.

[0037] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint, characterized in that, include: Acquire multi-view images of the target region for reconstruction, use a global motion structure recovery algorithm to obtain the initial point cloud information of the multi-view images, and establish a radiation field parameter model based on three-dimensional Gaussian mapping primitives; The coordinate system is transformed and the Gaussian element is reduced to a one-dimensional Gaussian element by performing coordinate system transformation and Gaussian element dimensionality reduction on the points on the radiation field ray, and the three-dimensional Gaussian element is transformed into a one-dimensional Gaussian element and a one-dimensional Gaussian normal is defined. The spatial distribution, scale characteristics, and opacity parameters of the three-dimensional Gaussian elements are used to define the bounding box for each Gaussian element and to establish tetrahedral mesh elements, wherein the opacity of the tetrahedral mesh elements decreases from the center of the element to the vertex of the bounding box. The relative positions of vertices and isosurfaces of tetrahedral mesh elements are evaluated using the normals of one-dimensional Gaussian elements. The tetrahedral mesh element is divided into two parts by taking the intersection plane of the normal and the one-dimensional Gaussian element as the zero plane. Points in the positive direction region of the normal are defined as external points, points in the negative direction region of the normal are defined as internal points, and points on the zero plane are defined as isosurface points. Thus, the state index of the tetrahedral mesh element is obtained and the final vertex coordinates are calculated. A joint evaluation model of one-dimensional Gaussian normals is established based on the final vertex coordinates. Mesh points are connected and surfaces are extracted from this joint evaluation model, thereby realizing the reconstruction of complex structural surfaces based on the joint multi-scale geometric features of one-dimensional Gaussian and three-dimensional Gaussian elements.

2. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: When using a global motion structure recovery algorithm to obtain the initial point cloud information of the multi-view image: global feature point extraction and registration are performed on the multi-view image of the reconstructed target area, the view pose is analyzed, and the spatial points are located and the initial point cloud is generated according to the triangulation measurement algorithm.

3. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: When establishing a radiation field parameter model based on 3D Gaussian mapping primitives: a 3D Gaussian distribution defined by the covariance matrix and mean is established based on the initial point cloud. This distribution is used as the mapping primitive. The color and opacity parameter information of the 3D Gaussian mapping primitives are obtained from multi-view images. The spatial set where the mapping primitives are located is defined as the radiation field. The covariance matrix includes a rotation matrix and a scaling matrix, which are used to control the rotation angle and scale characteristics of the Gaussian primitives. The mean is generated from the coordinate information within the point cloud and is used to define the spatial distribution of Gaussian elements.

4. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: The coordinate system of the points on the radiation field rays is transformed, changing the world coordinate system in which the imaging process takes place to the Gaussian coordinate system in which the mapping primitives are located.

5. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: When performing Gaussian dimensionality reduction on points on a radiation field ray: the three-dimensional Gaussian radiant is converted into a one-dimensional Gaussian radiant, and the normal is defined based on the one-dimensional Gaussian radiant, thereby achieving dimensionality reduction of the computation space and simplification of normal generation.

6. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: Each vertex of the tetrahedral mesh cell is taken from a 3D Gaussian bounding box, wherein the center opacity of the tetrahedral mesh cell is defined by the opacity of the source 3D Gaussian cell.

7. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 6, characterized in that: The three-dimensional Gaussian bounding box is generated based on the spatial distribution and scale characteristic parameters of the three-dimensional Gaussian elements.

8. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: The zero plane, as an iso-plane tangent to the surface of the mapping primitive, is used to evaluate the generation and optimization of tetrahedral mesh elements based on the intersection position of the plane and the 3D Gaussian bounding box. The tetrahedral mesh element state index stores the tetrahedral mesh intersection position in binary encoding for fast lookup when calculating the final vertex coordinates.

9. The surface reconstruction method based on cross-dimensional Gaussian multi-scale feature joint as described in claim 1, characterized in that: The method involves connecting the final vertex coordinates to form a mesh region based on the final vertex coordinates and the joint evaluation model. Then, the internal and external positional relationship between the mesh and the reconstructed surface is evaluated according to the one-dimensional Gaussian normal direction to guide optimization, thereby realizing the reconstruction of complex structural surfaces based on the joint multi-scale geometric features of one-dimensional Gaussian and three-dimensional Gaussian elements.

Images