Three-dimensional reconstruction method for boundary-free scene of extraterrestrial celestial body based on Gaussian radiation field

Abstract

The invention relates to an extraterrestrial celestial body boundary-free scene three-dimensional reconstruction method based on a Gaussian radiation field, and the method comprises the steps: extracting visual features of an input image, and obtaining internal parameters, external parameters and scene sparse point cloud of a camera through feature matching and cluster adjustment; an expanded dense point cloud is calculated according to a light plane intersection mode, a series of three-dimensional Gaussian distributions are constructed, and pixel colors in a given visual angle direction are rendered in a rasterization mode; aiming at a rendering result, guiding a Gaussian position to be accurately close to a scene surface by utilizing pre-estimated depth prior in an optimization process; a scene surface is accurately captured in combination with an explicit and implicit model joint representation mode; and rebuilding a fine and complete three-dimensional mesh model on the surface of the extraterrestrial celestial body by using a marching tetrahedron algorithm according to an optimization result. According to the method, fine three-dimensional reconstruction of the surface of the extraterrestrial celestial body in a boundary-free scene can be realized, the modeling capability of a Gaussian radiation field in the boundary-free scene is enhanced, and the precision and integrity of three-dimensional reconstruction of the surface of the extraterrestrial celestial body are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep space exploration, and in particular to a three-dimensional reconstruction method for a boundaryless scene of an extraterrestrial object based on a Gaussian radiation field. Background Art

[0002] 3D reconstruction using a sequence of images taken by a known camera is an advanced computer vision technique that can recover the 3D information of the original scene from input multi-view images, extracting and reconstructing a 3D mesh model of the scene surface. 3D reconstruction of the surface of extraterrestrial objects provides highly accurate topographic data, enabling researchers to conduct more quantitative and systematic landform analysis and infer the geological age, material composition, and internal structural evolution of extraterrestrial objects. This, in turn, helps deepen our understanding of the origin and evolution of the solar system and even the universe. This is a key research direction in deep space exploration and planetary science, and holds significant significance in both scientific research and engineering applications.

[0003] With the rapid development of deep learning technology, 3D reconstruction technology has also achieved new breakthroughs. Implicit neural radiance fields are widely used in the field of scene 3D reconstruction. However, such methods often rely on deep neural networks to fit scene information and optimize a large number of network parameters during training, resulting in serious problems such as long training time and low algorithm efficiency. Recently, the three-dimensional Gaussian radiance field (3DGS) has shown superior performance in model extraction and reconstruction tasks. This method represents the geometric structure and color information of the 3D scene by optimizing a set of discrete, dense anisotropic Gaussian ellipsoids in space. Combined with a fast and differentiable rasterization rendering method, it significantly improves the time efficiency of the algorithm.

[0004] While 3DGS is widely used in various real-world scenes and achieves excellent rendering and reconstruction results, it is primarily suitable for 3D reconstruction of surfaces in conventional indoor scenes and foreground surfaces centered on objects. In deep space exploration missions, extraterrestrial scenes often extend infinitely far away. The lack of explicit boundary conditions during 3D surface reconstruction makes it difficult to capture more geometric information in the scene. In particular, when reconstructing surfaces in distant areas, it is impossible to recover complete and fine texture details.

[0005] Regarding the research on 3D reconstruction methods for extraterrestrial scenes without boundaries, the shortcomings of existing technologies are mainly manifested in the following aspects: (1) long training time, high time cost, and severely limited reconstruction efficiency; (2) the surface of extraterrestrial objects is infinitely extended, and the reconstruction accuracy and completeness of distant areas are low; (3) there are a large number of weak texture areas in extraterrestrial scenes, which makes it difficult to provide sufficient visual information and geometric information. Summary of the Invention

[0006] The technical problem solved by the present invention is: to overcome the shortcomings of the existing technology and provide a three-dimensional reconstruction method for the boundaryless scene of an extraterrestrial object based on a Gaussian radiation field. On the basis of the existing method, the completeness and accuracy of the three-dimensional reconstruction of the surface of an extraterrestrial object are improved, and the problems of poor reconstruction details and missing geometric structures caused by the boundaryless scene are alleviated, thereby realizing the complete reconstruction of a high-precision three-dimensional grid model of an extraterrestrial object.

[0007] The technical solution of the present invention is:

[0008] A method for 3D reconstruction of an extraterrestrial object without a boundary scene based on a Gaussian radiation field comprises the following steps:

[0009] Step 1: Construct a Gaussian difference pyramid to extract the SIFT visual features of the input image, perform feature matching and bundle adjustment through motion recovery structure, and obtain camera internal parameters, external parameters and scene sparse point cloud;

[0010] Step 2: For the sparse point cloud of the scene obtained in step 1, the 3D coordinates of the expanded dense point cloud are calculated based on the intersection of light planes. A series of 3D Gaussian distributions representing the unbounded scene of extraterrestrial objects are constructed to form a Gaussian radiation field in space. The pixel color of the given viewing angle is rendered in combination with a differentiable rasterization rendering method.

[0011] Step 3: Optimize the rendering result of step 2. During the optimization process, the depth prior information calculated by the pre-trained monocular depth estimation model is used to guide the distribution of Gaussian primitives in the scene so that they are close to the scene surface.

[0012] Step 4: In the optimization process of step 3, the explicit Gaussian primitives and the implicit neural signed distance field are combined to capture the scene surface by using the joint representation of the explicit and implicit models;

[0013] Step 5: Calculate the Gaussian opacity field based on the Gaussian radiation field finally optimized in steps 3 and 4, and use the marching tetrahedron three-dimensional reconstruction method to extract the three-dimensional grid model of the surface of the extraterrestrial object.

[0014] A device for reconstructing a three-dimensional scene of an extraterrestrial object without boundaries based on a Gaussian radiation field, comprising:

[0015] The construction module is used to construct a Gaussian difference pyramid to extract SIFT visual features of the input image, perform feature matching and bundle adjustment through motion recovery structure, and obtain camera internal parameters, external parameters and scene sparse point cloud;

[0016] The rendering module calculates the 3D coordinates of the expanded dense point cloud based on the intersection of light planes for the sparse point cloud of the scene, constructs a series of 3D Gaussian distributions representing the unbounded scene of extraterrestrial objects, forms a Gaussian radiation field in space, and combines it with a differentiable rasterization rendering method to render the pixel color in a given viewing direction;

[0017] The Gaussian primitive distribution calculation module optimizes the rendering results. During the optimization process, it uses the depth prior information calculated by the pre-trained monocular depth estimation model to guide the Gaussian primitive distribution in the scene so that it is close to the scene surface.

[0018] The scene surface capture module, based on the optimization process, combines explicit Gaussian primitives with implicit neural signed distance fields to capture the scene surface using a joint representation of explicit and implicit models;

[0019] The three-dimensional mesh model extraction module calculates the Gaussian opacity field based on the final optimization result of the Gaussian radiation field, and uses the three-dimensional reconstruction method of the marching tetrahedron to extract the three-dimensional mesh model of the surface of the extraterrestrial object.

[0020] An electronic device, characterized in that it includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the described method.

[0021] A computer-readable storage medium is characterized in that executable instructions are stored thereon, and when the instructions are executed by a processor, the processor implements the method described.

[0022] The beneficial effects of the present invention compared with the prior art are:

[0023] (1) The present invention adopts a dense filling method for initializing point clouds. Several rays are emitted through the optical center of the camera to intersect with the fitting surface of the extraterrestrial object. The camera intrinsic parameters and camera extrinsic parameters are combined to perform projective transformation. The expanded point cloud is added to the area not covered by the initial sparse point cloud, which effectively expands the scene representation range of the Gaussian primitive and improves the accuracy of model reconstruction.

[0024] (2) This invention utilizes the depth prior information calculated by a pre-trained monocular depth estimation network to guide the Gaussian primitives in the scene to accurately approximate the surface of extraterrestrial objects, significantly enhancing the representational capabilities of the Gaussian primitives. Simultaneously, by combining the implicit neural signed distance field and the explicit Gaussian primitives, a joint representation of the explicit and implicit models is employed to reconstruct the 3D scene of extraterrestrial objects, which is beneficial for improving the accuracy of 3D reconstruction of unbounded extraterrestrial scenes.

[0025] In summary, the method adopted by the present invention is simple in principle and can achieve the purpose of completely extracting and reconstructing a high-precision three-dimensional grid model in a boundaryless scene of an extraterrestrial object. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a method for three-dimensional reconstruction of an extraterrestrial object without boundaries based on a Gaussian radiation field according to the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other. To achieve the above-mentioned objectives, the present invention adopts the following technical solutions.

[0028] like Figure 1 As shown, the present invention provides a method for 3D reconstruction of an extraterrestrial object without boundary scenes based on Gaussian radiation field, and the specific implementation steps are as follows:

[0029] Step 1: Construct a Gaussian difference pyramid to extract the SIFT visual features of the input image, perform feature matching and bundle adjustment through motion recovery structure, and obtain camera internal parameters, external parameters and scene sparse point cloud;

[0030] Step 2: For the sparse point cloud of the scene obtained in step 1, the 3D coordinates of the expanded dense point cloud are calculated based on the intersection of light planes. A series of 3D Gaussian distributions representing the unbounded scene of extraterrestrial objects are constructed to form a Gaussian radiation field in space. The pixel color of the given viewing angle is rendered in combination with a differentiable rasterization rendering method.

[0031] Step 3: Optimize the rendering result of step 2. During the optimization process, the depth prior information calculated by the pre-trained monocular depth estimation model is used to guide the distribution of Gaussian primitives in the scene so that they are close to the scene surface.

[0032] Step 4: In the optimization process of step 3, the explicit Gaussian primitives and the implicit neural signed distance field are combined to capture the scene surface by using the joint representation of the explicit and implicit models;

[0033] Step 5: Calculate the Gaussian opacity field based on the Gaussian radiation field finally optimized in steps 3 and 4, and use the marching tetrahedron three-dimensional reconstruction method to extract the three-dimensional grid model of the surface of the extraterrestrial object.

[0034] In step 1, a set of optical image sequences of extraterrestrial objects taken by the camera is input, the SIFT visual features of the input images are extracted, and the camera internal parameters, external parameters and scene sparse point cloud are obtained through feature matching and bundle adjustment.

[0035] Gaussian kernel functions with different blur scales are used to perform convolution processing on input images of different sizes, and the Gaussian image pyramid is constructed using the following formula:

[0036] ,

[0037] ,

[0038] in, The blur scale representing Gaussian smoothing is , coordinate position Gaussian kernel function at , represents the input optical image, Represents the constructed Gaussian image pyramid;

[0039] Perform difference operations on Gaussian smoothed images of the same size and adjacent blur scales to construct a Gaussian difference image pyramid:

[0040] ,

[0041] in, Represents the constructed Gaussian difference image pyramid, Indicates that the fuzzy scale constructed in the previous step is Gaussian image pyramid, Indicates that the fuzzy scale constructed in the previous step is Gaussian image pyramid, is the coefficient of variation of adjacent blur scales in each layer of the image pyramid;

[0042] Considering that directly extracting the extreme points inside each layer of the Gaussian difference image pyramid is not accurate enough, it is necessary to use the second-order Taylor expansion to perform curve interpolation on the scale space function to obtain the accurate key point position at the sub-pixel level:

[0043] ,

[0044] in, is the interpolation center, This is the constructed Gaussian difference image pyramid , express Relative interpolation center The offset, represents the first-order partial differential operation, represents the second-order partial differential operation;

[0045] In the above formula Derivative, get the deviation The extreme point of , the extreme value of the spatial scale function corresponding to the extreme point is:

[0046] ,

[0047] in, Represents the extreme point of the spatial scale function, that is, the final sub-pixel level accurate position of the key point;

[0048] After calculating the precise sub-pixel keypoint positions, the keypoints with low contrast are removed and the edge response is eliminated to avoid the interference of edge noise. Then, a sub-pixel keypoint with a radius of 1 is selected as the center. The pixel features in the local window are counted to determine the direction information of the key point. That is, the gradient amplitude and gradient direction of all pixels in the window are calculated. According to the distance between each pixel in the window and the key point, the amplitude is weighted and statistically analyzed to generate a directional gradient histogram. The direction corresponding to the peak in the gradient histogram is selected as the main direction of the key point, and the direction greater than 80% of the peak of the main direction is retained as the auxiliary direction of the key point:

[0049] ,

[0050] ,

[0051] in, Representing coordinates The gradient amplitude at , Representing coordinates The gradient direction at the key point can achieve more accurate feature point matching by integrating multiple directions into the key point.

[0052] After obtaining the position, scale and direction of the key points, The gradient histogram in the neighborhood of is calculated to obtain the following 128-dimensional feature point descriptor:

[0053] ,

[0054] in, Represents the 128-dimensional SIFT feature descriptor, Respectively represent the 128 components of the SIFT feature descriptor;

[0055] In order to remove the influence of image illumination changes, the SIFT feature descriptor vector is normalized:

[0056] ,

[0057] in, represents the 128-dimensional normalized SIFT feature descriptor, Represent the 128 components of the normalized SIFT feature descriptor, namely ;

[0058] After fully extracting SIFT feature points and descriptors, we perform feature matching and bundle adjustment optimization, and use structure from motion to obtain the camera's intrinsic parameters, extrinsic parameters, and sparse point cloud of the scene for the optical image of the extraterrestrial surface:

[0059] ,

[0060] Among them, K represents the internal parameter matrix of the camera, represents the external parameter matrix of the i-th camera, P represents the sparse point cloud of the scene reconstruction, represents the i-th optical image of the surface of an extraterrestrial object, SFM represents the algorithm for recovering structure from motion, Represents the total number of optical images of the surface of extraterrestrial objects.

[0061] In step 2, for the sparse point cloud of the scene obtained in step 1, the three-dimensional coordinates of the expanded dense point cloud are calculated based on the intersection of light planes, and a series of three-dimensional Gaussian distributions representing the unbounded scenes of extraterrestrial objects are constructed to form a Gaussian radiation field in space. Combined with the differentiable rasterization rendering method, the pixel color in a given viewing direction is calculated.

[0062] The dense point cloud is expanded by combining the geometric relationship between viewpoints and the surface characteristics of extraterrestrial objects. The least squares method is applied to fit an ideal geometric plane based on the sparsely reconstructed point cloud data of the scene:

[0063] ,

[0064] Where n represents the normal vector of the ideal geometric plane, c represents the intercept offset of the ideal geometric plane, f represents the least squares method, and P is the sparse reconstructed point cloud of the scene obtained by the motion recovery structure algorithm in step 1;

[0065] Several rays are emitted from the camera's optical center, passing through the centers of all pixels in the pixel plane. The depth information at the intersection is calculated by intersecting the rays with the scene surface. Then, based on the basic proportional relationship of projective geometry, the coordinate information of the expanded 3D point in the dense space can be obtained by simple calculation:

[0066] ,

[0067] ,

[0068] ,

[0069] Among them, n and c are the normal vector and intercept offset of the ideal geometric plane calculated in the previous step, p represents the three-dimensional coordinates of any position on the surface of an extraterrestrial object, r is the coordinate of the light emitted by the optical center in the world coordinate system, o represents the coordinate of the camera optical center, d represents the unit direction vector of the light emitted by the optical center, t represents the depth information of the light emitted by the optical center, and the way the light intersects with the scene surface is the constraint , solve the simultaneous equations to obtain depth information, It is a stable constant that prevents the denominator from being zero and ensures that the division operation is meaningful;

[0070] In order to improve the computational efficiency of the algorithm and process the correspondence between the sparse point cloud of the scene and the plane more accurately, the extraterrestrial scene representation space is discretized into voxels and the 1024 pixels are plotted on the fitted ideal geometric plane. The resolution of 1024 is divided into grids, and the point cloud is only densely expanded in the scene areas that are not covered by the initial sparse point cloud;

[0071] Use the expanded dense point cloud to initialize and construct a series of three-dimensional Gaussian distributions that represent the unbounded scenes of extraterrestrial objects. These anisotropic Gaussian primitives are semi-transparent, and each Gaussian primitive is parameterized by a three-dimensional covariance matrix and a central mean:

[0072] ,

[0073] Among them, the mean is considered as the center position of the three-dimensional Gaussian basis element, Represents the three-dimensional point coordinates close to the location, covariance matrix is decomposed into a scale matrix and a direction matrix ,Right now ; Each 3D Gaussian primitive also has a color and opacity , these two properties are used for differentiable point-based Mixed, the k-th 3D Gaussian primitive in the scene contains the following five attributes:

[0074] ,

[0075] in, is the quaternion representing the direction matrix R, is a three-dimensional vector representing the scaling matrix S, is the center of the three-dimensional Gaussian primitive, and the position and color information of the expanded dense point cloud are used to initialize the center of the three-dimensional Gaussian primitive respectively. and color , initialize the three-dimensional vector according to the distance from each point in the dense point cloud to the nearest point around it , for quaternions and opacity Perform zero initialization;

[0076] The standard Gaussian radiation field is locally approximated to the perspective camera projection through an affine transformation for efficient rasterization rendering; this involves first transforming each 3D Gaussian primitive to the camera coordinate system through a world-to-camera coordinate transformation matrix, and then projecting it to the image plane through a local affine transformation:

[0077] ,

[0078] Among them, W represents the transformation matrix from the world coordinate system to the camera coordinate system, which comes from the camera pose estimated by the SFM method, and J represents the local affine transformation, which can be calculated by the internal parameters of the camera. is the covariance matrix in the camera coordinate system. Skipping the last row and column will give the two-dimensional covariance matrix ,matrix Used to calculate the two-dimensional Gaussian distribution on the image plane , and will be in the subsequent point-based Use in mixtures;

[0079] The standard Gaussian radiation field method sorts the two-dimensional Gaussian distribution projected onto the image plane from front to back by depth, and then adopts a differentiable point-based Blending integrates weighted colors into the rendered image:

[0080] ,

[0081] in, is the final rendered pixel color, Indicates by color Gathered by the rasterizer The set of Gaussian basis elements, is the index of the sorted Gaussian basis, Defined as , as a blending weight for point-based rendering.

[0082] In step 3, for the pixel image rendering result of step 2, the depth prior information calculated by the pre-trained monocular depth estimation model is used to guide the distribution of Gaussian primitives in the scene during the optimization process, so that it is accurately close to the scene surface.

[0083] The pre-trained monocular depth estimation model Depth Anything V2 is used to process the input extraterrestrial optical image sequence and calculate the depth prior information of each image in the image sequence:

[0084] ,

[0085] in, Depth Anything V2, a monocular depth estimation model. is the input extraterrestrial optical image, Represents a monocular depth estimation model Pre-trained saved breakpoint parameters, Represents a monocular depth estimation model The calculated depth prior information;

[0086] The above pre-trained monocular depth estimation model Calculated depth prior information The depth D rendered by the algorithm during the training process in step 2 is processed in parallel in blocks. In the sense of relative scale, the monocular depth prior is used as the true depth to guide the 3D reconstruction of the unbounded extraterrestrial scene, maintain the topological consistency of the surface geometric structure of the unbounded extraterrestrial scene, and refine the texture details of the geometric surface reconstruction. The calculation formula of the depth loss is as follows:

[0087] ,

[0088] in, Represents the total number of image blocks, M represents the size of each image block, D and They represent the depth rendered by the algorithm during training and the pre-estimated prior depth information respectively. Mean and std represent the average value and standard deviation of the pixel grayscale on each image block respectively.

[0089] In step 4, during the optimization process of step 3, the explicit Gaussian primitive representation and the implicit neural signed distance field representation are combined to accurately capture the scene surface using the joint representation of explicit and implicit models, thereby improving the accuracy of surface reconstruction of unbounded scenes.

[0090] Based on the explicit Gaussian primitive representation of the scene in step 2, an implicit neural signed distance field is introduced. The signed distance field represents the three-dimensional scene as an implicit equation through a deep neural network, and maps the spatial position into a signed distance value from the geometric surface of the scene, so that the shortest distance from any three-dimensional space point to the surface can be directly calculated by multi-layer perceptron fitting, and the surface of the extraterrestrial scene can be accurately extracted and reconstructed. In the implicit neural signed distance field representation, the geometric surface of the scene can be reconstructed by extracting the zero level set of the signed distance field, which is mathematically expressed as:

[0091] ,

[0092] in, Represents the coordinates of any three-dimensional point in space, represents the implicit neural signed distance field, S is a set of all the equations satisfying A set of three-dimensional points, that is, all points in the set are exactly located on the surface of the scene;

[0093] The joint representation method of explicit and implicit models is similar to surface reconstruction from point clouds, which can provide clear 3D supervision information and jointly optimize the signed distance field network and Gaussian primitive parameters; specifically, for the discrete Gaussian primitive center positions distributed in the spatial scene in step 2, First, the signed distance field is used as a priori information to guide the distribution of Gaussian primitives in space to be close to the geometric surface of the scene, that is, surface position constraints are imposed on the Gaussian primitives and the signed distance field:

[0094] ,

[0095] Where N represents the total number of Gaussian primitives. represents an implicit signed distance field network, is the center position of the k-th Gaussian basis element, represents the surface position constraint loss;

[0096] Then, in order to encourage the signed distance field to approximate a valid signed distance function, the Eikonal equation is followed to penalize the gradient deviation of the signed distance field:

[0097] ,

[0098] Where N represents the total number of Gaussian primitives. represents the gradient of the signed distance field, is the center position of the k-th Gaussian basis element, represents the gradient bias loss.

[0099] In step 5, the Gaussian opacity field is calculated on the Gaussian radiation field finally optimized in steps 3 and 4, and the three-dimensional grid reconstruction method of the marching tetrahedron is used to extract and reconstruct a detailed and complete three-dimensional grid model of the unbounded surface of the extraterrestrial object.

[0100] A continuous Gaussian opacity field is constructed from discrete 3D Gaussian primitives using sampling along the ray direction. The specific process is as follows:

[0101] First, the world coordinate system is transformed into the local coordinate system of each Gaussian primitive and the scale is normalized:

[0102] ,

[0103] in, is the scaling matrix of the k-th Gaussian basis, is the direction matrix of the k-th Gaussian basis, is the center position of the k-th Gaussian basis element, 、 Represent the camera optical center and light direction in the world coordinate system respectively, t is the depth along the light direction, 、 、 They represent the camera optical center, the light direction, and the coordinates of the three-dimensional point along the light direction and at a depth of t in the local coordinate system of the k-th Gaussian basis element;

[0104] In the local coordinate system of the kth Gaussian basis element, the Gaussian value of a three-dimensional point at any depth along the direction of the light follows a one-dimensional Gaussian distribution and can be expressed as:

[0105] ,

[0106] For this one-dimensional Gaussian distribution, there is a closed-form solution to its maximum value. The calculation formula of the closed-form solution is:

[0107] ,

[0108] For the k-th Gaussian primitive, the opacity of a 3D point at any depth along the ray direction in space can be expressed as:

[0109] ,

[0110] Taking into account the influence of all N Gaussian basis elements in space on the opacity at any position, the opacity of a three-dimensional point in space at a depth t along the light direction r can be expressed as:

[0111] ,

[0112] in, 、 is the blending weight defined during the rendering process in step 2;

[0113] The opacity of any point in space is defined by the minimum opacity value of all training views and viewing directions, constructing a Gaussian opacity field with continuous distribution in space:

[0114] ,

[0115] Where x represents any point in three-dimensional space, Indicates the opacity value at that point;

[0116] After constructing the spatial Gaussian opacity field, for each three-dimensional Gaussian basis element Generate a 3D cube bounding box at the position, then calculate the opacity at the center of each Gaussian primitive and its bounding box vertex, and subtract 0.5 from the opacity value as the signed distance function (SDF) value, which describes the signed distance from any point in 3D space to the scene surface:

[0117] ,

[0118] Use the maximum scale of each Gaussian primitive As the scale of the Gaussian basis element center and its bounding box vertices, combined with the SDF value calculated according to the opacity, several spatial tetrahedrons are generated, and the marching tetrahedron algorithm is used to extract the three-dimensional mesh model; finally, the zero level set of the SDF is determined by the binary search method, and the three-dimensional points located on the scene surface are found. The three-dimensional model of the unbounded scene surface of the extraterrestrial object is extracted and reconstructed.

[0119] Any matters not described in detail in the present specification are known to those skilled in the art. Although the above description of the present invention is based on illustrative embodiments to facilitate understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the specific embodiments. It should be apparent to those skilled in the art that various modifications within the spirit and scope of the present invention as defined and established by the appended claims are readily apparent, and all inventions utilizing the present invention are protected.

Claims

1. A method for 3D reconstruction of extraterrestrial objects without boundaries based on Gaussian radiation field, characterized in that: include: Step 1: Construct a Gaussian difference pyramid to extract the SIFT visual features of the input image, perform feature matching and bundle adjustment through motion recovery structure, and obtain camera internal parameters, external parameters and scene sparse point cloud; Step 2: For the sparse point cloud of the scene obtained in step 1, the 3D coordinates of the expanded dense point cloud are calculated based on the intersection of light planes. A series of 3D Gaussian distributions representing the unbounded scene of extraterrestrial objects are constructed to form a Gaussian radiation field in space. The pixel color of the given viewing angle is rendered in combination with a differentiable rasterization rendering method. Step 3: Optimize the rendering result of step 2. During the optimization process, the depth prior information calculated by the pre-trained monocular depth estimation model is used to guide the distribution of Gaussian primitives in the scene so that they are close to the scene surface. Step 4: In the optimization process of step 3, the explicit Gaussian primitives and the implicit neural signed distance field are combined to capture the scene surface by using the joint representation of the explicit and implicit models; Step 5: Calculate the Gaussian opacity field based on the Gaussian radiation field finally optimized in steps 3 and 4, and use the marching tetrahedron three-dimensional reconstruction method to extract the three-dimensional grid model of the surface of the extraterrestrial object.

2. The method for 3D reconstruction of extraterrestrial objects without boundaries based on Gaussian radiation field according to claim 1, characterized in that: In step 1, input the optical image of the extraterrestrial object taken by the camera and construct the Gaussian image pyramid using the following formula: ， ， in, The blur scale representing Gaussian smoothing is , coordinate position Gaussian kernel function at , represents the input optical image, Represents the constructed Gaussian image pyramid; Perform difference operations on Gaussian smoothed images of the same size and adjacent blur scales to construct a Gaussian difference image pyramid: ， in, Represents the constructed Gaussian difference image pyramid, Indicates that the fuzzy scale constructed in the previous step is Gaussian image pyramid, Indicates that the fuzzy scale constructed in the previous step is Gaussian image pyramid, is the coefficient of variation of adjacent blur scales in each layer of the image pyramid; The scale space function is interpolated using the second-order Taylor expansion to obtain the key point position at the sub-pixel level: ， in, is the interpolation center, This is the constructed Gaussian difference image pyramid , express Relative interpolation center The offset, represents the first-order partial differential operation, represents the second-order partial differential operation; The above formula is correct Derivative, get the deviation The extreme point of , corresponding to the extreme value of the spatial scale function is: ， in, Represents the extreme point of the spatial scale function, that is, the final sub-pixel position of the key point; After calculating the positions of all sub-pixel key points, the key points with low contrast are removed and the edge response is eliminated. Then, a sub-pixel key point with a radius of 1 is selected as the center. The pixel features in the local window are counted to determine the direction information of the key point. That is, the gradient amplitude and gradient direction of all pixels in the window are calculated. According to the distance between each pixel in the window and the key point, the amplitude is weighted and statistically analyzed to generate a direction histogram. The direction corresponding to the peak in the histogram is selected as the main direction of the key point, and the direction greater than 80% of the peak of the main direction is retained as the auxiliary direction of the key point: ， ， in, Representing coordinates The gradient amplitude at , Representing coordinates The gradient direction at ; After obtaining the position, scale and direction of the key points, The gradient histogram in the neighborhood of is calculated to obtain the following 128-dimensional feature point descriptor: ， in, Represents the 128-dimensional SIFT feature descriptor, Respectively represent the 128 components of the SIFT feature descriptor; Normalize the SIFT feature descriptor vector: ， in, represents the 128-dimensional normalized SIFT feature descriptor, Represent the 128 components of the normalized SIFT feature descriptor, namely ; After extracting SIFT feature points and descriptors, we perform feature matching and bundle adjustment optimization, and use motion recovery structure to obtain the intrinsic parameters, extrinsic parameters, and sparse point cloud of the optical image of the extraterrestrial surface: ， Among them, K represents the internal parameter matrix of the camera, represents the external parameter matrix of the i-th camera, P represents the sparse point cloud of the scene reconstruction, represents the i-th optical image of the surface of an extraterrestrial body, SFM represents the algorithm for recovering structure from motion, Represents the total number of optical images of the surface of extraterrestrial objects.

3. The method for 3D reconstruction of extraterrestrial objects without boundaries based on Gaussian radiation field according to claim 2, characterized in that: In step 2, for the sparsely reconstructed point cloud of the scene obtained in step 1, the process of calculating the 3D coordinates of the expanded dense point cloud based on the intersection of light planes includes: The dense point cloud is expanded by combining the geometric relationship between viewpoints and the surface characteristics of extraterrestrial objects. The least squares method is applied to fit an ideal geometric plane based on the sparsely reconstructed point cloud data of the scene: ， Where n represents the normal vector of the ideal geometric plane, c represents the intercept offset of the ideal geometric plane, f represents the least squares method, and P is the sparse reconstructed point cloud of the scene obtained by the motion recovery structure algorithm in step 1; Several rays are emitted from the camera's optical center, passing through the centers of all pixels in the pixel plane. The depth information at the intersection is calculated by intersecting the rays with the scene surface. Then, the coordinate information of the expanded 3D point in dense space is obtained by simple calculation based on the basic proportional relationship of projective geometry: ， ， ， Among them, n and c are the normal vector and intercept offset of the ideal geometric plane calculated in the previous step, p represents the three-dimensional coordinates of any position on the surface of an extraterrestrial object, r is the coordinate of the light emitted by the optical center in the world coordinate system, o represents the coordinate of the camera optical center, d represents the unit direction vector of the light emitted by the optical center, t represents the depth information of the light emitted by the optical center, and the way the light intersects with the scene surface is the constraint , solve the simultaneous equations to obtain depth information, is the stability constant; The expanded dense point cloud is used to initialize a series of three-dimensional Gaussian distributions that represent the unbounded scenes of extraterrestrial objects, forming a Gaussian radiation field in space. Each Gaussian basis element in the Gaussian distribution is parameterized by a three-dimensional covariance matrix and a central mean: ， Among them, the mean is considered as the center position of the three-dimensional Gaussian basis element, Represents the three-dimensional point coordinates close to the location, covariance matrix is decomposed into a scale matrix and a direction matrix ,Right now ; Each 3D Gaussian primitive also has a color and opacity , these two properties are used for differentiable point-based Mixed, the k-th 3D Gaussian primitive in the scene contains the following five attributes: ， in, is the quaternion representing the direction matrix R, is a three-dimensional vector representing the scaling matrix S, is the center of the three-dimensional Gaussian primitive, and the position and color information of the expanded dense point cloud are used to initialize the center of the three-dimensional Gaussian primitive respectively. and color , initialize the three-dimensional vector according to the distance from each point in the dense point cloud to the nearest point around it , for quaternions and opacity Perform zero initialization; The standard Gaussian radiation field is locally approximated to the perspective camera projection through an affine transformation for rasterization rendering; this involves first transforming each 3D Gaussian primitive to the camera coordinate system through a world-to-camera coordinate transformation matrix, and then projecting it to the image plane through a local affine transformation: ， Among them, W represents the transformation matrix from the world coordinate system to the camera coordinate system, which comes from the camera pose estimated by the SFM method, and J represents the local affine transformation, which is calculated by the internal parameters of the camera. is the covariance matrix in the camera coordinate system. Skipping the last row and column will give the two-dimensional covariance matrix ,matrix Used to calculate the two-dimensional Gaussian distribution on the image plane , and will be in the subsequent point-based Use in mixtures; The standard Gaussian radiation field method sorts the 2D Gaussian distributions projected onto the image plane from front to back by depth, and then adopts a differentiable point-based Blending integrates weighted colors into the rendered image: ， in, is the final rendered pixel color, Indicates by color Gathered by the rasterizer The set of Gaussian basis elements, is the index of the sorted Gaussian basis, Defined as , as a blending weight for point-based rendering.

4. The method for 3D reconstruction of extraterrestrial objects without boundaries based on Gaussian radiation field according to claim 3, characterized in that: In step 3, for the pixel image rendering result of step 2, the depth prior information calculated by the pre-trained monocular depth estimation model is used during the optimization process to guide the distribution of Gaussian primitives in the scene so that it is accurately close to the scene surface, including: The pre-trained monocular depth estimation model is used to process the input extraterrestrial optical image sequence and calculate the depth prior information of the image sequence: ， in, represents the monocular depth estimation model, is the input extraterrestrial optical image, Represents a monocular depth estimation model Pre-trained saved breakpoint parameters, Represents a monocular depth estimation model The calculated depth prior information; The above pre-trained monocular depth estimation model The calculated depth prior information and the depth rendered by the algorithm during the training process in step 2 are processed in parallel in blocks. The monocular depth prior is used as the true depth to guide the 3D reconstruction. The depth loss is calculated as follows: ， in, Represents the total number of image blocks, M represents the size of each image block, D and They represent the depth rendered by the algorithm during training and the pre-estimated prior depth information respectively. Mean and std represent the average value and standard deviation of the pixel grayscale on each image block respectively.

5. The method for 3D reconstruction of extraterrestrial object boundaryless scenes based on Gaussian radiation field according to claim 4, characterized in that: In step 4, during the optimization process of step 3, the explicit Gaussian primitive representation and the implicit neural signed distance field representation are combined to capture the scene surface using a joint representation of explicit and implicit models, including: Based on the explicit Gaussian primitive representation of the scene in step 2, an implicit neural signed distance field is introduced. The signed distance field represents the three-dimensional scene as an implicit equation through a deep neural network, and maps the spatial position into a signed distance value from the geometric surface of the scene. The shortest distance from any three-dimensional space point to the surface is directly calculated by multi-layer perceptron fitting, and the surface of the extraterrestrial scene is extracted and reconstructed. In the implicit neural signed distance field representation, the geometric surface of the scene is reconstructed by extracting the zero level set of the signed distance field, which is mathematically expressed as: ， in, Represents the coordinates of any three-dimensional point in space, represents the implicit neural signed distance field, S is a set of all the equations satisfying A set of three-dimensional points, that is, all points in the set are exactly located on the surface of the scene; The center position of the discrete Gaussian primitives distributed in the spatial scene in step 2 First, the signed distance field is used as a priori information to guide the distribution of Gaussian primitives in space to be close to the geometric surface of the scene, that is, surface position constraints are imposed on the Gaussian primitives and the signed distance field: ， Where N represents the total number of Gaussian primitives. represents an implicit signed distance field network, is the center position of the k-th Gaussian basis element, represents the surface position constraint loss; The gradient deviation of the signed distance field is penalized according to the Eikonal equation: ， Where N represents the total number of Gaussian primitives. represents the gradient of the signed distance field, is the center position of the k-th Gaussian basis element, represents the gradient bias loss.

6. The method for 3D reconstruction of extraterrestrial object boundaryless scenes based on Gaussian radiation field according to claim 5, characterized in that: In step 5, the Gaussian opacity field is calculated on the Gaussian radiation field finally optimized in steps 3 and 4, and the 3D mesh model of the unbounded surface of the extraterrestrial object is extracted and reconstructed using the 3D mesh reconstruction method of marching tetrahedrons, including: A continuous Gaussian opacity field is constructed from discrete 3D Gaussian primitives using a sampling method along the ray direction. The specific process is as follows: First, the world coordinate system is transformed into the local coordinate system of each Gaussian primitive and the scale is normalized: ， in, is the scaling matrix of the k-th Gaussian basis, is the direction matrix of the k-th Gaussian basis, is the center position of the k-th Gaussian basis element, 、 Represent the camera optical center and light direction in the world coordinate system respectively, t is the depth along the light direction, 、 、 They represent the camera optical center, the light direction, and the coordinates of the three-dimensional point along the light direction and at a depth of t in the local coordinate system of the k-th Gaussian basis element; In the kth local coordinate system, the Gaussian value of a three-dimensional point at any depth along the direction of the light follows a one-dimensional Gaussian distribution and can be expressed as: ， For this one-dimensional Gaussian distribution, there is a closed-form solution to its maximum value. The calculation formula of the closed-form solution is: ， For the k-th Gaussian primitive, the opacity of a 3D point at any depth along the ray direction in space can be expressed as: ， The opacity of a three-dimensional point in space at a depth of t along the direction of light r is expressed as: ， in, 、 is the blending weight defined during the rendering process in step 2; Construct a spatial Gaussian opacity field by defining the opacity of any point in space by the minimum opacity value across all training views and viewing directions: ， Where x represents any point in three-dimensional space, Indicates the opacity value at that point; After constructing the spatial Gaussian opacity field, a three-dimensional cube bounding box is generated for each three-dimensional Gaussian primitive. The opacity at the center of each Gaussian primitive and the vertex of its bounding box is calculated, and the opacity value is subtracted by 0.5 as the signed distance function (SDF) value: ， Use the maximum scale of each Gaussian primitive The marching tetrahedron algorithm is used to extract the three-dimensional mesh model, taking the center of the Gaussian basis element and the scale of its bounding box vertices as the basis element, combined with the SDF value calculated according to the opacity. Finally, the zero level set of the SDF is determined by the binary search method to extract and reconstruct the three-dimensional model of the unbounded scene surface of the extraterrestrial object.

7. According to the Gaussian radiation field-based three-dimensional reconstruction method for extraterrestrial object boundless scenes, the extraterrestrial object scene representation space is discretized into voxels, and the point cloud is densely expanded only in the scene area not covered by the initial sparse point cloud.

8. A device for 3D reconstruction of extraterrestrial objects without boundaries based on Gaussian radiation field, characterized in that: include: The construction module is used to construct a Gaussian difference pyramid to extract SIFT visual features of the input image, perform feature matching and bundle adjustment through motion recovery structure, and obtain camera internal parameters, external parameters and scene sparse point cloud; The rendering module calculates the 3D coordinates of the expanded dense point cloud based on the intersection of light planes for the sparse point cloud of the scene, constructs a series of 3D Gaussian distributions representing the unbounded scene of extraterrestrial objects, forms a Gaussian radiation field in space, and combines it with a differentiable rasterization rendering method to render the pixel color in a given viewing direction; The Gaussian primitive distribution calculation module optimizes the rendering results. During the optimization process, it uses the depth prior information calculated by the pre-trained monocular depth estimation model to guide the Gaussian primitive distribution in the scene so that it is close to the scene surface. The scene surface capture module, based on the optimization process, combines explicit Gaussian primitives with implicit neural signed distance fields to capture the scene surface using a joint representation of explicit and implicit models; The three-dimensional grid model extraction module calculates the Gaussian opacity field on the Gaussian radiation field and uses the three-dimensional reconstruction method of the marching tetrahedron to extract the three-dimensional grid model of the surface of the extraterrestrial object.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 7.

Images