Illumination variation-oriented asteroid global terrain three-dimensional reconstruction method
Through adaptive Gaussian radiation field pruning and neural network optimization, combined with depth normal consistency loss and rotation variance constraints, the problems of large data volume, large calculation overhead and illumination change errors in the three-dimensional reconstruction of asteroid global terrain are solved, and efficient and accurate three-dimensional reconstruction of asteroids is achieved.
Patent Information
- Application Number
- CN202510409155.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-02
- Publication Date
- 2025-07-11
- Estimated Expiration
- 2045-04-02
AI Technical Summary
The prior art requires a large amount of data, high manual participation, long reconstruction cycle, long training time, large calculation overhead in three-dimensional reconstruction of asteroid global terrain, and it is difficult to deal with reconstruction errors caused by lighting changes.
Adaptive Gaussian radiation field pruning method is used, combined with SAM's segmentation mask and hybrid mask, and the Gaussian radiation field is optimized using depth normal consistency and rotation variance loss, and a three-dimensional model is generated in combination with the improved maneuvering tetrahedral algorithm.
The accuracy and speed of the three-dimensional reconstruction of the global terrain of asteroids is improved, noise interference is reduced, the degree of intelligence of the model is improved, and three-dimensional reconstruction under high-precision lighting changes is realized.
Smart Images

Figure CN120298613A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep space exploration, and particularly to a three-dimensional reconstruction method for the global terrain of an asteroid facing illumination changes. Background Art
[0002] Using optical images for three-dimensional reconstruction is a computer vision technology aimed at recovering the three-dimensional information of the original model from the input multi-view images. The three-dimensional reconstruction technology of the global terrain of asteroids is crucial in scientific research, helping to study the formation and evolution of the solar system and providing key support for planetary defense of the Earth. Traditional vision-based asteroid three-dimensional reconstruction methods, such as SPG and SPC, have problems such as large data requirements, high manual participation, and long reconstruction cycles. With the rapid development of asteroid deep space exploration technology, the demand for faster and more intelligent asteroid three-dimensional reconstruction is increasing.
[0003] Due to the wide application of deep learning technology, neural implicit three-dimensional reconstruction methods are used for the three-dimensional reconstruction of the global terrain of asteroids, but these methods have problems such as long training time and large computational overhead. In recent years, 3D Gaussian Splatting (3DGS) has shown excellent reconstruction speed and quality in three-dimensional modeling tasks. This method uses a series of Gaussian basis elements to efficiently represent the geometric structure and color information of the scene, and combines rasterization technology to greatly accelerate the training process.
[0004] However, the existing 3DGS methods are mainly designed for ground conventional scenes and have not been optimized for the characteristics of remote sensing images in deep space exploration tasks. In the deep space environment, asteroids are only illuminated by parallel sunlight, which is different from the situation of multiple light sources in the ground environment, resulting in a more complex and variable distribution of the reflected light intensity on the asteroid surface. In addition, the irregular shape of the asteroid causes occlusion, and the parallel incident sunlight casts shadows on its surface. As the asteroid rotates, the illumination intensity in the shadow area changes periodically, which leads to errors in surface reconstruction. Therefore, the reconstruction technology based on Gaussian splatting has great development potential in the research of three-dimensional reconstruction of the global terrain of asteroids in the deep space environment.
[0005] Regarding the research on the three-dimensional reconstruction method of the global terrain of asteroids, the deficiencies of the existing technologies are mainly manifested in the following aspects: (1) large data volume requirements, high dependence on manual intervention, low automation degree, and long reconstruction cycle; (2) long training time, large memory occupation, and large computational overhead; (3) it is difficult to model the complex illumination changes on the asteroid surface, and the reconstruction error in the shadow area with insufficient illumination is large. Summary of the Invention
[0006] The problem solved by the technology of the present invention is: to overcome the deficiencies of the prior art, provide a three-dimensional reconstruction method for the global terrain of asteroids facing illumination changes, and on the basis of the existing methods, improve the intelligence level and reconstruction speed of the model, solve the problem of large reconstruction errors caused by illumination changes, and use the optical images of asteroids taken by detectors to achieve high-precision three-dimensional reconstruction.
[0007] The technical solution of the present invention is a three-dimensional reconstruction method for the global terrain of asteroids facing illumination changes, including the following steps:
[0008] Step 1: Input the optical image of the asteroid, use the SFM method to estimate the camera pose and generate a sparse point cloud, and use the sparse point cloud to initialize the Gaussian radiation field to generate a set of initial Gaussian basis elements;
[0009] Step 2: For the set of initial Gaussian basis elements generated in Step 1, use the SAM method to output the segmentation mask of the asteroid optical image, and use the segmentation mask to prune the Gaussian radiation field;
[0010] Step 3: For the Gaussian radiation field pruned in Step 2, output the hybrid mask of the asteroid optical image, and combine the segmentation mask output by SAM in Step 2 to identify the shadow area on the asteroid surface;
[0011] Step 4: Use position encoding to process the coordinates of the camera center, use spherical harmonic coefficients to represent the initial color of the Gaussian basis elements, use hash grid to encode the coordinates of the Gaussian basis elements, and combine the angle of the incident light. Input these four into two neural networks, which are respectively used to regress the color values of the Gaussian basis elements in the illuminated and shadow areas;
[0012] Step 5: Add depth normal consistency loss, rotation variance and scale constraint loss to the loss function, and jointly supervise the optimization of the Gaussian radiation field in combination with photometric loss;
[0013] Step 6: For the final optimization result of the Gaussian radiation field in Step 5, calculate the Gaussian opacity field, and use the improved marching tetrahedron algorithm to generate a three-dimensional model of the global terrain of the asteroid.
[0014] The advantages of the present invention compared with the prior art are as follows:
[0015] (1) The present invention adopts an adaptive Gaussian radiation field pruning method, and improves the densification result of Gaussian basis elements through statistical filtering, effectively eliminating the interference of noise in the deep space environment. It also adopts depth normal consistency loss, rotation variance and scale constraint loss of Gaussian basis elements, improving the accuracy of three-dimensional reconstruction of the global terrain of asteroids.
[0016] (2) The present invention combines the segmentation mask of SAM with that of 3DGS The hybrid mask predicts the shadow areas on the asteroid surface and incorporates lighting information into the neural network to regress the color values of Gaussian basis elements. The surface model is extracted by calculating the Gaussian opacity field and the improved marching tetrahedra algorithm, improving the accuracy of 3D reconstruction of asteroids with surface lighting variations.
[0017] In summary, the method adopted in the present invention has a simple principle and can achieve the purpose of 3D reconstruction of the global terrain of asteroids using the optical images captured by the detector. Brief Description of the Drawings
[0018] Figure 1 It is a flowchart of a method for 3D reconstruction of the global terrain of an asteroid facing lighting variations according to the present invention. Detailed Embodiments
[0019] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above objectives, the present invention adopts the following technical solutions.
[0020] As Figure 1 shown, a method for 3D reconstruction of the global terrain of an asteroid facing lighting variations according to the present invention is specifically implemented as follows:
[0021] Step 1: Input the optical image of the asteroid, use the SFM method to estimate the camera pose and generate a sparse point cloud, and use the sparse point cloud to initialize the Gaussian radiation field to generate a set of initial Gaussian basis elements;
[0022] Step 2: For the set of initial Gaussian basis elements generated in Step 1, use the SAM method to output the segmentation mask of the asteroid optical image, and use the segmentation mask to prune the Gaussian radiation field;
[0023] Step 3: For the Gaussian radiation field pruned in Step 2, output the hybrid mask of the asteroid optical image, and identify the shadow areas on the asteroid surface in combination with the segmentation mask output by SAM in Step 2;
[0024] Step 4: Process the coordinates of the camera center using positional encoding, represent the initial color of the Gaussian basis element using spherical harmonic coefficients, encode the coordinates of the Gaussian basis element using a hash grid, and combine the angle of the incident light. Input these four into two neural networks, which are respectively used to regress the color values of the Gaussian basis elements in the illuminated and shadow areas;
[0025] Step 5: Add depth normal consistency loss, rotation variance, and scale constraint loss to the loss function, and jointly supervise the optimization of the Gaussian radiation field in combination with the photometric loss;
[0026] Step 6: For the final optimization result of the Gaussian radiation field in Step 5, calculate the Gaussian opacity field, and use the improved marching tetrahedra algorithm to generate a three-dimensional model of the global terrain of the asteroid.
[0027] In Step 1, first, input dozens of optical images of the asteroid taken by the detector. Use the Structure from Motion (SFM) method to estimate the pose of the moving camera from the multi-view optical images and generate a sparse point cloud of the asteroid model. Use this sparse point cloud to initialize the Gaussian radiation field.
[0028] The standard Gaussian radiation field method represents a 3D scene through a set of 3D Gaussian basis elements, which are translucent and anisotropic. Each 3D Gaussian basis element is explicitly parameterized by a 3D covariance matrix and a mean :
[0029] ,
[0030] where is defined as . Specifically, the 3D covariance matrix is decomposed into a scaling matrix and a rotation matrix for optimization. The mean can be regarded as the central position of the 3D Gaussian basis element, and is a 3D point near this position.
[0031] In addition, each 3D Gaussian basis element also has a color and an opacity , which are used for differentiable point-based blending. Therefore, the k-th 3D Gaussian basis element in the scene contains the following five attributes:
[0032] ,
[0033] where is the quaternion of the rotation matrix , is the three-dimensional vector of the scaling matrix , is the center of the 3D Gaussian basis element. Use the position and color information of the sparse point cloud to initialize the center and color of the 3D Gaussian basis element respectively, and calculate the distance from each point in the sparse point cloud to the nearest surrounding point to initialize the three-dimensional vector , in addition, the quaternion and opacity are initialized to zero.
[0034] The standard Gaussian radiation field is locally approximated by an affine transformation for perspective camera projection to achieve efficient rasterization rendering. Specifically, first, each 3D Gaussian basis element is transformed into the camera coordinate system through the world-to-camera transformation matrix , and then it is projected onto the image plane through a local affine transformation :
[0035] ,
[0036] where the transformation matrix comes from the camera pose estimated by the SFM method, and the affine transformation is calculated through the internal parameters of the camera. is the covariance matrix in the camera coordinate system, and then the 2D Gaussian covariance matrix is obtained by skipping the last row and the last column. The matrix is used to calculate on the image plane and will be used in the subsequent point-based blending.
[0037] The standard Gaussian radiation field method sorts the 2D Gaussians projected onto the image plane by depth, from front to back. Then, it uses differentiable point-based blending to integrate the weighted colors into the rendered image:
[0038] ,
[0039] where is the pixel color of the final rendering, represents the set of Gaussian basis elements aggregated by the rasterizer according to the color , is the index of the sorted Gaussian basis elements, is defined as , which serves as the blending weight for point-based rendering.
[0040] In step 2, a vision segmentation large model (SAM) is used to generate a segmentation mask for the foreground in the asteroid image:
[0041] ,
[0042] where is the l-th input RGB image, is the estimator of SAM, is the output segmentation mask from the l-th viewpoint. Pixels within the foreground are set to 1, while background pixels are set to 0. Next, the center of the 3D Gaussian basis element is projected from the three-dimensional world space to the two-dimensional image plane using the following formula:
[0043] ,
[0044] where, is the center of the k-th 3D Gaussian basis element in the scene, are the corresponding projection coordinates on the image. If lies within the foreground of the segmentation mask , the basis element is retained; otherwise, it is discarded. During the optimization process, ensure that each 3D Gaussian basis element appears in the foreground of all segmentation masks, thereby eliminating all background noise:
[0045] ,
[0046] where L is the sum of all viewpoints, represents the remaining 3D Gaussian basis elements. Directly pruning using the segmentation mask output by SAM would result in valid 3D Gaussian basis elements being wrongly removed. To retain valid 3D Gaussian basis elements, a morphological dilation method is adopted to obtain a segmentation mask with a larger foreground using the following formula:
[0047] ,
[0048] where, is the structuring element, are the pixel coordinates on the image, represents the dilation operation, represents translating the structuring element to the coordinate , is the dilated mask, generating a binary image.
[0049] Next, apply a statistical filtering method to adaptively prune 3D Gaussian basis elements with inappropriate scales and excessive dispersion. Assume there are K 3D Gaussian basis elements in the Gaussian radiation field, and calculate the mean and variance of their scales. The scale is defined as the average size of each 3D Gaussian basis element along the x, y, and z axes:
[0050] ,
[0051] ,
[0052] where, and are the sample mean and variance of the scales of the 3D Gaussian basis elements respectively, is the k-th basis element in Dimensions on the axis. For each 3D Gaussian basis element, when it is retained; otherwise, it is discarded. The thresholds and are calculated as follows:
[0053] ,
[0054] where is the adjustable coefficient of the scale variance, and can be regarded as the upper and lower limits of the scale.
[0055] Similarly, statistical filtering is applied to the positions of the 3D Gaussian basis elements. First, a neighborhood is defined for each basis element, and the H nearest basis elements are assigned to it. Then, the average distance between this basis element and all other basis elements in the neighborhood is calculated as follows:
[0056] ,
[0057] where is the distance between the k-th 3D Gaussian basis element and the h-th basis element in its neighborhood, is the new distance attribute of the k-th basis element. Next, the mean and variance of the distances of the K 3D Gaussian basis elements in the radiation field are calculated, as well as the thresholds and :
[0058] ,
[0059] ,
[0060] where and are the sample mean and variance of the distances, is the adjustable coefficient of the distance variance, and can be regarded as the adaptive thresholds of the Gaussian radiation field. For each 3D Gaussian basis element, when its distance attribute satisfies it is retained; otherwise, it is pruned to obtain the pruned Gaussian radiation field. This helps to prune the 3D Gaussian basis elements with overly scattered positions.
[0061] In step 3, for the pruned Gaussian radiation field, when the asteroid image from the training perspective is input, SAM can output a pixel-level segmentation mask. For the corresponding perspective of the input asteroid image, the pre-trained Gaussian radiation field can output a mixed mask as follows:
[0062] ,
[0063] ,
[0064] Among them, is the mixing weight defined during the rendering process of step 1. After preliminary training, the Gaussian radiation field has learned the approximate shape of the asteroid. Therefore, the mixing mask can be regarded as a mask at the instance level. By performing bitwise operations on the segmentation mask and the mixing mask the shadow region mask can be identified:
[0065] ,
[0066] Among them, is the index of the viewpoint, is 's complement. The 3D Gaussian basis elements projected onto the shadow region are marked, and a separate MLP is used later to regress their color values. For easy distinction, the MLP in the shadow region is called shadow-MLP, and the MLP for normal lighting is called lighting-MLP. The architectures of both are the same.
[0067] In step 4, although the reflectivities of the surfaces of different types of asteroids (C-type, S-type, M-type) are different, their surface illumination reflection models can be uniformly simplified to the Lambert reflection model, as follows:
[0068] ,
[0069] Among them, is the reflected light intensity, is the incident light intensity, is the diffuse reflection coefficient of the surface material, is the direction of the incident light source, is the direction of the surface normal. The incident light intensity and the surface diffuse reflection coefficient are regarded as fixed values for a given asteroid. However, since the angle between the surface normal and the incident light direction changes with the shape of the asteroid, the reflected light intensity will change. Calculate the angle between the incident light ray and each 3D Gaussian basis element and use it as one of the inputs of the MLP to regress the color:
[0070] ,
[0071] ,
[0072] Among them, is the center of the k-th 3D Gaussian basis element, is the center of the Gaussian radiation field, is the solar position calculated according to the orbit, is the angle between the k-th 3D Gaussian basis element and the incident light.
[0073] The extrinsic parameters of the camera are estimated using SFM, and the central position of the camera is used as another input to the MLP to regress the color of the 3D Gaussian basis elements. Position encoding (PE) is applied to the camera center:
[0074] ,
[0075] where is the scalar component of the camera center position coordinates. This process means mapping the low-dimensional space to the high-dimensional space .
[0076] In addition, hash encoding (HE) is performed on the multi-resolution voxel grid to represent the Gaussian radiation field. The voxels surrounding the 3D Gaussian basis elements are identified, and trilinear interpolation is performed on each basis element using the hash feature vectors at their corners. Then, the hash encodings at different resolutions are concatenated to obtain the hash feature:
[0077] ,
[0078] where R represents the multi-resolution level. At the same time, the initial color of the 3D Gaussian basis elements is represented by 16-dimensional spherical harmonic coefficients, and the spherical harmonic coefficients of each 3D Gaussian basis element are retained and optimized together with the hash feature during the training process. The angle between the incident light and the 3D Gaussian basis element, the position encoding of the camera center, the spherical harmonic coefficients, and the hash feature are respectively input into two shallow neural networks with the same structure to regress the color values of the Gaussian basis elements in the well-lit and shadow regions.
[0079] In step 5, the photometric loss is used to supervise the training of the Gaussian radiation field:
[0080] ,
[0081] where is the L1 loss between the real image and the rendered image, is the structural similarity index (SSIM) loss, is an adjustable coefficient used to balance the two losses. However, in the surface reconstruction task, the photometric loss alone lacks the necessary geometric constraints. Therefore, the depth-normal consistency loss is introduced to align the 3D Gaussian basis elements with the asteroid surface:
[0082] ,
[0083] where is the primitive index where the ray intersects along the way, is the mixing weight of 3D Gaussian primitives, is the normal of the i-th primitive, is the surface normal direction obtained by applying the finite difference method on the depth map.
[0084] Since the surface of the asteroid is usually covered by regolith composed of broken rocks and dust, the surface texture is weak and the details are small. Therefore, it is necessary to limit the size of 3D Gaussian primitives and penalize those primitives with too large scales to avoid generating overly smooth local regions. The scale constraint loss is expressed as:
[0085] ,
[0086] where, , that is, the scale of 3D Gaussian primitives is limited by the maximum size on the x, y, and z axes.
[0087] For the orientation loss of Gaussian primitives, the rotation quaternions of 3D Gaussian primitives are statistically analyzed to maximize their variance. The rotation variance loss is expressed as:
[0088] ,
[0089] where, represents the -th component of the quaternion of the k-th Gaussian primitive, is the calculation of the sample variance, which calculates the sample variance for the quaternion components of the same dimension of all Gaussian primitives.
[0090] Finally, the total loss function consists of four parts:
[0091] ,
[0092] where, represents the photometric loss, represents the depth-normal consistency loss, represents the scale constraint loss, represents the rotation variance loss, , , , are the weights of the corresponding loss functions.
[0093] Step 6, For the final optimization result of the Gaussian radiation field, calculate the Gaussian opacity field and use the improved marching tetrahedra algorithm to generate a three-dimensional model of the global terrain of the asteroid. A continuous Gaussian opacity field is constructed from discrete 3D Gaussian primitives using a ray-sampling-based rendering method. The specific process is as follows:
[0094] First, convert the world coordinate system to the local coordinate system of each primitive and normalize the scale:
[0095] ,
[0096] where ( ) are the attributes of the 3D Gaussian primitive, is the camera center, is the ray direction. Any point along the ray can be expressed as , where is the depth of the ray. After coordinate transformation, ( ) are the attributes in the corresponding Gaussian coordinate system.
[0097] In this local coordinate system, the Gaussian value of any point along the ray becomes a one-dimensional Gaussian and can be expressed as:
[0098] ,
[0099] For this one-dimensional Gaussian, the solution of its maximum value has a closed-form expression, and the calculation formula is as follows:
[0100] ,
[0101] For a single Gaussian primitive, the opacity of any point along the ray in space can be expressed as:
[0102] ,
[0103] This means that the opacity along the ray increases monotonically and remains constant after reaching the maximum value. Therefore, given a set of 3D Gaussian primitives, the opacity of any point along the ray can be expressed as:
[0104] ,
[0105] where is the blending weight defined in the rendering process of step 1. Since any point in space may be visible from any training view, the opacity of any point in space can be defined by the minimum opacity value of all training views and observation directions:
[0106] ,
[0107] After constructing the Gaussian opacity field, a 3D bounding box can be generated for the position of each 3D Gaussian primitive. Then, calculate the opacity at the center of each primitive and at the vertices of its bounding box, and subtract 0.5 from the opacity value as the signed distance function (SDF) value:
[0108] ,
[0109] Use the maximum scale of each primitive As the scale of the primitive center and the vertices of its bounding box, combined with the SDF value, the marching tetrahedron algorithm is used to extract the mesh. Finally, the binary search method is used to determine the zero level set, and the three-dimensional model of the global terrain of the asteroid is extracted.
Claims
1. A three-dimensional reconstruction method for the global terrain of asteroids facing illumination changes, characterized in that, The specific implementation steps are as follows: Step 1: Input the optical image of the asteroid, use the SFM method to estimate the camera pose and generate a sparse point cloud, and use this sparse point cloud to initialize the Gaussian radiation field to generate a set of initial Gaussian basis elements; Step 2: For the set of initial Gaussian basis elements generated in Step 1, use the SAM method to output the segmentation mask of the asteroid optical image, and use the segmentation mask to prune the Gaussian radiation field; Step 3: For the Gaussian radiation field after pruning in Step 2, output the hybrid mask, and identify the shadow area on the asteroid surface in combination with the segmentation mask output by SAM in Step 2; Step 4: Process the coordinates of the camera center using position encoding, represent the initial color of the Gaussian basis element using spherical harmonic coefficients, encode the coordinates of the Gaussian basis element using a hash grid, and combine the angle of the incident light. Input these four into two neural networks respectively, which are used to regress the color values of the Gaussian basis elements in the illuminated and shadow regions; Step 5: Add the depth normal consistency loss, rotation variance and scale constraint loss to the loss function, and jointly supervise the optimization of the Gaussian radiation field in combination with the photometric loss; Step 6: For the final optimized result of the Gaussian radiation field in Step 5, calculate the Gaussian opacity field, and use the improved marching tetrahedra algorithm to generate a three-dimensional model of the global terrain of the asteroid.
2. The three-dimensional reconstruction method of the global terrain of an asteroid facing light changes according to claim 1, wherein, In Step 1, first, input the optical image of the asteroid taken by the detector, use the Structure from Motion (SFM) method to estimate the pose of the moving camera from multi-view optical images and generate a sparse point cloud of the asteroid model, and use this sparse point cloud to initialize the Gaussian radiation field; The standard Gaussian radiation field method represents 3D scenes through a set of 3D Gaussian basis elements, which are translucent and anisotropic. Each 3D Gaussian basis element is explicitly parameterized by a 3D covariance matrix and a mean : , Among them, is defined as , where the 3D covariance matrix is decomposed into a scaling matrix and a rotation matrix , the mean is regarded as the central position of the 3D Gaussian basis element, is a 3D point near this position; each 3D Gaussian basis element also has a color and opacity , and these two attributes are used for differentiable point-based blending. The k-th 3D Gaussian basis element in the scene contains the following five attributes: , Among them, is the rotation matrix in quaternion form, is the scaling matrix in three-dimensional vector form, is the center of the 3D Gaussian basis element. The centers of the 3D Gaussian basis elements are initialized using the position and color information of the sparse point cloud and color respectively. The three-dimensional vector is initialized by calculating the distance from each point in the sparse point cloud to the nearest surrounding point. Additionally, the quaternion and opacity are initialized to zero; The standard Gaussian radiation field is locally approximated by an affine transformation for perspective camera projection to achieve efficient rasterization rendering; including, first, through the world-to-camera transformation matrix transform each 3D Gaussian basis element into the camera coordinate system, and then through a local affine transformation project it onto the image plane: , Among them, the transformation matrix comes from the camera pose estimated by the SFM method, and the affine transformation is calculated through the internal parameters of the camera. is the covariance matrix in the camera coordinate system, and then a 2D Gaussian covariance matrix is obtained by skipping the last row and the last column. , the matrix is used to calculate on the image plane and will be used in the subsequent point-based mixing; The standard Gaussian radiation field method sorts 2D Gaussians projected onto the image plane by depth, from front to back. Then, it uses a differentiable point-based blending to integrate the weighted colors into the rendered image: , Among them, is the pixel color of the final rendering, represents the set of Gaussian basis elements aggregated by the rasterizer according to the color , is the index of the sorted Gaussian basis elements, is defined as , which serves as the blending weight for point-based rendering.
3. A three-dimensional reconstruction method for the global terrain of asteroids facing light changes according to claim 2, characterized in that, In Step 2, use the large vision segmentation model SAM to generate the segmentation mask of the foreground in the asteroid image: , Among them, is the l-th input RGB image, is the estimator of SAM, is the output segmentation mask from the l-th viewpoint. Pixels within the foreground are set to 1, and background pixels are set to 0. Next, the centers of the 3D Gaussian basis elements are projected from the three-dimensional world space to the two-dimensional image plane using the following formula: , Among them, is the center of the k-th 3D Gaussian primitive in the scene, is the corresponding projection coordinate on the image. If is located within the foreground of the segmentation mask then retain the primitive; otherwise, discard it. During the optimization process, ensure that each 3D Gaussian primitive appears in the foreground of all segmentation masks, thereby eliminating all background noise: , where L is the sum of all viewpoints, represents the remaining 3D Gaussian basis elements; the morphological dilation method is used to dilate the 3D Gaussian basis elements to obtain a segmentation mask with a larger foreground, and the formula is as follows: , Among them, is a structural element, is the pixel coordinate on the image, represents the dilation operation, means translating the structural element to the coordinate , is the dilated mask, generating a binary image; Next, apply a statistical filtering method to adaptively prune 3D Gaussian basis elements with inappropriate scales and excessive dispersion; including: assuming there are K 3D Gaussian basis elements in the Gaussian radiation field, calculate the mean and variance of their scales. The scale is defined as the average size of each 3D Gaussian basis element on the x, y, and z axes: , , wherein, and are the sample mean and variance of the 3D Gaussian basis element scale respectively, is the size of the k-th basis element on the axis. For each 3D Gaussian basis element, when it is retained, otherwise it is discarded; Use statistical filtering on the positions of the 3D Gaussian basis elements. First, define a neighborhood for each basis element and assign the nearest H basis elements to it. Then, calculate the average distance between this basis element and all other basis elements in the neighborhood as follows: , Among them, is the distance between the k-th 3D Gaussian basis element and the h-th basis element in its neighborhood, is the new distance attribute of the k-th basis element. Next, calculate the distance mean, variance, and threshold and for the K 3D Gaussian basis elements in the radiation field. For each 3D Gaussian basis element, when its distance attribute satisfies , it is retained; otherwise, it is pruned to obtain the pruned Gaussian radiation field.
4. A three-dimensional reconstruction method for the global terrain of an asteroid facing illumination changes according to claim 3, characterized in that, In step 3, for the pruned Gaussian radiation field, input the asteroid image from the training perspective, the SAM outputs a pixel-level segmentation mask, input the corresponding perspective of the asteroid image, and the preliminarily trained Gaussian radiation field can output a mixed mask , as follows: , , Among them, is the mixing weight defined during the rendering process of step 1. After preliminary training, the Gaussian radiation field has learned the approximate shape of the asteroid. Therefore, the mixed mask is regarded as an instance-level mask. By performing bitwise operations on the segmentation mask and the mixed mask the shadow region mask is identified: , Among them, is the index of the viewpoint, is the complement set of, marking the 3D Gaussian basis elements projected onto the shadow area, and using a separate MLP to regress their color values subsequently. For easy distinction, the MLP in the shadow area is called shadow-MLP, while the MLP under normal illumination is called illumination-MLP, and the architectures of both are the same.
5. A three-dimensional reconstruction method of the global terrain of an asteroid facing light changes according to claim 4, characterized in that, In Step 4, the surface illumination reflection model of different types of asteroids is the Lambert reflection model, and the formula is as follows: , Among them, is the reflected light intensity, is the incident light intensity, is the diffuse reflection coefficient of the surface material, is the direction of the incident light source, is the direction of the surface normal. Regarding the incident light intensity and the surface diffuse reflection coefficient as fixed values of a given asteroid, calculate the angle between the incident light and each 3D Gaussian basis element, and use it as one of the inputs of the MLP to regress the color: , , Among them, is the center of the k-th 3D Gaussian basis element, is the center of the Gaussian radiation field, is the solar position calculated according to the orbit, is the angle between the k-th 3D Gaussian basis element and the incident light; Use SFM to estimate the external parameters of the camera, use the center position of the camera as another input of the MLP to regress the color of the 3D Gaussian basis element, and apply position encoding PE to the camera center: , Among them, is the scalar component of the camera center position coordinates, and this process means mapping the low-dimensional space to the high-dimensional space ; Hash encoding HE is performed on the multi-resolution voxel grid to represent the Gaussian radiation field, the voxels surrounding the 3D Gaussian primitive are identified, and trilinear interpolation is performed on each primitive using the hash feature vectors at their corners. Then, the hash encodings at different resolutions are concatenated to obtain the hash feature: , Among them, R represents the multi-resolution level. Use spherical harmonic coefficients to represent the initial color of the 3D Gaussian basis element. The spherical harmonic coefficients of each 3D Gaussian basis element are reserved and optimized together with the hash features during the training process. Input the angle between the incident light and the 3D Gaussian basis element, the position encoding of the camera center, spherical harmonic coefficients, and hash features into two shallow neural networks with the same structure respectively, which are used to regress the color values of the Gaussian basis elements in the illuminated and shadow regions.
6. A three-dimensional reconstruction method for the global terrain of asteroids facing light changes according to claim 5, characterized in that, In step 5, the photometric loss is used to supervise the training of the Gaussian radiation field: , wherein, is the L1 loss between the real image and the rendered image, is the structural similarity index loss, is an adjustable coefficient for balancing the two losses, and a depth-normal consistency loss is introduced to align the 3D Gaussian basis elements with the asteroid surface: , Among them, is the primitive index where the ray intersects along the way, is the mixing weight of the 3D Gaussian primitive, is the normal of the i-th primitive, is the surface normal direction obtained by applying the finite difference method on the depth map; The scale constraint loss is expressed as: , Among them, , that is, the scale of the 3D Gaussian basis element is restricted by the maximum dimensions on the x, y, and z axes; For the direction loss of the Gaussian basis element, statistically calculate the rotation quaternion of the 3D Gaussian basis element and maximize their variance. The rotation variance loss is expressed as: , Among them, represents the -th component of the quaternion representing the k-th Gaussian basis element, is the calculation of the sample variance, which calculates the sample variance for the quaternion components of the same dimension of all Gaussian basis elements; Finally, the total loss function includes four parts: , wherein, represents photometric loss, represents depth-normal consistency loss, represents scale constraint loss, represents rotation variance loss, , , , are the weights of the corresponding loss functions.
7. A three-dimensional reconstruction method for the global terrain of an asteroid facing illumination changes according to claim 6, characterized in that In Step 6, use the ray-sampling-based rendering method to construct a continuous Gaussian opacity field from discrete 3D Gaussian basis elements. The specific process is as follows: First, convert the world coordinate system to the local coordinate system of each primitive and normalize the scale: , Among them, ( ) is the property of the 3D Gaussian basis element, is the camera center, is the ray direction, and any point along the ray can be expressed as , where is the depth of the ray. After coordinate transformation, ( ) is the property in the corresponding Gaussian coordinate system; In this local coordinate system, the Gaussian value at any point along the ray becomes a one-dimensional Gaussian, expressed as: , For this one-dimensional Gaussian, the solution for its maximum value has a closed-form expression, and the calculation formula is as follows: , For a single Gaussian primitive, the opacity at any point along the ray in space is expressed as: , Given a set of 3D Gaussian primitives, the opacity at any point along the ray can be expressed as: , Among them, is the blending weight defined during the rendering process of step 1, and the opacity of any point in space is defined by the minimum opacity value of all training views and viewing directions: , After constructing the Gaussian opacity field, for each 3D Gaussian basis element's position, generate a 3D bounding box. Then, calculate the opacity at the center of each basis element and at the vertices of its bounding box, and subtract 0.5 from the opacity value as the signed distance function (SDF) value: , Using the maximum scale of each primitive As the primitive center and the scale of the vertices of its bounding box, combined with the SDF value, the marching tetrahedron algorithm is used to extract the mesh; finally, the binary search method is used to determine the zero level set, and the three-dimensional model of the global terrain of the asteroid is extracted.
8. A three-dimensional reconstruction method for the global terrain of an asteroid facing light changes according to claim 3, characterized in that Threshold and are calculated as follows: , Among them, is an adjustable coefficient of the scale variance, and can be regarded as the upper and lower limits of the scale.
9. The three-dimensional reconstruction method of the global terrain of an asteroid facing illumination changes according to claim 3, characterized in that Threshold value and are calculated as follows: , , Among them, and are the sample mean and variance of the distance, is the adjustable coefficient of the distance variance, and can be regarded as the adaptive threshold of the Gaussian radiation field.
10. A three-dimensional reconstruction method for the global terrain of an asteroid facing light changes according to claim 3, characterized in that Different types of asteroids include C-type, S-type, and M-type.
Citation Information
Patent Citations
Urban three-dimensional scene reconstruction method based on satellite images
CN118736135A
Asteroid surface fine three-dimensional reconstruction method based on 3D Gaussian
CN118864767A
Gaussian radiation field three-dimensional reconstruction method based on pre-training optical flow model geometric distillation
CN118887346A
Method and system for semantic-driven intelligent reconstruction of large-scene sparse light field
US20240078744A1
Dynamic scene three-dimensional reconstruction method and system based on multi-scale space-time coding
WO2024007181A1
Cited By
Panoramic image three-dimensional reconstruction method based on 3DGS
CN120526066A
Power grid digital space modeling and inverse rendering method based on three-dimensional Gaussian
CN121170158A
Three-dimensional gaussian-based power grid digital space modeling and inverse rendering method
CN121170158B