Asteroid surface fine vision three-dimensional reconstruction method fusing laser radar data

Abstract

The invention relates to an asteroid surface fine vision three-dimensional reconstruction method fusing laser radar data. The method comprises the following steps: collecting and sorting visual camera data and laser radar point cloud data; obtaining sparse point cloud for the filtered visual camera data, and initializing a 3D Gaussian grid center by using the sparse point cloud; generating an asteroid instance segmentation mask by combining a visual camera image with sparse point cloud projection point prompt; gaussian volume rendering: optimizing the difference value between the rendered image and the true-value visual camera image; the generated instance segmentation mask is subjected to background Gaussian rejection based on cumulative transparency, Gaussian surface geometric regularization is realized by applying Gaussian depth rasterization, and a rasterization depth value is optimized by taking laser radar point cloud data as supervision. According to the invention, the method can achieve the precise three-dimensional reconstruction of the surface of the asteroid in combination with the laser radar point cloud data under the condition of global or local visual imaging, reduces the manual participation, shortens the reconstruction period of the asteroid, and improves the precision of the three-dimensional reconstruction of the surface of the asteroid.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction, and specifically relates to a fine visual three-dimensional reconstruction method for asteroid surfaces that integrates lidar data. Background Art

[0002] Three-dimensional reconstruction based on visual images is a computer graphics and computer vision technology that aims to reconstruct the three-dimensional information of the original model from the input single-view or multi-view images and other data. The fine three-dimensional reconstruction technology for asteroid surfaces is of great significance in deep space exploration tasks such as asteroid landing and sampling. On the one hand, it helps to analyze key physical parameters such as the gravitational field of asteroids; on the other hand, it can provide accurate guidance for the navigation, landing, and sampling tasks of detectors.

[0003] Currently, the classic three-dimensional reconstruction algorithms for asteroid surfaces are mainly stereo photogrammetry (SPG) and stereo photometric mapping (SPC), etc. Among them, stereo photogrammetry uses geometric principles to perform three-dimensional reconstruction through the parallax information of multi-view images. Stereo photometric mapping mainly relies on the scattering characteristics of the target surface. This method calculates the surface slope using the brightness information in the image to reconstruct the terrain. During the reconstruction process of stereo photometric mapping, the global model is divided into multiple local regions, and then photometric mapping is used to estimate the local pixel-level slope, and finally merged into a complete global model. These two methods mainly rely on the extraction and matching of feature points, and are limited by factors such as lighting changes and the accuracy of prior estimates. To achieve a better reconstruction effect, a large amount of data is required, and the reconstruction effect in areas with insufficient observations is limited. The above two methods only use visual information, and it is difficult to apply lidar data, which is one of the other important data sources for asteroid detection, in these two methods. It is usually only used for verifying the reconstruction results. Moreover, traditional methods involve a large amount of manual interaction and have a long reconstruction cycle. With the rapid development of the deep space exploration and remote sensing mapping fields, the demand for faster and more intelligent three-dimensional reconstruction of asteroids is increasing.

[0004] With the development and gradual maturity of deep learning methods in computer vision and computer graphics, in recent years, scene reconstruction methods based on Neural Radiance Fields (NeRF) have been used for novel view synthesis based on ray tracing and volume rendering principles, which have the advantages of high rendering quality, realistic visual effects, and good continuity. Subsequently, in response to the slow ray tracing process and long training time of NeRF, the academic community proposed an explicit reconstruction method based on 3D Gaussian radiance fields, expressing the scene as a series of Gaussian basis elements, which greatly improved the rendering efficiency and training efficiency. Advanced neural differentiable rendering methods represented by NeRF and 3D Gaussian radiance fields provide new approaches for 3D reconstruction. Although significant progress has been made in 3D reconstruction technology, there are still challenges in some applications. For example, inaccuracies may occur when dealing with complex scenes, motion blur, or low-quality images, and the fusion methods with multi-modal data such as lidar are not yet mature. Therefore, the improvement and optimization of 3D reconstruction technology are still of great significance, especially in the fields of small celestial body exploration and remote sensing topographic mapping.

[0005] Regarding the research on the fine visual 3D reconstruction method of asteroid surfaces that integrates lidar information, the deficiencies of existing technologies are mainly manifested in the following aspects: (1) There are problems of single and repetitive texture features in asteroid visual images. Traditional methods rely on feature point extraction and matching, usually requiring a large amount of data, and the reconstruction effect in insufficient observation areas is limited; (2) Using complex multi-stage reconstruction processes, with low automation and long reconstruction time; (3) Existing traditional methods have difficulties in integrating lidar point cloud data in small celestial body reconstruction. Directly applying inaccurate point cloud depth values for supervision during reconstruction will damage the surface geometry and it is difficult to improve the reconstruction quality; (4) Existing 3D Gaussian radiance field methods have problems such as insufficient model expressiveness and obvious redundant noise in reconstruction, resulting in poor results and low reconstruction accuracy. Summary of the Invention

[0006] The technical problem solved by the present invention is: to overcome the deficiencies of existing technologies and provide a fine visual 3D reconstruction method for asteroid surfaces that integrates multi-modal lidar data. On the basis of existing methods, it improves the intelligence level and reconstruction speed of the model, while maintaining high reconstruction accuracy, realizes high-precision 3D reconstruction from input images with limited viewpoints and high-quality rendering of images from arbitrary viewpoints, and provides a conversion method from Gaussian representation to mesh representation.

[0007] The technical solution of the present invention is a fine visual 3D reconstruction method for asteroid surfaces that integrates lidar data, including the following steps:

[0008] Step (1): Collect and organize the visual camera data and lidar point cloud data obtained by the asteroid detector, and apply histogram-based statistical filtering to the deep space images to filter out the interference of bright background noise.

[0009] In step (2), for the visual camera data filtered in step (1), the Structure from Motion (SFM) method is used to obtain a sparse point cloud on the asteroid surface, and the center of the 3D Gaussian grid is initialized using the sparse point cloud.

[0010] In step (3), for the visual camera images in step (1), a large visual semantic segmentation model is applied, and combined with the projection point hint of the sparse point cloud, an instance segmentation mask of the deep space asteroid is generated.

[0011] In step (4), for the initialization result in step (2), the Gaussian sputtering rasterization method is applied to achieve Gaussian efficient volume rendering, and the backpropagation method is used to optimize the difference between the rendered image and the true visual camera image.

[0012] In step (5), for the optimization process in step (4), the instance segmentation mask generated in step (3) is used to perform background Gaussian elimination based on cumulative transparency, and Gaussian depth rasterization is applied to achieve surface geometry regularization of the Gaussian. With the lidar point cloud input as supervision, the rasterization depth is optimized to improve the reconstruction quality.

[0013] In step (6), for the optimized result in step (5), the triangulation method is used to initialize the distance field tetrahedral mesh based on the Gaussian center point, the ray tracing method is used to generate the distance field on the mesh, and then the fine mesh model of the asteroid surface is obtained by tracing the zero level set of the distance field.

[0014] The advantages of the present invention compared with the prior art are as follows:

[0015] (1) The present invention adopts an explicit scene representation method of 3D Gaussian based on grid aggregation, allowing the Gaussian to carry feature vectors, enhancing the expression ability of 3D Gaussian for scene details; pixel-by-pixel dense supervision avoids the extraction and matching of feature points in the dense reconstruction stage, adapts to the characteristics of single and repetitive asteroid texture features, has high data utilization rate, and still has a good reconstruction effect in areas with insufficient observations. The algorithm realizes end-to-end three-dimensional reconstruction of asteroids, improves the automation degree, reduces the manual participation degree, and enhances the reconstruction speed on the basis of maintaining the same reconstruction accuracy as traditional methods.

[0016] (2) The present invention is based on the depth rasterization method and the depth regularization method based on order relationship, realizes the effective utilization of lidar point cloud data, and improves the reconstruction accuracy. The application of cumulative transparency elimination based on semantic segmentation visual mask effectively eliminates the adverse effects of invalid Gaussians in the deep space background on the reconstruction geometric structure and grid extraction, and significantly improves the reconstruction effect of the 3D Gaussian radiation field method for single foreground objects with boundaries.

[0017] In summary, the method principle adopted by the present invention is simple, and it can achieve the purpose of fine visual three-dimensional reconstruction of the asteroid surface by fusing lidar data from a global or local perspective. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a method for fine visual three-dimensional reconstruction of the asteroid surface by fusing lidar data according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] As Figure 1 shown, a method for fine visual three-dimensional reconstruction of the asteroid surface by fusing lidar data according to the present invention is specifically implemented as follows:

[0021] Step (1): In the deep space image, the pixel gray values in the asteroid target area are relatively scattered, and the bright background noise often concentrates near the same gray value. First, the overall gray histogram of the visual camera image is statistically analyzed, and each gray level is divided into an interval, and a total of intervals are divided. The mean and variance of all pixel gray values are calculated:

[0022] ;

[0023] ;

[0024] wherein, represents the gray value of the th pixel in the th interval. Then, the mean and variance of the gray level in each interval are calculated, and the threshold for bright background adaptive filtering is calculated:

[0025] ;

[0026] ;

[0027] ;

[0028] The threshold It is calculated by weighting the mean and variance of the overall pixel grayscale. and are the mean and variance weights respectively. Intervals with variance greater than the threshold are treated as bright background noise in the deep space image. When the variance in the th interval is greater than the threshold , all pixel grayscale values in the interval are set to 0 to filter out the interference of bright background noise; finally, histogram equalization is performed on the filtered image to approximate the brightness and contrast of the grayscale histogram, alleviating the problems caused by the illumination change of the deep space image under different perspectives.

[0029] Step (2): After preprocessing the visual camera image, use the classical Structure from Motion (SFM) method based on feature point matching to estimate the initial sparse point cloud, and determine the pose relationship between different perspectives of the visual camera based on the epipolar geometry relationship and bundle adjustment optimization. The initial sparse point cloud is denoted as , where is the number of point clouds, is the three-dimensional coordinate of the th point in the world coordinate system. Use the point cloud to initialize the grid Gaussian. The Gaussian grid is a square grid with side length . Denote the set of initialized grid center coordinates as the result of removing duplicates of the center coordinates of the grid cells where the point cloud is located. The coordinate set is calculated as follows:

[0030] ;

[0031] Each grid cell carries a feature vector with dimension for generating Gaussians. Denote the center coordinate of the th grid cell. The offset of the mean of the th Gaussian generated is , the grid feature vector , and the camera observation direction vector . Then the parameters of the Gaussian generated by this grid cell and its color , transparency are calculated as follows:

[0032] ;

[0033] where are all MLP multi-layer perceptrons for decoding Gaussian parameters from the grid feature vector. It is a conversion function from quaternion to rotation matrix. By introducing grid representation and eigenvector decoding process, it overcomes the problem of insufficient expressive power of spherical harmonics in the naive 3D Gaussian radiation field method, while exploiting the spatial correlation of Gaussians, avoiding the storage overhead of per-Gaussian eigenvectors, and enhancing the view generalization ability.

[0034] Step (3): To effectively eliminate invalid Gaussians in the deep space background and remove artifacts such as floating objects, a visual semantic segmentation large model prompted by reference points is used to generate a target mask for the visual camera image, indicating the area where the target asteroid exists in the image. Denote the coordinates of the initial sparse point cloud in the camera coordinate system at the viewpoint as , where is the number of point clouds, is the three-dimensional coordinate of the th point in the world coordinate system. Use the visual large model to generate the asteroid segmentation mask:

[0035] ;

[0036] where, is the th visual camera image, is denoted as the projection position of the point cloud coordinates in the th visual camera image in the image coordinate system, is the camera focal length. Since the visual segmentation edge is not completely accurate, to avoid erroneously eliminating Gaussians at the edge, a morphological operator is used to dilate the mask .

[0037] Step (4): The 3D reconstruction method based on 3D Gaussian radiation field represents the scene and objects as a collection of a large number of 3D Gaussian ellipsoids, uses information such as the pixel color of the visual camera image to supervise and optimize the scene representation, and uses backpropagation method and gradient-based optimization method to optimize the eigenvectors, offsets, and parameters of the decoding MLP stored on the grid, so as to achieve an efficient and accurate scene representation.

[0038] In the 3D Gaussian radiation field method, for any th 3D Gaussian , its geometric shape and color information are parameterized by the mean , covariance , opacity , and color :

[0039] ;

[0040] The final pixel color is calculated by the volume rendering formula:

[0041] ;

[0042] Among them, represents the Gaussian set covering a certain pixel, represents the opacity of the th Gaussian on the current pixel multiplied by the density of the projected 2D Gaussian distribution at the pixel position, which can be directly calculated according to the pixel coordinates by ; represents the calculated color. To alleviate the attenuation problem of the Gaussian in the high-frequency domain, this project truncates the Gaussian kernel, destroying the continuity of the derivative of the Gaussian kernel function. The actual form of the Gaussian kernel used is as follows:

[0043] ;

[0044] Among them, the hyperparameter is the truncation coefficient;

[0045] Denote the image rendered from the perspective as , the ground truth of the visual camera image is , and optimize the mesh parameters and decoder parameters based on the following loss function:

[0046] ;

[0047] is the weight coefficient of the structural similarity index SSIM loss function.

[0048] Step (5), during the optimization process of the Gaussian radiation field, to avoid the appearance of invalid Gaussians in the deep space pure black background, which affects the view generalization and geometric extraction, use the cumulative depth regularization of the segmentation mask extracted in step (3) to eliminate the invalid Gaussians. The specific operation is to calculate the cumulative transparency at the pixel and apply the following cumulative opacity penalty term.

[0049] During the rendering process in step (4), the cumulative transparency at the pixel is calculated by the transparency of all Gaussians covering this pixel. Denote the opacity of all Gaussians contributing to the color of the ray on the ray from the optical center to the pixel as , then the cumulative opacity is denoted as:

[0050] ;

[0051] Then the cumulative transparency penalty term under the th visual camera view is as follows:

[0052] ;

[0053] Apply a transparency penalty to the Gaussian in the deep - space background area of the mask. Under the action of this penalty term, the opacity of the Gaussian appearing in the background area will continue to decrease, and the Gaussian at the edge will continue to approach the object area, realizing the elimination of ineffective Gaussians in the deep - space background area;

[0054] In the standard 3D Gaussian radiation field model, since only visual input is used as supervision, the distribution of Gaussians only meets the requirements of visual input rendering quality and does not guarantee the formation of a geometrically correct object surface representation, which is not conducive to the mesh extraction process. During the optimization process, in order to alleviate the problems that 3D Gaussians are difficult to align with the object surface and the surface distribution is not compact, and to ensure that Gaussians form a correct geometric surface during the reconstruction process, avoiding problems such as broken surfaces, floating artifacts, and non - manifold surfaces, a surface regularization strategy based on depth and normal vector rasterization is introduced on the basis of color loss;

[0055] The geometric representation of the 3D Gaussian radiation field is different from traditional representation methods such as meshes and voxels. It is difficult to define the location of the real substance of the object, and thus it is difficult to define concepts such as depth and normal vector in traditional computer graphics. In order to obtain reasonable depth and normal vector in the Gaussian model, it is stipulated that the ray from the optical center to the pixel first contacts the object's material reality at the point where the probability density of the ray in the contacted Gaussian first reaches the maximum, and the depth of this point is defined as the rasterization depth of the Gaussian. Given a camera center and a ray direction , the pixel corresponding to the ray , any point along the ray can be written as , where is the depth of the ray; First, move the camera center to the local coordinate system of the 3D Gaussian, and normalize the camera center and the ray direction by rotation and scale factor to obtain the camera center , ray direction and three - dimensional point coordinates in the local coordinate system of this 3D Gaussian:

[0056] ;

[0057] where, represents element - wise multiplication. In this local coordinate system, the 3D Gaussian along the ray becomes a 1D Gaussian, which is described as follows:

[0058] ;

[0059] Then, calculating the 3D Gaussian along the ray will generate a 1D Gaussian, which has a closed - form solution when solving for the maximum value:

[0060] ;

[0061] Ray depth Has reference frame invariance, so it is considered that the ray intersects with the Gaussian material at the point:

[0062] ;

[0063] Therefore, the corresponding pixel Depth value:

[0064] ;

[0065] And for the pixel The normal vector can be calculated by solving the partial derivative of the depth, as shown in the following formula:

[0066] ;

[0067] In actual implementation, the calculation of depth and normal vector can be efficiently carried out through 2D Gaussian information and projection Jacobian matrix. Next, the median depth of 3D Gaussian is defined separately, that is, the depth aggregation between Gaussians is calculated in the way of color mixing, and the specific form is as follows:

[0068] ;

[0069] Among them, Represents the mean depth value of the th Gaussian successively passed through along the ray from the optical center to infinity, Represents the opacity contribution of the th Gaussian successively passed through along the ray from the optical center to infinity during the volume rendering process. This depth is actually a depth image obtained by using the depth of the Gaussian mean as the color according to the volume rendering method, reflecting the depth weighted average of each Gaussian along the ray according to the transparency;

[0070] The form of the loss term for surface regularization is as follows:

[0071] ;

[0072] The specific function is to squeeze the opaque Gaussians on the ray to the surface to form a dense and continuous geometric surface structure, which is conducive to the extraction of the surface mesh;

[0073] The depth obtained by rasterization rendering is also used to supervise the depth of the lidar point cloud. Due to instrument errors, motion compensation errors, etc., the lidar point cloud inevitably has inaccuracies. Directly applying inaccurate depth values for supervision will damage the surface geometry. To alleviate this problem, the project further introduces a more robust regularization based on depth order relationship. The viewing angle The depth map obtained by rasterizing the medium Gaussian radiation field model is denoted as , and it is cut into a series of slices of size . The lidar point cloud data in the camera coordinate system at the viewing angle is denoted as , where is the number of point clouds, is the th three-dimensional coordinate of the point. The three-dimensional points of the point cloud existing in the th slice are denoted as , where is the number of point clouds existing in the th slice, . Randomly generate a permutation of , then the regularization loss of the depth order relationship within this slice is as follows:

[0074] ;

[0075] where is the sign function, is the ReLU activation function. The meaning is that when the depth size relationship between the point pair and in the patch does not match the depth size relationship between and , a non-zero penalty term is generated, and the surface depth size relationship within the slice is adjusted during optimization.

[0076] Step (6), after the optimization process is completed, use the Gaussian surface tracking method based on ray tracing for mesh extraction. First, initialize the tetrahedral mesh structure. For the th Gaussian , the three orthogonal eigenvectors of its covariance matrix are denoted as , and the eigenvalues are denoted as Initialize the mean as the center, the length, width, and height are parallel to the three eigenvectors, and the size is . Take the nine points including its eight vertices and the center point as the candidate tetrahedral mesh vertices. Collect the candidate mesh vertices of all Gaussians and generate a regular tetrahedral mesh on the subsequent mesh vertices using Delaunay triangulation for carrying the surface distance field;

[0077] Subsequently, use the ray tracing method to calculate the distance from each candidate vertex to the Gaussian surface. For the th camera viewing angle, generate a line segment connecting the given camera center and the candidate mesh vertex , and the line segment direction , the points on the line segment are represented as . Calculate the ray depths of all Gaussians along the ray according to the content described in step (4) , the th Gaussian 's cumulative contribution on the ray is defined as:

[0078] ;

[0079] Finally, calculate the sum of cumulative contributions along the camera ray, and obtain according to the volume rendering formula:

[0080] ;

[0081] Calculate the minimum sum of cumulative contributions for all viewpoints, and use it as the signed distance of the candidate grid vertices , to form the signed distance field on the grid M:

[0082] ;

[0083] Use the signed distance field on the grid to calculate the zero level set, obtain the vertex set on the edges in the tetrahedral grid, and obtain the rough grid extraction result. For the vertex set on the edges, repeat the above method to obtain the signed distance field distance values of the vertex set , and use the binary search method to determine the exact vertex set , as the refined grid extraction result, to obtain the fine three-dimensional model of the asteroid surface.

[0084] It can be seen that the present invention can effectively fuse lidar point cloud data under imaging conditions with medium and short distances and relatively rich target textures, and use the explicit scene representation method of 3D Gaussian radiation field to achieve end-to-end fine three-dimensional reconstruction of the asteroid surface, avoiding feature matching in the dense reconstruction stage, and still having an observable reconstruction effect in areas with insufficient observation. While maintaining the same accuracy as traditional asteroid three-dimensional reconstruction, it significantly shortens the reconstruction cycle, reduces the degree of manual participation, and realizes fine visual three-dimensional reconstruction of the asteroid surface by fusing lidar data.

[0085] The content not described in detail in the specification of the present invention belongs to the well-known technology of those skilled in the art. Although the above description of the illustrative specific embodiments of the present invention is provided for the convenience of those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are within the scope of protection.

Claims

1. A method for fine visual 3D reconstruction of an asteroid surface by integrating lidar data, characterized in that: The following steps are involved: Step (1) collect and organize the visual camera data and lidar point cloud data acquired by the asteroid probe, apply histogram-based statistical filtering to the deep space image, and filter out the interference of bright background noise; Step (2) for the visual camera data filtered in step (1), a sparse point cloud of the asteroid surface is obtained using the structure from motion (SFM) method, and the sparse point cloud is used to initialize the center of the 3D Gaussian grid; Step (3) for the visual camera image of step (1), apply the visual semantic segmentation model and combine the sparse point cloud projection point prompts to generate the asteroid instance segmentation mask; Step (4) for the initialization result of step (2), Gaussian sputtering rasterization method is applied to realize Gaussian volume rendering, and the difference between the rendered image and the true value visual camera image is optimized by back propagation method; Step (5) for the optimization process of step (4), apply the instance segmentation mask generated in step (3) to perform background Gaussian culling based on cumulative transparency, apply Gaussian deep rasterization to implement Gaussian surface geometry regularization, and use the lidar point cloud as supervision to optimize the rasterization depth value to improve the reconstruction quality; Step (6) is to use the triangulation method to initialize the distance field tetrahedral mesh based on the Gaussian center point for the result of step (5) optimization, and use the ray tracing method to generate the distance field on the mesh, and then obtain the fine mesh model of the asteroid surface by tracing the zero level set of the distance field.

2. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 1, characterized in that: The specific implementation method of step (1) is as follows: First, the grayscale histogram of the visual camera image is counted. Grayscale is one interval, divided into intervals, calculate the mean grayscale value of all pixels and variance : ； ； in, Indicates The first The gray value of each pixel is calculated. Then, the mean gray value in each interval is calculated. and variance , the interval variance is greater than the threshold Pixels are treated as deep space background: ； ； ； Threshold It is calculated by weighting the mean and variance of the overall pixel grayscale. and are the mean and variance The weight of Variance within the interval Greater than threshold When , all pixel grayscales in the interval are set to 0 to filter out the interference of bright background noise; finally, the filtered image is subjected to histogram equalization.

3. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 2, characterized in that: The specific implementation method of step (2) is as follows: After preprocessing the visual camera image, the initial sparse point cloud is estimated using the motion recovery structure (SFM) method based on feature point matching, and the pose relationship between each view of the visual camera is determined based on the epipolar geometry relationship and bundle adjustment optimization. The initial sparse point cloud is denoted as ,in is the number of point clouds, For the The three-dimensional coordinates of the points in the world coordinate system; use the point cloud to initialize the mesh Gaussian, the Gaussian mesh has a side length of The square grid, the initial grid center coordinate set is the result of removing the duplicate center coordinates of the grid unit where the point cloud is located, the coordinate set , calculated as: ； Each grid cell carries a dimension of The feature vector of Gauss; Grid cell center coordinates , the generated The shift of the mean of the Gaussians is , the feature vector carried by the grid , the camera observation direction vector , then the Gaussian generated by the grid cell Parameters and their colors ,transparency The calculation is as follows: ； in, Both are multi-layer perceptrons (MLPs), used to decode Gaussian parameters from grid feature vectors; The conversion function from quaternion to rotation matrix.

4. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 3, characterized in that: The specific implementation method of step (3) is as follows: In order to effectively remove invalid Gaussians in the deep space background and eliminate floating artifacts, a large model of visual semantic segmentation with reference point prompts is used to generate a target mask for the visual camera image, indicating the area where the target asteroid exists in the image; The coordinates of the initial sparse point cloud in the camera coordinate system under the viewing angle are ,in is the number of point clouds, For the The 3D coordinates of the points in the world coordinate system; use the visual large model to generate the asteroid segmentation mask: ； in, For the visual camera images, Recorded as The projection position of the point cloud coordinates in the camera coordinate system under the visual camera image perspective in the image coordinate system, is the camera focal length, Cue point set In the image A visual deep learning model for instance segmentation on the image; since the visual segmentation edges are not completely accurate, in order to avoid incorrectly removing the Gaussian at the edges, a morphological operator is used to mask the image. Perform expansion treatment.

5. According to the method for fine visual 3D reconstruction of the asteroid surface by integrating laser radar data as described in claim 4: the specific implementation method of step (4) is as follows: The 3D reconstruction method based on 3D Gaussian radiation field is to represent the scene and objects as a collection of a large number of 3D Gaussian ellipsoids, use the visual camera image pixel color information to supervise the optimization of scene representation, and use the back propagation method and gradient-based optimization method to optimize the feature vectors, offsets and parameters of the decoding MLP stored on the grid, so as to achieve efficient and accurate scene representation; In the 3D reconstruction based on 3D Gaussian radiation field, any 3D Gaussian The geometric shape and color information of , covariance , Opacity ,color Parameterization: ； Final pixel color Calculated by the volume rendering formula: ； in, represents the Gaussian set covering a pixel, Indicates The opacity of a Gaussian at the current pixel is given by Directly calculated based on pixel coordinates, Represents the color of Gaussian. In order to alleviate the attenuation problem of Gaussian in the high-frequency domain, the Gaussian kernel is truncated to destroy the continuity of the derivative of the Gaussian kernel function. The actual Gaussian kernel form used is as follows: ； Among them, the hyperparameters is the cutoff coefficient; Note the perspective The rendered image is , the true value of the visual camera image is , optimize the grid parameters and decoder parameters based on the following loss function: ； is the weight coefficient of the structural similarity index SSIM loss function.

6. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 5, characterized in that: The specific implementation method of step (5) is as follows: Based on the instance segmentation mask extracted in step (3), the cumulative depth regularization strategy is applied. The specific operation is to calculate the pixel At the cumulative transparency, the following cumulative opacity penalty term is applied; remember the visual camera optical center to pixel On a ray, the opacity of all Gaussians that contribute to the color of the ray is , then the cumulative opacity is recorded as: ； The first The cumulative transparency penalty from the visual camera perspective is as follows: 。 7. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 6, characterized in that: The step (5) further comprises: In addition, a surface regularization strategy based on depth and normal vector rasterization is introduced; given a camera center and a ray direction , the ray corresponds to the pixel , any point along the ray Can be written as ,in is the depth of the ray; first, the camera center Move to the local coordinate system of the 3D Gaussian and apply the rotation and scale factors Center of camera and the ray direction Normalize to get the camera center in the 3D Gaussian coordinate system , ray direction and the 3D point coordinates : ； in, represents the Gaussian mean coordinates in the world coordinate system, represents element-wise multiplication. In this local coordinate system, the 3D Gaussian at any point along the ray becomes a 1D Gaussian, which is described as follows: ； Computing the 3D Gaussian along the ray then produces a 1D Gaussian which, when solved for the maximum, has a closed form solution: ； Ray Depth With reference frame invariance, it is believed that the ray and Gaussian matter actually intersect at the point: ； So the corresponding pixel Depth value: ； The pixels The normal vector can be calculated by solving the partial derivative of the depth, as follows: ； The median depth of the 3D Gaussian is defined separately, that is, the depth aggregation between Gaussians is calculated by color mixing. The specific form is as follows: ； in, represents the first The mean depth value of Gaussians, represents the first The opacity contribution of a Gaussian in the volume rendering process; The loss term used for surface regularization is of the following form: ； Further introduce a more robust regularization based on deep order relationship; The depth map obtained by rasterizing the Gaussian radiation field model is denoted as , cut it into a series of sizes Slices ; Perspective The lidar point cloud data in the camera coordinate system is recorded as ,in, is the number of point clouds, For the The three-dimensional coordinates of the point, The three-dimensional points of the point cloud in the slice are recorded as ,in, For the The number of point clouds in the slice, ; Randomly generated Arrangement , then the depth order regularization loss within the slice is as follows: ； in, is the symbolic function, is the ReLU activation function.

8. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 7, characterized in that: The specific implementation method of step (6) is as follows: First, initialize the tetrahedral grid structure; for the Gauss , the three orthogonal eigenvectors of its covariance matrix are recorded as , the eigenvalue is recorded as Initialize the mean Centered on, with length, width and height parallel to the three eigenvectors, and size The cuboid has eight vertices and the center point, a total of nine points, as candidate tetrahedral mesh vertices. All Gaussian candidate mesh vertices are collected, and Dirichlet triangulation is used to generate regular tetrahedral meshes on the mesh vertices. .

9. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 8, characterized in that: Further, the step (6) comprises: The distance from each candidate vertex to the Gaussian surface is calculated using the ray tracing method. Visual camera perspective, generating a given camera center and candidate mesh vertices Connected line segments, line segment direction , the points on the line segment are represented as , calculate the ray depth of all Gaussians along the ray according to step (4) , No. Gauss The cumulative contribution on a ray is defined as: ； Finally, the cumulative contribution is calculated along the camera ray and obtained according to the volume rendering formula: ； Calculate the minimum cumulative contribution sum under all viewing angles as The signed distance of constitutes the signed distance field on the grid M: 。 10. The method for fine visual 3D reconstruction of an asteroid surface by fusing laser radar data according to claim 9, characterized in that: Further, the step (6) comprises: Utilizing the Grid The zero level set is calculated from the signed distance field on the tetrahedral mesh to obtain the vertex set on the edge, and the mesh rough extraction result is obtained. , repeat the above method to obtain the vertex set The distance value of the directed distance field is determined by using the binary search method to determine the exact vertex set , as the result of mesh refinement extraction, a detailed three-dimensional model of the asteroid surface is obtained.

Images