Multi-feature 3D (three-dimensional) Gaussian reconstruction method based on laser vision

Abstract

A multi-feature three-dimensional reconstruction 3D Gaussian method based on laser vision comprises the steps that laser radar point cloud and camera images are aligned through space-time calibration, and a unified coordinate system is established; extracting geometric features by using point cloud data acquired by the Lidar point cloud, and initializing a Gaussian ellipsoid according to the Lidar point cloud; optimizing the brightness, the contrast ratio and the structural similarity of the rendered image and the real image by combining the mean absolute error L1 and the structural similarity SSIM; the curvatures of Gaussian ellipsoids of K-nearest neighbors are forced to be consistent, and long and short axes and line and surface features of the Gaussian ellipsoids are aligned to reduce geometric distortion; the distribution density of 3D Gaussian is dynamically adjusted through line / surface features and visual structure information extracted by Lidar, and balance between geometric detail enhancement and calculation efficiency is achieved. According to the method, the position, the scale and the rotation parameters of Gaussian are uniformly optimized, and the details and the calculation efficiency of the model are balanced while the consistency of the model structure is improved.

Description

TECHNICAL FIELD

[0001] The application relates to a multi-feature three-dimensional reconstruction 3D Gaussian method based on laser vision, and belongs to the technical field of three-dimensional modeling and real-time rendering. BACKGROUND

[0002] In recent years, three-dimensional reconstruction technology based on 3D Gaussian (3DGS) representation has attracted much attention in the field of computer vision and graphics due to its efficient rendering capability and high-quality detail recovery advantage. The technology realizes rendering by representing a scene as a large number of anisotropic Gaussian ellipsoids. However, the reconstruction of a high-quality 3DGS model highly depends on the accuracy of initialization and the effectiveness of optimization strategies.

[0003] Currently, mainstream visual-based initialization methods heavily rely on sparse point clouds output by motion recovery structure algorithms such as Colmap. Such methods are prone to feature matching failure, geometric distortion, and point cloud holes when dealing with low-texture areas or highly similar structures, which limits the geometric accuracy of initialization and subsequent optimization and makes it difficult to ensure the consistency of three-dimensional structures. To solve the inherent defects of visual initialization, multi-modal methods that fuse laser radar and visual images have become a research trend. For example, LOD-3DGS directly initializes the center position of the Gaussian ellipsoid using LiDAR point clouds, partially alleviating the dependence on Colmap point clouds. However, existing fusion methods still have the following limitations: (1) insufficient geometric constraints: existing methods fail to fully utilize the rich geometric structure information contained in LiDAR point clouds to finely constrain the scale and rotation parameters of Gaussian ellipsoids, resulting in loose structure and loss of geometric details of the reconstructed model; (2) lack of adaptive optimization: the optimization of 3DGS models relies on dynamic splitting and merging operations of Gaussian distribution to control density. Most existing strategies use a globally uniform density adjustment threshold, which cannot perceive local structural characteristics, leading to excessive splitting of Gaussians in flat areas, causing computational redundancy; in structural edges or texture complex areas, the density is insufficient, resulting in loss of geometric sharpness and details; (3) weak coordination of multi-modal features: existing technologies do not deeply explore effective collaborative optimization mechanisms for laser precise geometric features and visual rich gradient / texture features. The lack of such feature fusion makes it difficult for the model to meet both high-precision geometric structure requirements and high-quality photometric consistency, and details are easily lost or artifacts are easily produced in geometric-texture coupled areas; (4) computational efficiency challenge: the introduction of large-scale LiDAR point clouds improves geometric potential, but also puts higher requirements on computing resources, and more efficient optimization strategies are urgently needed. SUMMARY

[0004] In view of the problems in the prior art, the application provides a multi-feature three-dimensional reconstruction 3D Gaussian method based on laser vision, which can uniformly optimize the position, scale and rotation parameters of the Gaussian, balance the consistency of the model structure and the details and calculation efficiency of the model, and improve the consistency of the model structure.

[0005] In order to achieve the above object, the application provides a multi-feature three-dimensional reconstruction 3D Gaussian method based on laser vision, comprising the following steps:

[0006] S1, 3D Gaussian initialization: aligning the laser radar point cloud and the camera image through space-time calibration, establishing a unified coordinate system; extracting geometric features from the point cloud data obtained by the Lidar point cloud, and initializing the Gaussian ellipsoid according to the Lidar point cloud;

[0007] S2, establishing photometric consistency and geometric consistency constraints: combining the mean absolute error L1 and the structural similarity SSIM to optimize the brightness, contrast and structural similarity of the rendered image and the real image; forcing the Gaussian ellipsoid curvature consistency of the K nearest neighbors and aligning the major and minor axes of the Gaussian ellipsoid with the line and surface features, and reducing the geometric distortion;

[0008] S3, adaptive density control combined with Gaussian ellipsoid line and surface features: dynamically adjusting the distribution density of the 3D Gaussian by the line / surface features extracted by the Lidar and the visual structure information, realizing the balance between the geometric detail enhancement and the calculation efficiency.

[0009] Further, the specific process of S1 is:

[0010] S1.1, using a three-dimensional Gaussian function to represent a three-dimensional point, parameterizing the 3D Gaussian ellipsoid as position mu, opacity alpha, anisotropy and spherical harmonic function representing the view-dependent color, and the 3D Gaussian is defined as:

[0011]

[0012] Wherein, x=[x,y,z] T , represents the position information of the three-dimensional point, and Sigma is the covariance matrix;

[0013] The nonlinear formula for projecting the 3D Gaussian to the 2D image plane is:

[0014]

[0015] Wherein, x img ,y img represents the pixel coordinates of the image, f x ,f y is the focal length, c x ,c y is the optical center coordinate, and x, y and z represent the three-dimensional coordinates of the three-dimensional point in the camera coordinate system.

[0016] For simplicity of calculation, a first order Taylor expansion is performed on the non-linear projection with the center u cam = (x0, y0, z0) as the expansion point, approximated by the perspective projection matrix P and the Jacobian matrix J:

[0017] ∑′ = JP′P T J T ;

[0018] where J is the local linear approximation of the projection transformation:

[0019]

[0020] By applying the covariance matrix, the standard 3D Gaussian ellipsoid is transformed to an arbitrary ellipsoid. To ensure the positive semi-definiteness of the covariance matrix during the optimization process, the covariance matrix is calculated by:

[0021] ′ = RSS T R T ;

[0022] where S is the parameterized three-dimensional vector and R is the unitized rotation quaternion;

[0023] S1.2, 3D Gaussian parameter initialization:

[0024] Given a Lidar point p i ∈ P {p1, p2,... p n}, the nearest neighbor distance of p i is calculated:

[0025]

[0026] The initial scaling factor is calculated by:

[0027] Copy the scaling factor to the three axes: S i = [s i , s i , s i ];

[0028] Initialize the rotation parameters using the unit quaternion:

[0029] q i = [1, 0, 0, 0].

[0030] Further, the specific process of the S2 is:

[0031] S2.1, 3D Gaussian is transformed to a 2D plane by a projection transformation, and the pixel color C k is calculated by alpha blending after sorting by depth:

[0032]

[0033] where c i is the color of the ith Gaussian, a i is used to control the transparency, G i (x) is the projected 2D Gaussian function, and C k is the output rendered image, C k gt is the image obtained by vision, and the photometric consistency constraint module optimizes the Gaussian parameters by minimizing the average absolute error L1 and the structural similarity SSIM error, wherein the L1 error is used to calculate the absolute difference between the rendered pixels and the real pixels, and the formula is as follows:

[0034]

[0035] where N is the total number of pixels;

[0036] The L1 error is robust to outliers, but lacks the constraint of structural consistency. By adding a weighted structural similarity error, the brightness, contrast and structural information of the local area of the image are compared, which can well enhance the fidelity of texture and geometric details.

[0037] The formula of the structural similarity SSIM error is as follows:

[0038]

[0039] where v is the mean value, sigma is the standard deviation, c1 and c2 are stable constants, and the loss of D-SSIM is defined as:

[0040]

[0041] where I render and I gt represent the image rendered by the 3D Gaussian and the real image at a specific viewing angle respectively, and are defined as:

[0042]

[0043] where is used to measure the difference between the image rendered by the 3D Gaussian model and the real reference image, and kappa is a hyperparameter, generally kappa = 0.2;

[0044] S2.2, establish geometric consistency constraint: force to align the major and minor axes of the Gaussian ellipsoid and the curvature of the adjacent neighbor, and improve the geometric distribution of the Gaussian ellipsoid, the specific process is as follows:

[0045] S2.2-1, establish plane constraint and line constraint: wherein the construction process of the plane feature constraint is as follows:

[0046] Let Lidar point p i The neighborhood point set of p near is P n ={p1, p2,...p i}, and the centroid of the neighborhood points is calculated:

[0047]

[0048] The covariance matrix C is constructed:

[0049]

[0050] The eigenvalue decomposition of the covariance matrix C is performed, and the equation is solved:

[0051] det(C-λI)=0;

[0052] where λ is the eigenvector, I is the unit matrix, and the eigenvalues λ1≥λ2≥λ3 are obtained. If λ3 / λ1<ε and λ2 / λ1>ζ are satisfied, it is considered that the point cloud to which the neighborhood point belongs is a plane, where ε and ζ are hyperparameters that are adjusted according to the actual scene;

[0053] The construction process of the linear feature constraint is as follows: the method of constructing the plane feature constraint is also used to construct the Lidar point p i The covariance matrix is constructed and eigenvalue decomposition is performed to obtain the eigenvalues λ1≥λ2≥λ3. If λ3 / λ1<ε and λ2 / λ1<γ are satisfied, it is considered that the point cloud to which the neighborhood point belongs is a line, where ε and γ are hyperparameters that are adjusted according to the actual scene;

[0054] Let the normal vector of Lidar point p i be Define the short axis of the Gaussian ellipsoid as the normal vector of the Gaussian ellipsoid. For a plane Gaussian ellipsoid, align the point cloud surface by minimizing the difference between and the normal vector of the Gaussian ellipsoid , which is specifically:

[0055] For a Lidar point p i belonging to a surface feature, the covariance matrix of its neighborhood points is calculated, and the eigenvalue decomposition of the covariance matrix is represented as:

[0056]

[0057] where R i is a rotation matrix, the column vectors of which are eigenvectors, representing the principal directions of the ellipsoid, is the smallest direction vector, and Λ iFor a diagonal matrix, the diagonal elements are eigenvalues, which represent the variance of the scaling size of the Gaussian ellipsoid along the three axis directions, and the plane loss is achieved by aligning the Lidar point normal vector with the Gaussian ellipsoid normal vector to realize the geometric optimization of the Gaussian ellipsoid, and the plane loss is defined as follows:

[0058]

[0059] where ∈ is a small value to ensure that the denominator is not zero, and ||n g ||, ||n p || are the norms of the Gaussian ellipsoid normal vector and the plane normal vector, respectively, and their calculation formulas are as follows:

[0060]

[0061] where n g,x , n g,y , n g,z represent the components of the Gaussian ellipsoid normal vector on the x, y, and z axes, respectively, and n p,x , n p,y , n p,z represent the components of the plane normal vector on the x, y, and z axes, respectively.

[0062] For a point belonging to a line feature, let the line point direction vector be The covariance matrix is calculated from its neighborhood points, and the eigenvalue decomposition of the covariance matrix is represented as:

[0063]

[0064] where R i is a rotation matrix, and its column vectors are eigenvectors, which represent the principal direction of the ellipsoid, is the direction vector with the largest value, and the long axis direction vector of the Gaussian ellipsoid is defined as Λ i is a diagonal matrix, and the line loss is optimized by aligning the principal axis direction of the stretched ellipsoid with the line point cloud direction to reduce geometric distortion, and the definition of the line loss is as follows:

[0065]

[0066] where ∈ is a small value to ensure that the denominator is not zero.

[0067] S2.2-2, a curvature consistency constraint is constructed by constraining the Gaussian ellipsoid curvature smooth model: for the Lidar point cloud which is neither a plane point nor a line point in Lidar, a minimum k Gaussian ellipsoid curvature difference smooth model is adopted to reduce ambiguity, which is specifically:

[0068] For a Lidar point pi , p i is the geometric center of the Gaussian ellipsoid, and the three principal axes of the Gaussian ellipsoid are obtained by the covariance C decomposition, and the covariance matrix is expressed as:

[0069]

[0070] where Λ i is a diagonal matrix, and let κ1≥κ2≥κ3 be the three eigenvalues of the diagonal matrix Λ i , and the joint ∑=RSS T R T and Solve: Λ i =SS T , that is The lengths of the three principal axes of the Gaussian ellipsoid are Let For curvature consistency, only the curvature of the Gaussian ellipsoid in the direction of the shortest axis is constrained, and the calculation formula of the curvature is:

[0071]

[0072] Let the lengths of the three principal axes of the current Gaussian ellipsoid be The scale parameters of the nearest neighbor k Gaussian ellipsoids are where s i,j =(s i,j,1 ,s i,j,2 ,s i,j,3 );

[0073] The curvature of the current point s i is: The curvature of the neighbor point s i,j is: The curvature difference is Δκ i,j =κ i -κ i,j ;

[0074] To reduce the influence of outliers, the Huber loss form is adopted to robustly punish the curvature difference, and the expression is as follows:

[0075]

[0076] By weighting the curvature difference of different regions with adaptive weights, the smaller the neighborhood scale, that is, the more sensitive the geometry, the greater the weight, and the specific process is:

[0077] Filter invalid scales:

[0078] where is an indicator function, ∧ is a logical and operator, s i,j,2 ,s i,j,3the second and third principal axes of the j-th neighbor ellipsoid of the i-th Gaussian ellipsoid;

[0079] Weight assignment based on neighborhood scale stability:

[0080] Weighted loss on curvature difference:

[0081] Take the average of all points and neighborhoods, and the final curvature consistency loss is:

[0082] The final geometric loss is defined by the weighted plane constraint, line constraint and curvature consistency constraint:

[0083]

[0084] where ω, β, χ are hyperparameters, which are adjusted according to actual conditions;

[0085] The final total loss function is defined as the sum of photometric consistency loss and geometric consistency loss, that is:

[0086]

[0087] Further, the specific process of S3 is:

[0088] S3.1, the line feature area increases the Gaussian ellipsoid density to retain sharp geometry, and the surface feature area reduces the Gaussian density to avoid redundant calculation, specifically:

[0089] For each Gaussian ellipsoid p i , the density weight w i is assigned according to the feature type:

[0090]

[0091] where, is the visual gradient amplitude, is the local curvature estimation, w max , w min is the reference weight of line / surface feature; θ, ψ are the mixing coefficients of other regions and satisfy θ+ψ=1, is a classification function, which determines the geometric type of the Gaussian point by eigenvalue decomposition;

[0092] S3.2, adaptive density control of line / surface features, specifically:

[0093] For line Gaussians: if w i > w high and det(∑ i )>δsplit , split into 2 sub-ellipsoids along the direction of visual gradient, the expression is:

[0094]

[0095] For plane Gaussian: if or w i <w low and ‖μ i -μ j ‖2<r merge , merge coplanar ellipsoids, the expression is:

[0096]

[0097] Where w high , w low are the minimum weight value and the maximum weight value triggering the split and merge of Gaussian ellipsoid respectively, det(∑ i ) represents the determinant value of the covariance of the i-th Gaussian ellipsoid, u new represents the spatial position of the newly split Gaussian ellipsoid, ι is a step coefficient for adjusting the moving distance of the new Gaussian ellipsoid, τ img is the image gradient threshold, δ split is the split scale threshold, r merge is the merging distance threshold, and n gradient is the unit vector of the gradient direction.

[0098] Through the geometric consistency constraint and the multi-feature adaptive density, the 3D Gaussian modeling accuracy can be effectively improved, the geometric distortion can be reduced, and the detail level can be enriched.

[0099] The core mechanism of combining geometric loss and photometric consistency loss is introduced, and multi-feature guided adaptive density control is introduced; the geometric loss based on the core mechanism utilizes the local curvature information of the laser radar point cloud to regularize the shape parameters of the 3D Gaussian ellipsoid, and the model structure consistency is significantly improved; through the line and surface features extracted by the laser radar, the principal axis direction of the Gaussian ellipsoid is optimized, and the geometric distortion is reduced; the photometric loss optimizes the color and transparency parameters of the 3D Gaussian by combining the pixel level and structure level information of the visual image, and a high-fidelity rendering result is obtained; the multi-feature guided adaptive density control dynamically adjusts the distribution density of the 3D Gaussian through the line / surface features extracted by the Lidar and the visual structure information, while preserving the scene details, the calculation efficiency is improved, the balance between geometric detail enhancement and calculation efficiency is realized. The present application significantly improves the reconstruction accuracy and robustness of the 3D Gaussian by fusing the laser radar and visual data, combining the geometric feature constraint and photometric consistency optimization. BRIEF DESCRIPTION OF DRAWINGS

[0100] Figure 1is a workflow diagram of the method of the present application;

[0101] Figure 2 is an effect diagram of feature extraction of the method of the present application;

[0102] Figure 3 is a comparison diagram of the method of the present application and other methods for reconstructing the overall effect of a certain underground garage. DETAILED DESCRIPTION

[0103] The present application will be further described below in conjunction with the accompanying drawings.

[0104] As shown in Figure 1 , a multi-feature three-dimensional reconstruction 3D Gaussian method based on laser vision includes the following steps:

[0105] S1, 3D Gaussian initialization: align the Lidar point cloud and the camera image through space-time calibration to establish a unified coordinate system; extract geometric features from the point cloud data obtained by the Lidar point cloud, and initialize the Gaussian ellipsoid according to the Lidar point cloud;

[0106] S2, establish photometric consistency and geometric consistency constraints: combine the average absolute error L1 and the structural similarity SSIM to optimize the brightness, contrast and structural similarity of the rendered image and the real image; force the Gaussian ellipsoid curvature consistency of the K-nearest neighbor and align the major and minor axes of the Gaussian ellipsoid with the line and surface features to reduce geometric distortion;

[0107] S3, adaptive density control combined with Gaussian ellipsoid line and surface features: dynamically adjust the distribution density of the 3D Gaussian through the line / surface features extracted by the Lidar and the visual structure information to achieve a balance between geometric detail enhancement and computational efficiency.

[0108] To verify the effectiveness of the method of the present application, as shown in Figure 2 , the surface point cloud and the line point cloud extracted according to the method of the present application are shown, wherein, Figure 2 (a) the red point cloud is the surface point cloud, Figure 2 (b) the green point cloud is the line point cloud, and from the effect diagram of the adaptive density control of the line / surface features, it can be seen that the present application effectively reduces the number of point clouds and reduces floating artifacts through adaptive density control of the line / surface features;

[0109] As shown in Figure 3 , (a), (b) and (c) are comparison diagrams of the method of the present application and other methods for reconstructing the overall effect of a certain underground garage, wherein, Figure 3 (a) is the reconstruction result of 3DGS, Figure 3 (b) is the reconstruction result of LOD-3DGS, Figure 3 (c) is the reconstruction result of the method of the present application; from the diagram, it can be seen that the peripheral floating artifacts of the present method are significantly reduced compared with other methods;

[0110] At the same time, as shown in Table 1, the geometric precision of the method of the application and other methods is compared:

[0111] Table 1 Comparison table of geometric precision of the method of the application and other methods

[0112]

[0113] From the data comparison in Table 1, it can be seen that the geometric precision indexes of the method of the application are obviously better than those of other methods;

[0114] As shown in Table 2, the rendering indexes and final point numbers of the method of the application and other methods are compared:

[0115] Table 2 Comparison table of rendering indexes and final point numbers of the method of the application and other methods

[0116] LOSS point cloud number L1↓ PSNR↑ RGB↓ LPIPS↓ 3DGS 627376 0.011 34.796 0.017 0.178 LOD-3DGS 727150 0.011 34.692 0.017 0.174 the present invention 310110 0.013 32.087 0.021 0.211

[0117] From the data comparison in Table 2, it can be seen that the number of Gaussian ellipsoids used by the method of the application is obviously reduced in the case of slightly reducing the rendering index, which is conducive to the lightweight expression of the model.

Claims

1. A multi-feature 3D Gaussian reconstruction method based on laser vision, characterized in that, Includes the following steps: S1, 3D Gaussian initialization: Align the LiDAR point cloud and camera image through spatiotemporal calibration to establish a unified coordinate system; extract geometric features using the point cloud data obtained from the LiDAR point cloud, and initialize the Gaussian ellipsoid based on the LiDAR point cloud. S2. Establish photometric and geometric consistency constraints: Combine mean absolute error L1 and structural similarity SSIM to optimize the brightness, contrast and structural similarity between the rendered image and the real image; force the curvature of the Gaussian ellipsoid of the K nearest neighbors to be consistent and align the major and minor axes of the Gaussian ellipsoid with line and surface features to reduce geometric distortion; S3. Adaptive density control by combining Gaussian ellipsoidal line and surface features: By using line / surface features extracted from LiDAR and visual structural information, the distribution density of 3D Gaussian is dynamically adjusted to achieve a balance between geometric detail enhancement and computational efficiency.

2. The multi-feature 3D Gaussian reconstruction method based on laser vision according to claim 1, characterized in that, The specific process of S1 is as follows: S1.

1. Using a 3D Gaussian function to represent 3D points, the 3D Gaussian ellipsoid is parameterized by position μ, opacity α, anisotropy, and spherical harmonics to represent viewpoint-dependent color. The 3D Gaussian ellipsoid is defined as: Where x = [x, y, z] T , represents the position information of three-dimensional points, and ∑ is the covariance matrix; The nonlinear formula for projecting a 3D Gaussian image onto a 2D image plane is: Where, x img ,y img f represents the pixel coordinates of the image. x ,f y It's the focal length, c x ,c y These are the optical center coordinates, where x, y, and z represent the three-dimensional coordinates of a point in the camera coordinate system. To simplify the calculation, a first-order Taylor expansion is performed on the nonlinear projection, with the Gaussian center u. cam = (x0, y0, z0) is the expansion point, approximated by the perspective projection matrix P and the Jacobian matrix J: ∑′=JP∑P T J T ; Where J is a local linear approximation of the projection transformation: By applying the covariance matrix, the standard 3D Gaussian ellipsoid is transformed to an arbitrary ellipsoid. To ensure the positive semidefiniteness of the covariance matrix during the optimization process, the covariance matrix is ​​calculated by the following formula: ∑=RSS T R T ; Where S is a parameterized three-dimensional vector and R is a normalized rotation quaternion; S1.2, 3D Gaussian parameter initialization: Let Lidar point p i ∈P{p1,p2,...p n }, calculate p i nearest neighbor distance: The initial scaling factor is calculated by the following formula: Copy the scaling factor to all three axes: S i =[s i ,s i ,s i ]; Initialize rotation parameters using a unity quaternion: q i ＝[1,0,0,0]。 3. The multi-feature 3D Gaussian reconstruction method based on laser vision according to claim 1, characterized in that, The specific process of S2 is as follows: S2.1, 3D Gaussian is projected onto a 2D plane, sorted by depth, and then pixel color C is calculated using α-mixing. k : Among them, c i It is the color of the i-th Gaussian, α i Used to control transparency, G i (x) is the projected 2D Gaussian function, let C k For the output rendered image, C k gt For the visually acquired image, the photometric consistency constraint module optimizes the Gaussian parameters by minimizing the mean absolute error L1 and the structural similarity error SSIM. The L1 error is used to calculate the absolute difference between the rendered pixel and the real pixel, as shown in the following formula: Where N is the total number of pixels; The formula for the SSIM error is as follows: Where v is the mean, σ is the standard deviation, c1 and c2 are stability constants, and the loss of D-SSIM is defined as: Among them, I render ,I gt These represent the 3D Gaussian-rendered image and the real image, respectively, defined as: in, κ is a hyperparameter used to measure the difference between an image rendered by a 3D Gaussian model and a real reference image. It is typically set to κ = 0.

2. S2.2 Establishing Geometric Consistency Constraints: This involves forcibly aligning the major and minor axes of the Gaussian ellipsoid with the curvature of its nearest neighbors to improve the geometric distribution of the Gaussian ellipsoid. The specific process is as follows: S2.2-1. Establishing Planar and Linear Constraints: The process for constructing planar feature constraints is as follows: Let Lidar point p i The neighborhood point set of P near ={p1,p2,...p n } Calculate the centroid of the neighborhood points: Construct the covariance matrix C: Perform eigenvalue decomposition on the covariance matrix C and solve the equation: det(C-λI) = 0; Where λ is the eigenvector and I is the identity matrix, the eigenvalues ​​are obtained as follows: λ1≥λ2≥λ3. If λ3 / λ1<ε and λ2 / λ1>ζ, then the point cloud to which the neighboring point belongs is considered to be a plane. Here, ε and ζ are hyperparameters that are adjusted according to the actual scene. The process of constructing linear feature constraints is as follows: the method of constructing planar feature constraints is also applied to LiDAR points p. i Construct the covariance matrix and perform eigenvalue decomposition to obtain eigenvalues: λ1≥λ2≥λ3. If λ3 / λ1<ε and λ2 / λ1<γ, then the point cloud of the neighboring points is considered to be a line. Here, ε and γ are hyperparameters that are adjusted according to the actual scene. Let Lidar point p i The normal vector is Define the minor axis of the Gaussian ellipsoid Let be the normal vector of the Gaussian ellipsoid. For a planar Gaussian ellipsoid, minimize Normal vector of Gaussian ellipsoid The differences between the aligned point cloud surfaces are as follows: For a LiDAR point p i As a surface feature, its covariance matrix is ​​calculated from its neighboring points. The eigenvalue decomposition of the covariance matrix is ​​expressed as: Among them, R i It is a rotation matrix, and its column vectors are eigenvectors, representing the principal directions of the ellipsoid. Let Λ be the smallest direction vector among them. i This is a diagonal matrix, where the diagonal elements are eigenvalues ​​representing the variance of the Gaussian ellipsoid's scaling along the three axes. The plane loss achieves geometric optimization of the Gaussian ellipsoid by aligning the Lidar point normal vectors with the Gaussian ellipsoid normal vector. The plane loss is defined as follows: Where ∈ is a local minimum value used to ensure that the denominator is not zero, ||n g ||,||n p || represents the norm of the Gaussian ellipsoid normal vector and the plane normal vector, respectively, and their calculation formulas are as follows: Where, n g,x n g,y n g,z Let n represent the components of the Gaussian ellipsoid normal vector on the x, y, and z axes, respectively. p,x n p,y n p,z These represent the components of the plane normal vector on the x, y, and z axes, respectively. For a point that belongs to the characteristics of a line, let the line point direction vector be... Its covariance matrix is ​​calculated from its neighborhood points, and the eigenvalue decomposition of the covariance matrix is ​​expressed as: Among them, R i It is a rotation matrix, and its column vectors are eigenvectors, representing the principal directions of the ellipsoid. The direction vector of the major axis of the Gaussian ellipsoid is defined as the largest of these direction vectors. Λ i As a diagonal matrix, the linear loss optimizes the Gaussian ellipsoid by aligning the principal axis of the stretched ellipsoid with the direction of the line point cloud, thereby reducing geometric distortion. The linear loss is defined as follows: Where ∈ is a local minimum value used to ensure that the denominator is not zero; S2.2-2. Constructing curvature consistency constraints through a constrained Gaussian ellipsoid curvature smoothing model: For LiDAR point clouds that are neither planar nor linear points, a smoothing model minimizing the curvature differences of k Gaussian ellipsoids is adopted to reduce ambiguity, specifically: For a LiDAR point p that is neither a planar point nor a line i , with p i The three principal axes of a Gaussian ellipsoid centered at its geometry are obtained by decomposing its covariance C. Similarly, the eigenvalue decomposition of the covariance matrix is ​​expressed as: Among them, Λ i Let κ1≥κ2≥κ3 be a diagonal matrix Λ i The three eigenvalues ​​on the ∑ = RSS T R T and Solve for: Λ i =SS T ,Right now The lengths of the three principal axes of the Gaussian ellipsoid are make For curvature consistency, only the curvature of the Gaussian ellipsoid along the shortest axis is constrained. The formula for calculating curvature is: Let the lengths of the three principal axes of the current Gaussian ellipsoid be... The scale parameters of the k nearest neighbor Gaussian ellipsoids are: Among them, s i,j =(s i,j,1 ,s i,j,2 ,s i,j,3 ); Current point s i The curvature is: Neighboring point s i,j The curvature is: The curvature difference is: Δκ i,j =κ i -κ i,j ; To reduce the impact of outliers, a Huber loss form is adopted to robustly penalize curvature differences, as shown in the following expression: By adaptively weighting the curvature differences in different regions, the smaller the neighborhood scale (i.e., the more geometrically sensitive), the larger the weight. The specific process is as follows: Filtering invalid criteria: in, is an indicator function, ∧ is the logical AND operator, s i,j,2 ,s i,j,3 Let represent the second and third principal axes of the j-th neighbor ellipsoid of the i-th Gaussian ellipsoid; Weight allocation is based on neighborhood scale stability: Weighted loss for curvature differences: The final curvature consistency loss, averaged over all points and their neighborhoods, is: The final geometric loss, through weighted plane constraints, line constraints, and curvature consistency constraints, is defined as: Where ω, β, and χ are hyperparameters that are adjusted according to the actual situation; The final total loss function is defined as the sum of the photometric consistency loss and the geometric consistency loss, that is:

4. The multi-feature 3D Gaussian reconstruction method based on laser vision according to claim 1, characterized in that, The specific process of S3 is as follows: S3.

1. For line feature regions, the Gaussian ellipsoid density is increased to preserve sharp geometry, while for surface feature regions, the Gaussian density is reduced to avoid redundant calculations. Specifically: For each Gaussian ellipsoid p i Assign density weights w based on feature type i : in, It is the visual gradient magnitude. It is a local curvature estimation, w max ,w min It is the baseline weight of line / surface features; ψ is the mixing coefficient of other regions and satisfies This is a classification function that determines the geometric type of a Gaussian point through eigenvalue decomposition. S3.2, Perform adaptive density control for line / surface features, specifically as follows: For Gaussian lines: if w i >w high And det(∑ i )>δ split It splits into two sub-ellipsoids along the visual gradient direction, as expressed by: For Gaussian surfaces: if or w i <w low And ||μ i -μ j ||2<r merge The expression for merging coplanar ellipsoids is: Among them, w high ,w low These are the minimum and maximum weight values ​​that trigger the splitting and merging of the Gaussian ellipsoid, respectively, det(∑ i ) represents the determinant value of the covariance of the i-th Gaussian ellipsoid, u new The new Gaussian ellipsoid represents its spatial position, ι is a step size coefficient used to adjust the distance the new Gaussian ellipsoid moves, and τ is the step size coefficient. img δ is the image gradient threshold. split r is the splitting scale threshold. merge To determine the merging distance threshold, n gradient It is the unit vector of the gradient direction.

Images