Automatic space coordinate registration method and device based on 3D Gaussian splash model

Abstract

The invention provides an automatic space coordinate registration method and device based on a 3D Gaussian splash model, and the method takes RTK data as a strong constraint to be embedded into SFM optimization, and achieves the deep fusion. During coordinate registration, coarse registration and fine registration are combined, scale and global alignment is solved through coarse registration, and detail optimization is achieved through appearance-geometry double constraints in fine registration; finally, a differentiable optimization framework is put forward, the differentiable rendering characteristic of the 3DGS is utilized, joint optimization and deep coupling are carried out on the space projection loss and the normal vector constraint loss of the fine registration loss function and the 3D Gaussian rendering loss function, the registration error directly guides Gaussian parameter adjustment, meanwhile, the model rendering effect reversely constrains the registration precision, and the precision of the 3D Gaussian rendering is improved. And cooperative improvement of registration precision-model details is realized. The 3D Gaussian splash model is not only a result of coordinate registration, but also a participant of registration optimization, and the problems that a traditional method depends on manual registration, and the cross-coordinate system deviation is large are solved.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling and spatial coordinate registration technology, and in particular to an automated spatial coordinate registration method and apparatus based on a 3D Gaussian splash model. Background Technology

[0002] In scenarios requiring high-precision positioning and detailed measurement, such as geographic information systems, urban planning, architectural design, and environmental monitoring, the UAV spatial coordinate registration method can capture ground or aerial images, collect data, and generate high-precision 3D models.

[0003] Traditional methods for registering spatial coordinates of UAVs typically rely on manually setting up ground control points or manual alignment later, resulting in low efficiency and high costs. The registration process for a single scene often takes hours or even days and requires significant manpower and equipment investment. Furthermore, due to the lack of automated coordination mechanisms, traditional methods are prone to large deviations when registering across coordinate systems (such as local model coordinates and geographic coordinates), making it difficult to meet the requirements of high-precision scenarios and limiting their feasibility in large-scale and high-precision applications.

[0004] On the other hand, 3D Gaussian Splatting (3DGS) technology has significant advantages in generating high-precision 3D models. This technology can efficiently construct accurate 3D models, but its limitation lies in the fact that the generated model coordinates are only in a local coordinate system, lacking the ability to automatically correlate them with real geographic coordinates (such as latitude, longitude, and elevation). Therefore, current 3DGS technology still relies on external tools for manual registration, resulting in poor adaptability and an inability to achieve efficient automated conversion across coordinate systems.

[0005] Therefore, how to solve the deviation problem in cross-coordinate system registration and improve the registration accuracy and automation level has become an urgent technical problem to be solved. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide an automated spatial coordinate registration method and apparatus based on a 3D Gaussian splash model to solve the problem of low efficiency in manual registration of 3D Gaussian splash models.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] The first aspect of this invention discloses an automated spatial coordinate registration method based on a 3D Gaussian splash model, the method comprising:

[0009] Collect aerial survey data and construct a joint position-attitude optimization model based on the plane coordinates and rotation matrix in the preprocessed aerial survey data;

[0010] The sparse point cloud generated by the position-attitude joint optimization model is mapped to the global geographic coordinate system, and registration is performed based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; the fine registration loss function is determined based on spatial projection loss and surface normal vector.

[0011] Using the finely registered global sparse point cloud and the finely registered loss function, as well as the registered planar coordinates and rotation matrix, the parameters of the initialized 3D Gaussian splash model are iteratively optimized to obtain the converged 3D Gaussian splash model parameters.

[0012] Model rendering is performed based on the converged 3D Gaussian splash model parameters to obtain a 3D Gaussian splash binary format file with spatial coordinate registration.

[0013] Preferably, the step of collecting aerial survey data and constructing a joint position-attitude optimization model based on the planar coordinates and rotation matrix in the preprocessed aerial survey data includes:

[0014] Collect aerial survey data, and extract the geographic coordinates of each camera and the Euler angle attitude data corresponding to each image from the aerial survey data;

[0015] The geographic coordinates are converted into planar coordinates through multi-level curvature correction;

[0016] Convert the Euler angle attitude data into a rotation matrix;

[0017] The planar coordinates and the rotation matrix are used as strong constraints and embedded in the motion inference structure optimization framework to construct a joint position-attitude optimization model.

[0018] Preferably, the step of mapping the sparse point cloud generated by the position-attitude joint optimization model to the global geographic coordinate system, and performing registration based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix, includes:

[0019] The sparse point cloud is generated using the aforementioned position-attitude joint optimization model;

[0020] Based on the planar coordinates and the rotation matrix, the sparse point cloud is mapped to the global geographic coordinate system to obtain a coarsely registered global sparse point cloud.

[0021] The fine registration loss function is determined based on the spatial projection loss and the surface normal vector;

[0022] Based on the fine registration loss function, spatial projection loss iterative optimization and surface normal vector constraint processing are performed on the coarse registration global sparse point cloud, the planar coordinates, and the rotation matrix to obtain the fine registration global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix. The spatial projection loss iterative optimization process aims to minimize the error in projecting the coarse registration global sparse point cloud coordinates to image pixels; the surface normal vector constraint processing aims to minimize the error between the normal vector in the local coordinate system and the normal vector in the global geographic coordinate system.

[0023] Preferably, the step of iteratively optimizing the parameters of the initialized 3D Gaussian splash model using the finely registered global sparse point cloud, the finely registered loss function, and the registered planar coordinates and rotation matrix to obtain the converged 3D Gaussian splash model parameters includes:

[0024] Initialize the 3D Gaussian splash model parameters based on the finely registered global sparse point cloud;

[0025] Initialize the coordinate registration parameters of the 3D Gaussian splash model based on the registered planar coordinates and rotation matrix;

[0026] Based on the fine registration loss function and the preset 3D Gaussian rendering loss function, a joint loss function is constructed;

[0027] Based on the initialized Gaussian center coordinate registration parameters and the joint loss function, the parameters of the initialized 3D Gaussian splash model are iteratively optimized to obtain the converged 3D Gaussian splash model parameters.

[0028] Preferably, the step of rendering the model based on the converged 3D Gaussian splash model parameters to obtain a 3D Gaussian splash binary format file with spatial coordinate registration includes:

[0029] The converged 3D Gaussian splash model parameters are stored in a structured manner.

[0030] The converged 3D Gaussian splash model parameters are imported into the 3D Gaussian splash model rendering framework for model rendering, resulting in a 3D Gaussian splash binary format file with spatial coordinate registration.

[0031] A second aspect of this invention discloses an automated spatial coordinate registration device based on a 3D Gaussian splash model, the device comprising:

[0032] The building unit is used to collect aerial survey data and construct a position-attitude joint optimization model based on the planar coordinates and rotation matrix in the preprocessed aerial survey data;

[0033] The registration unit is used to map the sparse point cloud generated by the position-attitude joint optimization model to the global geographic coordinate system, and to perform registration based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; the fine registration loss function is determined based on the spatial projection loss and the surface normal vector.

[0034] The iterative optimization unit is used to iteratively optimize the parameters of the initialized 3D Gaussian splash model using the finely registered global sparse point cloud, the finely registered loss function, and the registered planar coordinates and rotation matrix, so as to obtain the converged 3D Gaussian splash model parameters.

[0035] The model rendering unit is used to render the model based on the converged 3D Gaussian splash model parameters, and obtain a 3D Gaussian splash binary format file after spatial coordinate registration.

[0036] Preferably, the building unit includes:

[0037] The extraction module is used to collect aerial survey data and extract the geographic coordinates of each camera and the Euler angle attitude data corresponding to each image from the aerial survey data;

[0038] The first conversion module is used to convert the geographic coordinates into planar coordinates through multi-level curvature correction;

[0039] The second conversion module is used to convert the Euler angle attitude data into a rotation matrix;

[0040] The first construction module is used to embed the planar coordinates and the rotation matrix as strong constraints into the motion inference structure optimization framework to construct a position-attitude joint optimization model.

[0041] Preferably, the registration unit includes:

[0042] The generation module is used to generate sparse point clouds using the position-attitude joint optimization model;

[0043] The mapping module is used to map the sparse point cloud to the global geographic coordinate system according to the planar coordinates and the rotation matrix to obtain a coarsely registered global sparse point cloud.

[0044] The determination module is used to determine the fine registration loss function based on the spatial projection loss and the surface normal vector;

[0045] The registration module is used to perform spatial projection loss iterative optimization and surface normal vector constraint processing on the coarse-registered global sparse point cloud, the planar coordinates, and the rotation matrix based on the fine registration loss function, to obtain the fine-registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; wherein, the spatial projection loss iterative optimization process aims to minimize the error of projecting the coordinates of the coarse-registered global sparse point cloud onto image pixels; the surface normal vector constraint processing aims to minimize the error between the normal vector in the local coordinate system and the normal vector in the global geographic coordinate system.

[0046] Preferably, the iterative optimization unit includes:

[0047] The first initialization module is used to initialize the parameters of the 3D Gaussian splash model based on the finely registered global sparse point cloud.

[0048] The second initialization module is used to initialize the coordinate registration parameters of the 3D Gaussian splash model based on the registered planar coordinates and rotation matrix.

[0049] The second construction module is used to construct a joint loss function based on the fine registration loss function and the preset 3D Gaussian rendering loss function;

[0050] The iterative optimization module is used to iteratively optimize the parameters of the initialized 3D Gaussian splash model based on the initialized Gaussian center coordinate registration parameters and the joint loss function, so as to obtain the converged 3D Gaussian splash model parameters.

[0051] Preferably, the model rendering unit includes:

[0052] The storage module is used to structure and store the converged 3D Gaussian splash model parameters.

[0053] The model rendering module is used to import the converged 3D Gaussian splash model parameters into the 3D Gaussian splash model rendering framework for model rendering, and obtain a 3D Gaussian splash binary format file after spatial coordinate registration.

[0054] Based on the above embodiments of the present invention, an automated spatial coordinate registration method and apparatus based on a 3D Gaussian splash model is provided. This invention embeds RTK data as a strong constraint into SFM optimization to achieve deep fusion. During coordinate registration, coarse and fine registration are combined. Coarse registration solves scale and global alignment, while fine registration achieves detail optimization through "appearance-geometric" dual constraints. Finally, a differentiable optimization framework is proposed. Utilizing the differentiable rendering characteristics of 3DGS, the spatial projection loss and normal constraint loss of the fine registration loss function are jointly optimized with the 3D Gaussian rendering loss function, achieving deep coupling. This allows the registration error to directly guide the adjustment of Gaussian parameters, while the model rendering effect inversely constrains the registration accuracy, achieving a synergistic improvement in "registration accuracy and model details." This makes the 3D Gaussian splash model not only the result of coordinate registration but also a "participant" in registration optimization, solving the problems of traditional methods relying on manual registration and large cross-coordinate system deviations. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 A flowchart illustrating an automated spatial coordinate registration method based on a 3D Gaussian splash model, provided in an embodiment of the present invention;

[0057] Figure 2 This is a structural block diagram of an automated spatial coordinate registration device based on a 3D Gaussian splash model, provided in an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0060] As the background technology shows, 3D Gaussian Splatting (3DGS) technology has significant advantages in generating high-precision 3D models. However, the coordinates of the models it generates are only local coordinate systems and lack the ability to automatically associate with real geographic coordinates (such as latitude, longitude, and elevation). It still needs to rely on external tools for manual registration, has poor adaptability, and cannot achieve efficient and automated conversion across coordinate systems.

[0061] Therefore, this invention provides an automated spatial coordinate registration method and apparatus based on a 3D Gaussian splash model. This invention embeds RTK data as a strong constraint into SFM optimization to achieve deep fusion. During coordinate registration, coarse and fine registration are combined. Coarse registration solves scale and global alignment, while fine registration achieves detail optimization through "appearance-geometric" dual constraints. Finally, a differentiable optimization framework is proposed. Utilizing the differentiable rendering characteristics of 3DGS, the spatial projection loss and normal constraint loss of the fine registration loss function are jointly optimized with the 3D Gaussian rendering loss function, achieving deep coupling. This allows the registration error to directly guide Gaussian parameter adjustment, while the model rendering effect inversely constrains the registration accuracy, achieving a synergistic improvement in "registration accuracy and model details." This makes the 3D Gaussian splash model not only the result of coordinate registration but also a "participant" in registration optimization, solving the problems of traditional methods relying on manual registration and large cross-coordinate system deviations.

[0062] See Figure 1 The diagram shows a flowchart of an automated spatial coordinate registration method based on a 3D Gaussian splash model provided by an embodiment of the present invention. This method is applicable to high-precision positioning devices (such as Real-Time Kinematic (RTK) drones).

[0063] It should be noted that RTK technology uses a ground-based reference station for calibration, providing centimeter-level precision absolute geographic coordinates (longitude, latitude, altitude) and high-precision attitude information (pitch, roll, heading) for the exposure points of each image taken by the UAV. This data is directly written into the EXIF ​​information of the image.

[0064] This automated spatial coordinate registration method includes:

[0065] Step S101: Collect aerial survey data and construct a position-attitude joint optimization model based on the planar coordinates and rotation matrix in the preprocessed aerial survey data.

[0066] In the specific implementation step S101, aerial survey data such as RGB imagery, geographic coordinates (B,L,H), altitude, and Euler angles (heading angle, pitch angle, roll angle) are collected, and these aerial survey data are preprocessed. Then, a position-attitude joint optimization model is constructed based on the plane coordinates and rotation matrix in the preprocessed aerial survey data.

[0067] Specifically, it includes the following processes (processes A1 to A4):

[0068] Process A1: Collect aerial survey data and extract the geographic coordinates of each camera and the Euler angle attitude data corresponding to each image from the aerial survey data.

[0069] In implementing process A1, aerial survey data is collected using a drone equipped with real-time dynamic positioning (RTK) technology. The geographic coordinates of each camera and the Euler angle attitude data corresponding to each image are extracted from the aerial survey data for preprocessing.

[0070] Process A2: Convert geographic coordinates to planar coordinates through multi-level curvature correction.

[0071] It is understandable that, for each camera, the geographic coordinates (B, L, H) are converted to planar coordinates (E, N, H) through multi-level curvature correction, and all cameras follow a unified geographic coordinate reference during the coordinate transformation. The specific transformation is shown in formulas (1) and (2):

[0072] (1)

[0073] (2)

[0074] The transformed plane coordinates are .

[0075] In formulas (1) and (2), The reference value for the eastern coordinates is usually 500,000m (to make all eastern coordinates positive). The reference value is the North coordinate system, with 0m for the Northern Hemisphere and 10,000,000m for the Southern Hemisphere. The projection scale factor for the central meridian; , representing the difference in longitude between the target point and the central meridian, where Longitude of the central meridian; The meridian arc length, from the equator to latitude The arc length of the meridian.

[0076] , For the second eccentricity ( ), The first eccentricity ( ),in It is the semi-major axis of the ellipsoid. It is the minor semi-axis of the ellipsoid.

[0077] Process A3: Convert Euler angle attitude data into rotation matrix.

[0078] It is understandable that Euler angle attitude data is converted into a rotation matrix. This provides attitude constraints for subsequent point cloud generation.

[0079] It should be noted that the conversion is performed by the Euler angle rotation order (usually Yaw-Pitch-Roll, i.e. ZYX order), as shown in formulas (3) to (5).

[0080] Rotate around the z-axis by the yaw angle (yaw angle), as shown in formula (3):

[0081] (3)

[0082] Rotate the pitch angle around the y-axis, as shown in formula (4):

[0083] (4)

[0084] The Roll angle (roll angle) is calculated around the x-axis, as shown in formula (5):

[0085] (5)

[0086] The transformed rotation matrix is .

[0087] Process A4: Using planar coordinates and rotation matrices as strong constraints, embed them into the motion inference structure optimization framework to construct a joint position-attitude optimization model.

[0088] When implementing process A4, the planar coordinates and rotation matrix As a strong constraint, it is embedded in the motion inference structure optimization framework to explicitly constrain the camera extrinsic parameters (rotation matrix R). i Translation vector t i The model needs to closely approximate the RTK measurement values ​​in order to construct a joint position-attitude optimization model.

[0089] It should be noted that the position-attitude joint optimization model is also known as the Structure from Motion (SFM) joint optimization model with RTK priors. It is an optimization model that combines high-precision RTK positioning information with SFM 3D reconstruction method.

[0090] The loss function of the position-attitude joint optimization model is shown in Equation (6).

[0091]

[0092] In formula (6), and For weights ( Greater than Because RTK positioning accuracy is higher than attitude accuracy, the camera pose is forced to approximate the real geographic coordinates. For the first The rotation matrix of each camera (describing the camera pose). For the first Translation vectors of each camera (describing the camera position). For the first Point cloud coordinates (point cloud to be reconstructed).

[0093] In formula (6), For image projection error, 3D points Projected to the Predicted location on Zhang image Compared with actual observed pixels The error between them.

[0094] in, For the first Zhang's image is the first The pixel coordinates of each feature point. For projection functions, 3D points in the world coordinate system are projected. Through camera internal parameters and camera external parameters ( Projected onto the pixel plane. It is the square of the Euclidean distance, used to measure the pixel position deviation. : indicates that the summation is performed only on the set of interior points filtered by RANSAC (Random Sample Consensus). This represents the set of interior points after RANSAC filtering.

[0095] Understandably, RANSAC is used to remove mismatched points (such as dynamic objects and repetitive textures) to ensure the geometric consistency of the point cloud.

[0096] In formula (6), For RTK pose constraints, constrain the camera rotation matrix. Attitude matrix near RTK measurement Ensure that the reconstructed camera orientation matches the actual measurement.

[0097] in, Given the square of the Frobenius norm, calculate the overall difference between the matrix elements. The weight parameter controls the strength of the attitude constraint and is usually set according to the RTK attitude measurement accuracy (e.g., 0.1~100).

[0098] In formula (6), For RTK position constraints, constrain the camera position. Location close to RTK measurement This ensures that the reconstructed camera location matches the actual geographic coordinates.

[0099] in, It is the square of the Euclidean norm, which measures the positional deviation in three-dimensional space. The weight parameter controls the strength of the position constraint and is usually much larger than μ because the position accuracy (centimeter level) of RTK is usually higher than the attitude accuracy (degree level).

[0100] Step S102: Map the sparse point cloud generated by the position-attitude joint optimization model to the global geographic coordinate system, and perform registration based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix.

[0101] Understandably, the fine registration loss function is determined based on the spatial projection loss and the surface normal vector.

[0102] In step S102, a collaborative strategy of "coarse registration + fine registration" is adopted for coordinate registration. First, the sparse point cloud generated by the position-attitude joint optimization model is mapped to the global geographic coordinate system to achieve RTK prior global alignment; then, based on the fine registration loss function, a dual-objective coordinate registration of "projection loss + normal vector constraint" is performed to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix.

[0103] The specific implementation process is as follows (processes B1 to B4):

[0104] Process B1: Generate sparse point clouds using a position-attitude joint optimization model.

[0105] That is, using the RTK-SFM joint optimization model to generate sparse point clouds.

[0106] Process B2: Based on the planar coordinates and rotation matrix, the sparse point cloud is mapped to the global geographic coordinate system to obtain a coarsely registered global sparse point cloud.

[0107] It is understandable that process B2 implements coarse registration, that is, RTK prior global alignment, as shown in formula (7).

[0108] (7)

[0109] In formula (7), This represents the global geographic coordinates of the i-th coarsely registered global sparse point after RTK prior global alignment. Sparse point cloud coordinates generated by the RTK-SFM joint optimization model; It is a rotation matrix; These are planar coordinates.

[0110] It should be noted that the implementation logic of formula (7) is through a rotation matrix. and planar coordinates The sparse points in the local coordinate system generated by the RTK-SFM joint optimization model Mapped to coordinates in the global geographic coordinate system This enables global alignment of sparse point clouds, quickly eliminates scale bias, and achieves preliminary spatial location matching.

[0111] Process B3: Determine the fine registration loss function based on the spatial projection loss and the surface normal vector.

[0112] It should be noted that, in this embodiment of the invention, for the residual RTK error (such as attitude drift) and model detail deviation from coarse registration, the fine registration loss function for the two targets is determined by iteratively optimizing the spatial projection loss and surface normal vector constraints, thereby further eliminating RTK error and model detail deviation. Specifically, the fine registration loss function (as shown in formula (8)) is used to further eliminate RTK error and model detail deviation.

[0113]

[0114] In formula (8), These are the normal vector constraint weight parameters. This is the fine registration loss function.

[0115] In formula (8), (Spatial projection loss) minimizes the error in projecting sparse point cloud coordinates to image pixels, ensuring the model's appearance matches the image (for all camera views). and all sparse point clouds In real-world scenarios, summing operations may involve multiple images from different perspectives (corresponding to different camera views). (This involves summing all these combinations to comprehensively consider the projection error of the entire model relative to the image in all views).

[0116]

[0117] In formula (9), To calculate the projected pixel coordinates ( ) and original pixel coordinates The squared distance between them is used to achieve better model alignment by minimizing the sum of squared distances. This is the projection function. For the first Rotation matrix of each camera. For the first The translation vector of each camera. For the registration of the first The three-dimensional coordinates of a sparse point cloud. This is the intrinsic parameter matrix of the camera. In the first Under the camera view, corresponding to the first camera view The pixel coordinates of a sparse point cloud.

[0118] In formula (8), (Surface normal vector constraint) is used to force the surface normal vector of the model to be consistent with multi-view observation, so as to ensure geometric authenticity.

[0119]

[0120] In formula (10), This indicates the calculation of vector difference ( This is used to measure the difference between the normal vector in the local coordinate system and the normal vector in the global coordinate system. The consistency of the normal vectors on the model surface is ensured by minimizing the sum of squared differences. For the first The normal vector of a point cloud in the local coordinate system. For the first The point at the th The observation normal vectors from each viewpoint are estimated using a stereo matching algorithm from images from different viewpoints. This represents the rotation matrix from the local coordinate system to the global coordinate system (obtained during the coarse registration stage), and... Consistency is ensured to guarantee a unified global coordinate system.

[0121] Process B4: Based on the fine registration loss function, the coarse registration global sparse point cloud, planar coordinates, and rotation matrix are subjected to spatial projection loss iterative optimization and surface normal vector constraint processing to obtain the fine registration global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix.

[0122] Among them, the spatial projection loss iterative optimization process aims to minimize the error of coarse registration of global sparse point cloud coordinates projected to image pixels; the surface normal vector constraint process aims to minimize the error between the normal vector in the local coordinate system and the normal vector in the global geographic coordinate system.

[0123] In implementing process B4, coarse registration of the global sparse point cloud is used. As input to the model, spatial projection loss is iteratively optimized and surface normal vector constraints are applied through a fine registration loss function to achieve fine alignment between coordinates and geometry. After optimization, a finely registered global sparse point cloud is obtained, providing a high-precision coordinate reference for subsequent 3D Gaussian splash model modeling. At the same time, the planar coordinates and rotation matrix are also finely registered to obtain the registered planar coordinates and rotation matrix.

[0124] It should be noted that the fine registration loss function is calculated in each iteration of the optimization process. right The partial derivatives are used to adjust the parameters along the negative gradient direction to reduce the loss value. For example, when there are 3 consecutive iterations... When the change is less than the threshold (0.00000001) or the number of iterations reaches the maximum threshold (200 times), the optimization stops and the finely registered global sparse point cloud is output.

[0125] Understandably, process B4 is an optimization process for fine registration, which is essentially an optimization of the coarse registration of the global sparse point cloud. Make localized detail corrections, through eliminate The projection deviation, through Correction Geometric deviations.

[0126] by For example, The physical meaning of "this" is "the global sparse point coordinates during the fine registration optimization process," and the starting point of these coordinates is... The core logic of precise registration is "based on..." Based on this, fine-tuning is performed iteratively. "By gradually reducing the projection error using the numerical values," the final finely registered point cloud coordinates are... The results of the optimization and correction.

[0127] Step S103: Using the fine registration of the global sparse point cloud and the fine registration loss function, as well as the registered planar coordinates and rotation matrix, the parameters of the initialized 3D Gaussian splash model are iteratively optimized to obtain the converged 3D Gaussian splash model parameters.

[0128] In the specific implementation step S103, the differentiable rendering characteristics of the 3D Gaussian splash model are utilized to jointly optimize the spatial projection loss of coordinate registration, the normal constraint loss, and the rendering loss of the 3D Gaussian splash model. This allows the registration error to directly guide the adjustment of Gaussian parameters, while the model rendering effect inversely constrains the registration accuracy. This deeply couples the fine registration loss function with the optimization of 3D Gaussian splash model parameters, achieving a synergistic improvement in "registration accuracy - model details." The specifics are shown in processes C1 to C4 below.

[0129] Process C1: Initialize the parameters of the 3D Gaussian splash model based on the finely registered global sparse point cloud.

[0130] In the implementation of process C1, the finely registered global sparse point cloud is used as the initial value of the Gaussian center coordinates for initializing the 3D Gaussian splash model.

[0131] Process C2: Initialize the coordinate registration parameters of the 3D Gaussian splash model based on the registered planar coordinates and rotation matrix.

[0132] In other words, the camera extrinsic parameters after fine registration (registered planar coordinates and rotation matrix) are used as the initial values ​​of the coordinate registration parameters for the 3D Gaussian splash model.

[0133] Process C3: Construct a joint loss function based on the fine registration loss function and the preset 3D Gaussian rendering loss function.

[0134] In implementing process C3, a "precise registration loss function" is constructed. +Preset 3D Gaussian rendering loss function The joint loss function is as follows. Among them, the fine registration loss function is based on the projection loss and normal vector constraint calculated by the 3D Gaussian center coordinates, which replaces the traditional sparse point cloud and realizes the coupling of registration and modeling; the preset 3D Gaussian rendering loss function is used to calculate the sum of squared pixel Euclidean distances between the rendered image of the 3D Gaussian splash model and the original image, ensuring that the rendering effect is consistent with the real image.

[0135] Specifically, joint loss function As shown in formula (11).

[0136] (11)

[0137] In formula (11), This is a weighting parameter used to balance registration accuracy and model rendering quality. The loss function for fine registration of 3D Gaussian points (spatial projection loss + surface normal vector constraint) is as follows: Spatial projection loss and surface normal vector constraint Both calculations are based on 3D Gaussian points (the initial coordinates of the Gaussian center of the 3D Gaussian splash model are the finely registered global sparse point cloud after fine registration in the hierarchical registration stage). This is similar to the fine registration loss for the sparse point cloud in the previous hierarchical spatial coordinate registration stage, but the optimization object changes from the sparse point cloud to 3D Gaussian points, and the coordinate alignment changes from point cloud coordinate alignment to Gaussian model coordinate alignment. In other words, .

[0138] In formula (11), This is a preset 3D Gaussian rendering loss function used to measure the pixel difference between the 3D Gaussian splash model rendered image and the original aerial survey image.

[0139] Specifically, As shown in formula (12).

[0140] (12)

[0141] In formula (12), For the 3D Gaussian splash model in camera pose ( The rendered image under ) The raw RGB image captured by the drone.

[0142] Process C4: Iteratively optimize the initialized 3D Gaussian splash model parameters based on the initialized Gaussian center coordinate registration parameters and joint loss function to obtain the converged 3D Gaussian splash model parameters.

[0143] In implementing process C4, backpropagation is performed to co-optimize the parameters of the initialized 3D Gaussian splash model based on the initialized Gaussian center coordinate registration parameters and the joint loss function. The joint loss function is optimized using gradient descent. Differentiation is performed to achieve bidirectional parameter iterative updates.

[0144] Specifically, calculating the joint loss function For the gradients of all parameters, two types of parameters are updated synchronously along the negative gradient direction. The first type of parameter is the coordinate registration parameter: fine-tuning the camera extrinsic parameters, i.e., the camera pose. Reduce the fine registration loss function of 3D Gaussian points. The second type of parameter is the 3D Gaussian splash model parameter: adjusting the Gaussian center coordinates (correcting registration bias), scale (the covariance matrix of the 3D Gaussian distribution, describing the diffusion range of 3D Gaussian points in three-dimensional space), transparency (the transparency parameter of 3D Gaussian points) (eliminating redundant Gaussian points), and color (spherical harmonic coefficients), reducing the preset 3D Gaussian rendering loss function. .

[0145] For example, when the joint loss function is used for 5 consecutive iterations The change is less than the threshold (0.00000001), and the preset 3D Gaussian rendering loss function... When the error is less than the set threshold (image pixel error less than 0.5), the 3D Gaussian splash model parameters are considered to have converged.

[0146] In practical applications, the output is a converged set of 3D Gaussian splash model parameters, which includes the center coordinates, scale (covariance matrix), transparency, and color (spherical harmonic coefficients) of all Gaussian points.

[0147] It should be added that during the iterative optimization process, registration errors drive Gaussian position adjustments, and rendering blur, in turn, corrects the camera pose. Registration errors (such as...) Increasing the value will push the Gaussian position towards a better coordinate, while rendering blur (such as...) Increasing the position will inversely correct the camera pose parameters. ), rendering loss The gradient will directly affect the camera pose parameters. When the rendered image differs significantly from the original image, the gradient direction points towards the pose change direction that makes the 3D Gaussian point projection more closely match the pixel coordinates. This is achieved by fine-tuning the rotation matrix. Translation vector Reduce spatial projection loss This achieves a reverse constraint between "rendering quality" and "pose accuracy," forming a closed-loop optimization.

[0148] Understandably, by introducing a reverse constraint mechanism of "rendering loss → pose parameters → projection accuracy," the traditional sequential process of "first performing point cloud registration, then independently building the model" is broken. This mechanism combines the rendering quality of the 3D Gaussian splash model with coordinate registration optimization, enabling simultaneous optimization of registration accuracy and model details, thereby improving the overall rendering effect and modeling accuracy, achieving a dual breakthrough in rendering and accuracy.

[0149] Step S104: Render the model based on the converged 3D Gaussian splash model parameters to obtain a 3D Gaussian splash binary format file and complete the automated spatial coordinate registration.

[0150] In the specific implementation step S104, the converged 3D Gaussian splash model parameters (such as center coordinates, scale, transparency, and color) are stored in a structured manner; for example, a binary format is used for structured storage, recording the parameters of each Gaussian point. Then, the 3DGS differentiable rendering module is integrated, importing the converged 3D Gaussian splash model parameters, that is, the structured stored Gaussian parameters (center coordinates, scale, transparency, and color), into the 3D Gaussian splash model rendering framework for model rendering. The differentiable rendering logic realizes the transformation of Gaussian parameters into 3D model visualization; finally, a 3D Gaussian splash binary format file is obtained, completing automated spatial coordinate registration.

[0151] In this embodiment of the invention, RTK data is embedded as a strong constraint in SFM optimization to achieve deep fusion. During coordinate registration, coarse and fine registration are combined. Coarse registration solves scale and global alignment, while fine registration achieves detail optimization through "appearance-geometric" dual constraints. Finally, a differentiable optimization framework is proposed. Utilizing the differentiable rendering characteristics of 3DGS, the spatial projection loss and normal constraint loss of the fine registration loss function are jointly optimized with the 3D Gaussian rendering loss function, achieving deep coupling. This allows the registration error to directly guide the adjustment of Gaussian parameters, while the model rendering effect inversely constrains the registration accuracy, achieving a synergistic improvement in "registration accuracy and model details." This makes the 3D Gaussian splash model not only the result of coordinate registration but also a "participant" in registration optimization, solving the problems of traditional methods such as reliance on manual registration, large cross-coordinate system deviations, and poor compatibility with 3DGS technology.

[0152] Corresponding to the automated spatial coordinate registration method based on a 3D Gaussian splash model provided in the above embodiments of the present invention, see also... Figure 2 The diagram shows a structural block diagram of an automated spatial coordinate registration device based on a 3D Gaussian splash model provided by an embodiment of the present invention.

[0153] The device includes: a construction unit 201, a registration unit 202, an iterative optimization unit 203, and a model rendering unit 204.

[0154] The construction unit 201 is used to collect aerial survey data and construct a position-attitude joint optimization model based on the planar coordinates and rotation matrix in the preprocessed aerial survey data.

[0155] The registration unit 202 is used to map the sparse point cloud generated by the position-attitude joint optimization model to the global geographic coordinate system, and to perform registration based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; the fine registration loss function is determined based on the spatial projection loss and the surface normal vector.

[0156] The iterative optimization unit 203 is used to iteratively optimize the parameters of the initialized 3D Gaussian splash model by using the finely registered global sparse point cloud and the finely registered loss function, as well as the registered planar coordinates and rotation matrix, to obtain the converged 3D Gaussian splash model parameters.

[0157] Model rendering unit 204 is used to render the model based on the converged 3D Gaussian splash model parameters to obtain a 3D Gaussian splash binary format file after spatial coordinate registration.

[0158] In this embodiment of the invention, RTK data is embedded as a strong constraint in SFM optimization to achieve deep fusion. During coordinate registration, coarse and fine registration are combined. Coarse registration solves scale and global alignment, while fine registration achieves detail optimization through "appearance-geometric" dual constraints. Finally, a differentiable optimization framework is proposed. Utilizing the differentiable rendering characteristics of 3DGS, the spatial projection loss and normal constraint loss of the fine registration loss function are jointly optimized with the 3D Gaussian rendering loss function, achieving deep coupling. This allows the registration error to directly guide the adjustment of Gaussian parameters, while the model rendering effect inversely constrains the registration accuracy, achieving a synergistic improvement in "registration accuracy and model details." This makes the 3D Gaussian splash model not only the result of coordinate registration but also a "participant" in registration optimization, solving the problems of traditional methods such as reliance on manual registration, large cross-coordinate system deviations, and poor compatibility with 3DGS technology.

[0159] Combination Figure 2 The content shown, the construction unit 201, includes: an extraction module, a first conversion module, a second conversion module and a first construction module.

[0160] The extraction module is used to collect aerial survey data and extract the geographic coordinates of each camera and the Euler angle attitude data corresponding to each image from the aerial survey data.

[0161] The first conversion module is used to convert geographic coordinates into planar coordinates through multi-level curvature correction.

[0162] The second conversion module is used to convert Euler angle attitude data into rotation matrices.

[0163] The first building module is used to embed planar coordinates and rotation matrices as strong constraints into the motion inference structure optimization framework to build a joint position-attitude optimization model.

[0164] Combination Figure 2 The content shown, the registration unit 202, includes: a generation module, a mapping module, a determination module and a registration module.

[0165] The generation module is used to generate sparse point clouds using a joint position-attitude optimization model.

[0166] The mapping module is used to map sparse point clouds to the global geographic coordinate system based on planar coordinates and rotation matrices, thereby obtaining a coarsely registered global sparse point cloud.

[0167] The determination module is used to determine the fine registration loss function based on the spatial projection loss and the surface normal vector.

[0168] The registration module performs iterative optimization of spatial projection loss and surface normal vector constraint processing on the coarsely registered global sparse point cloud, planar coordinates, and rotation matrix based on the fine registration loss function. This results in a finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix. The spatial projection loss iterative optimization minimizes the error of projecting the coarsely registered global sparse point cloud coordinates onto image pixels. The surface normal vector constraint processing minimizes the error between the normal vector in the local coordinate system and the normal vector in the global geographic coordinate system.

[0169] Combination Figure 2 The content shown, the iterative optimization unit 203, includes: a first initialization module, a second initialization module, a second construction module and an iterative optimization module.

[0170] The first initialization module is used to initialize the parameters of the 3D Gaussian splash model based on the finely registered global sparse point cloud.

[0171] The second initialization module is used to initialize the coordinate registration parameters of the 3D Gaussian splash model based on the registered planar coordinates and rotation matrix.

[0172] The second construction module is used to construct a joint loss function based on the fine registration loss function and the preset 3D Gaussian rendering loss function.

[0173] The iterative optimization module is used to iteratively optimize the parameters of the initialized 3D Gaussian splash model based on the initialized Gaussian center coordinate registration parameters and joint loss function, so as to obtain the converged 3D Gaussian splash model parameters.

[0174] Combination Figure 2 The content shown is model rendering unit 204, which includes: storage module and model rendering module.

[0175] The storage module is used to structure and store the parameters of the converged 3D Gaussian splash model.

[0176] The model rendering module is used to import the converged 3D Gaussian splash model parameters into the 3D Gaussian splash model rendering framework for model rendering, and obtain a 3D Gaussian splash binary format file after spatial coordinate registration.

[0177] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0178] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0179] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automated spatial coordinate registration method based on a 3D Gaussian splash model, characterized in that, The method includes: Collect aerial survey data and construct a joint position-attitude optimization model based on the plane coordinates and rotation matrix in the preprocessed aerial survey data; The sparse point cloud generated by the position-attitude joint optimization model is mapped to the global geographic coordinate system, and registration is performed based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; the fine registration loss function is determined based on spatial projection loss and surface normal vector. Using the finely registered global sparse point cloud and the finely registered loss function, as well as the registered planar coordinates and rotation matrix, the parameters of the initialized 3D Gaussian splash model are iteratively optimized to obtain the converged 3D Gaussian splash model parameters. Model rendering is performed based on the converged 3D Gaussian splash model parameters to obtain a 3D Gaussian splash binary format file with spatial coordinate registration.

2. The method according to claim 1, characterized in that, The process of collecting aerial survey data and constructing a joint position-attitude optimization model based on the plane coordinates and rotation matrix in the preprocessed aerial survey data includes: Collect aerial survey data, and extract the geographic coordinates of each camera and the Euler angle attitude data corresponding to each image from the aerial survey data; The geographic coordinates are converted into planar coordinates through multi-level curvature correction; Convert the Euler angle attitude data into a rotation matrix; The planar coordinates and the rotation matrix are used as strong constraints and embedded in the motion inference structure optimization framework to construct a joint position-attitude optimization model.

3. The method according to claim 1, characterized in that, The process of mapping the sparse point cloud generated by the position-attitude joint optimization model to the global geographic coordinate system, and performing registration based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix, includes: The sparse point cloud is generated using the aforementioned position-attitude joint optimization model; Based on the planar coordinates and the rotation matrix, the sparse point cloud is mapped to the global geographic coordinate system to obtain a coarsely registered global sparse point cloud. The fine registration loss function is determined based on the spatial projection loss and the surface normal vector; Based on the fine registration loss function, spatial projection loss iterative optimization and surface normal vector constraint processing are performed on the coarse registration global sparse point cloud, the planar coordinates, and the rotation matrix to obtain the fine registration global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix. The spatial projection loss iterative optimization process aims to minimize the error in projecting the coarse registration global sparse point cloud coordinates to image pixels; the surface normal vector constraint processing aims to minimize the error between the normal vector in the local coordinate system and the normal vector in the global geographic coordinate system.

4. The method according to claim 1, characterized in that, The parameters of the initialized 3D Gaussian splash model are iteratively optimized using the finely registered global sparse point cloud, the finely registered loss function, and the registered planar coordinates and rotation matrix to obtain the converged 3D Gaussian splash model parameters, including: Initialize the 3D Gaussian splash model parameters based on the finely registered global sparse point cloud; Initialize the coordinate registration parameters of the 3D Gaussian splash model based on the registered planar coordinates and rotation matrix; Based on the fine registration loss function and the preset 3D Gaussian rendering loss function, a joint loss function is constructed; Based on the initialized Gaussian center coordinate registration parameters and the joint loss function, the parameters of the initialized 3D Gaussian splash model are iteratively optimized to obtain the converged 3D Gaussian splash model parameters.

5. The method according to claim 1, characterized in that, The model rendering based on the converged 3D Gaussian splash model parameters yields a 3D Gaussian splash binary format file with spatial coordinate registration, including: The converged 3D Gaussian splash model parameters are stored in a structured manner. The converged 3D Gaussian splash model parameters are imported into the 3D Gaussian splash model rendering framework for model rendering, resulting in a 3D Gaussian splash binary format file with spatial coordinate registration.

6. An automated spatial coordinate registration device based on a 3D Gaussian splash model, characterized in that, The device includes: The building unit is used to collect aerial survey data and construct a position-attitude joint optimization model based on the planar coordinates and rotation matrix in the preprocessed aerial survey data; The registration unit is used to map the sparse point cloud generated by the position-attitude joint optimization model to the global geographic coordinate system, and to perform registration based on the fine registration loss function to obtain the finely registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; the fine registration loss function is determined based on the spatial projection loss and the surface normal vector. The iterative optimization unit is used to iteratively optimize the parameters of the initialized 3D Gaussian splash model using the finely registered global sparse point cloud, the finely registered loss function, and the registered planar coordinates and rotation matrix, so as to obtain the converged 3D Gaussian splash model parameters. The model rendering unit is used to render the model based on the converged 3D Gaussian splash model parameters, and obtain a 3D Gaussian splash binary format file after spatial coordinate registration.

7. The apparatus according to claim 6, characterized in that, The building unit includes: The extraction module is used to collect aerial survey data and extract the geographic coordinates of each camera and the Euler angle attitude data corresponding to each image from the aerial survey data; The first conversion module is used to convert the geographic coordinates into planar coordinates through multi-level curvature correction; The second conversion module is used to convert the Euler angle attitude data into a rotation matrix; The first construction module is used to embed the planar coordinates and the rotation matrix as strong constraints into the motion inference structure optimization framework to construct a position-attitude joint optimization model.

8. The apparatus according to claim 6, characterized in that, The registration unit includes: The generation module is used to generate sparse point clouds using the position-attitude joint optimization model; The mapping module is used to map the sparse point cloud to the global geographic coordinate system according to the planar coordinates and the rotation matrix to obtain a coarsely registered global sparse point cloud. The determination module is used to determine the fine registration loss function based on the spatial projection loss and the surface normal vector; The registration module is used to perform spatial projection loss iterative optimization and surface normal vector constraint processing on the coarse-registered global sparse point cloud, the planar coordinates, and the rotation matrix based on the fine registration loss function, to obtain the fine-registered global sparse point cloud in the global geographic coordinate system, as well as the registered planar coordinates and rotation matrix; wherein, the spatial projection loss iterative optimization process aims to minimize the error of projecting the coordinates of the coarse-registered global sparse point cloud onto image pixels; the surface normal vector constraint processing aims to minimize the error between the normal vector in the local coordinate system and the normal vector in the global geographic coordinate system.

9. The apparatus according to claim 6, characterized in that, The iterative optimization unit includes: The first initialization module is used to initialize the parameters of the 3D Gaussian splash model based on the finely registered global sparse point cloud. The second initialization module is used to initialize the coordinate registration parameters of the 3D Gaussian splash model based on the registered planar coordinates and rotation matrix. The second construction module is used to construct a joint loss function based on the fine registration loss function and the preset 3D Gaussian rendering loss function; The iterative optimization module is used to iteratively optimize the parameters of the initialized 3D Gaussian splash model based on the initialized Gaussian center coordinate registration parameters and the joint loss function, so as to obtain the converged 3D Gaussian splash model parameters.

10. The apparatus according to claim 6, characterized in that, The model rendering unit includes: The storage module is used to structure and store the converged 3D Gaussian splash model parameters. The model rendering module is used to import the converged 3D Gaussian splash model parameters into the 3D Gaussian splash model rendering framework for model rendering, and obtain a 3D Gaussian splash binary format file after spatial coordinate registration.

Images