Pose prior-based Gaussian sputtering three-dimensional reconstruction improvement method

Abstract

The invention provides a gauss sputtering three-dimensional reconstruction improvement method based on pose prior, and relates to the field of gauss sputtering three-dimensional reconstruction, and the method comprises the following steps: using a MarsLogger mobile phone application program to collect and obtain multi-modal information, the multi-modal information comprising video data and inertial measurement unit data; according to the inertial measurement unit data, estimating and obtaining pose information of a corresponding time point; performing frame extraction on the video data, converting the video data into a picture, and performing accurate matching on the picture and a pose by adopting a three-dimensional linear interpolation method and a three-dimensional spherical interpolation method according to a timestamp so as to obtain a matched picture pose; performing COLMAP sparse reconstruction based on pose priori according to the matched picture poses to obtain a sparse reconstruction point cloud; and performing 3DGS reconstruction according to the sparse reconstruction point cloud. The method is used for solving the problem that the reconstruction effect is poor in a complex environment (such as a mine tunnel) in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of Gaussian sputtering three-dimensional reconstruction, and in particular to an improved method for Gaussian sputtering three-dimensional reconstruction based on posture prior. Background Art

[0002] In recent years, with the development of computer graphics and computer vision technology, 3D reconstruction and new perspective synthesis technologies have received widespread attention. Among them, Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) are two representative technologies.

[0003] NeRF optimizes neural networks to represent the volume density and color information of a scene, enabling high-quality synthesis of new perspectives. However, NeRF's training and rendering processes typically require significant time and computing resources, making it difficult to meet real-time requirements. Furthermore, NeRF is prone to overfitting and detail loss when processing complex scenes.

[0004] In comparison, 3DGS has significant advantages. It uses a representation method based on 3D Gaussian distribution, which enables efficient rendering and new perspective synthesis. 3DGS renders much faster than NeRF and can provide high-quality visual effects in real-time applications. In addition, 3DGS has better flexibility and scalability when representing complex scenes, and can better capture the details and structure of the scene.

[0005] However, the input of 3DGS usually relies on sparse point clouds generated by structure-from-motion (SfM) algorithms such as COLMAP. In complex environments such as mine tunnels, due to poor lighting conditions and simple texture features, COLMAP's feature extraction and sparse point cloud reconstruction are often unsatisfactory. This is mainly because the mine tunnel environment lacks sufficient texture information, making it difficult to accurately detect and match feature points. In addition, the complex geometric structure and occlusion in the mine tunnel further increase the difficulty of feature extraction. This unsatisfactory sparse point cloud reconstruction result directly affects the 3DGS 3D reconstruction effect, resulting in errors and incompleteness in the reconstructed 3D model.

[0006] Therefore, how to improve the sparse point cloud reconstruction quality of COLMAP in complex environments such as mine tunnels, so as to provide more accurate input for 3DGS, is one of the urgent problems to be solved in the current 3D reconstruction field. In response to this problem, the present invention proposes an improved 3DGS 3D reconstruction method based on pose prior. By introducing pose prior information, the sparse point cloud reconstruction process of COLMAP is optimized, thereby improving the 3D reconstruction effect of 3DGS. Summary of the Invention

[0007] The present invention provides an improved Gaussian sputtering three-dimensional reconstruction method based on posture prior, which is used to solve the problem that the existing technology has poor reconstruction effect in complex environments (such as mine tunnels).

[0008] In one aspect, the present invention provides an improved method for 3D reconstruction of Gaussian sputtering based on pose prior, comprising:

[0009] Using the MarsLogger mobile app to collect multimodal information, including video data and inertial measurement unit data;

[0010] According to the inertial measurement unit data, the pose information at the corresponding time point is estimated;

[0011] Convert video data frames into images, and use 3D linear interpolation and 3D spherical interpolation methods to accurately match the images with the poses based on the timestamps to obtain the matched image poses.

[0012] According to the matched image pose, perform COLMAP sparse reconstruction based on pose prior to obtain sparse reconstructed point cloud;

[0013] 3DGS reconstruction is performed based on the sparse reconstructed point cloud.

[0014] Furthermore, based on the inertial measurement unit data, the pose information at the corresponding time point is estimated, including:

[0015] Acquiring inertial measurement unit data and preprocessing it to obtain a preprocessed inertial measurement unit data set, wherein the inertial measurement unit data includes a timestamp sequence, a gyroscope three-axis angular velocity, and an accelerometer three-axis acceleration in the inertial measurement unit's own coordinate system;

[0016] Analyzing the acceleration data in the static state according to the inertial measurement unit data set to detect the static state of the inertial measurement unit to obtain a static state detection result;

[0017] According to the static state detection results, the gravity acceleration measured by the accelerometer in the static state is analyzed to calculate the initial attitude of the inertial measurement unit;

[0018] During motion, the initial attitude of the inertial measurement unit is updated using the gyroscope and accelerometer data, and the gain K of the acceleration data is set to zero when stationary. P The assignment is normal, K in non-stationary state P Set to 0 to get the initial attitude of the updated inertial measurement unit;

[0019] According to the updated initial attitude of the inertial measurement unit, the velocity and displacement relative to the initial position in the earth coordinate system are calculated to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point.

[0020] Furthermore, based on the inertial measurement unit data set, the acceleration data in the stationary state is analyzed to detect the stationary state of the inertial measurement unit to obtain a stationary state detection result, including:

[0021] Based on the inertial measurement unit data set, the acceleration amplitude is calculated;

[0022] According to the acceleration amplitude, the cutoff frequency is set to remove the gravity acceleration and static acceleration components to obtain the acceleration amplitude after high-pass filtering;

[0023] According to the acceleration amplitude after high-pass filtering, the absolute value is taken and low-pass filtering is performed to obtain the acceleration amplitude after low-pass filtering;

[0024] According to the acceleration amplitude after low-pass filtering, a threshold detection is performed. If the acceleration amplitude is less than the threshold, it is marked as a static state. If the acceleration amplitude is not less than the threshold, it is marked as a moving state to obtain a static state detection result;

[0025] When the motion state is detected, the state changes within the preset time range before and after are checked, and the switching between the static state and the motion state is smoothed by setting the edge value.

[0026] Furthermore, based on the static state detection result, the gravity acceleration measured by the accelerometer in the static state is analyzed to calculate the initial attitude of the inertial measurement unit, including:

[0027] According to the static state detection result, assuming that the initial angle is 0, use Quaternion = q = [1, 0, 0, 0] to represent the quaternion, where Quaternion is Represents the quaternion of the sensor relative to the earth, q is The quaternion representing the earth relative to the sensor is obtained according to the conjugate relationship of the quaternion: Quaternion = q * =[q 0’ -q 1’ -q 2’ -q3], where q0 is the real part, and q1, q2, and q3 are the imaginary parts;

[0028] According to the static state detection results, the time window of the initial static state of the inertial measurement unit is selected, the average value of the acceleration vector measured in the static state within the window is calculated and normalized to obtain the unitized acceleration, where the acceleration measured in the static state is The unitized acceleration is Represents the unit component of the gravity direction in the inertial measurement unit coordinate system, a x 、a y 、az is the acceleration in the x, y, and z directions in the inertial measurement unit coordinate system;

[0029] According to the quaternion and the unitized acceleration, when the carrier is stationary, use Calculate the relationship between the accelerometer output and the acceleration due to gravity and use The direction of gravity is estimated, where a x 、a y 、a z is the acceleration in the x, y, and z directions in the inertial measurement unit coordinate system, q0 is the real part, q1, q2, and q3 are the imaginary parts, is the direction of gravity, g is the acceleration due to gravity, is the relationship between the accelerometer output and the acceleration due to gravity;

[0030] The angle error is obtained by cross product calculation based on the gravity direction of the normalized acceleration and the estimated gravity direction;

[0031] According to the angle error, use InitError=K i InitError+K P Error corrects the angular velocity of the gyroscope and uses Update the attitude of the inertial measurement unit, where IntError is the error integral term, K p and K i Respectively represent the calculation error gain and the integral error gain, Error is the error, is the updated quaternion, Δt is the time interval, t is the time, Represents the derivative of the quaternion, and the rate of change of the current posture quaternion is calculated using quaternion multiplication: represents quaternion multiplication, s w t is the constructed angular velocity quaternion;

[0032] The above steps are iterated until the error between the estimated gravity direction and the actual gravity direction is minimized to obtain an accurate initial attitude estimate, that is, the initial attitude of the inertial measurement unit.

[0033] Furthermore, based on the updated initial attitude of the inertial measurement unit, the velocity and displacement relative to the initial position in the earth coordinate system are calculated to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point, including;

[0034] According to the updated initial attitude of the inertial measurement unit, the acceleration measured by the inertial measurement unit is used Convert to the Earth reference system and get the acceleration in the Earth reference system, where Ea represents the acceleration in the Earth reference frame, S a represents the acceleration in the inertial measurement unit reference frame, is the updated quaternion, represents quaternion multiplication, Represents the quaternion in the earth reference frame;

[0035] According to the acceleration in the Earth's reference frame, use v t =v t-1 + E a t ·Δt calculates the velocity in the Earth coordinate system, where E a t represents the acceleration at time t, v t represents the speed at time t, and Δt is the time interval;

[0036] According to the velocity in the earth coordinate system, combined with the rate of change of the velocity and the period of motion, the velocity drift is calculated using linear interpolation, and the velocity drift is subtracted from the velocity in the earth coordinate system to obtain the corrected velocity;

[0037] According to the corrected velocity, the position matrix is initialized to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point.

[0038] Furthermore, the video data is converted into images by extracting frames, and the images are accurately matched with the poses using 3D linear interpolation and 3D spherical interpolation methods based on the timestamps to obtain the matched image poses, including:

[0039] Convert video data frames into images to obtain a list of sampled video timestamps;

[0040] According to the sampled video timestamp list, the video frame timestamps that are not within the timestamp range of the inertial measurement unit are removed to obtain a valid video frame timestamp list;

[0041] According to the valid video frame timestamp list, based on the position and attitude of the inertial measurement unit at adjacent moments, the matched image pose is obtained. The position is achieved through three-dimensional linear interpolation, and the attitude is achieved through quaternion interpolation. The quaternion interpolation adopts spherical linear interpolation. The formula is: Where s is the interpolation factor, q0 is the inertial measurement unit attitude corresponding to the adjacent previous time stamp, q1 is the inertial measurement unit attitude corresponding to the adjacent next time stamp, and θ is the angle between q0 and q1.

[0042] Furthermore, according to the matched image pose, COLMAP sparse reconstruction based on pose prior is performed to obtain a sparse reconstructed point cloud, including:

[0043] According to the matched image pose, the key points and descriptors are extracted from the input image through the detection algorithm. The corresponding feature point pairs are matched by the similarity of the descriptors, and geometric verification is performed to obtain the matched feature point pairs.

[0044] Sort all images according to the matched feature point pairs, select two images with more matched feature point pairs to initialize, and obtain the initialized images;

[0045] Based on the initialized image, the image with the most matching points or the widest distribution of visible points is selected, and the pose of the new image is adjusted and optimized to obtain the incrementally reconstructed image;

[0046] Repeat the above steps to optimize all generated 3D points and camera poses as a whole, and filter and eliminate the optimized point cloud and image to obtain a sparse reconstructed point cloud.

[0047] Furthermore, 3DGS reconstruction is performed based on the sparsely reconstructed point cloud, including:

[0048] According to the sparse reconstruction point cloud, the point cloud and camera parameters are obtained by sparse reconstruction using COLMAP, and the point cloud is initialized to a 3D Gaussian distribution, and the 3D Gaussian distribution is G i ={μ i ,Σ i ,α i ,c i}, where μ i is the position, Σ i is the covariance matrix, α i is the opacity, c i are the spherical harmonic coefficients;

[0049] By minimizing the error between the reconstructed image and the training view according to the 3D Gaussian distribution, we use Optimize the parameters of the Gaussian distribution to obtain the optimized 3D Gaussian distribution parameters, where is the L1 loss, is the D-SSIM loss, λ is the balance parameter;

[0050] During the 3D Gaussian distribution parameter optimization process, the number and density of Gaussian distributions are dynamically adjusted through cloning and splitting operations;

[0051] According to the optimized 3D Gaussian distribution parameters, the tile rasterization algorithm is used for real-time rendering to obtain the rendered 3D Gaussian distribution;

[0052] Repeat the above steps of 3D Gaussian distribution parameter optimization and real-time rendering until the preset number of iterations or convergence condition is reached.

[0053] Furthermore, during the 3D Gaussian distribution parameter optimization process, the number and density of Gaussian distributions are dynamically adjusted through cloning and splitting operations, including:

[0054] If the small-scale Gaussian distribution position gradient is large, it means that the area needs more geometric details. Increase the number of Gaussian distributions through cloning operations:

[0055] And α i <θ α , then clone

[0056] If the large-scale Gaussian distribution has a large position gradient, it means that the area needs to be divided more finely. A large Gaussian distribution is split into two small Gaussian distributions through a split operation:

[0057] And α i ≥θ α , then split

[0058] Among them, μ i is the position, α i is the opacity, θ pos is the gradient threshold, θ α is the opacity threshold.

[0059] Furthermore, based on the optimized 3D Gaussian distribution parameters, a tile rasterization algorithm is used for real-time rendering to obtain the rendered 3D Gaussian distribution, including:

[0060] Divide the screen into 16×16 pixel tiles based on the optimized 3D Gaussian distribution parameters;

[0061] Use GPU quick sorting algorithm to perform deep sorting on the Gaussian distribution in each tile to obtain the sorted Gaussian distribution;

[0062] Each pixel is rasterized according to the sorted Gaussian distribution and the Calculates the cumulative value of color and opacity, where p is the pixel position, C(p) is the color value of the pixel, N is the total number of samples, i is the index, and μ i is the position, α i is the opacity, c i are the spherical harmonic coefficients, and T is time.

[0063] The improved Gaussian sputtering 3D reconstruction method based on pose prior provided by the present invention significantly improves the reconstruction efficiency and accuracy by introducing IMU data as pose prior. Figure 7 (a) is the corridor model diagram, Figure 7(b) is the sparse point cloud generated by traditional COLMAP. Due to the lack of texture features in the corridor, the COLMAP reconstruction result only has one side of the corridor, and the other side of the corridor and the connection are discarded. Figure 7 (c) is the reconstruction result of the present invention. When the prior pose is added, the structure of the corridor is well restored.

[0064] By comparison, it can be seen intuitively that the reconstruction results of the present invention have significantly improved in terms of structural integrity and detail restoration. Specifically, the point cloud generated by traditional COLMAP may have problems of sparse point cloud and incomplete structure in some areas, especially in complex environments, which is prone to reconstruction deviation caused by insufficient feature matching. The optimized point cloud (due to the introduction of the pose prior information provided by the IMU) has obvious improvements in point cloud density, structural integrity and geometric accuracy. In terms of accuracy, the point cloud generated by traditional COLMAP may cause uneven point cloud distribution or deviation in some areas due to pose estimation errors. The point cloud constructed by the present invention has a more uniform point cloud distribution and higher geometric accuracy due to the optimization of the pose prior, and can more accurately restore the real structure of the three-dimensional scene. In addition, the point cloud constructed by the present invention has also significantly improved in reconstruction efficiency. The traditional COLMAP requires complex feature matching and geometric verification processes, while the optimized method simplifies the calculation process through known pose prior information and significantly shortens the reconstruction time.

[0065] Through this comparison, we can clearly see the optimization effect of introducing pose prior information on COLMAP sparse point cloud reconstruction. Especially in three-dimensional reconstruction in complex environments, the optimized point cloud of the present invention shows higher reconstruction efficiency and accuracy, providing higher quality input for subsequent 3DGS three-dimensional reconstruction, and further improving the overall reconstruction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 1 is a flow chart of an improved method for 3D reconstruction using Gaussian sputtering based on pose priors provided by an embodiment of the present invention;

[0068] Figure 2 It is the algorithm flow chart of the present invention;

[0069] Figure 3 It is a flow chart of prior pose estimation;

[0070] Figure 4 It is a schematic diagram of three-dimensional linear interpolation;

[0071] Figure 5 It is a schematic diagram of three-dimensional spherical interpolation;

[0072] Figure 6 It is a sparse reconstruction flowchart of pose prior;

[0073] Figure 7 This is a comparison chart of sparse reconstruction effects. DETAILED DESCRIPTION

[0074] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0075] Figure 1 This is one of the flow charts of the improved method for Gaussian sputtering 3D reconstruction based on pose prior provided in an embodiment of the present invention.

[0076] like Figure 1 As shown, the improved method for Gaussian sputtering 3D reconstruction based on pose prior provided by an embodiment of the present invention mainly includes the following steps:

[0077] 11. Use the MarsLogger mobile app to collect multimodal information, including video data and inertial measurement unit data;

[0078] 12. Estimate the position and posture information at the corresponding time point based on the inertial measurement unit data;

[0079] 13. Convert the video data frames into images, and use 3D linear interpolation and 3D spherical interpolation methods to accurately match the images with the poses based on the timestamps to obtain the matched image poses.

[0080] 14. According to the matched image pose, perform COLMAP sparse reconstruction based on pose prior to obtain sparse reconstructed point cloud;

[0081] 15. Reconstruct 3DGS based on the sparse reconstructed point cloud.

[0082] In an embodiment of the present invention, IMU data is introduced into the COLMAP sparse point cloud reconstruction process to estimate the image pose. The estimated pose prior information is then input into the COLMAP algorithm to optimize the sparse point cloud reconstruction result, thereby improving the accuracy and completeness of 3DGS three-dimensional reconstruction. Based on the optimized sparse point cloud, the 3DGS algorithm is used to perform efficient and high-quality three-dimensional reconstruction, ultimately obtaining an accurate three-dimensional model. This comprehensive method not only significantly improves the quality of sparse point cloud reconstruction, but also greatly improves the efficiency and reliability of 3DGS three-dimensional reconstruction, effectively solving the problem of poor reconstruction effect of existing technologies in complex environments. In traditional 3DGS reconstruction, COLMAP feature extraction and reconstruction processes are indispensable links, but they require complex feature matching and geometric verification to estimate the camera pose. This process is not only computationally intensive but also easily affects the reconstruction accuracy due to the accumulation of pose estimation errors. Although the present invention also requires feature extraction and reconstruction processes, it does not require complex rotation and translation calculations because the pose prior provided by the known IMU can be directly used for reconstruction. Specifically, Once the camera pose is known, there is no need to rely on complex feature matching and geometric verification processes to estimate the pose, which not only simplifies the calculation process but also greatly speeds up the overall reconstruction speed. In terms of accuracy, traditional methods often lead to a decrease in reconstruction accuracy due to the error accumulation problem of pose estimation. The present invention uses the known accurate pose to effectively avoid this problem and significantly improve the reconstruction accuracy. In addition, the prior pose is also optimized in combination with image data. By using the pose prior as a strong constraint, the optimization algorithm can converge to a more accurate solution more quickly, further improving the accuracy of the reconstruction results. At the same time, the present invention also simplifies the reconstruction process, reduces the dependence on feature matching and initialization, and reduces the risk of reconstruction failure due to pose estimation failure, making the entire reconstruction process more stable and reliable, and providing a more effective solution for high-efficiency and high-precision 3D reconstruction. Therefore, by combining pose prior information estimation, COLMAP sparse point cloud reconstruction optimization and 3DGS 3D reconstruction, the present invention achieves efficient and accurate reconstruction of 3D scenes in complex environments such as mine tunnels, providing strong technical support for research and application in related fields.

[0083] like Figure 1 As shown in 12, based on the inertial measurement unit data, the pose information of the corresponding time point is estimated, including:

[0084] 121. Acquire and preprocess inertial measurement unit data to obtain a preprocessed inertial measurement unit data set, wherein the inertial measurement unit data includes a timestamp sequence, a gyroscope three-axis angular velocity, and an accelerometer three-axis acceleration in the inertial measurement unit's own coordinate system;

[0085] 122. Analyze the acceleration data in the static state according to the inertial measurement unit data set to detect the static state of the inertial measurement unit to obtain a static state detection result;

[0086] 123. Analyze the gravity acceleration measured by the accelerometer in the static state according to the static state detection result, and calculate the initial attitude of the inertial measurement unit;

[0087] 124, during the motion, use the gyroscope and accelerometer data to update the initial attitude of the inertial measurement unit, and in the static state, use the acceleration data K P The assignment is normal, K in non-stationary state P Set to 0 to get the initial attitude of the updated inertial measurement unit;

[0088] 125, based on the updated initial attitude of the inertial measurement unit, the velocity and displacement relative to the initial position in the earth coordinate system are calculated to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point.

[0089] In an embodiment of the present invention, an IMU raw measurement data set is obtained, which includes a timestamp sequence, the gyroscope's three-axis angular velocity, and the accelerometer's three-axis acceleration in the IMU's own coordinate system. To ensure data consistency, the data is converted to units, where time is measured in seconds (s), angular velocity is expressed in degrees per second (deg / s), and acceleration is converted to gravitational acceleration (g). In addition, to reduce measurement noise and improve signal quality, a low-pass filter is applied to the acceleration data to smooth high-frequency noise, and a high-pass filter is applied to the angular velocity data to remove low-frequency drift. Finally, an appropriate data range is manually selected based on actual needs to ensure the accuracy and stability of the analysis.

[0090] The raw data is obtained from the inertial measurement unit (IMU), including the timestamp sequence, the three-axis angular velocity of the gyroscope and the three-axis acceleration of the accelerometer in the IMU's own coordinate system, and these raw data are preprocessed to ensure the accuracy and consistency of the data; the preprocessed inertial measurement unit data set is obtained, which provides a reliable basis for subsequent analysis and processing. The preprocessing step improves the accuracy of the data and reduces the error accumulation caused by noise or calibration errors; based on the preprocessed inertial measurement unit data set, the characteristics of the acceleration data in the static state are analyzed, and by detecting the stability or specific pattern of the acceleration data, it is determined whether the IMU is in a static state; the static state detection result is obtained, which provides a basis for subsequent attitude initialization and motion state judgment. Accurate detection of the static state helps to improve the accuracy and stability of attitude estimation; in the static state, the acceleration measured by the accelerometer mainly reflects the acceleration of gravity. By analyzing the component of the acceleration of gravity in the IMU coordinate system, the initial attitude of the IMU is calculated, and the inertial measurement unit is obtained. The initial attitude provides a starting reference for subsequent motion tracking. Accurate initial attitude estimation helps reduce the cumulative error during motion. During motion, the IMU attitude is continuously updated using data from the gyroscope and accelerometer. In a stationary state, the acceleration data reflects the acceleration of gravity, so its gain is kept at a normal value. In a non-stationary state, in order to avoid the dynamic component in the accelerometer data affecting the attitude estimation, its gain is set to 0, and the updated initial attitude of the inertial measurement unit is obtained, which can reflect the motion state of the IMU in real time. By reasonably processing the accelerometer data, the accuracy and robustness of the attitude estimation are improved. The updated inertial measurement unit attitude is used in combination with the data from the gyroscope and accelerometer to calculate the velocity and displacement of the IMU in the earth coordinate system. The velocity is obtained by integrating the acceleration data, and the displacement is obtained by integrating the velocity data, thereby reconstructing the motion trajectory of the IMU. The reconstructed motion trajectory, that is, the posture information at the corresponding time point, is obtained, providing accurate and continuous motion trajectory data for applications such as navigation, positioning, and motion analysis.

[0091] like Figure 1 As shown, 122, based on the inertial measurement unit data set, analyzing the acceleration data in the static state to detect the static state of the inertial measurement unit to obtain a static state detection result, including:

[0092] 1221, calculate the acceleration amplitude according to the inertial measurement unit data set;

[0093] 1222, according to the acceleration amplitude, set the cutoff frequency to remove the gravity acceleration and the static acceleration components to obtain the acceleration amplitude after high-pass filtering;

[0094] 1223, taking the absolute value of the acceleration amplitude after high-pass filtering and performing low-pass filtering to obtain the acceleration amplitude after low-pass filtering;

[0095] 1224, performing a threshold detection based on the acceleration amplitude after low-pass filtering. If the acceleration amplitude is less than the threshold, it is marked as a stationary state. If the acceleration amplitude is not less than the threshold, it is marked as a moving state, thereby obtaining a stationary state detection result.

[0096] 1225, when the motion state is detected, the state change within the preset time range before and after is checked, and the switching between the static state and the motion state is smoothed by setting the edge value.

[0097] In an embodiment of the present invention, in a stationary state, the accelerometer of the IMU mainly measures the acceleration of gravity, and the angular velocity of the gyroscope should be close to 0; by analyzing the acceleration data in the stationary state, the initial posture of the IMU can be calculated; in addition, during motion, the data in the stationary state can be used as a reference value for detecting and correcting errors, thereby improving the accuracy and robustness of posture estimation; the present invention uses an accelerometer to detect whether the IMU is in a stationary state, and the specific process is as follows: first, the acceleration amplitude is calculated, and then the calculated acceleration amplitude is high-pass filtered, and an extremely low cutoff frequency (0.001Hz) is set to remove the acceleration of gravity and the static acceleration component; then the absolute value is taken and low-pass filtered to remove sudden noise; Finally, a threshold detection is performed. If the amplitude is less than 0.03g, it is marked as a stationary state (1); otherwise, it is marked as a moving state (0). In order to further improve the accuracy and robustness of the stationary state detection, front and back edge smoothing is adopted. Specifically, when the moving state is detected, the state changes within a certain time range before and after are checked, and the switching between the stationary state and the moving state is smoothed by setting the margin value. In the present invention, the margin value is set to 10% of the sampling rate, that is, 100 milliseconds. This can effectively reduce the misjudgment caused by instantaneous noise or movement changes in a short period of time, thereby improving the stability of state recognition and ensuring that the consistency of the stationary or moving state can be maintained when the acceleration amplitude fluctuates slightly in a short period of time.

[0098] like Figure 1 As shown in 123, according to the static state detection result, the gravity acceleration measured by the accelerometer in the static state is analyzed and the initial attitude of the inertial measurement unit is calculated, including:

[0099] 1231, according to the static state detection result, assuming that the initial angle is 0, use Quaternion = q = [1, 0, 0, 0] to represent the quaternion, where Quaternion is Represents the quaternion of the sensor relative to the earth, q is The quaternion representing the earth relative to the sensor is obtained according to the conjugate relationship of the quaternion: Quaternion = q * =[q 0’ -q 1’ -q 2’ -q3], where q0 is the real part, and q1, q2, and q3 are the imaginary parts;

[0100] 1232. According to the static state detection result, select the time window of the initial static state of the inertial measurement unit, calculate the average value of the acceleration vector measured in the static state within the window and normalize it to obtain the unitized acceleration, where the acceleration measured in the static state is The unitized acceleration is Represents the unit component of the gravity direction in the inertial measurement unit coordinate system, a x 、a y 、a z is the acceleration in the x, y, and z directions in the inertial measurement unit coordinate system; z

[0101] 1233, based on the quaternion and the unitized acceleration, when the carrier is stationary, use Calculate the relationship between the accelerometer output and the acceleration due to gravity and use The direction of gravity is estimated, where a x 、a y 、a z is the acceleration in the x, y, and z directions in the inertial measurement unit coordinate system, q0 is the real part, q1, q2, and q3 are the imaginary parts, is the direction of gravity, g is the acceleration due to gravity, is the relationship between the accelerometer output and the acceleration due to gravity;

[0102] 1234, the angle error is calculated by cross product based on the gravity direction of the normalized acceleration and the estimated gravity direction;

[0103] 1235, according to the angle error, use InitError = K i InitError+K P Error corrects the angular velocity of the gyroscope and uses Update the attitude of the inertial measurement unit, where IntError is the error integral term, K p and K i Respectively represent the calculation error gain and the integral error gain, Error is the error, is the updated quaternion, Δt is the time interval, t is the time, Represents the derivative of the quaternion, and the rate of change of the current posture quaternion is calculated using quaternion multiplication: represents quaternion multiplication, s w t is the constructed angular velocity quaternion;

[0104] 1236, iteratively perform the above steps until the error between the estimated gravity direction and the actual gravity direction is minimized, so as to obtain an accurate initial attitude estimate, that is, the initial attitude of the inertial measurement unit.

[0105] In the embodiment of the present invention, through the x, y coordinates of the point P (x, y, z), the algorithm can accurately calculate the position (gi, gj) in the grid, and then determine the grid index, so that the subsequent patch information of the corresponding column can be accurately obtained from the texture, providing a basis for binary search; using the xyGridTexture texture, the algorithm can quickly obtain the patch information of the column, including the data position datapos, the number of patches interfaceNum and the in-order sequence index treeIndex, which provides the necessary data support for the subsequent binary search, so that the search process can be carried out efficiently; through two binary searches, the algorithm can accurately locate the position of the sampling point curPos in the patch sequence, the first The first binary search determines the patch range where the sampling point is located, and the second binary search finds the last patch that is different from the previous state, thereby determining the up-down relationship of the sampling points. This precise positioning enables the algorithm to accurately obtain the corresponding attribute values from the InterfaceUpAndDownProps array; the tree-based binary sampling method uses textures instead of arrays, fully utilizing the parallel processing capabilities of the GPU to improve the efficiency of data query. Accurate attribute value acquisition provides rich data support for volume rendering, making the rendered images more realistic and detailed; the tree-based binary sampling method improves the efficiency and effectiveness of volume rendering through precise positioning, efficient acquisition and accurate query, providing strong support for the visualization of complex three-dimensional data.

[0106] like Figure 1 As shown, 124, according to the updated initial posture of the inertial measurement unit, the velocity and displacement relative to the initial position in the earth coordinate system are calculated to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point, including;

[0107] 1241, according to the updated initial attitude of the inertial measurement unit, the acceleration measured by the inertial measurement unit is used Convert to the Earth reference system and get the acceleration in the Earth reference system, where E a represents the acceleration in the Earth reference frame, S a represents the acceleration in the inertial measurement unit reference frame, is the updated quaternion, represents quaternion multiplication, Represents the quaternion in the earth reference frame;

[0108] 1242, according to the acceleration in the Earth reference frame, use v t =v t-1 + E a t ·Δt calculates the velocity in the Earth coordinate system, where E a t represents the acceleration at time t, v t represents the speed at time t, and Δt is the time interval;

[0109] 1243, according to the velocity in the earth coordinate system, combined with the rate of change of the velocity and the period of motion, the velocity drift is calculated using linear interpolation, and the velocity drift is subtracted from the velocity in the earth coordinate system to obtain the corrected velocity;

[0110] 1244, initialize the position matrix according to the corrected velocity to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point.

[0111] In the embodiment of the present invention, for the entire motion process, the quaternion is updated using the gyroscope and accelerometer data. Since the acceleration data is more reliable in the static state, K p The assignment is normal, K in non-stationary state p Set to 0, and the rest of the update process is the same as step 3. In addition, for the convenience of subsequent calculations, the conjugate of the quaternion is calculated and stored.

[0112] like Figure 1 As shown in 13, the video data is extracted and converted into an image, and the image and the pose are accurately matched using the three-dimensional linear interpolation and three-dimensional spherical interpolation methods according to the timestamp to obtain the matched image pose, including:

[0113] 131, converting the video data frames into images to obtain a sampled video timestamp list;

[0114] 132, according to the sampled video timestamp list, remove the video frame timestamps that are not within the timestamp range of the inertial measurement unit to obtain a valid video frame timestamp list;

[0115] 133. Based on the valid video frame timestamp list, the position and attitude of the inertial measurement unit at adjacent moments are used to obtain the matched image pose. The position is achieved through three-dimensional linear interpolation, and the attitude is achieved through quaternion interpolation. The quaternion interpolation adopts spherical linear interpolation. The formula is: Where s is the interpolation factor, q0 is the inertial measurement unit attitude corresponding to the adjacent previous time stamp, q1 is the inertial measurement unit attitude corresponding to the adjacent next time stamp, and θ is the angle between q0 and q1.

[0116] In an embodiment of the present invention, the input video is converted into an image frame, the video sampling rate is about 30Hz, and one frame is saved every six frames, which approximately achieves 5 frames per second, thereby reducing the amount of data, and returning a sampled video timestamp list, which reflects the time flow of the video and provides basic data for subsequent timestamp screening and posture matching; the video frame timestamp is compared with the timestamp range of the inertial measurement unit (IMU), and those video frame timestamps that are not within the IMU timestamp range are eliminated, and a valid video frame timestamp list synchronized with the IMU data time is obtained, which ensures the timeliness consistency of the video frame and the IMU data in the subsequent posture matching process; according to the valid video frame timestamp list, based on the inertial at adjacent moments, The position and attitude of the measurement unit are used to obtain the matched image pose. Through three-dimensional linear interpolation, the interpolated position corresponding to the video frame timestamp is calculated based on the position data of the IMU at adjacent moments. Spherical linear interpolation (Slerp) is used to calculate the interpolated attitude corresponding to the video frame timestamp based on the attitude data (quaternion representation) of the IMU at adjacent moments. The video frame and IMU data are accurately matched in time and space, and the corresponding position and attitude are calculated for each frame of video image. A set of image pose data corresponding to the video frame timestamp is generated. These data fuse the information of the video and IMU, providing accurate spatial and temporal positioning information for subsequent visual-inertial fusion, three-dimensional reconstruction, augmented reality and other applications.

[0117] like Figure 1 As shown in 14, according to the matched image pose, COLMAP sparse reconstruction based on pose prior is performed to obtain a sparse reconstructed point cloud, including:

[0118] 141, according to the matched image pose, extract key points and descriptors from the input image through the detection algorithm, match the corresponding feature point pairs based on the similarity of the descriptors, and perform geometric verification to obtain the matched feature point pairs;

[0119] 142, sorting all images according to the matched feature point pairs, selecting two images with more matched feature point pairs to initialize, and obtaining the initialized images;

[0120] 143, based on the initialized image, select the image with the most matching points or the widest distribution of visible points, adjust and optimize the pose of the new image to obtain an incrementally reconstructed image;

[0121] 144, repeat the above steps to optimize all generated 3D points and camera poses as a whole, and filter and eliminate the optimized point cloud and image to obtain a sparse reconstructed point cloud.

[0122] In an embodiment of the present invention, a detection algorithm is used to extract representative key points and descriptors from an image, feature point pairs are matched by the similarity of the descriptors, and the accuracy of the matching is ensured by geometric verification, thereby obtaining a set of verified and accurate matching feature point pairs. These point pairs provide a reliable basis for subsequent image sorting, initialization, and incremental reconstruction; the images are sorted according to the number of matching feature point pairs, and the two images with the most matching points are selected for initialization, providing a stable starting point for subsequent incremental reconstruction, thereby obtaining an initialized image pair, and the relative position and feature point correspondence between the two images has been determined, providing a reliable initial condition for the subsequent reconstruction process; based on the initialization, the image with the most matching points or the widest distribution of visible points is selected as the next step. The goal of one-step reconstruction is to achieve incremental 3D reconstruction by adjusting and optimizing the pose of the new image, and obtain incrementally reconstructed images. These images have been accurately aligned and fused with the previous images in 3D space to form a more complete 3D scene; by repeating the above steps, the 3D reconstructed scene is continuously expanded and improved, and finally all generated 3D points and camera poses are optimized as a whole to improve the accuracy and consistency of reconstruction. At the same time, the optimized point cloud and image are filtered and culled to remove noise and redundant information, and sparsely reconstructed point clouds are obtained. These point clouds accurately represent the key structures and features in the 3D scene, providing a reliable foundation for subsequent dense reconstruction, texture mapping and other applications. At the same time, the quality and availability of the point cloud are improved through filtering and culling.

[0123] like Figure 1 As shown in 15, 3DGS reconstruction is performed based on the sparse reconstructed point cloud, including:

[0124] 151, based on the sparse reconstruction point cloud, use COLMAP sparse reconstruction to obtain point cloud and camera parameters, and initialize the point cloud to a 3D Gaussian distribution, the 3D Gaussian distribution is G i ={μ i ,Σ i ,α i ,c i}, where μ i is the position, Σ i is the covariance matrix, α i is the opacity, c i are the spherical harmonic coefficients;

[0125] 152, according to the 3D Gaussian distribution, by minimizing the error between the reconstructed image and the training view, using Optimize the parameters of the Gaussian distribution to obtain the optimized 3D Gaussian distribution parameters, where is the L1 loss, is the D-SSIM loss, λ is the balance parameter;

[0126] 153, in the process of 3D Gaussian distribution parameter optimization, the number and density of Gaussian distribution are dynamically adjusted through cloning and splitting operations;

[0127] 154, performing real-time rendering using a tile rasterization algorithm according to the optimized 3D Gaussian distribution parameters to obtain a rendered 3D Gaussian distribution;

[0128] 155, repeat the above steps of 3D Gaussian distribution parameter optimization and real-time rendering until a preset number of iterations or convergence condition is reached.

[0129] In an embodiment of the present invention, the COLMAP tool is used for sparse reconstruction to obtain point cloud and camera parameters, providing basic data for subsequent 3D Gaussian distribution initialization, initializing the point cloud to a 3D Gaussian distribution, providing a mathematical model for further optimization and rendering of the point cloud, and obtaining initial 3D Gaussian distribution parameters, including position, covariance matrix, opacity and spherical harmonic coefficients, which provide a starting point for subsequent optimization and rendering processes; by minimizing the error between the reconstructed image and the training view, the parameters of the 3D Gaussian distribution are optimized, so that the reconstruction result is closer to the real scene, and the optimized 3D Gaussian distribution parameters, including more accurate position, covariance matrix, opacity and spherical harmonic coefficients, are obtained, thereby improving the accuracy and realism of the reconstructed point cloud; during the optimization process, the parameters of the 3D Gaussian distribution are dynamically adjusted according to the complexity of the scene and the detail requirements. The number and density of Gaussian distributions are adjusted to meet the reconstruction needs of different scenes. Through cloning and splitting operations, the number and density of Gaussian distributions are flexibly increased, and the detail performance and scene adaptability of the reconstructed point cloud are improved; using the tile rasterization algorithm, real-time rendering is performed according to the optimized 3D Gaussian distribution parameters to generate a realistic 3D scene, and the rendered 3D Gaussian distribution is obtained, presenting a 3D scene with delicate details and realism, providing a basis for subsequent visual display and interaction; through continuous iterative optimization and rendering steps, the accuracy and realism of the reconstructed point cloud are gradually improved until the preset number of iterations or convergence conditions are reached, and the final optimized 3D Gaussian distribution parameters and rendering results are obtained. The accuracy and realism of the reconstructed point cloud are significantly improved, meeting the preset reconstruction requirements and standards.

[0130] like Figure 1 As shown,153, during the 3D Gaussian distribution parameter optimization process, the number and density of Gaussian distributions are dynamically adjusted through cloning and splitting operations, including:

[0131] 1531, if the small-scale Gaussian distribution position gradient is large, it means that the area needs more geometric details. Increase the number of Gaussian distributions through cloning operations:

[0132] And α i <θ α , then clone

[0133] 1532. If the position gradient of the large-scale Gaussian distribution is large, it means that the area needs to be divided more finely. The large Gaussian distribution is split into two small Gaussian distributions through the split operation:

[0134] And α i ≥θ α , then split

[0135] Among them, μ i is the position, α i is the opacity, θ pos is the gradient threshold, θ α is the opacity threshold.

[0136] In an embodiment of the present invention, during the 3D Gaussian distribution parameter optimization process, the number and density of Gaussian distributions are dynamically adjusted through cloning and splitting operations. If the position gradient of the small-scale Gaussian distribution is large, that is, the position change in the area is more drastic, indicating that the area requires more geometric details to accurately describe, the number of Gaussian distributions is increased through the cloning operation, that is, based on the original small-scale Gaussian distribution, more similar Gaussian distributions are copied to more finely describe the geometric features of the area. The cloning operation increases the number of Gaussian distributions in areas requiring more geometric details, thereby improving the detail performance and accuracy of the reconstructed point cloud, making the reconstruction result closer to the real scene, especially in areas with complex geometric structures. If the position gradient of the large-scale Gaussian distribution is large, that is, although the overall range of the area is large, the local changes are also more drastic, indicating that the area requires finer division to capture local features, the splitting operation splits a large Gaussian distribution into two small Gaussian distributions, that is, according to the direction of the position gradient, the original large-scale Gaussian distribution is split into two small Gaussian distributions with smaller scales and more specific position information. The splitting operation increases the density of the Gaussian distribution in areas requiring finer division, thereby more accurately capturing local geometric features. This helps improve the overall quality and realism of the reconstructed point cloud, especially in scenes with large scales but complex local variations. By dynamically adjusting the number and density of Gaussian distributions through cloning and splitting operations, the Gaussian distributions can be flexibly increased or decreased according to the actual needs of the scene to improve the accuracy and realism of the reconstructed point cloud. This dynamic adjustment mechanism makes the reconstruction process more adaptable to the complexity and detail requirements of different scenes, thereby generating higher quality 3D reconstruction results.

[0137] like Figure 1 As shown in FIG154 , according to the optimized 3D Gaussian distribution parameters, a tile rasterization algorithm is used for real-time rendering to obtain a rendered 3D Gaussian distribution, including:

[0138] 1541, based on the optimized 3D Gaussian distribution parameters, the screen is divided into 16×16 pixel tiles;

[0139] 1542, perform deep sorting on the Gaussian distribution in each tile using the GPU quick sort algorithm to obtain a sorted Gaussian distribution;

[0140] 1543, according to the sorted Gaussian distribution, each pixel is rasterized and used Calculates the cumulative value of color and opacity, where p is the pixel position, C(p) is the color value of the pixel, N is the total number of samples, i is the index, and μ i is the position, α i is the opacity, c i are the spherical harmonic coefficients, and T is time.

[0141] In an embodiment of the present invention, a tile rasterization algorithm is used to perform real-time rendering based on optimized 3D Gaussian distribution parameters to obtain a rendered 3D Gaussian distribution; based on the optimized 3D Gaussian distribution parameters, the screen is divided into 16×16 pixel tiles. This segmentation method helps to divide the rendering task into smaller units, which is convenient for parallel processing and optimization, while reducing the number of Gaussian distributions in each tile and improving processing efficiency; the Gaussian distribution in each tile is depth-sorted using a GPU quick sorting algorithm to obtain a sorted Gaussian distribution. The depth sorting ensures that the occlusion relationship between Gaussian distributions can be correctly handled during rendering, so that the Gaussian distribution closer to the camera is rendered first, thereby generating a correct visual effect and improving the accuracy and realism of the rendering result; based on the sorted Gaussian distribution, each pixel is rasterized, and the cumulative value of color and opacity is calculated using the formula, where is the pixel position, is the color value of the pixel, is the total number of samples, i is the index, μ i is the position, α i is the opacity, c i are spherical harmonic coefficients, and T is time. The rasterization process converts the Gaussian distribution into color and transparency information on the screen pixels. By accumulating color and transparency, a rendering result with delicate details and realism can be generated. The use of spherical harmonic coefficients may be used to process complex visual effects such as lighting changes or surface reflections, while the time parameter may be used to achieve dynamic effects or animations.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An improved method for 3D reconstruction of Gaussian sputtering based on pose prior, comprising: Using the MarsLogger mobile app to collect multimodal information, including video data and inertial measurement unit data; According to the inertial measurement unit data, the pose information at the corresponding time point is estimated; Convert video data frames into images, and use 3D linear interpolation and 3D spherical interpolation methods to accurately match the images with the poses based on the timestamps to obtain the matched image poses. According to the matched image pose, perform COLMAP sparse reconstruction based on pose prior to obtain sparse reconstructed point cloud; 3DGS reconstruction is performed based on the sparse reconstructed point cloud.

2. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 1, characterized in that: According to the inertial measurement unit data, the pose information of the corresponding time point is estimated, including: Acquiring inertial measurement unit data and preprocessing it to obtain a preprocessed inertial measurement unit data set, wherein the inertial measurement unit data includes a timestamp sequence, a gyroscope three-axis angular velocity, and an accelerometer three-axis acceleration in the inertial measurement unit's own coordinate system; Analyzing the acceleration data in the static state according to the inertial measurement unit data set to detect the static state of the inertial measurement unit to obtain a static state detection result; According to the static state detection results, the gravity acceleration measured by the accelerometer in the static state is analyzed to calculate the initial attitude of the inertial measurement unit; During motion, the gyroscope and accelerometer data are used to update the initial attitude of the inertial measurement unit, and the acceleration data K is used to update the initial attitude of the inertial measurement unit when the vehicle is stationary. P Set to 0, K in non-stationary state P Assign values as normal to get the updated inertial measurement unit attitude; According to the updated attitude of the inertial measurement unit, the velocity and displacement relative to the initial position in the earth coordinate system are calculated to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point.

3. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 2, characterized in that: Based on the IMU data set, the acceleration data in the static state is analyzed to detect the static state of the IMU, thereby obtaining the static state detection result, including: Based on the inertial measurement unit data set, the acceleration amplitude is calculated; According to the acceleration amplitude, the cutoff frequency is set to remove the gravity acceleration and static acceleration components to obtain the acceleration amplitude after high-pass filtering; According to the acceleration amplitude after high-pass filtering, the absolute value is taken and low-pass filtering is performed to obtain the acceleration amplitude after low-pass filtering; According to the acceleration amplitude after low-pass filtering, a threshold detection is performed. If the acceleration amplitude is less than the threshold, it is marked as a static state. If the acceleration amplitude is not less than the threshold, it is marked as a moving state to obtain a static state detection result; When the motion state is detected, the state changes within the preset time range before and after are checked, and the switching between the static state and the motion state is smoothed by setting the edge value.

4. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 2, characterized in that: Based on the static state detection results, analyze the gravitational acceleration measured by the accelerometer in the static state and calculate the initial attitude of the inertial measurement unit, including: According to the static state detection result, assuming that the initial angle is 0, use Quaternion = q = [1, 0, 0, 0] to represent the quaternion, where Quaternion is Represents the quaternion of the sensor relative to the earth, q is The quaternion representing the earth relative to the sensor is obtained according to the conjugate relationship of the quaternion: Quaternion = q * =[q0, -q1, -q2, -q3], where q0 is the real part and q1, q2, q3 are the imaginary parts; According to the static state detection results, the time window of the initial static state of the inertial measurement unit is selected, the average value of the acceleration vector measured in the static state within the window is calculated and normalized to obtain the unitized acceleration, where the acceleration measured in the static state is The unitized acceleration is Represents the unit component of the gravity direction in the inertial measurement unit coordinate system, a x 、a y 、a z is the acceleration in the x, y, and z directions in the inertial measurement unit coordinate system; According to the quaternion and the unitized acceleration, when the carrier is stationary, use Calculate the relationship between the accelerometer output and the acceleration due to gravity and use Estimate the direction of gravity, where a x 、a y 、a z is the acceleration in the x, y, and z directions in the inertial measurement unit coordinate system, q0 is the real part, q1, q2, and q3 are the imaginary parts, is the direction of gravity, g is the acceleration due to gravity, is the relationship between the accelerometer output and the acceleration due to gravity; The angle error is obtained by cross product calculation based on the gravity direction of the normalized acceleration and the estimated gravity direction; According to the angle error, use InitError=K i InitError+K P Error corrects the angular velocity of the gyroscope and uses Update the attitude of the inertial measurement unit, where IntError is the error integral term, K p and K i Respectively represent the calculation error gain and the integral error gain, Error is the error, is the updated quaternion, Δt is the time interval, t is the time, Represents the derivative of the quaternion, and the rate of change of the current posture quaternion is calculated using quaternion multiplication: represents quaternion multiplication, s w t is the constructed angular velocity quaternion; The above steps are iterated until the error between the estimated gravity direction and the actual gravity direction is minimized to obtain an accurate initial attitude estimate, that is, the initial attitude of the inertial measurement unit.

5. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 2, characterized in that: Based on the updated initial attitude of the inertial measurement unit, the velocity and displacement relative to the initial position in the Earth coordinate system are calculated to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point, including; According to the updated initial attitude of the inertial measurement unit, the acceleration measured by the inertial measurement unit is used Convert to the Earth reference system and get the acceleration in the Earth reference system, where E a represents the acceleration in the Earth reference frame, S a represents the acceleration in the inertial measurement unit reference frame, is the updated quaternion, represents quaternion multiplication, Represents the quaternion in the earth reference frame; According to the acceleration in the Earth's reference frame, use v t =v t-1 + E a t ·Δt calculates the velocity in the Earth coordinate system, where E a t represents the acceleration at time t, v t represents the speed at time t, and Δt is the time interval; According to the velocity in the earth coordinate system, combined with the rate of change of the velocity and the period of motion, the velocity drift is calculated using linear interpolation, and the velocity drift is subtracted from the velocity in the earth coordinate system to obtain the corrected velocity; According to the corrected velocity, the position matrix is initialized to obtain the reconstructed motion trajectory, that is, the posture information at the corresponding time point.

6. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 5, characterized in that: Convert video data frames into images, and accurately match the images with poses using 3D linear interpolation and 3D spherical interpolation based on timestamps to obtain the matched image poses, including: Convert video data frames into images to obtain a list of sampled video timestamps; According to the sampled video timestamp list, the video frame timestamps that are not within the timestamp range of the inertial measurement unit are removed to obtain a valid video frame timestamp list; According to the valid video frame timestamp list, based on the position and attitude of the inertial measurement unit at adjacent moments, the matched image pose is obtained. The position is achieved through three-dimensional linear interpolation, and the attitude is achieved through quaternion interpolation. The quaternion interpolation adopts spherical linear interpolation. The formula is: Where s is the interpolation factor, q0 is the inertial measurement unit attitude corresponding to the adjacent previous time stamp, q1 is the inertial measurement unit attitude corresponding to the adjacent next time stamp, and θ is the angle between q0 and q1.

7. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 6, characterized in that: According to the matched image pose, perform COLMAP sparse reconstruction based on pose prior to obtain a sparse reconstructed point cloud, including: According to the matched image pose, the key points and descriptors are extracted from the input image through the detection algorithm. The corresponding feature point pairs are matched by the similarity of the descriptors, and geometric verification is performed to obtain the matched feature point pairs. Sort all images according to the matched feature point pairs, select two images with more matched feature point pairs to initialize, and obtain the initialized images; Based on the initialized image, the image with the most matching points or the widest distribution of visible points is selected, and the pose of the new image is adjusted and optimized to obtain the incrementally reconstructed image; Repeat the above steps to optimize all generated 3D points and camera poses as a whole, and filter and eliminate the optimized point cloud and image to obtain a sparse reconstructed point cloud.

8. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 7, characterized in that: 3DGS reconstruction is performed based on the sparse reconstructed point cloud, including: According to the sparse reconstruction point cloud, the point cloud and camera parameters are obtained by sparse reconstruction using COLMAP, and the point cloud is initialized to a 3D Gaussian distribution, and the 3D Gaussian distribution is G i ={μ i ,Σ i ,α i ,c i }, where μ i is the position, Σ i is the covariance matrix, α i is the opacity, c i are the spherical harmonic coefficients; By minimizing the error between the reconstructed image and the training view according to the 3D Gaussian distribution, we use Optimize the parameters of the Gaussian distribution to obtain the optimized 3D Gaussian distribution parameters, where is the L1 loss, is the D-SSIM loss, λ is the balance parameter; During the 3D Gaussian distribution parameter optimization process, the number and density of Gaussian distributions are dynamically adjusted through cloning and splitting operations; According to the optimized 3D Gaussian distribution parameters, the tile rasterization algorithm is used for real-time rendering to obtain the rendered 3D Gaussian distribution; Repeat the above steps of 3D Gaussian distribution parameter optimization and real-time rendering until the preset number of iterations or convergence condition is reached.

9. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 8, characterized in that: During the 3D Gaussian distribution parameter optimization process, the number and density of Gaussian distributions are dynamically adjusted through cloning and splitting operations, including: If the small-scale Gaussian distribution position gradient is large, it means that the area needs more geometric details. Increase the number of Gaussian distributions through cloning operations: And α i <θ α , then clone If the large-scale Gaussian distribution has a large position gradient, it means that the area needs to be divided more finely. A large Gaussian distribution is split into two small Gaussian distributions through a split operation: And α i ≥θ α , then split Among them, μ i is the position, α i is the opacity, θ pos is the gradient threshold, θ α is the opacity threshold.

10. The improved method for 3D reconstruction based on Gaussian sputtering based on pose prior according to claim 8, characterized in that: Based on the optimized 3D Gaussian distribution parameters, a tile rasterization algorithm is used for real-time rendering to obtain the rendered 3D Gaussian distribution, including: Divide the screen into 16×16 pixel tiles based on the optimized 3D Gaussian distribution parameters; Use GPU quick sorting algorithm to perform deep sorting on the Gaussian distribution in each tile to obtain the sorted Gaussian distribution; Each pixel is rasterized according to the sorted Gaussian distribution and the Calculates the cumulative value of color and opacity, where p is the pixel position, C(p) is the color value of the pixel, N is the total number of samples, i is the index, and μ i is the position, α i is the opacity, c i are the spherical harmonic coefficients, and T is time.

Images