Three-dimensional mapping method and system for underground long-distance pipelines based on three-dimensional Gaussian sputtering

Abstract

The present invention provides a method and system for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering, the method comprising: collecting color images, depth images, and IMU data; performing pixel coordinate conversion to obtain point cloud coordinates; inputting the depth image, color image, and IMU data into a pose estimation algorithm, processing to obtain the predicted pose of the IMU data and visual pose estimation and visual pose estimation information; initializing the 3D Gaussian, performing back propagation through the 3D Gaussian loss function, and optimizing to obtain a suitable 3D model; based on the initialized 3D Gaussian, using the GS-SLAM algorithm, updating the suitable 3D model and visual pose estimation information, and using the 3D Gaussian loss function to process to obtain the optimized camera pose; updating the initial camera pose through Kalman filtering to complete the pipeline mapping operation. The technical problems of large device memory, high labor and time costs, easy error accumulation, easy pose offset, and difficulty in maintaining a high accuracy of the model are solved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional modeling, and in particular to a three-dimensional mapping method and system for underground long-distance pipelines based on three-dimensional Gaussian sputtering. Background Art

[0002] As an important transportation device, pipelines are widely used in power transmission, urban sewage, and petrochemical industries. In the planning, design, maintenance, and management of underground pipelines, 3D mapping of underground pipelines can not only improve the efficiency and accuracy of pipeline management, but also play an important role in emergency response, planning and design, asset management, and intelligent operation. With the advancement of technology and the reduction of costs, 3D mapping technology has become a core tool for underground pipeline management, promoting the digitalization and intelligent management of underground infrastructure. However, traditional pipeline 3D mapping methods often have the following limitations. For example, in the face of massive point cloud data collected by pipeline robots, traditional methods are difficult to process efficiently, resulting in slow model reconstruction and rendering that cannot meet real-time requirements; the internal structure of the pipeline is complex and changeable, and traditional methods perform poorly in dealing with noise interference, occluded areas, and other problems, resulting in insufficient accuracy of model reconstruction and rendering results; for different types of pipelines, including straight pipes, curved pipes, and branch pipes, it is difficult to achieve accurate mapping.

[0003] The existing public document "Three-dimensional reconstruction of urban drainage pipelines based on multi-view image matching of low-cost panoramic cameras" proposes to use low-cost video panoramic cameras as pipeline inspection tools and use spherical panoramic videos to achieve three-dimensional reconstruction of pipeline scenes. The panoramic camera is installed at a reasonable position of the inspection equipment so that the camera captures the omnidirectional field of view of the pipeline; when the inspection equipment moves in the pipeline, the video recording function of the camera is turned on. After the equipment is recovered, a panoramic video is generated, and a spherical panoramic image set is obtained by frame extraction. The edge geometric distortion of the panoramic image is corrected by generating a perspective image set using multi-view projection. The reprojected image set is processed by photogrammetry, and multi-view image matching, sfM (Structure from Motion, motion-based reconstruction) reconstruction, dense matching, meshing, and texture mapping are performed to obtain a three-dimensional scene model with real pipeline texture. However, this existing method requires data to be collected manually or by machine in advance, and cannot achieve real-time mapping of long-distance pipelines.

[0004] The existing public document "Research on the Three-Dimensional Mapping Method of Underground Tunnel Based on Lidar" provides a three-dimensional mapping method for underground tunnels in coal mines based on Lidar. This existing method builds a three-dimensional laser scanning system based on the characteristics of underground tunnels in coal mines and the intelligent requirements of cantilever tunneling machines; based on the characteristics of point cloud data of coal mine tunnels and the requirements of tunnel mapping, a cloth filtering algorithm is used to achieve the segmentation of point cloud data of the working surface of coal mine tunnels. However, the three-dimensional images obtained by the above-mentioned existing technologies have a large deviation from the actual results, and cannot meet the requirements of real-time mapping of long-distance pipelines.

[0005] The existing invention patent application document "Three-dimensional reconstruction method of the inner wall of the pipeline" with publication number CN109544679A, the existing scanning method includes: controlling the laser device and the camera device to move synchronously in the pipeline; controlling the laser device to emit laser to the inner wall of the pipeline to form laser stripes on the inner wall of the pipeline, and controlling the camera device to synchronously obtain the image information of the laser stripes at each moment from two different angles, and record the posture parameters of the camera device at each moment; obtaining the three-dimensional coordinates of the center point of the laser stripes at each moment in the coordinate system of the camera device corresponding to each moment according to the image information; reconstructing the inner wall of the pipeline in three dimensions according to the three-dimensional coordinates at multiple moments and the posture parameters of the camera device. By controlling the laser device and the camera device to move in the pipeline, the real-time image information of the laser stripes is dynamically obtained, and the image information of the inner wall of the pipeline is obtained in real time. However, the above-mentioned prior art requires a lot of manual work, the data volume required for reconstruction is large, and the memory and computing power of the equipment are required to be high.

[0006] The existing invention patent application document "Low-cost 360-degree panoramic video camera urban pipeline 3D reconstruction method and system" with publication number CN111462326A, the existing method uses a low-cost 360-degree panoramic video camera to perform 3D reconstruction of urban pipelines, uses a panoramic camera equipped with a fisheye lens to enter the drainage pipeline to shoot a 360-degree panoramic video, and extracts a single frame of 360-degree panoramic image from the panoramic video, and uses a sequence of panoramic images to perform 3D reconstruction of the pipeline, thereby constructing a 3D model of the underground pipeline. However, the above-mentioned existing method is prone to posture deviation in the 3D reconstruction of underground long-distance pipelines, and at the same time, the accuracy of the model obtained by the above-mentioned existing 3D reconstruction method is low.

[0007] Existing technology defects:

[0008] (1) The traditional method of 3D reconstruction of underground pipeline networks uses 3D laser scanning technology, which requires large equipment memory, high labor and time costs, and is prone to error accumulation.

[0009] 3D laser scanning technology can obtain a large amount of high-precision point cloud data, but this data is usually very large and requires a lot of computing resources and high-cost software tools to process. It also has high requirements for device memory, especially when the data quality is not high and the acquisition environment is complex. For complex underground pipe network scenes, data processing and analysis are difficult. Underground pipe networks usually require extensive scanning coverage, involving a large amount of data collection, processing and subsequent analysis, which is a large workload. Therefore, when using 3D laser scanning technology, the manpower and time costs required are also high. Due to the complex environment and structure of the underground pipe network, the accuracy of laser scanning may be limited by the accuracy of the equipment itself and the choice of scanning position and direction. In some complex underground environments, measurement errors may accumulate, affecting the accuracy of the final 3D model.

[0010] (2) The original data processing incremental sfM algorithm is prone to posture deviation in long-distance single scenes, and it is difficult for the model to maintain a high level of accuracy.

[0011] When the scene is too large, there is a lack of real depth information, and the sfM algorithm camera pose estimation is inaccurate. When errors occur in the camera position calculation of certain viewing angles, the visualization of the entire scene will be affected, and the intuitive display is that the ratio of the camera to the model will be abnormal. In addition, the features of underground pipeline data are fuzzy, similar, and ambiguous. When the features are not obvious enough and lead to matching errors, the accuracy of pose estimation is difficult to guarantee, resulting in pose offset, and ultimately making the 3D Gaussian render an incorrect and blurred scene. Image noise, lighting changes, or occlusion factors will affect feature extraction and matching.

[0012] In summary, the existing technology has technical problems such as large device memory, high labor and time costs, easy error accumulation, easy posture deviation, and difficulty in maintaining a high accuracy of the model. Summary of the invention

[0013] The technical problem to be solved by the present invention is: how to solve the technical problems in the prior art that the device has large memory, high labor and time costs, easy error accumulation, easy posture deviation, and it is difficult for the model to maintain a high accuracy.

[0014] The present invention adopts the following technical solutions to solve the above technical problems: A three-dimensional mapping method for underground long-distance pipelines based on three-dimensional Gaussian sputtering includes:

[0015] S1, collect color images, depth images, and IMU (Inertial Measurement Unit) data;

[0016] S2, using the depth map, color map and camera internal parameters, through pixel coordinate conversion operation, obtain the point cloud coordinates;

[0017] S3, input the depth map, color map and IMU data into the pose estimation algorithm, process the predicted pose of the IMU data and the visual pose estimation, and process the visual pose estimation information;

[0018] S4, inputting the point cloud coordinates, visual pose estimation information and color map into the three-dimensional Gaussian to obtain an initialized three-dimensional Gaussian, performing back propagation through the three-dimensional Gaussian loss function, performing adaptive density control, and optimizing to obtain a suitable three-dimensional model;

[0019] S5. Based on the initialized 3D Gaussian, use the GS-SLAM (Gaussian Splatting Simultaneous Localization and Mapping) algorithm to update the applicable 3D model and visual pose estimation information, and use the 3D Gaussian loss function to perform optimization operations to obtain the optimized camera pose.

[0020] S6. Through Kalman filtering, the camera pose is optimized and fused, the initial camera pose is updated, and the pipeline mapping operation is completed.

[0021] The present invention aims to solve the problems of low model accuracy, accumulated posture errors, and low real-time performance in long-distance pipeline mapping, and constructs an integrated method for achieving real-time rendering of three-dimensional images. The present invention uses three-dimensional Gaussian sputtering for three-dimensional mapping, which solves the problems of low accuracy and long time consumption of three-dimensional reconstruction in traditional methods, and ensures the accuracy and completeness of long pipeline mapping; Kalman filtering is used for posture update to solve the problems of posture offset and error accumulation of camera posture in long pipeline mapping, and at the same time solves the real-time problem of model reconstruction, thereby achieving long-distance accurate real-time mapping. The present invention uses Gaussian sputtering to map long-distance underground pipelines, which improves the quality and efficiency of mapping.

[0022] The present invention simplifies the operation steps of three-dimensional mapping of underground long-distance pipelines, reduces the dependence on professionals, and improves the efficiency of management and maintenance of underground pipeline networks.

[0023] In a more specific technical solution, in S2, the coordinate relationship between the pixel plane and the three-dimensional space point cloud is expressed; based on the coordinate relationship, the point cloud coordinates are derived; according to the point cloud coordinates, the first 3D points .

[0024] In a more specific technical solution, in S3, in the pose estimation algorithm ORB-SLAM3 (Oriented FAST and Rotated BRIEF Simultaneous Localization and Mapping3, synchronous positioning and map data for feature extraction of accelerated switching platforms and rotation binary descriptors), the angular velocity and acceleration data in the IMU data are used to calculate the position of the object in time. For the current The group pose is predicted to obtain the predicted pose of the IMU data; according to the predicted pose, the graph optimization operation is used to process the visual pose estimation information.

[0025] In a more specific technical solution, in S31, it is assumed that: ; Set from the Group to Group IMU data elapsed time , processed to obtain the predicted pose.

[0026] In a more specific technical solution, in S32, from the color map Extract ORB features from the image, perform pose estimation by optimizing the 2D reprojection error corresponding to the image, and jointly optimize the predicted pose and visual pose estimation to obtain the initial camera pose. .

[0027] In a more specific technical solution, in S4, Gaussian parameters are initialized for the point cloud coordinates, and a Gaussian ellipsoid set is generated based on the initialized sparse point cloud; the following logic is used to express the three-dimensional Gaussian initialization equation: :

[0028]

[0029]

[0030] in, are the coordinates of the center point of the Gaussian ellipsoid, Represents the rotation matrix of the three-dimensional Gaussian point relative to the world coordinate system, Represents the scaling matrix of the three coordinate axes of the three-dimensional Gaussian relative to the world coordinate system, Represents the transpose of the scaling matrix of the three coordinate axes of the three-dimensional Gaussian relative to the world coordinate system, represents the transpose of the rotation matrix of the three-dimensional Gaussian point relative to the world coordinate system, is the circumference parameter, represents the error covariance matrix, Represents a 3D point;

[0031] Set up Group images include Gaussian balls, define the colors of Gaussian balls and N-order Gaussian balls;

[0032] Using the three-dimensional Gaussian loss function Back propagation is used to complete the model optimization operation and output a suitable Gaussian model, in which the network gradient is updated by the back propagation algorithm and the network weight parameters are updated using the standard gradient descent optimization algorithm to minimize the three-dimensional Gaussian loss function , tune the rendered image, and use the following logic to calculate the loss function value between the rendered image and the original image :

[0033]

[0034]

[0035]

[0036] in, is the depth map reconstructed by the model, is the real depth map, is the structural similarity between the two images, The similarity parameter between the depth map reconstructed by the model and the real depth map, represents the loss function, represents the first loss function, represents the L1 norm of the vector, is the loss constant.

[0037] The present invention uses the back propagation algorithm to update the gradient in the network and uses the standard gradient descent optimization algorithm to update the weight parameters of the network to minimize the loss function. , thereby gradually improving the quality of the rendered image, making it closer to the original image.

[0038] By simulating the Gaussian distribution characteristics, the present invention can achieve high-precision pipe network positioning and imaging in complex underground environments, effectively improving the accuracy of mapping, while greatly reducing manual intervention, making the mapping process more automated and intelligent. In the mapping of underground pipe networks of the same length, the present invention significantly improves mapping efficiency and reduces energy consumption.

[0039] In a more specific technical solution, Group image The color of a Gaussian ball is represented by :

[0040]

[0041] in, represents the order of spherical harmonics, represents the maximum spherical harmonic order, Indicates the degree of the current order. The value range of the degree of the current order includes: [ -l,l ], Represents the order of spherical harmonics , the number of times at the current order The corresponding The spherical harmonic coefficients to be optimized are is the degree of the current order , the order of spherical harmonics The corresponding spherical harmonic basis functions , Represents the IMU measurement angle in the IMU data, is the azimuth, Indicates the IMU measurement angle and azimuth corresponding to the first The color of the Gaussian sphere;

[0042] Nth order The color of the Gaussian sphere It is expressed as:

[0043]

[0044]

[0045]

[0046] in, For the The opacity of the Gaussian sphere , For the The attenuation coefficient of a Gaussian sphere , For the The propagation distance of a Gaussian sphere , For the the opacity of the group image, N represents the Gaussian sphere set, Represents the number of Gaussian spheres.

[0047] In order to meet the demand for long-distance mapping of underground pipeline networks, the present invention adopts three-dimensional Gaussian and synchronous positioning and mapping methods to solve the problems of low accuracy and short reconstruction distance of traditional three-dimensional reconstruction models, and realizes accurate mapping of long-distance underground pipeline networks.

[0048] In a more specific technical solution, in S5, the GS-SLAM algorithm is used to convert the Initialize the three-dimensional Gaussian The optimized three-dimensional sputtering model is obtained by unbiased and lossless splicing of initialized three-dimensional Gaussians. and optimize the camera pose ;

[0049] Initialize Three-dimensional Gaussian to The pose of a three-dimensional Gaussian is: ,but , can get the first Applicable 3D Model for Group Prediction ,in, represents the initial rotation matrix, Indicates the initial time, Indicates the update time. represents the updated rotation matrix, Indicates Three-dimensional Gaussian to The error covariance matrix corresponding to the three-dimensional Gaussian is Represents the scaling matrix of the three coordinate axes of the three-dimensional Gaussian relative to the world coordinate system, represents the transpose of the scaling matrix, represents the transpose of the initial rotation matrix, Indicates the opacity of the group image, Indicates the Nth order The color of the Gaussian sphere;

[0050] Using the following logic, we define the Gaussian splicing loss function in the unbiased lossless splicing operation: :

[0051]

[0052]

[0053]

[0054] in, For the Gaussian model for group prediction, It is the first reconstruction by S4 Group 3D Gaussian, is the pose after splicing optimization, argmin represents the minimization optimization function, represents the minimization optimization function corresponding to the three-dimensional Gaussian. Represents the minimization optimization function corresponding to updating the rotation matrix and time t.

[0055] The present invention combines the three-dimensional Gaussian and simultaneous positioning and mapping methods, and uses the GS-SLAM algorithm to perform three-dimensional mapping of underground pipe networks, which greatly improves the accuracy and completeness of model reconstruction and ensures the accuracy of long-distance pipe network mapping. The present invention has lower computing power requirements for equipment than traditional three-dimensional reconstruction methods and is more suitable for the field of long-distance image rendering.

[0056] In a more specific technical solution, in S6, a Kalman filter is used to Initialize the camera pose and optimize the camera pose Fusion is performed to obtain the updated camera pose , in the Kalman filter, the Initialize the camera pose Set as the state vector and use the following logic to express the state transition model:

[0057]

[0058] in, For the Three-dimensional Gaussian to is the state transfer matrix of the three-dimensional Gaussian, Indicates The group initializes the camera pose, Indicates Process noise;

[0059] In the Kalman filter, the observation model is defined using the following logic:

[0060]

[0061] in, For the The observation matrix is ​​used to represent the Initialize the camera pose To Observation data The linear mapping of It is Observation noise in group IMU data;

[0062] Before each frame of data is updated, the Kalman filter predicts the current state through the state transition model:

[0063]

[0064]

[0065] in, The Kalman filter Group three-dimensional Gaussian to The predicted pose of the three-dimensional Gaussian group, Indicates The prediction error covariance matrix of the set of three-dimensional Gaussians, It is Group three-dimensional Gaussian to The prediction error covariance matrix of the three-dimensional Gaussian group represents the uncertainty of the prediction results. It is The covariance matrix of the process noise describes the size of the model prediction error. Indicates Three-dimensional Gaussian to The transpose of the state transfer matrix of a three-dimensional Gaussian;

[0066] In the Observation data When it reaches the Kalman filter, the Kalman filter updates the pose estimate based on the observed data using the following logic:

[0067]

[0068]

[0069]

[0070] in, To optimize the Kalman gain of the i-th three-dimensional sputtering pose, To optimize the Kalman gain of the i+1th three-dimensional sputtering posture, Observation data The degree of impact on status updates, Indicates +1 observation data, is the covariance matrix of the ith observation noise, describing the error of the observation data, is the predicted Kalman filter The predicted pose of the three-dimensional Gaussian group, is the identity matrix, is the i+1th observation matrix;

[0071] Using the following logic, the Kalman gain is processed to update the initial pose :

[0072]

[0073]

[0074] in, express, represents the i-th rotation matrix, represents the i-th updated rotation matrix, represents the i-th predicted rotation matrix, Indicates the Kalman gain to update the initial pose The corresponding moment, represents the i-th prediction moment, represents the i-th update time, Represents the identity matrix.

[0075] The present invention synchronously updates the initial camera pose through Kalman filtering, reduces the accumulation of pose errors in the mapping process, realizes real-time mapping of underground pipeline networks, saves the time cost of mapping, and ensures the accuracy and efficiency of long-distance mapping.

[0076] In response to the demand for real-time mapping of underground pipe networks, the present invention synchronously updates the camera pose through Kalman filtering, solves the problem of pose error accumulation during model reconstruction, and realizes real-time and accurate mapping of underground pipe networks.

[0077] In a more specific technical solution, the underground long-distance pipeline 3D mapping system based on 3D Gaussian sputtering includes:

[0078] Data acquisition module, used to collect color images, depth images, and IMU data;

[0079] A coordinate conversion module is used to obtain point cloud coordinates by using the depth map, color map and camera internal parameters through pixel coordinate conversion operations. The coordinate conversion module is connected to the data acquisition module.

[0080] A visual pose estimation module is used to input the depth map, color map and IMU data into the pose estimation algorithm, process it to obtain the IMU predicted pose and visual pose estimation, and process it to obtain the visual pose estimation information. The visual pose estimation module is connected to the coordinate conversion module;

[0081] The 3D model optimization module is used to input the point cloud coordinates, visual pose estimation information and color map into the 3D Gaussian to obtain the initialized 3D Gaussian, perform back propagation through the 3D Gaussian loss function, perform adaptive density control, and optimize to obtain a suitable 3D model. The 3D model optimization module is connected to the visual pose estimation module;

[0082] The model updating and posture optimization module is used to update the applicable 3D model and visual posture estimation information based on the initialization of 3D Gaussian and using the GS-SLAM algorithm, and to perform optimization operations using the 3D Gaussian loss function to obtain the optimized camera posture. The model updating and posture optimization module is connected to the 3D model optimization module;

[0083] The pipeline mapping module is used to complete the pipeline mapping operation by fusing and optimizing the camera pose through Kalman filtering and updating the initial camera pose. The pipeline mapping module is connected with the model update and pose optimization module.

[0084] Compared with the prior art, the present invention has the following advantages:

[0085] The present invention aims to solve the problems of low model accuracy, accumulated posture errors, and low real-time performance in long-distance pipeline mapping, and constructs an integrated method for achieving real-time rendering of three-dimensional images. The present invention uses three-dimensional Gaussian sputtering for three-dimensional mapping, which solves the problems of low accuracy and long time consumption of three-dimensional reconstruction in traditional methods, and ensures the accuracy and completeness of long pipeline mapping; Kalman filtering is used for posture update to solve the problems of posture offset and error accumulation of camera posture in long pipeline mapping, and at the same time solves the real-time problem of model reconstruction, thereby achieving long-distance accurate real-time mapping. The present invention uses Gaussian sputtering to map long-distance underground pipelines, which improves the quality and efficiency of mapping.

[0086] The present invention simplifies the operation steps of three-dimensional mapping of underground long-distance pipelines, reduces the dependence on professionals, and improves the efficiency of management and maintenance of underground pipeline networks.

[0087] The present invention uses the back propagation algorithm to update the gradient in the network and uses the standard gradient descent optimization algorithm to update the weight parameters of the network to minimize the loss function. , thereby gradually improving the quality of the rendered image, making it closer to the original image.

[0088] By simulating the Gaussian distribution characteristics, the present invention can achieve high-precision pipe network positioning and imaging in complex underground environments, effectively improving the accuracy of mapping, while greatly reducing manual intervention, making the mapping process more automated and intelligent. In the mapping of underground pipe networks of the same length, the present invention significantly improves mapping efficiency and reduces energy consumption.

[0089] In order to meet the demand for long-distance mapping of underground pipeline networks, the present invention adopts three-dimensional Gaussian and synchronous positioning and mapping methods to solve the problems of low accuracy and short reconstruction distance of traditional three-dimensional reconstruction models, and realizes accurate mapping of long-distance underground pipeline networks.

[0090] The present invention combines the three-dimensional Gaussian and simultaneous positioning and mapping methods, and uses the GS-SLAM algorithm to perform three-dimensional mapping of underground pipe networks, which greatly improves the accuracy and completeness of model reconstruction and ensures the accuracy of long-distance pipe network mapping. The present invention has lower computing power requirements for equipment than traditional three-dimensional reconstruction methods and is more suitable for the field of long-distance image rendering.

[0091] The present invention synchronously updates the initial camera pose through Kalman filtering, reduces the accumulation of pose errors in the mapping process, realizes real-time mapping of underground pipeline networks, saves the time cost of mapping, and ensures the accuracy and efficiency of long-distance mapping.

[0092] In response to the demand for real-time mapping of underground pipe networks, the present invention synchronously updates the camera pose through Kalman filtering, solves the problem of pose error accumulation during model reconstruction, and realizes real-time and accurate mapping of underground pipe networks.

[0093] The present invention solves the technical problems existing in the prior art, such as large device memory, high labor and time costs, easy error accumulation, easy posture deviation, and difficulty in maintaining a high accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 This is a schematic diagram of the basic steps of a method for three-dimensional mapping of underground long-distance pipelines based on three-dimensional Gaussian sputtering according to Example 1 of the present invention;

[0095] Figure 2 This is a comparison diagram of the actual underground long-distance pipeline and the underground long-distance pipeline modeling effect in Example 1 of the present invention. DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] The example of the present application solves the problem of low accuracy and inability to render long-distance pipeline models in real time by providing a long-distance pipeline real-time mapping method and system based on three-dimensional Gaussian sputtering.

[0098] Example 1

[0099] like Figure 1 As shown, the method for constructing three-dimensional underground long-distance pipelines based on three-dimensional Gaussian sputtering provided by the present invention includes the following basic steps:

[0100] S1, collect color images, depth images, and IMU data;

[0101] In this embodiment, the camera intrinsic parameters of the depth camera and the RGB (Red Green Blue) camera are calibrated, the color image is collected by the RGB camera, the depth image is collected by the depth camera, and the IMU data is collected by the gyroscope;

[0102] In this embodiment, due to the requirement for the same intrinsic parameters of the depth camera and the RGB camera, the intrinsic parameter matrices of the RGB camera and the depth camera are calibrated as follows: :

[0103]

[0104] In this embodiment, the RGB camera is used to obtain the color image and the depth camera is used to obtain the depth image; the gyroscope is used to obtain the linear acceleration and angular velocity of the accelerometer:

[0105]

[0106] In the application scenario of long pipeline mapping, it is necessary to ensure the smooth movement of the equipment. .

[0107] S2, using the depth map, the color map and the camera intrinsic parameters, through pixel coordinate conversion operation, to obtain point cloud coordinates;

[0108] In this embodiment, the camera's intrinsic parameter matrix , depth map, and color map are converted into pixel coordinates to obtain the point cloud coordinates in three-dimensional space.

[0109] In this embodiment, the following logic is used to express the coordinate relationship between the pixel plane and the three-dimensional space point cloud:

[0110]

[0111]

[0112]

[0113] in, is the focal length of the camera on the horizontal and vertical axes, is the center of the camera's aperture The corresponding principal point position, is the scaling factor of the depth map, z is the vertical coordinate value of the point cloud in three-dimensional coordinates, Represents the horizontal coordinate value of the point cloud in three-dimensional coordinates, Represents the vertical coordinate value of the point cloud in three-dimensional coordinates, Represents the pixel depth value, represents the coordinates of the center point of the Gaussian ellipsoid, represents the observation noise;

[0114] Based on the coordinate relationship, the point cloud coordinates are derived:

[0115]

[0116]

[0117]

[0118] According to the point cloud coordinates, the 3D point parameters :

[0119]

[0120] in, is the two-dimensional pixel coordinate of the color image, Indicates A color map, Indicates The two-dimensional pixel coordinates correspond to the transpose of the matrix, , represents a sequence of pixel depth values, Indicates Pixel depth value , is the camera's internal parameter matrix, including the camera's focal length on the horizontal and vertical axes and the main point position , Represents the inverse matrix of the camera's intrinsic parameter matrix.

[0121] S3, input the depth map, color map and IMU data into the pose estimation algorithm, process the predicted pose of the IMU data and the visual pose estimation, and process the visual pose estimation information;

[0122] In this embodiment, the depth map , color map , IMU data Input the pose estimation algorithm and initialize the pose of the RGB camera, denoted as .

[0123] In the pose estimation algorithm ORB-SLAM3, the angular velocity and acceleration data provided by the IMU are first used to calculate the position of the object in time. For the current The group pose is predicted, assuming: . Assume that from Group to Group IMU data elapsed time , using the following logic, the predicted pose is obtained:

[0124]

[0125]

[0126]

[0127] in, For the The measured angle increment in the group IMU data, For the The observation noise increment in the group IMU data, It is The predicted pose from the group IMU data, Indicates Linear acceleration in group IMU data, is the gyroscope in the IMU coordinate system Linear acceleration on the axis, Indicates IMU measurement angle in group IMU data , Represents the abscissa component of the IMU measurement angle, Represents the ordinate component of the IMU measurement angle, Represents the vertical coordinate component of the IMU measurement angle, Indicates Group to The time variation of the group IMU data, It is The group initializes the camera pose, Indicates An exponential function of the measured angle increments in the group IMU data.

[0128] In this embodiment, based on the IMU predicted pose, the visual pose estimation is obtained by using graph optimization.

[0129] From the color map Extract ORB (Oriented FAST and Rotated BRIEF, oriented to the accelerated switching platform and rotated binary descriptor) features, and perform pose estimation by optimizing the 2D reprojection error corresponding to the image:

[0130]

[0131]

[0132]

[0133] in, is the observation noise of each frame The corresponding posture, Indicates Observation noise The corresponding initialization camera pose, is the initial value of the custom pose, represents the observation noise The corresponding initial pose is Express posture, Indicates The estimated 3D points in space, is the 3D point Color map 2D projection on is a mapping function that projects a 3D point in space onto a 2D plane. is the camera’s intrinsic parameter matrix, Express posture With 3D points Multiply, is a sequence of 3D points, Indicates the degree of the current order, represents the pose sequence, Indicates the frame number. argmin represents the minimization optimization function, Represents the L2 norm of a vector;

[0134] In this embodiment, the predicted pose and visual pose estimation are jointly optimized to obtain the initial camera pose :

[0135]

[0136]

[0137] in, Pose estimation error for predicted pose and visual pose estimation , It is The predicted pose from the group IMU data, Indicates time The corresponding camera extrinsics.

[0138] S4, inputting the obtained point cloud data, pose data and color image into the three-dimensional Gaussian, initializing the three-dimensional Gaussian and optimizing the three-dimensional model;

[0139] In this embodiment, the point cloud data, posture data and color Figure 1 And input into the three-dimensional Gaussian, initialize the three-dimensional Gaussian, optimize the scene through the adaptive density control strategy of the back propagation of the three-dimensional Gaussian loss function, and obtain a three-dimensional model with higher accuracy;

[0140] In this embodiment, the Group of point clouds , color map and pose data Input 3D Gaussian, initialize 3D Gaussian , where the maximum number of iterations is set .

[0141] Initialize the Gaussian parameters of the point cloud coordinates and generate a Gaussian ellipsoid set based on the initialized sparse point cloud; the following formula is used to express the three-dimensional Gaussian initialization equation :

[0142]

[0143]

[0144] Set up Group images include Gaussian balls, define the colors of Gaussian balls and N-order Gaussian balls, where: ;No. Group image The color of a Gaussian ball is represented by :

[0145]

[0146] Nth order The color of the Gaussian sphere It is expressed as:

[0147]

[0148]

[0149]

[0150] In this embodiment, the loss function is used Back propagation is used to optimize the model and output a Gaussian model.

[0151] In this embodiment, the gradient in the network is updated by the back propagation algorithm, and the weight parameters of the network are updated using the standard gradient descent optimization algorithm to minimize the loss function , thereby gradually improving the quality of the rendered image, making it closer to the original image.

[0152] Using the three-dimensional Gaussian loss function Back propagation is used to complete the model optimization operation and output a suitable Gaussian model, in which the network gradient is updated by the back propagation algorithm and the network weight parameters are updated using the standard gradient descent optimization algorithm to minimize the three-dimensional Gaussian loss function , tune the rendered image, and use the following logic to calculate the loss function value between the rendered image and the original image :

[0153]

[0154]

[0155]

[0156] S5. Based on the initialization of the three-dimensional Gaussian, using the GS-SLAM algorithm, update the applicable three-dimensional model and visual pose estimation information, and use the three-dimensional Gaussian loss function to perform optimization operations to obtain the optimized camera pose; Initialize the three-dimensional Gaussian The optimized three-dimensional sputtering model is obtained by unbiased and lossless splicing of initialized three-dimensional Gaussians. and optimize the camera pose ;

[0157] Initialize Three-dimensional Gaussian to The pose of a three-dimensional Gaussian is: ,but , can get the first Applicable 3D Model for Group Prediction ,in, represents the initial rotation matrix, Indicates the initial time, Indicates the update time. represents the updated rotation matrix, Indicates Three-dimensional Gaussian to The error covariance matrix corresponding to the three-dimensional Gaussian is Represents the scaling matrix of the three coordinate axes of the three-dimensional Gaussian relative to the world coordinate system, represents the transpose of the scaling matrix, represents the transpose of the initial rotation matrix, Indicates the opacity of the group image, Indicates the Nth order The color of the Gaussian sphere;

[0158] In this embodiment, the Group 3D points of point cloud , color map and pose data Input 3D Gaussian.

[0159] After the three-dimensional reconstruction of S4, we get Gaussian Model .

[0160] In this embodiment, the GS-SLAM method is used to Gauss and Gaussians are unbiased and lossless spliced ​​to obtain an optimized three-dimensional sputtering model and optimize the camera pose .

[0161] Initialize Three-dimensional Gaussian to The pose of a three-dimensional Gaussian is: ,but , can get the first Applicable 3D Model for Group Prediction .

[0162] Define the Gaussian splicing loss function in the unbiased lossless splicing operation :

[0163]

[0164]

[0165]

[0166] S6. Through Kalman filtering, the camera pose is optimized and fused, the initial camera pose is updated, and the pipeline mapping operation is completed;

[0167] In this embodiment, the camera pose optimized by the three-dimensional Gaussian loss function in S5 and the initial camera pose are fused through Kalman filtering, and the initial camera pose in the algorithm is updated, thereby achieving accurate mapping of long-distance underground pipelines.

[0168] In this embodiment, a Kalman filter is used to Initialize the camera pose and optimize the camera pose Fusion is performed to obtain the updated camera pose .

[0169] In the Kalman filter, Initialize the camera pose Set as the state vector and use the following logic to express the state transition model:

[0170]

[0171] in, is the state transfer matrix, which describes the transition from the first Group to The group's status changes, is process noise, which represents random errors that cannot be fully described in the model prediction.

[0172] In the aforementioned Kalman filter, the observation model is defined as:

[0173]

[0174] Before each frame of data is updated, the Kalman filter predicts the current state through the state transition model:

[0175]

[0176]

[0177] When the observation data When it reaches the Kalman filter, the Kalman filter updates the pose estimate based on the observed data:

[0178]

[0179]

[0180]

[0181] Update the initial pose using the Kalman gain obtained above :

[0182]

[0183]

[0184] In this embodiment, the initial posture is updated, that is, the initial posture data is updated. Figure 2 The left figure shows the modeling effect of the actual pipeline. Figure 2 Picture on the right.

[0185] In summary, the present invention aims to solve the problems of low model accuracy, accumulated posture errors, and low real-time performance in long-distance pipeline mapping, and constructs an integrated method for realizing real-time rendering of three-dimensional images. The present invention uses three-dimensional Gaussian sputtering for three-dimensional mapping, which solves the problems of low three-dimensional reconstruction accuracy and long time consumption in traditional methods, and ensures the accuracy and completeness of long pipeline mapping; Kalman filtering is used for posture update to solve the problems of posture offset and error accumulation of camera posture in long pipeline mapping, and at the same time solves the real-time problem of model reconstruction, thereby realizing long-distance accurate real-time mapping. The present invention uses Gaussian sputtering to map long-distance underground pipelines, which improves the quality and efficiency of mapping.

[0186] The present invention simplifies the operation steps of three-dimensional mapping of underground long-distance pipelines, reduces the dependence on professionals, and improves the efficiency of management and maintenance of underground pipeline networks.

[0187] The present invention uses the back propagation algorithm to update the gradient in the network and uses the standard gradient descent optimization algorithm to update the weight parameters of the network to minimize the loss function. , thereby gradually improving the quality of the rendered image, making it closer to the original image.

[0188] By simulating the Gaussian distribution characteristics, the present invention can achieve high-precision pipe network positioning and imaging in complex underground environments, effectively improving the accuracy of mapping, while greatly reducing manual intervention, making the mapping process more automated and intelligent. In the mapping of underground pipe networks of the same length, the present invention significantly improves mapping efficiency and reduces energy consumption.

[0189] In order to meet the demand for long-distance mapping of underground pipeline networks, the present invention adopts three-dimensional Gaussian and synchronous positioning and mapping methods to solve the problems of low accuracy and short reconstruction distance of traditional three-dimensional reconstruction models, and realizes accurate mapping of long-distance underground pipeline networks.

[0190] The present invention combines the three-dimensional Gaussian and simultaneous positioning and mapping methods, and uses the GS-SLAM algorithm to perform three-dimensional mapping of underground pipe networks, which greatly improves the accuracy and completeness of model reconstruction and ensures the accuracy of long-distance pipe network mapping. The present invention has lower computing power requirements for equipment than traditional three-dimensional reconstruction methods and is more suitable for the field of long-distance image rendering.

[0191] The present invention synchronously updates the initial camera pose through Kalman filtering, reduces the accumulation of pose errors in the mapping process, realizes real-time mapping of underground pipeline networks, saves the time cost of mapping, and ensures the accuracy and efficiency of long-distance mapping.

[0192] In response to the demand for real-time mapping of underground pipe networks, the present invention synchronously updates the camera pose through Kalman filtering, solves the problem of pose error accumulation during model reconstruction, and realizes real-time and accurate mapping of underground pipe networks.

[0193] The present invention solves the technical problems existing in the prior art, such as large device memory, high labor and time costs, easy error accumulation, easy posture deviation, and difficulty in maintaining a high accuracy of the model.

[0194] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D mapping method for underground long-distance pipelines based on 3D Gaussian sputtering, characterized in that: The method comprises: S1, collect color images, depth images, and IMU data; S2, using the depth map, the color map and the camera internal parameters, through pixel coordinate conversion operation, to obtain point cloud coordinates; S3, input the depth map, color map and IMU data into the pose estimation algorithm, process the predicted pose of the IMU data and the visual pose estimation, and process the visual pose estimation information; S4, inputting the point cloud coordinates, visual pose estimation information and the color map into a three-dimensional Gaussian to obtain an initialized three-dimensional Gaussian, performing back propagation through a three-dimensional Gaussian loss function, performing adaptive density control, and optimizing to obtain a suitable three-dimensional model; Among them, the Gaussian parameters of the point cloud coordinates are initialized, and a Gaussian ellipsoid set is generated based on the initialized sparse point cloud; Set up Group images include Gaussian balls, define the colors of Gaussian balls and N-order Gaussian balls; Using the three-dimensional Gaussian loss function Back propagation is performed to complete the model optimization operation and output a Gaussian model. S5, based on the initialization of three-dimensional Gaussian, using the GS-SLAM algorithm, updating the applicable three-dimensional model and visual pose estimation information, using the three-dimensional Gaussian loss function to perform optimization operations, and obtaining the optimized camera pose; S6. Through Kalman filtering, the camera pose is optimized and fused, the initial camera pose is updated, and the pipeline mapping operation is completed.

2. The method for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering according to claim 1 is characterized in that: In S2, the coordinate relationship between the pixel plane and the three-dimensional space point cloud is expressed; based on the coordinate relationship, the point cloud coordinates are derived; according to the point cloud coordinates, the first 3D points .

3. The method for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering according to claim 1 is characterized in that: In S3, in the posture estimation algorithm ORB-SLAM3, the angular velocity and acceleration data in the IMU data are used to calculate the position of the pose in time. For the current The group pose is predicted to obtain the predicted pose of the IMU data; and according to the predicted pose, a graph optimization operation is used to process and obtain the visual pose estimation information.

4. The method for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering according to claim 3 is characterized in that: Set from Group to Set the IMU data to elapsed time , and process to obtain the predicted pose.

5. The method for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering according to claim 3 is characterized in that: From the color map Extract ORB features, perform pose estimation by optimizing the 2D reprojection error corresponding to the image; jointly optimize the predicted pose and visual pose estimation to obtain the initial camera pose .

6. The method for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering according to claim 1, characterized in that: In S4, the following logic is used to express the three-dimensional Gaussian initialization equation: : In the formula, are the coordinates of the center point of the Gaussian ellipsoid, Represents the rotation matrix of the three-dimensional Gaussian point relative to the world coordinate system, Represents the scaling matrix of the three coordinate axes of the three-dimensional Gaussian relative to the world coordinate system, represents the transpose of the scaling matrix of the three coordinate axes of the three-dimensional Gaussian with respect to the world coordinate system, represents the transpose of the rotation matrix of the three-dimensional Gaussian point relative to the world coordinate system, is the circumference parameter, represents the error covariance matrix, Represents a 3D point; The network gradient is updated by the back propagation algorithm, and the network weight parameters are updated using the standard gradient descent optimization algorithm to minimize the three-dimensional Gaussian loss function. , tune the rendered image, and use the following logic to calculate the loss function value between the rendered image and the original image : In the formula, is the depth map reconstructed by the model, is the real depth map, is the structural similarity between the two images, A similarity parameter representing the depth map reconstructed by the model and the real depth map, represents the loss function, represents the first loss function, represents the L1 norm of the vector, is the loss constant.

7. The method for constructing 3D underground long-distance pipelines based on 3D Gaussian sputtering according to claim 6, characterized in that: The said Group image The color of the Gaussian ball is expressed as : In the formula, represents the order of spherical harmonics, represents the maximum spherical harmonic order, Indicates the number of times at the current order, and the value range of the number of times at the current order includes: [ -l,l ], represents the order of the spherical harmonics , the number of times under the current order The corresponding The spherical harmonic coefficients to be optimized are is the degree of the current order , the order of the spherical harmonics The corresponding spherical harmonic basis functions , Represents the IMU measurement angle in the IMU data, is the azimuth, Indicates the IMU measurement angle in the IMU data, the first the color of the Gaussian sphere; Nth order The color of the Gaussian sphere It is expressed as: In the formula, For the The opacity of the Gaussian sphere , For the The attenuation coefficient of the Gaussian sphere is , For the The propagation distance of the Gaussian sphere , For the the opacity of the group image, N represents the Gaussian sphere set, Represents the number of Gaussian spheres.

8. The method for constructing three-dimensional mapping of underground long-distance pipelines based on three-dimensional Gaussian sputtering according to claim 1, characterized in that: In S5, the GS-SLAM algorithm is used to convert the The initialized three-dimensional Gaussian The initialized three-dimensional Gaussian is unbiased and lossless splicing is performed to obtain an optimized three-dimensional sputtering model. And the optimized camera pose ; Initialize Three-dimensional Gaussian to The pose of a three-dimensional Gaussian is: ,but , can get the first Applicable 3D Model for Group Prediction ,in, represents the initial rotation matrix, Indicates the initial time, Indicates the update time. represents the updated rotation matrix, Indicates The three-dimensional Gaussian The error covariance matrix corresponding to the three-dimensional Gaussian is, Represents the scaling matrix of the three coordinate axes of the three-dimensional Gaussian relative to the world coordinate system, represents the transpose of the scaling matrix, represents the transpose of the initial rotation matrix, Indicates the opacity of the group image, Indicates the Nth order the color of the Gaussian sphere; Using the following logic, the Gaussian splicing loss function in the unbiased lossless splicing operation is defined as : In the formula, For the Gaussian model for group prediction, is reconstructed by S4 Group 3D Gaussian, is the pose after splicing optimization, argmin represents the minimization optimization function, represents the minimization optimization function corresponding to the three-dimensional Gaussian, represents the updated rotation matrix and the minimized optimization function corresponding to the time t.

9. The method for 3D mapping of underground long-distance pipelines based on 3D Gaussian sputtering according to claim 1, characterized in that: In S6, a Kalman filter is used to convert the first Initialize the camera pose And the optimized camera pose Fusion is performed to obtain the updated camera pose In the Kalman filter, the Initialize the camera pose Set as the state vector and use the following logic to express the state transition model: In the formula, For the Three-dimensional Gaussian to is the state transfer matrix of the three-dimensional Gaussian, Indicates Set the initial camera pose, Indicates Process noise; In the Kalman filter, the observation model is defined using the following logic: In the formula, For the The observation matrix is ​​used to represent the Initialize the camera pose To Observation data The linear mapping of It is Observation noise in the IMU data of the group; Before each frame of data is updated, the Kalman filter predicts the current state through the state transition model: In the formula, is the first Group three-dimensional Gaussian to The predicted pose of the three-dimensional Gaussian group, Indicates The prediction error covariance matrix of the three-dimensional Gaussian is, It is Group the three-dimensional Gaussian into The prediction error covariance matrix of the three-dimensional Gaussian group represents the uncertainty of the prediction results, It is The covariance matrix of the process noise describes the size of the model prediction error. Indicates Three-dimensional Gaussian to The transpose of the state transfer matrix of a three-dimensional Gaussian; In the The observed data When it reaches the Kalman filter, the Kalman filter updates the pose estimate based on the observation data using the following logic: In the formula, To optimize the Kalman gain of the i-th three-dimensional sputtering pose, To optimize the Kalman gain of the i+1th three-dimensional sputtering posture, The observed data The degree of impact on status updates, Indicates +1 observation data, is the covariance matrix of the ith observation noise, describing the error of the observation data, The Kalman filter is the predicted The predicted pose of the three-dimensional Gaussian group, is the identity matrix, is the i+1th observation matrix; Using the following logic, the Kalman gain is processed to update the initial pose : In the formula, express, represents the i-th rotation matrix, represents the rotation matrix of the ith update, represents the i-th predicted rotation matrix, Indicates the Kalman gain to update the initial pose The corresponding moment, represents the i-th prediction moment, represents the i-th update time, Represents the identity matrix.

10. A 3D mapping system for underground long-distance pipelines based on 3D Gaussian sputtering, used to execute the 3D mapping method for underground long-distance pipelines based on 3D Gaussian sputtering as claimed in any one of claims 1 to 9, characterized in that: The system comprises: Data acquisition module, used to collect color images, depth images, and IMU data; A coordinate conversion module, used to obtain point cloud coordinates by using the depth map, the color map and the camera intrinsic parameters through pixel coordinate conversion operation, and the coordinate conversion module is connected to the data acquisition module; A visual pose estimation module, used for inputting the depth map, the color map and the IMU data into a pose estimation algorithm, processing to obtain the IMU predicted pose and the visual pose estimation, and processing to obtain the visual pose estimation information, wherein the visual pose estimation module is connected to the coordinate conversion module; A three-dimensional model optimization module, used for inputting the point cloud coordinates, the visual pose estimation information and the color map into a three-dimensional Gaussian to obtain an initialized three-dimensional Gaussian, performing back propagation through a three-dimensional Gaussian loss function, performing adaptive density control, and optimizing to obtain a suitable three-dimensional model, wherein the three-dimensional model optimization module is connected to the visual pose estimation module; A model updating and posture optimization module, which is used to update the applicable three-dimensional model and visual posture estimation information based on the initialized three-dimensional Gaussian using the GS-SLAM algorithm, and to perform optimization operations using the three-dimensional Gaussian loss function to obtain an optimized camera posture. The model updating and posture optimization module is connected to the three-dimensional model optimization module; The pipeline mapping module is used to fuse the optimized camera pose through Kalman filtering and update the initial camera pose to complete the pipeline mapping operation. The pipeline mapping module is connected to the model update and pose optimization module.

Images