Electromechanical pipeline model construction system and method based on oblique photography of unmanned aerial vehicle

Abstract

The invention relates to the technical field of three-dimensional modeling, in particular to an electromechanical pipeline model construction system and method based on unmanned aerial vehicle oblique photography. The method comprises the following steps: collecting electromechanical pipeline images; extracting feature points from the image by using an SfM technology, matching a multi-view image, reconstructing a sparse point cloud, and obtaining an initial three-dimensional structure and a camera track of a scene; initializing the point cloud into a 3D Gaussian ellipsoid, and decomposing a covariance matrix through a rotation matrix and a scaling matrix to optimize calculation; projecting the 3D ellipsoid to an image plane based on the view transformation matrix and the Jacobian matrix to realize rasterization rendering; calculating an error between a rendered image and a real image by adopting a mixed loss function, optimizing Gaussian primitive parameters through back propagation, and performing adaptive density control at the same time; a high-precision three-dimensional model is generated, BIM or CAD format export is supported, and visual display can be performed in a VR / AR environment; according to the invention, intelligent management and monitoring of electromechanical pipelines are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and particularly to a system and method for constructing an electromechanical pipeline model based on drone oblique photography. Background Art

[0002] In modern building construction and infrastructure operation and maintenance, the reasonable layout and efficient management of electromechanical pipelines are crucial. However, traditional electromechanical pipeline modeling methods mainly rely on manual measurement, laser scanning, and total station surveying. These methods not only have high costs and low efficiency but also face difficulties in data collection in complex building environments and large-space scenarios, making it difficult to obtain complete three-dimensional structure information. In recent years, due to its high efficiency and flexibility, drone oblique photography technology has been widely used in topographic surveying and building three-dimensional modeling, greatly improving the efficiency and accuracy of electromechanical pipeline three-dimensional modeling.

[0003] A Chinese invention patent with the publication number CN113920261B discloses a drone aerial photography modeling system and its modeling method. The system includes a control module, a wireless communication module, an aerial photography measurement module, a differential GPS module, a three-dimensional modeling module, and a storage module. The method includes controlling the control module through the wireless communication module and planning the flight trajectory of the drone device; by sleeving a rotating connecting column inside a hollow ring, binding the drone device and the shooting and measuring device together, the bearing connecting plate always remains parallel to the ground under the gravity of the load-bearing connecting block, panoramic camera, and laser rangefinder. The shaking, turning, etc. of the drone device during flight will not affect the state of the bearing connecting plate, thus ensuring the stability and accuracy of the panoramic camera and laser rangefinder when shooting and measuring the regional topography.

[0004] Traditional drone photogrammetry methods usually rely on GNSS signals for positioning. In GPS-free environments such as tunnels, underground pipe galleries, and inside buildings, due to the lack of high-precision position information, the modeling accuracy decreases and the error accumulates seriously, unable to meet the requirements of engineering applications. In addition, in these environments, electromechanical pipelines usually have a high density and complex spatial layout. Conventional structured light scanning or laser point cloud acquisition devices have high computational costs, cumbersome data processing processes, and are easily affected by factors such as light changes and occlusion interference when processing large-scale data, thus affecting the modeling quality. Summary of the Invention

[0005] The object of the present invention is to address the problems in the background art and propose a system and method for constructing an electromechanical pipeline model based on drone oblique photography.

[0006] The technical solution of the present invention: A method for constructing an electromechanical pipeline model based on drone oblique photography includes the following specific implementation steps:

[0007] S1. Select the electromechanical pipeline area as the modeling target, plan the UAV flight path based on the on-site drawings and pre-measured data, and instruct the UAV to collect continuous images;

[0008] S2. Extract feature points from the collected images and perform matching, calculate the fundamental matrix based on the epipolar constraint, convert it to the essential matrix using the intrinsic matrix, decompose the essential matrix to obtain the rotation matrix and translation vector, construct the projection matrix, solve for the 3D points using the triangulation method, and adopt bundle adjustment to minimize the reprojection error, optimize the camera parameters and 3D point positions, and finally output the sparse point cloud data;

[0009] S3. Initialize the 3D Gaussian ellipsoid for the point cloud through the Structure from Motion (SfM) technique, optimize the calculation using covariance matrix decomposition, represent the color and opacity using spherical harmonic functions, project it onto the 2D plane based on the view transformation and Jacobian matrix, render it through the rasterization method, calculate the pixel color and opacity, construct a loss function by combining the gray error and SSIM, and optimize the parameters using backpropagation until convergence, and then output the 3D model of the electromechanical pipeline;

[0010] S4. Dynamically adjust the density of Gaussian primitives through error analysis. For under-reconstructed areas, detect them using loss gradient analysis, and enhance the local coverage through primitive cloning and position fine-tuning. For over-reconstructed areas, count the primitive density and perform merging or splitting to optimize the calculation. Update the number of primitives based on the error feedback in each iteration to ensure complete modeling details and efficient calculation;

[0011] S5. Integrate the optimized 3D Gaussian ellipsoid parameters, generate the 3D model of the electromechanical pipeline, convert it to the BIM or CAD format, and simultaneously perform interactive 3D display.

[0012] Preferably, the generation process of the sparse point cloud data is as follows:

[0013] S21. Arbitrarily select two images. The feature points detected in the two images are: x = [u, v, 1] T and x' = [u', v', 1] T ;

[0014] where [u, v] and [u', v'] respectively represent the pixel coordinates on the image plane; T represents the transpose operation of the matrix;

[0015] S22. For any pair of matching points x and x', satisfy the epipolar constraint defined by the fundamental matrix F: (x') T Fx = 0, and thus obtain the fundamental matrix F;

[0016] S23. Using the intrinsic matrix K, convert the fundamental matrix F to the essential matrix E that describes the camera motion information: E = (K') T FK;

[0017] Among them, K and K' are the intrinsic parameter matrices of two images respectively;

[0018] S24. Decompose the essential matrix E into a rotation R and a translation component, and its theoretical expression is: E = [t] × R;

[0019] Among them, R is a 3×3 rotation matrix; t is a translation vector; [t] × represents the skew-symmetric matrix of t;

[0020] S25. For the matching points x and x' that satisfy: x = PX, x' = P'X, construct a linear equation system to solve the optimal estimate of X;

[0021]

[0022] Among them, and respectively represent the i-th row of the projection matrices P and P';

[0023] S26. Minimize the reprojection error of all points:

[0024] Among them, x ij represents the image coordinate of the i-th feature point observed in the i-th camera; represents the point reprojected onto the image plane according to the currently estimated 3D point X i and the projection matrix of camera j;

[0025] S27. Output sparse point cloud data.

[0026] Preferably, the construction process of the 3D model of the mechanical and electrical pipeline is as follows:

[0027] S31. Initialize each sparse point as a 3D Gaussian ellipsoid, and use a 3D Gaussian distribution to describe the local spatial information;

[0028] S32. Project the covariance matrix:

[0029] ∑ 2D = JW∑W T J T ;

[0030] In the formula, ∑ represents the covariance matrix of the original 3D Gaussian ellipsoid; W represents the view transformation matrix; J represents the Jacobian matrix of the projection transformation; ∑ 2D represents the covariance matrix of the 2D Gaussian ellipsoid after projection onto the image plane; T represents the transpose operation of the matrix;

[0031] S33. Use the reconstructed 2D Gaussian ellipsoid information to generate a rendered image through a rasterization method;

[0032] S34. Reverse-optimize the Gaussian ellipsoid parameters by calculating the error between the rendered image and the real captured image;

[0033] S35. Output the structural similarity loss and construct the 3D model of the mechanical and electrical pipelines.

[0034] Preferably, the initialization process of initializing each sparse point as a 3D Gaussian ellipsoid is as follows:

[0035] S41. Extract the sparse point cloud data and initialize the sparse point cloud number as a 3D Gaussian ellipsoid:

[0036]

[0037] where x represents the coordinates of any point in space; μ represents the center of the 3D Gaussian ellipsoid; ∑ represents the covariance matrix;

[0038] S42. Decompose the covariance matrix ∑ into the product of a rotation matrix R and a scaling matrix S, that is: ∑ = RSS T R T ;

[0039] where R represents the rotation matrix; S represents the scaling matrix;

[0040] S43. Color and opacity representation: color information and opacity α.

[0041] Preferably, the rendering process of generating the rendered image by the rasterization method is as follows:

[0042] S51. Point sputtering and pixel synthesis: Divide the entire image into image blocks of a fixed size. For each image block, select all the Gaussian basis elements within the viewing cone and with a confidence level higher than the set threshold, and calculate the pixel color layer by layer after sorting by depth:

[0043]

[0044] In the formula, c i represents the color information of the i-th Gaussian basis element; C(p) represents the final color of pixel p; N p represents the set of Gaussian basis elements that affect pixel p; w i represents the weight;

[0045] S52. Opacity calculation: The opacity α of each Gaussian basis element i plays a role in shielding the subsequent basis elements during rendering and adopts:

[0046] where, represents the initial opacity parameter; G i(x) represents the Gaussian response value of the primitive at pixel x.

[0047] Preferably, the optimization process for reverse-optimizing the Gaussian ellipsoid parameters is as follows:

[0048] S61. Construct the loss function: L = (1 - A)L y + AL ssim ;

[0049] where L represents the total loss; A represents the balance coefficient; L ssim represents the structural similarity loss; L y represents the absolute error of all pixels; L ssim represents the structural similarity loss;

[0050] S62. Calculate the absolute error of all pixels in the image:

[0051] where M represents the total number of pixels in the image; y i represents the gray value of the i-th pixel in the rendered image; represents the gray value of the i-th pixel in the corresponding real captured image;

[0052] S63. Calculate the structural similarity loss L ssim .

[0053] Preferably, the calculation process of the structural similarity loss is as follows:

[0054] S71. Arbitrarily select two images, define I as the data of the first image, as the data of the second image;

[0055] S72. Calculate the mean μ I and variance σ I and as well as the correlation

[0056] S73. Calculate the luminance similarity C luminance , contrast similarity C contrast and structural similarity C structure :

[0057]

[0058] C1 = (K1L) 2 ;

[0059] C2 = (K2L) 2 ;

[0060]

[0061] Among them, C1, C2, and C3 are small constants for stable calculation to avoid a zero denominator; L is the dynamic range of pixel values; K1 and K2 are very small constants;

[0062] S74. Calculate the structural similarity loss L ssim :

[0063]

[0064] The technical solution of the present invention: A system for constructing an electromechanical pipeline model based on UAV oblique photography, which is used to execute the above method for constructing an electromechanical pipeline model based on UAV oblique photography, includes:

[0065] A UAV cruise data acquisition module, which is used to autonomously cruise in the target area based on a UAV equipped with an optical camera to obtain electromechanical pipeline image data;

[0066] An SfM point cloud initialization module, which is used to recover a sparse point cloud from the images taken by the UAV based on the Structure from Motion (SfM) technology and provide initial position information;

[0067] A 3D modeling module, which is used to construct a high-precision electromechanical pipeline model using the 3D Gaussian Splatting (3DGS) technology, and perform point cloud optimization and rendering;

[0068] An adaptive density optimization module, which is used to improve the modeling accuracy and detail retention by automatically cloning Gaussian points and adjusting the rendering resolution strategy;

[0069] A model output and visualization module, which is used to store the generated three-dimensional model in the BIM format, support docking with CAD / BIM software, and perform engineering management visualization.

[0070] Compared with the prior art, the above technical solution of the present invention has the following beneficial technical effects:

[0071] The present invention designs a system and method for constructing an electromechanical pipeline model based on UAV oblique photography. Through technologies such as SfM point cloud reconstruction, 3D Gaussian Splatting rendering, and adaptive density optimization, it realizes high-precision, automated, and intelligent electromechanical pipeline modeling, and has broad engineering application value:

[0072] (1) Adapt to the GPS-free environment and achieve high-precision 3D modeling: By using SfM point cloud reconstruction + 3D Gaussian Splatting rendering, it can accurately restore the spatial layout and structural details of electromechanical pipelines in GPS-free signal environments such as underground tunnels, indoor electromechanical wells, and large factories, overcoming the limitations of traditional laser scanning and surveying methods;

[0073] (2) Automated data collection and modeling to improve efficiency: Through autonomous drone cruising + oblique photography, manual measurement intervention is significantly reduced, enabling efficient data acquisition. Combining with SfM to automatically calculate the camera trajectory without manual calibration improves modeling accuracy and work efficiency;

[0074] (3) 3D Gaussian Splatting (3DGS) to enhance rendering quality: The Gaussian ellipsoid modeling method effectively avoids topological discontinuity problems during triangular mesh reconstruction, ensuring the smoothness of pipeline surface details. Based on rasterization-based parallel rendering, it can efficiently generate high-quality 3D visualization models of mechanical and electrical pipelines, suitable for application scenarios such as engineering management, construction monitoring, and intelligent operation and maintenance;

[0075] (4) Adaptive density optimization to ensure refined modeling: Adopting the adaptive cloning + point cloud segmentation strategy, automatic supplementation of under-reconstructed areas and optimization and compression of over-reconstructed areas are achieved, improving the integrity of modeling details and computational efficiency. Combining error feedback to adjust the modeling density reduces the computational burden while ensuring model accuracy;

[0076] (5) Compatible with BIM / CAD systems to support intelligent engineering management: The generated 3D mechanical and electrical pipeline model can be converted into BIM formats (such as Revit, Navisworks, AutoCAD), supporting applications such as construction management, completion acceptance, and post-maintenance, promoting the digital management of mechanical and electrical pipelines. Brief Description of the Drawings

[0077] Figure 1 It is the system architecture diagram of a mechanical and electrical pipeline model construction system based on drone oblique photography proposed by the present invention;

[0078] Figure 2 It is the principle block diagram of a method for constructing a mechanical and electrical pipeline model based on drone oblique photography proposed by the present invention. Detailed Embodiments

[0079] Example 1, as Figure 1 shown, a mechanical and electrical pipeline model construction system based on drone oblique photography proposed by the present invention includes: a drone cruising data collection module, an SfM point cloud initialization module, a 3D modeling module, an adaptive density optimization module, and a model output and visualization module.

[0080] The drone cruising data collection module configures a drone with an optical camera to autonomously cruise in the target area to obtain mechanical and electrical pipeline image data;

[0081] The SfM point cloud initialization module, based on SfM (Structure from Motion) technology, recovers sparse point clouds from the images captured by the drone to provide initial position information;

[0082] The 3D modeling module uses 3DGS (3D Gaussian Splatting) technology to construct a high-precision electromechanical pipeline model, and performs point cloud optimization and rendering;

[0083] The adaptive density optimization module improves the modeling accuracy and detail retention by automatically cloning Gaussian points and adjusting the rendering resolution strategy;

[0084] The model output and visualization module stores the generated 3D model in BIM format, supports the docking of CAD / BIM software, and realizes the visualization of project management.

[0085] Example 2, as Figure 2 shown, a method for constructing an electromechanical pipeline model based on drone oblique photography proposed by the present invention is applied to a system for constructing an electromechanical pipeline model based on drone oblique photography proposed in Example 1, and its specific implementation steps are as follows:

[0086] S1. Select the electromechanical pipeline area as the modeling target, and plan the drone flight path according to the on-site drawings and pre-measured data to ensure full coverage of the pipeline and key connection parts;

[0087] It should be noted that the selected electromechanical pipeline area is a representative electromechanical pipeline area in an indoor, underground tunnel or an environment without GPS signal;

[0088] Accordingly, the drone automatically flies according to the preset path, and uses a high-resolution oblique photography camera, an inertial measurement unit (IMU) and a visual inertial odometer (VIO) to collect continuous images at different angles including but not limited to top view, side view and oblique view.

[0089] S2. The SfM point cloud initialization module performs multi-view image matching and sparse point cloud reconstruction. Specifically:

[0090] S21. Extract representative local features from multiple images collected by the drone and match them between different images. Let the feature points detected in two images be: x = [u, v, 1] T and x' = [u', v', 1] T ;

[0091] where [u, v] and [u', v'] respectively represent the pixel coordinates on the image plane; T represents the transpose operation of the matrix;

[0092] During the matching process, corresponding point pairs between images are established by comparing the similarity of descriptors;

[0093] S22. For any pair of matching points x and x', they satisfy the epipolar constraint defined by the fundamental matrix F: (x') T Fx = 0, and based on this, the fundamental matrix F is obtained;

[0094] S23. Since the images collected by the drone are usually calibrated by the camera and its internal parameter matrix K is known, using the internal parameter matrix, the fundamental matrix F is converted into the essential matrix E that describes the camera motion information: E = (K') T FK;

[0095] where K and K' are the internal parameter matrices of the two images respectively (if the cameras are the same, then K = K');

[0096] Accordingly: The essential matrix E captures the information of the relative pose (rotation and translation) between the cameras;

[0097] S24. Decompose the essential matrix E into the rotation R and the translation component, and its theoretical expression is: E = [t] × R;

[0098] where R is a 3×3 rotation matrix that describes the rotation from the first image to the second image; t is the translation vector (direction information) that represents the relative translation between the two camera centers; [t] × represents the skew-symmetric matrix of t;

[0099] Accordingly: Four sets of possible solutions are obtained based on the SVD decomposition of E, and the correct R and t are finally determined by combining the positive depth constraint (i.e., the reconstructed 3D points are in front of the camera);

[0100] S25. Assume that the projection matrix of the first image is P = K[I|0], and the second image is P' = K'[R|t]. Calculate the three-dimensional position X of the corresponding matching points through triangulation, specifically:

[0101] Assume that for the matching points x and x' satisfy: x = PX, x' = P'X;

[0102] where P is the projection matrix;

[0103] To solve for X (in homogeneous coordinates), construct a linear equation system (construct an overdetermined system):

[0104]

[0105] where, represents the i-th row of the projection matrix (such as is the third row);

[0106] Accordingly: Solve the optimal estimate of X by the least squares method;

[0107] S26. To optimize all camera parameters (rotation R, translation t) and the position of the 3D points X simultaneously, the bundle adjustment method is adopted, and its objective is to minimize the reprojection error of all points:

[0108]

[0109] where x ij represents the image coordinates of the i-th feature point observed in the i-th camera; represents the point reprojected onto the image plane according to the currently estimated 3D point X i and the projection matrix of camera j;

[0110] Accordingly: The bundle adjustment is solved by a non-linear least squares optimization algorithm (Levenberg-Marquardt) to minimize the overall reprojection error, thereby improving the accuracy of point cloud and camera pose estimation;

[0111] S27. Output sparse point cloud data.

[0112] S3. The 3D modeling module uses 3DGS (3D Gaussian Splatting) technology to construct a high-precision electromechanical pipeline model and perform point cloud optimization and rendering, specifically as follows:

[0113] S31. Initialization of 3D Gaussian ellipsoid set: To transform the SfM point cloud into a more suitable expression for refined modeling, each sparse point is initialized as a 3D Gaussian ellipsoid, that is, a 3D Gaussian distribution is used to describe the local spatial information, specifically as follows:

[0114] S3101. Standard 3D Gaussian distribution formula: Extract the sparse point cloud data and initialize the sparse point cloud as a 3D Gaussian ellipsoid. Specifically, for each point cloud data, a 3D Gaussian function containing position information and shape information is used to generate the 3D Gaussian ellipsoid:

[0115]

[0116] where x represents the coordinates of any point in space; μ represents the center (mean vector) of the 3D Gaussian ellipsoid; ∑ represents the covariance matrix, which reflects the shape of the Gaussian ellipsoid (including but not limited to the major and minor axes, rotation angle), and determines the distribution width of the local point cloud in each direction;

[0117] S3102. Decompose the covariance matrix ∑ into the product of a rotation matrix R and a scaling matrix S, that is: ∑ = RSS T R T ;

[0118] Among them, R represents the rotation matrix, which is used to align the ellipsoid with a certain reference axis in the world coordinate system; S represents the scaling matrix (in this embodiment, it is a diagonal matrix), which describes the scale of the ellipsoid in the directions of each main axis and reflects the "ductility" of the local area;

[0119] Accordingly: Rotate the 3D Gaussian ellipsoid to be flush with the elliptical world, then scale along the axis, and then rotate it back. This decomposition method can not only ensure the positive semi-definite property of the covariance matrix but also reduce the computational difficulty between matrices;

[0120] S3103. Color and opacity representation:

[0121] Color information: For the color (RGB value) of each Gaussian ellipsoid, continuous spherical harmonic functions are used for expression. The spherical harmonic functions, as a set of basis functions expanded on the sphere, describe the color changes under different viewing angles;

[0122] Opacity α: It is directly set as a learnable parameter, which describes the contribution of this Gaussian basis element to the final pixel color during rendering;

[0123] S32. To achieve image rendering, the 3D Gaussian ellipsoid must be projected onto the image plane of the camera. Specifically: According to the perspective projection theory in computer graphics, the following formula is used to project the covariance matrix:

[0124] ∑ 2D =JW∑W T J T ;

[0125] In the formula, ∑ represents the covariance matrix of the original 3D Gaussian ellipsoid; W represents the view transformation matrix, which converts the world coordinate system to the camera coordinate system, and its parameters are obtained by camera calibration; J represents the Jacobian matrix of the projection transformation, which is used to make a local linear approximation for the non-linear perspective projection to ensure the accuracy of the 2D projection result; ∑ 2D represents the covariance matrix of the 2D Gaussian ellipsoid projected onto the image plane and is used for subsequent rasterization rendering;

[0126] Accordingly: Maintain the accurate expression of the 3D local structure in the 2D image, so that subsequent rendering can truly reflect the spatial geometry and visual details;

[0127] S33. Use the reconstructed 2D Gaussian ellipsoid information to generate a rendered image through the rasterization method. Specifically:

[0128] S3301. Point sputtering and pixel synthesis: Divide the entire image into image blocks of a fixed size (in this embodiment, 16×16). For each image block, select all Gaussian basis elements within the viewing cone and with a confidence level higher than the set threshold, and calculate the pixel color layer by layer in depth order after sorting by depth:

[0129]

[0130] In the formula, c i represents the color information of the i-th Gaussian basis element; C(p) represents the final color of pixel p; N p represents the set of Gaussian basis elements that affect pixel p; w i represents the weight, which is determined by the opacity of the basis element and the coverage degree after projection;

[0131] S3302. Opacity calculation: The opacity α of each Gaussian basis element i plays a role in shielding the subsequent basis elements during rendering, and is adopted as:

[0132] wherein, represents the initial opacity parameter; G i (x) represents the Gaussian response value of the basis element at pixel x, reflecting the attenuation degree of its contribution to the pixel;

[0133] S34. By calculating the error between the rendered image and the real captured image, the Gaussian ellipsoid parameters are optimized inversely, specifically:

[0134] S3401. Construction of loss function: A hybrid loss function is adopted, taking into account the pixel-level gray error and structural similarity (SSIM), and the formula is: L = (1 - A)L y + AL ssim ;

[0135] wherein, L represents the total loss; A represents the balance coefficient (0 - 1), which is used to adjust the importance of the two parts of the loss; L y represents the gray error loss; L ssim represents the structural similarity loss; L ssim represents the structural similarity loss;

[0136] S3402. Gray error loss: Calculate the absolute error of all pixels in the image:

[0137] Calculate the absolute error of all pixels in the image:

[0138] wherein, M represents the total number of pixels in the image; y i represents the gray value of the i-th pixel in the rendered image; represents the gray value of the i-th pixel in the corresponding real captured image;

[0139] S3403. Structural similarity loss L ssim : According to the SSIM index, measure the differences in image brightness, contrast and structure, and its calculation process includes but is not limited to:

[0140] Brightness comparison: Using the image mean for comparison;

[0141] Contrast comparison: Using the image standard deviation for comparison;

[0142] Structure comparison: Using the covariance between pixels for comparison;

[0143] Specifically:

[0144] (1) Let I be the data of the first image, and

[0145] (2) Calculate the mean μ I and variance σ I and as well as the correlation between images

[0146] (3) Calculate the brightness similarity C luminance , contrast similarity C contrast and structure similarity C structure :

[0147]

[0148] C1 = (K1L) 2 ;

[0149] C2 = (K2L) 2 ;

[0150]

[0151] Among them, C1, C2, and C3 are small constants used for stable calculation to avoid the denominator being zero, given by the definition of the SSIM formula; L is the dynamic range of pixel values (for example, L = 255 for an 8-bit image); K1 and K2 are very small constants (in this embodiment, 0.01 and 0.03 are taken respectively);

[0152] (4) Calculate the structure similarity loss L ssim :

[0153]

[0154] S3404. Through the backpropagation algorithm, the gradients of the loss function with respect to each parameter (including but not limited to μ, R, S, color, and opacity) are passed, and Adam is used for update. The optimization process is iterated until the error between the rendered image and the real image reaches the preset convergence criterion;

[0155] S35. Output the structure similarity loss L ssim , and construct a three-dimensional model of mechanical and electrical pipelines.

[0156] S4. During the parameter optimization process of the adaptive density optimization module, to prevent insufficient or redundant local detail reconstruction, the density of Gaussian basis elements is adaptively adjusted, specifically as follows:

[0157] Case 1: Processing of under-reconstructed regions:

[0158] Detection method: Through loss gradient analysis, find the regions with large errors and determine them as under-reconstructed regions;

[0159] Optimization strategy: Clone (copy) the existing Gaussian basis elements in these regions and slightly adjust the positions of the newly generated basis elements to increase local coverage;

[0160] Case 2: Processing of over-reconstructed regions:

[0161] Detection method: Statistically analyze the density of Gaussian basis elements within the region. When the density exceeds a certain threshold, it is determined as an over-reconstructed region;

[0162] Optimization strategy: Merge or split the severely overlapping basis elements to reduce redundant calculations and improve the overall rendering efficiency;

[0163] Accordingly: In each optimization iteration, dynamically update the number of basis elements in each region according to the error feedback to ensure that the overall modeling has both sufficient details and maintains computational efficiency.

[0164] S5. The model output and visualization module integrates all the 3D Gaussian ellipsoid parameters after multiple iterations of optimization to generate a complete 3D model of the mechanical and electrical pipelines. The model not only contains fine geometric shapes but also integrates visual information such as color and opacity;

[0165] And convert the generated model into BIM or CAD formats (including but not limited to the formats supported by Revit, Navisworks, AutoCAD) for project management and subsequent applications, and at the same time, an interactive 3D display can be carried out using the VR / AR platform.

[0166] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to this. Various changes can be made without departing from the spirit of the present invention within the knowledge scope of those skilled in the relevant art.

Claims

1. A method for constructing an electromechanical pipeline model based on drone oblique photography, characterized in that, It includes the following specific implementation steps: S1. Select the electromechanical pipeline area as the modeling target, plan the UAV flight path based on the on-site drawings and pre-measured data, and command the UAV to collect continuous images; S2. Extract feature points from the collected images and perform matching, calculate the fundamental matrix based on the epipolar constraint, and convert it into the essential matrix using the intrinsic matrix. Decompose the essential matrix to obtain the rotation matrix and translation vector, construct the projection matrix, solve for the 3D points using the triangulation method, and use bundle adjustment to minimize the reprojection error, optimize the camera parameters and the 3D point positions, and finally output the sparse point cloud data; S3. Initialize the 3D Gaussian ellipsoid for the point cloud through the structure from motion technology, optimize the calculation using covariance matrix decomposition, and represent the color and opacity using spherical harmonics. After projecting onto the 2D plane based on the view transformation and Jacobian matrix, render through the rasterization method, calculate the pixel color and opacity, construct a loss function combining the gray error and SSIM, and optimize the parameters using backpropagation until convergence and then output the 3D model of the electromechanical pipeline; S4. Dynamically adjust the Gaussian basis element density through error analysis. For under-reconstructed areas, detect using loss gradient analysis, and enhance the local coverage through basis element cloning and position fine-tuning. For over-reconstructed areas, count the basis element density and perform merging or splitting to optimize the calculation. Update the number of basis elements based on the error feedback in each iteration to ensure complete modeling details and efficient calculation; S5. Integrate the optimized 3D Gaussian ellipsoid parameters, generate the 3D model of the electromechanical pipeline, and convert it into the BIM or CAD format, and at the same time perform interactive 3D display.

2. The method for constructing an electromechanical pipeline model based on drone oblique photography according to claim 1, wherein The generation process of the sparse point cloud data is as follows: S21. Arbitrarily select two images, and the feature points detected in the two images are respectively: x = [u, v, 1] T and x' = [u', v', 1] T ; Among them, [u, v] and [u', v'] respectively represent the pixel coordinates on the image plane; T represents the transpose operation of the matrix; S22. For any pair of matching points x and x', they satisfy the epipolar constraint defined by the fundamental matrix F: (x') T Fx = 0, and based on this, the fundamental matrix F is obtained; S23. The internal parameter matrix K is used to convert the fundamental matrix F into the essential matrix E that describes the camera motion information: E = (K') T FK; Among them, K and K' are the intrinsic matrices of the two images respectively; S24. Decompose the essential matrix E into a rotation R and a translation component, and its theoretical expression is: E = [t] × R; × R; where R is a 3×3 rotation matrix; t is a translation vector; [t] × denotes the skew-symmetric matrix of t; S25. For the matching points x and x' satisfying: x = PX, x' = P'X, construct a linear equation system to solve for the optimal estimate of X; where, P i T and respectively represent the i-th row of the projection matrices P and P'; S26. Minimize the reprojection error of all points: where, x ij represents the image coordinates of the i-th feature point observed in the i-th camera; represents the point reprojected onto the image plane according to the currently estimated 3D point X i and the projection matrix of camera j; S27. Output the sparse point cloud data.

3. A method for constructing an electromechanical pipeline model based on drone oblique photography according to claim 1, characterized in that, The construction process of the 3D model of the electromechanical pipeline is as follows: S31. Initialize each sparse point as a 3D Gaussian ellipsoid, and use the 3D Gaussian distribution to describe the local spatial information; S32. Project the covariance matrix: ∑ 2D = JW∑W T J T ; where, ∑ represents the covariance matrix of the original 3D Gaussian ellipsoid; W represents the view transformation matrix; J represents the Jacobian matrix of the projection transformation; ∑ 2D represents the covariance matrix of the 2D Gaussian ellipsoid projected onto the image plane; T represents the transpose operation of the matrix; S33. Use the reconstructed 2D Gaussian ellipsoid information to generate a rendered image through the rasterization method; S34. Reverse-optimize the Gaussian ellipsoid parameters by calculating the error between the rendered image and the real collected image; S35. Output the structural similarity loss and construct the 3D model of the electromechanical pipeline.

4. A method for constructing an electromechanical pipeline model based on drone oblique photography according to claim 3, characterized in that, The initialization process of initializing each sparse point as a 3D Gaussian ellipsoid is as follows: S41. Extract the sparse point cloud data and initialize the sparse point cloud number as a 3D Gaussian ellipsoid: Among them, x represents the coordinate of any point in space; μ represents the center of the 3D Gaussian ellipsoid; ∑ represents the covariance matrix; S42. Decompose the covariance matrix ∑ into the product of a rotation matrix R and a scaling matrix S, i.e., ∑ = RSS T R T ; Among them, R represents the rotation matrix; S represents the scaling matrix; S43. Color and opacity representation: color information and opacity α.

5. A method for constructing an electromechanical pipeline model based on drone oblique photography according to claim 3, characterized in that, The rendering process of generating a rendered image through the rasterization method is as follows: S51, Point Sputtering and Pixel Synthesis: Divide the entire image into image blocks of a fixed size. For each image block, select all Gaussian basis elements within the viewing cone and with a confidence level higher than the set threshold, and calculate the pixel color by cumulative calculation layer by layer after sorting by depth: where c i represents the color information of the i-th Gaussian basis element; C(p) represents the final color of pixel p; N p represents the set of Gaussian basis elements that affect pixel p; w i represents the weight; S52. Opacity calculation: The opacity α of each Gaussian primitive i acts as a mask for the subsequent primitive during rendering and is calculated using: Among them, represents the initial opacity parameter; G i (x) represents the Gaussian response value of the primitive at pixel x.

6. The method for constructing an electromechanical pipeline model based on drone oblique photography according to claim 3, wherein, The optimization process for reverse optimization of Gaussian ellipsoid parameters is as follows: S61. Construct a loss function: L = (1 - A)L y + AL ssim ; Among them, L represents the total loss; A represents the balance coefficient; L ssim represents the structural similarity loss; L y represents the absolute error of all pixels; L ssim represents the structural similarity loss; S62. Calculate the absolute error of all pixels in the image: Among them, M represents the total number of image pixels; y i represents the gray value of the i-th pixel in the rendered image; represents the gray value of the i-th pixel in the corresponding real captured image; S63. Calculate the structural similarity loss L ssim .

7. A method for constructing an electromechanical pipeline model based on drone oblique photography according to claim 6, characterized in that, The calculation process of the structural similarity loss is as follows: S71. Optionally select two images, define I as the data of the first image, and as the data of the second image; S72, calculate the mean value μ of the image I and variance σ I and as well as the correlation between images S73. Calculate the luminance similarity C luminance , the contrast similarity C contrast and the structural similarity C structure : C1 = (K1L) 2 ; C2 = (K2L) 2 ; Among them, C1, C2, and C3 are small constants used for stable calculation to avoid a zero denominator; L is the dynamic range of pixel values; K1 and K2 are very small constants; S74. Calculate the structural similarity loss L ssim :

8. An electromechanical pipeline model construction system based on UAV oblique photography, which is used to execute the method for constructing an electromechanical pipeline model based on UAV oblique photography according to any one of claims 1 to 7, characterized in that, It includes: An unmanned aerial vehicle (UAV) cruise data acquisition module, which is used to autonomously cruise in the target area based on a UAV equipped with an optical camera to obtain electromechanical pipeline image data; An SfM point cloud initialization module, which is used to recover a sparse point cloud from the images taken by the UAV based on the structure from motion (SfM) technology and provide initial position information; A 3D modeling module, which is used to construct a high-precision electromechanical pipeline model using the 3D Gaussian scattering technology (3DGS) and perform point cloud optimization and rendering; An adaptive density optimization module, which is used to improve the modeling accuracy and detail retention by automatically cloning Gaussian points and adjusting the rendering resolution strategy; A model output and visualization module, which is used to store the generated three-dimensional model in the BIM format, support docking with CAD / BIM software, and perform engineering management visualization.

Images