Unmanned aerial vehicle-based condensing tower automatic reconstruction and defect identification method

Abstract

The invention relates to the technical field of power generation equipment detection, in particular to a condensing tower automatic reconstruction and defect identification method based on an unmanned aerial vehicle. Comprising the following steps: performing full-coverage scanning on a condensing tower by using an unmanned aerial vehicle, and performing data acquisition; the collected data is used for 3DGS reconstruction; a rendering camera pose is automatically generated; carrying out 3DGS rendering; post-processing the rendered image; carrying out defect identification through a defect identification module; by means of the mode, the defect area on the surface of the condensing tower can be accurately recognized, and the reliability and accuracy of detection are effectively guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power generation equipment detection, and particularly relates to a condensing tower automatic reconstruction and defect identification method based on a UAV. BACKGROUND

[0002] As the core structure of the water circulation cooling system of a large power plant, the hyperbolic condensing tower is directly related to the stable operation of the power plant in terms of structural safety and usability. However, due to long-term use and natural environmental factors, defects such as cracks, concrete spalling, and steel bar corrosion are prone to occur on the surface of the condensing tower, which adversely affects the structural performance of the condensing tower.

[0003] The traditional detection method relies on manual completion and requires a large amount of time and labor cost, and the detection efficiency is low. In order to improve this situation, a simple-to-operate UAV is gradually applied to the defect detection of the condensing tower, which effectively reduces the workload of the detection personnel and saves time cost.

[0004] However, in the actual application process, when the UAV collects the defect image on the surface of the condensing tower, it is difficult to achieve complete shooting for large-scale defects, and defect omission problems frequently occur, which is specifically manifested as partial defect omission shooting or incomplete shooting of large-scale defects, seriously affecting the accuracy of the detection result. SUMMARY

[0005] The present application relates to the technical field of power generation equipment detection, and particularly relates to a condensing tower automatic reconstruction and defect identification method based on a UAV.

[0006] To achieve the above-mentioned purpose, a condensing tower automatic reconstruction and defect identification method based on a UAV is adopted, which comprises the following steps: The condensing tower is fully covered and scanned by the UAV for data collection. The collected data is used for 3DGS reconstruction. The camera pose is automatically generated. 3DGS rendering is performed. The rendered image is post-processed. Defect identification is performed through a defect identification module.

[0007] In the step of fully covering and scanning the condensing tower by the UAV for data collection: The collected data includes image sequence data and its corresponding pose, laser 3D point cloud map, wherein the pose and map coordinate system are aligned with the gravity direction.

[0008] In the step of using the collected data for 3DGS reconstruction: 3DGS initialization is performed by initializing 3D Gaussian parameters using a laser 3D point cloud map. The 3D Gaussian parameters include position, rotation, scale, and spherical harmonic coefficients sh. 3DGS training was performed, and the condensing tower was reconstructed in three dimensions using 3DGS tools, laser 3D point cloud maps, and image sequence data to obtain the condensing tower 3DGS reconstruction file 3dgsMap.

[0009] In the step of automatically generating and rendering camera poses: Project all 3D Gaussian positions of 3dgsMap onto the XOY plane along the gravity Z-axis to obtain a top-view projection of the condenser tower. Circle detection is achieved using OpenCV's cv2.HoughCircles() function, resulting in two sets of circle parameters: Circle1(o1, r1) and Circle2(o2, r2). The circle parameters include the center o and the radius r. The circle is divided into four equal and overlapping regions, which are S1+S2, S2+S3, S3+S4, and S4+S1. For any of the four overlapping regions, perform a front view projection; The virtual camera placement parameters are automatically determined from the obtained projected front view; Process S1+S2, S2+S3, and S4+S1; Virtual camera data organization.

[0010] Among the steps in 3DGS rendering: Using the 3DGS rendering engine, render frames one by one to obtain the sequence of rendered images (Renders); Renders={Render_S1+S2,Render_S2+S3,Render_S3+S4,Render_S4+S1}; Render_S1+S2 represents the overlapping region S1+S2, which corresponds to the image sequence [I_1,...,I_N] rendered by the virtual camera using the 3DGS rendering engine.

[0011] Among the steps in post-processing the rendered image: The rendered image sequences of the four overlapping regions S1+S2, S2+S3, S3+S4, and S4+S1 are stitched together to form a larger image. Where: I=verticalConcat(I_1,...,I_N), where hConcat represents the rendering images being stitched together in the vertical direction. In the specific implementation, the OpenCV function cv::hconcat is used to achieve this, thus obtaining the stitched images of 4 overlapping regions I_S1+S2, I_S2+S3, I_S3+S4, and I_S4+S1.

[0012] In the defect identification step using the defect identification module: Defect identification is performed on four overlapping region images I_S1+S2, I_S2+S3, I_S3+S4, and I_S4+S1 respectively. Taking an arbitrary overlapping region image I as an example, the explanation is as follows.

[0013] This invention discloses an automatic reconstruction and defect identification method for a cooling tower based on a drone, comprising the following steps: First, a drone is used to perform a full-coverage scan of the cooling tower to collect data; then, the collected data is used for 3DGS reconstruction; the rendering camera pose is automatically generated; 3DGS rendering is then performed; followed by post-processing of the rendered image; finally, defect identification is performed through a defect identification module. Specifically, by using 3DGS technology to achieve three-dimensional reconstruction of the cooling tower and then re-rendering the cooling tower image based on a preset camera pose, seamless coverage of the cooling tower can be directly achieved. This not only avoids the additional shooting steps added by segmented shooting but also completely avoids the complexity of traditional image stitching processes and the problem of stitching failure or errors due to weak textures. Furthermore, to further improve the defect identification effect, a defect identification model based on a transformer architecture is designed. Through the above methods, accurate identification of defect areas on the surface of the cooling tower is achieved, effectively ensuring the reliability and accuracy of the detection. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a top view projection of the condenser tower of the present invention.

[0016] Figure 2 This is a frontal projection view of the overlapping area (S3+S4) of the condenser tower of the present invention.

[0017] Figure 3 This is a schematic diagram of the virtual camera placement in the overlapping area (S3+S4) of the condenser tower according to the present invention.

[0018] Figure 4 This invention relates to a defect identification model based on the Transformer architecture.

[0019] Figure 5 This is a flowchart of the steps of the automatic reconstruction and defect identification method for condensing towers based on unmanned aerial vehicles (UAVs) of the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0021] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0023] Please see Figures 1-5 This invention provides a method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles (UAVs), comprising the following steps: Drones were used to conduct a full-coverage scan of the condensation tower to collect data. The collected data will be used for 3DGS reconstruction. Automatically generate and render camera poses; Perform 3DGS rendering; Perform post-processing on the rendered image; Defects are identified using a defect identification module.

[0024] In this embodiment, a drone is first used to perform a full-coverage scan of the cooling tower to collect data. The collected data is then used for 3DGS reconstruction, automatically generating the rendering camera pose. 3DGS rendering is then performed, followed by post-processing of the rendered image. Finally, a defect recognition module is used for defect identification. The 3D reconstruction of the cooling tower using 3DGS technology, followed by re-rendering the image based on the preset camera pose, directly achieves seamless coverage of the cooling tower. This not only avoids the additional shooting steps added by segmented shooting but also completely avoids the complexity of traditional image stitching processes and the problem of stitching failure or errors due to weak textures. Furthermore, to further improve the defect recognition effect, a defect recognition model based on a transformer architecture is designed. Through the above methods, accurate identification of defect areas on the surface of the cooling tower is achieved, effectively ensuring the reliability and accuracy of the detection.

[0025] Furthermore, in the step of using drones to conduct a full-coverage scan of the condensation tower and collect data: A drone equipped with lidar, camera, IMU sensors and SLAM positioning and mapping capabilities can complete a full-coverage scan of the condensation tower and collect data under manual control or automatic navigation.

[0026] The collected data includes image sequence data (Images) and their corresponding poses (Poses) and laser 3D point cloud map (Map). The pose and map coordinate system (UAV positioning and mapping coordinate system) are aligned with the direction of gravity, that is, one axis of the coordinate system is the same as the direction of Earth's gravity (this process is implemented by the SLAM positioning and mapping algorithm). In specific implementation, the Z-axis is the same as the direction of gravity.

[0027] Furthermore, in the step of using the acquired data for 3DGS reconstruction: Input: Image sequence data (Images) and their corresponding poses (Poses), and laser 3D point cloud map (Map).

[0028] Output: 3DGS reconstruction file 3dgsMap.

[0029] 3DGS initialization: Initialize 3D Gaussian parameters using a laser 3D point cloud map. The 3D Gaussian parameters include position, rotation, scale, and spherical harmonic coefficients sh. 3DGS training was performed, and the condensing tower was reconstructed in three dimensions using 3DGS tools, laser 3D point cloud maps, and image sequence data to obtain the condensing tower 3DGS reconstruction file 3dgsMap.

[0030] Furthermore, in the step of automatically generating and rendering camera poses: (1) Projected top view: Project all 3D Gaussian positions p of the 3dgsMap along the gravity Z-axis (positive Z-axis downwards) onto the XOY plane to obtain the top view projection of the condensing tower, as shown below. Figure 1 As shown; (2) Circle center detection: The cv2.HoughCircles() function of OpenCV is used to detect circles, and two sets of circle parameters Circle1(o1,r1) and Circle2(o2,r2) are obtained. The circle parameters include the center o and the radius r. Figure 1 The center coordinates of the circle are O1 = (o1 + o2) / 2 = (x0, y0, 0). We also assume that the new coordinate axis parallel to the gravity Z-axis and passing through O1 is G (with the positive direction the same as the Z-axis). (3) Determine the dividing line: Divide the circle into four equal overlapping regions, such as Figure 1 As shown. The dividing lines are L1 (parallel to the X-axis) and L2 (parallel to the Y-axis), the unit vector corresponding to L1 is nL1=(1,0,0), the unit vector corresponding to L2 is nL2=(0,1,0), and the four overlapping equal regions are S1+S2, S2+S3, S3+S4, and S4+S1. (4) Projected Orthographic View: For any of the four overlapping regions, project an orthographic view. Taking the overlapping region S3+S4 as an example, project all 3D Gaussian positions p of the 3dgsMap onto the plane formed by L1 and the G axis (i.e., the XOZ plane) to obtain the projected orthographic view of the overlapping region, as shown below. Figure 2 As shown, the approximate height H of the condenser tower can then be obtained from the projection diagram. Simultaneously, from the projected front view, the Z-coordinate of point O is z0, i.e., O = (x0, y0, z0). (See...) Figure 3 ; (5) Automatically calculate virtual camera placement parameters. For any of the four overlapping regions, perform orthographic projection. Taking the overlapping region S3+S4 as an example, the virtual camera placement parameters are automatically determined using the projected orthographic view obtained in (4). Assume that the width of the virtual camera rendering image is w=4000 pixels, the distance from the virtual camera to the G-axis is F=20m, and N virtual cameras are placed along the G-axis direction of the condensing tower. The width of the rectangular area captured by each virtual camera is Width=2*r2, and the height is Height=H / N. In the specific implementation, the number of N is determined by the height H of the condensing tower and the radius r2, and the formula is N=floor(H / (2*r2)). Based on the virtual camera and the parameters of the rectangular area captured, it is necessary to further determine the intrinsic parameters of the virtual camera and the camera pose, as follows: (5.1) Virtual camera intrinsic parameters: including camera focal length f and optical center (cx, cy), calculated as follows: f = F * w / Width = 40000 / r2 cx=w / 2=2000 cy=w*Height / (2*Width)=2000*H / (r2*N); Where f: camera focal length, cx, cy: optical centers of the camera along the x and y axes, F: distance from the virtual camera to the G axis, w: width of the virtual camera rendered image, r2: radius of the condenser tower base, etc. Figure 1 / 2 / 3 label, H: total height of the condenser tower, Width: width of the rectangular area captured by the virtual camera, Height: height of the rectangular area captured by the virtual camera, N: number of virtual cameras placed.

[0031] (5.2) Virtual camera pose: includes position P_i and orientation R_i, where i=1,2,...,N, ,in It is a vector outer product, nG=(0,0,1). R_i = twoVector2Rot(nG, nDirXOZ), where nDir = (0, -1, 0) represents the camera looking in the negative Y-axis direction, and twoVector2Rot represents the rotation matrix calculated using two vectors. In practice, this is calculated using Eigen's Quaternion::FromTwoVectors() function, where x0, y0, z0 are the coordinates of the origin O of the condenser tower, and i is the number of the i-th virtual camera. Figure 3 As shown; (6) Process S1+S2, S2+S3, and S4+S1 using (4) and (5). The virtual camera intrinsic parameters, P_i, and overlapping region S3+S4 are the same. R_i corresponds to different overlapping regions, and their calculation formulas are the same, but nDir is different, as follows: In the region S1+S2, nDir=(0,1,0) In the S2+S3 region, nDir=(1,0,0) In the region S3+S4, nDir=(0,-1,0) In the S4+S1 region, nDir=(-1,0,0).

[0032] (7) Virtual camera data organization: Virtual camera intrinsics: Cams={f,cx,cy} Virtual camera pose: Poses={Pose_S1+S2,Pose_S2+S3,Pose_S3+S4,Pose_S4+S1}, Pose_S1+S2 represents the virtual camera pose sequence [(P_1,R_1),...,(P_N,R_N)] corresponding to the overlapping region S1+S2.

[0033] Furthermore, in the 3DGS rendering process: Input: Virtual camera parameters Cams and their pose sequences Poses, 3DGS reconstruction file of the condenser tower 3dgsMap; Output: Rendered composite image sequence Renders. Specific steps: Using the 3DGS rendering engine, render frame by frame according to camera parameters (cam) and pose to obtain the rendered image sequence (Renders). Renders={Render_S1+S2,Render_S2+S3,Render_S3+S4,Render_S4+S1}.

[0034] Render_S1+S2 represents the overlapping region S1+S2, which corresponds to the image sequence [I_1,...,I_N] rendered by the virtual camera using the 3DGS rendering engine.

[0035] Furthermore, in the post-processing steps of rendering the image: The rendered image sequences of the four overlapping regions S1+S2, S2+S3, S3+S4, and S4+S1 are stitched together to form a larger image. Specifically: I = verticalConcat(I_1,...,I_N), where hConcat represents the rendering images being stitched together in the vertical direction. In practice, the OpenCV function cv::hconcat is used to achieve this, resulting in a stitched image of four overlapping regions: I_S1+S2, I_S2+S3, I_S3+S4, and I_S4+S1.

[0036] Furthermore, in the step of defect identification through the defect identification module: Defect identification is performed on four overlapping region images I_S1+S2, I_S2+S3, I_S3+S4, and I_S4+S1 respectively. Taking an arbitrary overlapping region image I as an example, the explanation is as follows.

[0037] Image patch defect identification, see Figure 4 : Image Patch Input: Input image I into the model as P*P image patches. Specifically, first, resize image I, with the new image width W' = P × (W / / P) and height H' = P × (H / / P), while recording the scaling ratios ratioW = W / W' and ratioH = H / H'; then, divide image I into P*P image patches and input them into the model, as follows: Figure 4 As shown. In a specific embodiment, P is set to 28; MLP Embedding Layer: Each image patch is fed into the MLP embedding layer, which encodes the image patch into a vector of length Length. In this example, Length is set to 64. Image patch position encoding: The position of each image patch is encoded using the RoPoE (Relative Position Encoding) algorithm, with an encoding length of LengthPoE. In this example, LengthPoE is set to 16. The output vector of the MLP embedding layer and the image patch location encoding vector are concatenated to form a new vector with a length of Length + LengthPoE; Next, vector (4) is input into N encoder layers, each consisting of a self-attention mechanism and an MLP feedforward mechanism. In this embodiment, N is 10. The output vector (5) is fed into the prediction layer. The prediction layer consists of an MLP (Multi-Layer Fully Connected Network) and a Softmax operation. The final output contains Num identification results, each including the defect type and confidence level. There are 5 types of defects during implementation, including 0. Condensation tower is normal, 1. Condensation tower has cracks, 2. Condensation tower has corrosion, 3. Condensation tower outer skin peels off, and 4. Other defects.

[0038] Restore the defect identification results: Each image patch corresponds to a defect type, and each image patch needs to be mapped back to the original image. Specifically, first, an image typeI' of W'xH' is created, then the image value of the corresponding patch is filled with the defect type type identified in (1), and finally typeI' is resized to the original image size to obtain typeI. Here, typeI is the final defect identification result, which marks the defect type of each patch in the original image on the corresponding original image.

[0039] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0040] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Drones were used to conduct a full-coverage scan of the condensation tower to collect data. The collected data will be used for 3DGS reconstruction. Automatically generate and render camera poses; Perform 3DGS rendering; Perform post-processing on the rendered image; Defects are identified using a defect identification module.

2. The method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles as described in claim 1, characterized in that, In the step of using drones to conduct a full-coverage scan of the condensation tower and collect data: The acquired data includes image sequence data and their corresponding poses and laser 3D point cloud maps, where the poses and map coordinate systems are aligned with the direction of gravity.

3. The method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles as described in claim 2, characterized in that, In the step of using the acquired data for 3DGS reconstruction: 3DGS initialization is performed by initializing 3D Gaussian parameters using a laser 3D point cloud map. The 3D Gaussian parameters include position, rotation, scale, and spherical harmonic coefficients sh. 3DGS training was performed, and the condensing tower was reconstructed in three dimensions using 3DGS tools, laser 3D point cloud maps, and image sequence data to obtain the condensing tower 3DGS reconstruction file 3dgsMap.

4. The method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles as described in claim 3, characterized in that, In the step of automatically generating and rendering camera poses: Project all 3D Gaussian positions of 3dgsMap onto the XOY plane along the gravity Z-axis to obtain a top-view projection of the condenser tower. Circle detection is achieved using OpenCV's cv2.HoughCircles() function, resulting in two sets of circle parameters: Circle1(o1, r1) and Circle2(o2, r2). The circle parameters include the center o and the radius r. The circle is divided into four equal and overlapping regions, which are S1+S2, S2+S3, S3+S4, and S4+S1. For any of the four overlapping regions, perform a front view projection; The virtual camera placement parameters are automatically determined from the obtained projected front view; Process S1+S2, S2+S3, and S4+S1; Virtual camera data organization.

5. The method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles as described in claim 4, characterized in that, In the steps of 3DGS rendering: Using the 3DGS rendering engine, render frames one by one to obtain the sequence of rendered images (Renders); Renders={Render_S1+S2,Render_S2+S3,Render_S3+S4,Render_S4+S1}; Render_S1+S2 represents the overlapping region S1+S2, which corresponds to the image sequence [I_1,...,I_N] rendered by the virtual camera using the 3DGS rendering engine.

6. The method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles as described in claim 5, characterized in that, In the post-processing steps of rendering images: The rendered image sequences of the four overlapping regions S1+S2, S2+S3, S3+S4, and S4+S1 are stitched together to form a larger image. Where: I=verticalConcat(I_1,...,I_N), where hConcat represents the rendering images being stitched together in the vertical direction. In the specific implementation, the OpenCV function cv::hconcat is used to achieve this, thus obtaining the stitched images of 4 overlapping regions I_S1+S2, I_S2+S3, I_S3+S4, and I_S4+S1.

7. The method for automatic reconstruction and defect identification of condensing towers based on unmanned aerial vehicles as described in claim 6, characterized in that, In the defect identification process using the defect identification module: Defect identification is performed on four overlapping region images I_S1+S2, I_S2+S3, I_S3+S4, and I_S4+S1 respectively. Taking an arbitrary overlapping region image I as an example, the explanation is as follows.

Images