Unmanned aerial vehicle inspection method for steel structure cooling tower

Abstract

The invention relates to the technical field of computers, and discloses an unmanned aerial vehicle inspection method for a steel structure cooling tower, and the method comprises the steps: obtaining a high-angle image of a to-be-inspected steel structure cooling tower; performing segmentation processing by using the segmentation model to obtain a steel structure cooling tower region and a background region; performing edge extraction by using an edge detection algorithm to obtain an original outer contour of the steel structure cooling tower image; performing smooth optimization on the original outer contour by using a non-parametric smoothing technology; the method comprises the following steps: calculating the flight distance and waypoint information of an unmanned aerial vehicle by combining the parameters of a load camera of the unmanned aerial vehicle, determining an inspection route, and controlling the unmanned aerial vehicle to execute an inspection task according to the inspection route. And the unmanned aerial vehicle load camera parameters are combined, the inspection route is determined, the steel structure cooling method structure is adapted, the inspection blind area is eliminated, and the inspection efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers. Background Technology

[0002] Currently, the energy construction industry has gradually transformed towards intelligence and digitalization, and the construction of smart construction sites has become the core direction for improving construction management efficiency. However, inspection, as a key link in ensuring construction safety, quality and progress, still faces many bottlenecks.

[0003] Traditional manual inspection has the following significant limitations in complex construction site scenarios: the construction site covers a wide area and has a dense structure, including many high-risk areas such as deep foundation pits, high formwork, lifting machinery, and temporary power supply. Manual inspection is not only time-consuming and labor-intensive, but also has a limited daily coverage area and is difficult to reach dangerous areas such as high altitudes and narrow spaces, which can easily create blind spots in supervision.

[0004] Against this backdrop, drones, with their low-altitude flight flexibility and multi-dimensional data collection capabilities, have become the core technology carrier for smart construction site inspection. Equipped with payloads such as high-definition cameras, infrared thermal imagers, and lidar, they can quickly acquire data such as panoramic images of the construction site, equipment operating status, and personnel work behavior.

[0005] One of the inspection tasks during the construction of power structures in thermal power plants is the inspection of hyperbolic steel cooling towers. These towers are 210 meters high and approximately 110 meters in diameter, with a curved outer surface. How to utilize drones for inspection during the construction of power structures in thermal power plants has become an urgent problem to be solved. Summary of the Invention

[0006] This invention provides a method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers to solve the problem of how to use UAVs for inspection.

[0007] In a first aspect, the present invention provides a method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers, the method comprising: Obtain an overhead image of the steel cooling tower to be inspected; A segmentation model was used to segment the overhead image of the steel cooling tower, separating the cooling tower area from the background area. Edge detection algorithms are used to extract edges from the segmented image to obtain the original outer contour of the steel structure cooling tower image; The original outer contour of the steel structure cooling tower image was smoothed and optimized using nonparametric smoothing techniques. Based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the parameters of the UAV's onboard camera, the flight distance and waypoint information of the UAV are calculated to determine the inspection route, and the UAV is controlled to perform the inspection task according to the inspection route.

[0008] This invention obtains the outer contour of a steel cooling tower by segmenting, extracting edges, and smoothing the outer contour of an overhead image of the tower to be inspected. Combined with the parameters of the camera mounted on the UAV, an inspection route is determined to adapt to the steel cooling tower structure, eliminate blind spots, and improve inspection efficiency.

[0009] In one optional implementation, the overhead image of the steel structure cooling tower is segmented using a segmentation model, including: Pre-train the segmentation model using deep learning algorithms; Set the key point coordinates of the steel structure cooling tower, input the key point coordinates into the pre-trained segmentation model, predict the target area, and mark the key point positions; The pixels of the key points are classified and labeled, and the steel structure cooling tower area and the background area are segmented based on the classification and labeling results.

[0010] This invention segments the steel structure cooling tower area and the background area using a segmentation model. It leverages the powerful zero-sample transfer capability of the segmentation model, eliminating the need for retraining to adapt to new scenarios. By utilizing key point input and target area prediction, it effectively improves segmentation accuracy and focuses on the key areas of the steel structure cooling tower.

[0011] In one optional implementation, the pixels of the marked key points are classified and labeled, and the steel structure cooling tower area and the background area are segmented according to the classification and labeling results, including: Using segmentation masking technology, a two-dimensional pixel matrix corresponding to the key points is generated. The two-dimensional pixel matrix consists of steel structure cooling tower labels and background labels. Areas labeled with "steel structure cooling tower" are designated as "steel structure cooling tower areas," and areas labeled with "background" are designated as "background areas."

[0012] This invention utilizes segmentation masking technology to generate a two-dimensional pixel matrix, achieving pixel-level region division, thereby determining the steel structure cooling tower area and the background area, and realizing automated region segmentation.

[0013] In one optional implementation, an edge detection algorithm is used to extract edges from the segmented image to obtain the original outer contour of the steel structure cooling tower image, including: The segmented image is then subjected to Gaussian smoothing. The Sobel algorithm was used to calculate the horizontal gradient magnitude, vertical gradient magnitude, and gradient direction of the image of the steel structure cooling tower, and to initially locate the outer contour of the steel structure cooling tower. Non-maximum suppression is used to scan pixels along the image gradient direction, preserving local maxima, to refine the edge of the initially located steel cooling tower's outer contour; A dual threshold method is used to filter out the outer edges of steel structure cooling towers whose pixel gradient magnitudes meet preset conditions from the refined outer contour. The connectivity of the outer edges of the selected steel structure cooling towers is determined, and the original outer contour of the steel structure cooling tower image is selected based on the connectivity determination.

[0014] This invention utilizes Gaussian smoothing to filter out image noise, employs the Sobel algorithm to initially locate the outer contour of the steel structure cooling tower, accurately captures the directional features of the outer contour, uses non-maximum suppression for edge refinement, eliminates redundant pixels, uses dual thresholds to filter edges, filters edge pixels in stages, and then performs connectivity judgment to eliminate isolated noise points, ensuring the integrity and realism of the contour.

[0015] In one optional implementation, the Sobel algorithm is used to calculate the horizontal gradient magnitude, vertical gradient magnitude, and gradient direction of the steel structure cooling tower image, including: By convolving the Sobel operator with the grayscale matrix of the original steel structure cooling tower image, the horizontal and vertical gradient magnitudes of the steel structure cooling tower image are obtained. Calculate the ratio of the horizontal gradient magnitude to the vertical gradient magnitude, and use the arctangent function to calculate the angle corresponding to the ratio to determine the gradient direction.

[0016] This invention employs dual gradient calculations, including horizontal and vertical gradients, to accurately capture contour grayscale changes and adapt to the complex curved surface structure of steel cooling towers.

[0017] In one optional implementation, the dual threshold includes a high threshold and a low threshold. The dual threshold is used to filter out the outer edges of the steel structure cooling tower from the edge-refined outer contour where the pixel gradient magnitude meets preset conditions, including: Edges with pixel gradient magnitudes greater than a high threshold are marked as sharp edges, and sharp edges are retained. Edges with pixel gradient magnitudes between the low and high thresholds are marked as blurred edges, and the blurred edges are temporarily stored. Edges with pixel gradient magnitudes less than a low threshold are marked as invalid edges, and invalid edges are deleted.

[0018] This invention employs a hierarchical judgment of pixel gradient amplitude, processes edges with different pixel gradient amplitudes separately, retains clear edges, and filters out invalid edges, thereby improving the accuracy of the outer edge of the steel structure cooling tower.

[0019] In one alternative implementation, the original outer contour of the steel structure cooling tower image is smoothed and optimized using nonparametric smoothing techniques, including: Identify non-smooth areas in the original outer contour of an image of a steel structure cooling tower; The loess function in R is used to perform local weighted regression smoothing on non-smooth regions.

[0020] This invention uses the loess function in R language to perform local weighted regression smoothing to eliminate jagged edges on the outer contour, providing accurate data for subsequent route planning.

[0021] Secondly, the present invention provides a drone inspection device for steel structure cooling towers, the device comprising: The acquisition module is used to acquire overhead images of the steel structure cooling tower to be inspected. The segmentation module is used to segment the overhead image of the steel structure cooling tower using a segmentation model. The segmentation model is used to segment the steel structure cooling tower area and the background area. The edge extraction module is used to extract edges from the segmented image using an edge detection algorithm to obtain the original outer contour of the steel structure cooling tower image; The smoothing optimization module is used to smooth and optimize the original outer contour of the image of the steel structure cooling tower using non-parametric smoothing technology; The inspection module is used to calculate the UAV's flight distance and waypoint information based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the UAV's on-board camera parameters, to determine the inspection route and control the UAV to perform inspection tasks according to the inspection route.

[0022] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the above-described first aspect or any corresponding embodiment of the unmanned aerial vehicle inspection method for steel structure cooling towers.

[0023] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the unmanned aerial vehicle (UAV) inspection method for steel structure cooling towers described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a method for unmanned aerial vehicle (UAV) inspection of a steel structure cooling tower according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the outer contour of a steel structure cooling tower according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the parameters of the top of the steel structure cooling tower and the parameters of the UAV according to an embodiment of the present invention; Figure 4 This is a structural block diagram of a drone inspection device for steel structure cooling towers according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0028] Traditional manual inspections have many limitations. They are not only time-consuming and labor-intensive, but also have limited daily coverage and are difficult to reach dangerous areas such as high altitudes and narrow passages, easily creating blind spots in supervision. In addition, manual records rely on paper ledgers or simple electronic forms, resulting in poor data real-time performance and easy data distortion, leading to delays in hazard identification and difficulties in tracing responsibility.

[0029] Against this backdrop, drones have become the core technology carrier for smart construction site inspection. Combining them with intelligent algorithms to inspect steel structure cooling towers has become an important technical path to solve the pain points of traditional inspection and improve the level of intelligent construction.

[0030] This invention provides a method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers, which is applied to a UAV inspection system for steel structure cooling towers. The system includes a system management module, a flight management module, a data management module, etc., to realize intelligent inspection of special structures such as cooling towers, thereby achieving dynamic monitoring and automatic early warning of construction.

[0031] The hardware deployment of this system requires corresponding network and power support. Airport deployment includes, but is not limited to, airport civil construction, electrical and network installation, and installation of airport surveillance cameras.

[0032] The system software installation requires the installation of a MySQL database, Minio file server, Nacos, and Redis, as well as the configuration and installation of JDK. For front-end deployment, navigate to the nginx configuration folder and extract the front-end project package to the current folder. Open a command prompt, navigate to the nginx folder, and type "nginx -s reload" to start the service. You can then access the system by entering the corresponding address in your browser.

[0033] System management includes menu management, equipment management, and performance monitoring, managing the system's basic configurations. The menu management list displays all menus in the system, allowing users to configure them, including adding, modifying, deleting, searching, and resetting. Equipment management primarily manages different categories of equipment within the system, including adding, modifying, deleting, querying, and resetting equipment. The performance monitoring page displays the activation status of various system services, including normal and abnormal, and also shows the system's CPU, GPU, and disk usage information.

[0034] Flight management includes route management, mission management, and flight control modules, which are used to manage the information required during the execution of flight missions.

[0035] The construction of hyperbolic steel structure cooling towers requires inspection and supervision of project progress, quality, and safety, as well as recording of images during construction. Due to the irregular curved shape of the steel tower and the presence of multiple tower cranes operating in the complex working space, inspection tasks include: assessing the quality of welds between high-altitude components, verifying that safety measures for high-altitude workers meet specifications, identifying any violations, and ensuring project progress meets requirements. Drones need to identify these scenarios from different heights and angles, resulting in diverse drone inspection routes that require systematic management.

[0036] Once the flight path is planned, task management is required. This includes specific tasks such as taking photos, recording videos, collecting orthophotos, collecting digital ground models, real-world models, and laser point cloud information using drones, as well as other technical requirements such as shooting accuracy, shooting direction, and specific shooting content.

[0037] Flight control mainly includes flight time, i.e., whether it is a certain time of day, week, or month; which route to follow: direct route, spiral route, or curved route, etc.; and whether to use wide-angle, infrared, or zoom lenses during flight.

[0038] Among them, the route management is used to maintain the system's route library for inspection tasks, including three types of routes: waypoint flights, mapping aerial photography, and steel tower inspection. The route management interface enables the management of routes, including operations such as adding, modifying, deleting, searching, and resetting.

[0039] The task management function manages drone inspection tasks, including operations such as adding, modifying, deleting, searching, and resetting tasks.

[0040] According to an embodiment of the present invention, an embodiment of a UAV inspection method for steel structure cooling towers is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0041] This embodiment provides a method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers. Figure 1 This is a flowchart of a drone inspection method for steel structure cooling towers according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain an overhead image of the steel structure cooling tower to be inspected.

[0042] In this embodiment of the invention, a drone is used to take a vertical downward shot from directly above the steel cooling tower to be inspected, obtaining an aerial image of the steel cooling tower. During the shooting, the camera's optical axis is kept as plumb as possible to reduce the shooting angle deviation and avoid image distortion.

[0043] Step S102: Use the segmentation model to segment the overhead image of the steel structure cooling tower.

[0044] In this embodiment of the invention, a segmentation model is introduced. This segmentation model is based on a deep learning model that can prompt segmentation tasks to segment images of steel structure cooling towers. This segmentation model is used to segment the steel structure cooling tower area and the background area.

[0045] Step S103: Use an edge detection algorithm to extract edges from the segmented image to obtain the original outer contour of the steel structure cooling tower image.

[0046] In this embodiment of the invention, an edge detection algorithm is used to process the segmented image. The edge detection algorithm includes, but is not limited to, the Canny edge detection algorithm. The core advantage of the Canny edge detection algorithm is that it can still stably extract the image contour of the steel structure cooling tower in the uneven lighting environment of the construction scene, thereby ensuring the accuracy and robustness of the edge information.

[0047] Step S104: Use nonparametric smoothing technology to smooth and optimize the original outer contour of the steel structure cooling tower image.

[0048] In this embodiment of the invention, since the original outer contour extracted by the edge detection algorithm has edge curvature, a smoothing technique is used to smooth and optimize the original outer contour. A schematic diagram of the outer contour of the steel structure cooling tower is shown below. Figure 2 As shown. Specifically, nonparametric smoothing technology is selected, and smoothing values ​​are calculated by fitting a local regression model through local weighted least squares regression, thereby improving the accuracy and smoothness of the outer contour of the steel structure cooling tower.

[0049] Step S105: Based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the parameters of the UAV's on-board camera, calculate the UAV's flight distance and waypoint information, determine the inspection route, and control the UAV to perform the inspection task according to the inspection route.

[0050] In this embodiment of the invention, after smoothing and optimizing the outer contour of the steel cooling tower, the flight distance of the UAV around the cooling tower is determined based on the outer contour of the cooling tower and the parameters of the camera mounted on the UAV. This flight distance is the range of distances between the UAV and the cooling tower when the UAV collects images of the cooling tower, including the minimum distance for safe flight and the maximum distance to ensure optimal image resolution. After determining the flight distance of the UAV, the number of waypoints required for the UAV to collect images also needs to be determined. An inspection route is then determined, and the UAV performs the inspection task according to this route, thereby achieving UAV inspection.

[0051] The UAV inspection method for steel structure cooling towers provided in this embodiment obtains the outer contour of the steel structure cooling tower by segmenting, extracting edges, and smoothing the outer contour of the aerial image of the steel structure cooling tower to be inspected. Combined with the parameters of the camera on the UAV, the inspection route is determined to adapt to the steel structure cooling tower, eliminate blind spots, and improve inspection efficiency.

[0052] This embodiment provides a method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers, the process of which includes the following steps: Step S201: Obtain an overhead image of the steel structure cooling tower to be inspected.

[0053] Please see details Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0054] Step S202: Use the segmentation model to segment the overhead image of the steel structure cooling tower.

[0055] Specifically, step S202 includes: Step S2021: Pre-train the segmentation model using a deep learning algorithm.

[0056] Step S2022: Set the key point coordinates of the steel structure cooling tower, input the key point coordinates into the pre-trained segmentation model, predict the target area, and mark the key point positions.

[0057] Step S2023: Classify and label the pixels of the marked key points, and segment the steel structure cooling tower area and the background area according to the classification and labeling results.

[0058] In this embodiment of the invention, the segmentation model is a deep learning model based on cue-based segmentation tasks. This model is trained on a large-scale segmentation dataset, which includes a large amount of masking and image information, improving the segmentation model's generalization ability to segment irregular structures and backgrounds in complex environments. Compared with traditional models, the segmentation model exhibits superior performance in image analysis accuracy and processing speed, can process high-resolution images promptly, and possesses excellent zero-shot transfer capability, adapting to new task scenarios without retraining.

[0059] The segmentation model leverages its interactive segmentation capabilities, using simple annotations to quickly and accurately segment the graphic information of the steel cooling tower. An encoder receives various input cues, which are converted into specific embedding representations. These cues are then combined with image embeddings generated by a pre-trained image encoder and fed into a lightweight mask decoder. This mask decoder, based on the Transformer architecture, utilizes self-attention and cross-attention mechanisms to effectively fuse image and cue information, ultimately generating a segmentation mask for the target object quickly.

[0060] First, a segmentation model is pre-trained using deep learning algorithms. This pre-trained model is then loaded and deployed onto a GPU, leveraging its parallel computing capabilities to improve segmentation speed. Next, the coordinates of key points on the steel cooling tower are set and input into the segmentation model to predict the target region. The key point locations are then labeled to guide the model to focus on the steel cooling tower area for prediction. Finally, the pixels of the labeled key points are classified and labeled to generate corresponding tags, thus distinguishing the steel cooling tower area from the background area.

[0061] The segmentation model is used to segment the steel cooling tower area and the background area. By leveraging the powerful zero-sample transfer capability of the segmentation model, it is possible to adapt to new scenes without retraining. By using key point input and target area prediction, the segmentation accuracy is effectively improved, focusing on the key areas of the steel cooling tower.

[0062] Specifically, step S2023 above includes: Step S20231: Using segmentation masking technology, generate a two-dimensional pixel matrix corresponding to the key points.

[0063] Step S20232: The area with the steel structure cooling tower label is defined as the steel structure cooling tower area, and the area with the background label is defined as the background area.

[0064] In this embodiment of the invention, multiple segmentation masks similar to key point features are generated. The core function of the segmentation mask is to classify and label each pixel as a "target region" and a "background region", generating a two-dimensional pixel matrix corresponding to the key point. The two-dimensional pixel matrix uses the number "1" to represent the steel structure cooling method and the number "0" to represent the background region.

[0065] In addition, each segmentation mask is scored to evaluate its segmentation accuracy. The purpose is to select the most accurate mask and visualize the segmentation results, saving them in a lossless compressed bitmap format for further analysis.

[0066] Specifically, segmentation masking is a computer vision processing technique used to accurately separate objects in an image from the background. It achieves fine-grained segmentation of image regions by classifying and labeling each pixel. Each pixel is assigned a label indicating whether it belongs to the target, the background, or a different object category. This labeling information forms a two-dimensional matrix, known as a segmentation mask.

[0067] It should be noted that when using the segmentation model to segment aerial images of steel cooling towers, the aerial images are first converted to a color gamut format. Specifically, the aerial images of the steel cooling towers are converted to a color gamut format compatible with the segmentation model (such as GRB, HSV, etc.). This is essentially to avoid segmentation errors caused by format incompatibility.

[0068] By using segmentation masking technology to generate a two-dimensional pixel matrix, pixel-level region division is achieved, thereby determining the steel structure cooling tower area and the background area, and realizing automated region segmentation.

[0069] Step S203: Use an edge detection algorithm to extract edges from the segmented image to obtain the original outer contour of the steel structure cooling tower image.

[0070] Specifically, step S203 includes: Step S2031: Perform Gaussian smoothing on the segmented image.

[0071] Step S2032: The Sobel algorithm is used to calculate the horizontal gradient magnitude, vertical gradient magnitude, and gradient direction of the steel structure cooling tower image to initially locate the outer contour of the steel structure cooling tower.

[0072] Step S2033: Non-maximum suppression is used to scan pixels along the image gradient direction, retaining local maxima, and the outer contour of the initially located steel structure cooling tower is refined.

[0073] Step S2034: Using dual thresholds, the outer edge of the steel structure cooling tower that meets the preset conditions for pixel gradient magnitude is selected from the outer contour after edge refinement.

[0074] Step S2035: Perform connectivity judgment on the outer edge of the selected steel structure cooling tower, and filter the original outer contour of the steel structure cooling tower image based on the connectivity judgment.

[0075] In this embodiment of the invention, the Canny edge detection algorithm is used to extract the edges of a steel structure cooling tower image. This algorithm can stably extract the outline of a steel structure cooling tower image in complex noisy environments. The core idea of ​​the edge detection algorithm is to identify significant edge features and remove false edges through a series of image processing steps to ensure the accuracy and robustness of the detection results.

[0076] The first step is to perform Gaussian smoothing on the segmented image, using a Gaussian filter to remove noise. The equation for generating the Gaussian filter kernel is as follows:

[0077] in, For the Gaussian filter kernel located at row index i and column indexes j The weight value at that location, Let be the standard deviation of the Gaussian distribution. i , j This is the position index of the current pixel in the filter kernel. m To control the size of the filter kernel, This is the normalization coefficient.

[0078] The second step is to use the Sobel algorithm to obtain the horizontal and vertical gradient magnitudes of the image of the steel structure cooling tower, and then calculate the gradient direction of the cooling tower image.

[0079] The third step involves using Non-Maximum Suppression (NMS) to scan pixels along the image gradient direction, retaining only local maxima. This refines the image and boundary boundaries and removes redundant noise information. The expression for this step is shown below:

[0080] The fourth step is to set two thresholds, namely a high threshold R. H and low threshold R LThe outer edge of the steel structure cooling tower that meets the preset conditions is selected from the outer contour after edge refinement.

[0081] The fifth step is to determine the connectivity of the outer edges of the selected steel structure cooling tower, remove isolated noise, and obtain the original outer contour of the steel structure cooling tower image.

[0082] Gaussian smoothing is used to filter out image noise. The Sobel algorithm is used to initially locate the outer contour of the steel structure cooling tower, accurately capture the directional features of the outer contour, use non-maximum suppression for edge refinement, remove redundant pixels, use double threshold to filter edges, filter edge pixels in stages, and then perform connectivity judgment to eliminate isolated noise points, ensuring the integrity and authenticity of the contour.

[0083] Specifically, step S2032 includes: Step S20321: Perform a convolution operation between the Sobel operator and the grayscale matrix of the original steel structure cooling tower image to obtain the horizontal and vertical gradient magnitudes of the steel structure cooling tower image.

[0084] Step S20322: Calculate the ratio of the horizontal gradient magnitude to the vertical gradient magnitude, and use the arctangent function to calculate the angle corresponding to the ratio to determine the gradient direction.

[0085] In an embodiment of the present invention, x and y The Sobel operators for the directions are as follows:

[0086]

[0087] in, B This is the grayscale matrix of the segmented image of the steel structure cooling method.

[0088] Calculate the gradient magnitude of the image:

[0089] Calculate the gradient direction:

[0090] in, S x for x Sobel operator for direction, S y for y Sobel operator for direction, S xy For image gradient magnitude, This represents the image gradient direction.

[0091] The system employs dual gradient calculations, using both horizontal and vertical gradients, to accurately capture changes in the grayscale of the contour, adapting to the complex curved surface structure of steel cooling towers.

[0092] Specifically, step S2034 above includes: Step S20341: Mark edges with pixel gradient magnitude greater than the high threshold as clear edges and retain clear edges.

[0093] Step S20342: Mark the edges whose pixel gradient magnitude is between the low threshold and the high threshold as blurred edges and temporarily store the blurred edges.

[0094] Step S20343: Mark edges with pixel gradient magnitude less than the low threshold as invalid edges and delete invalid edges.

[0095] In this embodiment of the invention, if S(i,j) > R H If R is clearly marked as the edge of the steel structure cooling tower image; L <S(i,j)<R H If S(i,j) < R, then mark the edges of the blurred steel cooling tower image; L If a pixel is marked as an edge pixel and can be deleted, it is considered invalid.

[0096] Step S204: Use nonparametric smoothing technology to smooth and optimize the original outer contour of the steel structure cooling tower image.

[0097] By employing a hierarchical judgment of pixel gradient amplitude, edges with different pixel gradient amplitudes are processed separately, retaining clear edges and filtering invalid edges, thereby improving the accuracy of the outer edge of the steel structure cooling tower.

[0098] Specifically, step S204 includes: Step S2041: Identify the non-smooth areas in the original outer contour of the steel structure cooling tower image.

[0099] Step S2042: Use the loess function in R language to perform local weighted regression smoothing on the non-smooth regions.

[0100] In this embodiment of the invention, since the original outer contour extracted by the edge detection algorithm has edge curvature, it is necessary to further smooth the original outer contour.

[0101] The loess function in R language is used. This method is a nonparametric smoothing technique that fits a local regression model through locally weighted least squares regression and calculates the smoothing value.

[0102] Given the coordinate data points of all pixels on the original outer contour of the steel structure cooling tower ( x i ,y i At a certain smooth point x At position 0, the following cubic weighting function is used:

[0103]

[0104] in, t This is the bandwidth parameter (used to control the width of the weight distribution). x 0 is the target point. x i For data points within the neighborhood.

[0105] Local weighted regression smoothing is performed using the loess function in R to eliminate jagged edges on the outer contour, providing accurate data for subsequent route planning.

[0106] Step S205: Based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the parameters of the UAV's on-board camera, determine the minimum and maximum distances for the UAV to fly, calculate the UAV waypoint information, determine the inspection route, and control the UAV to perform the inspection task according to the inspection route.

[0107] Specifically, the top diameter of the steel structure cooling tower is L. Based on the UAV's flight altitude and the parameters of the onboard camera, the following equation holds: L=(Lsd*N1*D1) / f=(Lsd*N2*D2) / f N1*D1=N2*D2 N1*(H1-H)=N2*(H2-H) H = H1 - (H2 - H1) / [(N1 / N2) - 1] Where Lsd is the distance at which the image is captured by the camera sensor, the focal length of the camera is f, and N1 and N2 are the pixel values ​​of the two captured photos, respectively.

[0108] To improve the accuracy of estimating the height of steel cooling towers from acquired images, the average height of the steel cooling tower from multiple captured images is calculated. For example... Figure 3 As shown, taking two sets of graphics as examples, Figure 3 This is a schematic diagram showing the parameters of the top of the steel structure cooling tower and the parameters of the drone.

[0109] The following formula is formed from n sets of graphic height information:

[0110] in, , , , Let N1 and N2 represent the pixel values ​​corresponding to H1 and H2 in the i-th image group.

[0111] The flight distance of the drone around the steel cooling tower needs to be determined, taking into account both the minimum safe flight distance and the maximum distance to ensure optimal image resolution. The minimum distance is determined based on the drone model, and the maximum distance is calculated using the following formula: D max =Losd*f / Lsd Losd represents the spatial distance between adjacent pixels of the steel cooling tower, i.e., the resolution.

[0112] Determine the number of waypoints required for the UAV to collect images. The number of waypoints is divided into horizontal waypoint spacing and vertical waypoint spacing.

[0113] The horizontal waypoint spacing is calculated according to the following formula:

[0114] Among them, D h D is the waypoint spacing. w Working distance of the drone-borne camera from the surface of the steel cooling tower, S w The width of the lens sensor, This represents the horizontal overlap rate of the image.

[0115] The above calculations can optimize the number of waypoints on the flight path, reduce the taking of unnecessary photos, and improve data collection efficiency.

[0116] The vertical waypoint spacing is calculated using the following formula:

[0117] Where Sh is the width of the lens sensor. This represents the vertical overlap ratio of the image.

[0118] The UAV inspection method for steel structure cooling towers provided in this embodiment has the following advantages: (1) It has strong adaptability and a dedicated route around the cooling tower covers the cooling tower, solving the problem of blind spots in traditional inspections; (2) Combining the Canny algorithm and the nonparametric smoothing algorithm, the outer contour edge of the steel structure cooling tower is accurately extracted and optimized; (3) Highly intelligent, requiring no drawings, and performs automated route planning.

[0119] This embodiment also provides a drone inspection device for steel structure cooling towers. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] This embodiment provides a drone inspection device for steel structure cooling towers, such as... Figure 4 As shown, it includes: The acquisition module 401 is used to acquire overhead images of the steel structure cooling tower to be inspected.

[0121] The segmentation module 402 is used to segment the overhead image of the steel structure cooling tower using a segmentation model. The segmentation model is used to segment the steel structure cooling tower area and the background area.

[0122] The edge extraction module 403 is used to extract edges from the segmented image using an edge detection algorithm to obtain the original outer contour of the steel structure cooling tower image.

[0123] The smoothing optimization module 404 is used to smooth and optimize the original outer contour of the steel structure cooling tower image using nonparametric smoothing technology.

[0124] The inspection module 405 is used to calculate the UAV's flight distance and waypoint information based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the UAV's on-board camera parameters, to determine the inspection route and control the UAV to perform the inspection task according to the inspection route.

[0125] In some alternative implementations, segmentation module 402 includes: The model training unit is used to pre-train the segmentation model using deep learning algorithms.

[0126] The annotation unit is used to set the key point coordinates of the steel structure cooling tower, input the key point coordinates into the pre-trained segmentation model, predict the target area, and annotate the key point positions.

[0127] The segmentation unit is used to classify and label the pixels of the labeled key points, and to segment the steel structure cooling tower area and the background area based on the classification and labeling results.

[0128] In some alternative implementations, the segmentation unit includes: The matrix generation sub-unit is used to generate a two-dimensional pixel matrix corresponding to key points using segmentation masking technology. The two-dimensional pixel matrix consists of steel structure cooling tower labels and background labels.

[0129] Define sub-units to identify areas with steel structure cooling tower labels as steel structure cooling tower areas and areas with background labels as background areas.

[0130] In some alternative implementations, the edge extraction module 403 includes: The smoothing unit is used to perform Gaussian smoothing on the segmented image.

[0131] The calculation unit is used to calculate the horizontal gradient magnitude, vertical gradient magnitude, and gradient direction of the steel structure cooling tower image using the Sobel algorithm, and to initially locate the outer contour of the steel structure cooling tower.

[0132] The edge refinement unit is used to refine the outer contour of the initially located steel cooling tower by employing non-maximum suppression to scan pixels along the image gradient direction and retaining local maxima.

[0133] The filtering unit is used to filter out the outer edge of the steel structure cooling tower from the edge-refined outer contour using dual thresholds, where the pixel gradient magnitude meets the preset conditions.

[0134] The connectivity judgment unit is used to judge the connectivity of the outer edge of the screened steel structure cooling tower and to filter the original outer contour of the steel structure cooling tower image based on the connectivity judgment structure.

[0135] In some alternative implementations, the computing unit includes: The first computational subunit is used to perform a convolution operation between the Sobel operator and the grayscale matrix of the original steel structure cooling tower image to obtain the horizontal and vertical gradient magnitudes of the steel structure cooling tower image.

[0136] The second calculation subunit is used to calculate the ratio of the horizontal gradient magnitude to the vertical gradient magnitude, and uses the arctangent function to calculate the angle corresponding to the ratio to determine the gradient direction.

[0137] In some alternative implementations, the filtering unit includes: The first marking subunit is used to mark edges with pixel gradient magnitudes greater than a high threshold as sharp edges and retain sharp edges.

[0138] The second marking subunit is used to mark edges whose pixel gradient magnitude is between the low threshold and the high threshold as blurred edges and temporarily store the blurred edges.

[0139] The third marking subunit is used to mark edges with pixel gradient magnitudes less than a low threshold as invalid edges and delete invalid edges.

[0140] In some alternative implementations, the smoothing optimization module 404 includes: The recognition unit is used to identify uneven areas in the original outer contour of an image of a steel structure cooling tower.

[0141] The smoothing optimization unit is used to perform local weighted regression smoothing on unsmooth regions using the R language's loess function.

[0142] The UAV inspection device for steel structure cooling towers provided in this embodiment of the invention can execute the UAV inspection method for steel structure cooling towers provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0143] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0144] The following is a detailed reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0145] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0146] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the UAV inspection method for steel structure cooling towers according to embodiments of the present invention.

[0147] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0148] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the UAV inspection method for steel structure cooling towers shown in the above embodiments is implemented.

[0149] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0150] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection of steel structure cooling towers, characterized in that, The method includes: Obtain an overhead image of the steel cooling tower to be inspected; The overhead image of the steel structure cooling tower is segmented using a segmentation model, which is used to segment the steel structure cooling tower area and the background area. Edge detection algorithms are used to extract edges from the segmented image to obtain the original outer contour of the steel structure cooling tower image; The original outer contour of the steel structure cooling tower image was smoothed and optimized using nonparametric smoothing techniques. Based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the parameters of the UAV's onboard camera, the flight distance and waypoint information of the UAV are calculated to determine the inspection route, and the UAV is controlled to perform the inspection task according to the inspection route.

2. The method according to claim 1, characterized in that, The segmentation process of the overhead image of the steel cooling tower using a segmentation model includes: Pre-train the segmentation model using deep learning algorithms; Set the key point coordinates of the steel structure cooling tower, input the key point coordinates into the pre-trained segmentation model, predict the target area, and mark the key point positions; The pixels of the key points are classified and labeled, and the steel structure cooling tower area and the background area are segmented based on the classification and labeling results.

3. The method according to claim 2, characterized in that, The process of classifying and labeling the pixels of the marked key points, and segmenting the steel structure cooling tower area and the background area based on the classification and labeling results, includes: Using segmentation masking technology, a two-dimensional pixel matrix corresponding to key points is generated. The two-dimensional pixel matrix consists of a steel structure cooling tower label and a background label. Areas labeled with "steel structure cooling tower" are designated as "steel structure cooling tower areas," and areas labeled with "background" are designated as "background areas." 4. The method according to claim 1, characterized in that, The process of extracting edges from the segmented image using an edge detection algorithm to obtain the original outer contour of the steel structure cooling tower image includes: The segmented image is then subjected to Gaussian smoothing. The Sobel algorithm was used to calculate the horizontal gradient magnitude, vertical gradient magnitude, and gradient direction of the image of the steel structure cooling tower, and to initially locate the outer contour of the steel structure cooling tower. Non-maximum suppression is used to scan pixels along the image gradient direction, preserving local maxima, to refine the edge of the initially located steel cooling tower's outer contour; A dual threshold method is used to filter out the outer edges of steel structure cooling towers whose pixel gradient magnitudes meet preset conditions from the refined outer contour. The connectivity of the outer edges of the selected steel structure cooling towers is determined, and the original outer contour of the steel structure cooling tower image is selected based on the connectivity determination.

5. The method according to claim 4, characterized in that, The Sobel algorithm is used to calculate the horizontal gradient magnitude, vertical gradient magnitude, and gradient direction of the image of the steel structure cooling tower, including: By convolving the Sobel operator with the grayscale matrix of the original steel structure cooling tower image, the horizontal and vertical gradient magnitudes of the steel structure cooling tower image are obtained. Calculate the ratio of the horizontal gradient magnitude to the vertical gradient magnitude, and use the arctangent function to calculate the angle corresponding to the ratio to determine the gradient direction.

6. The method according to claim 4, characterized in that, The dual thresholds include a high threshold and a low threshold. The process of using dual thresholds to filter the outer edges of the steel structure cooling tower from the refined outer contour where the pixel gradient magnitude meets preset conditions includes: Edges with pixel gradient magnitudes greater than a high threshold are marked as sharp edges, and these sharp edges are retained. Edges with pixel gradient magnitudes between low and high thresholds are marked as blurred edges, and these blurred edges are temporarily stored. Edges with pixel gradient magnitudes less than a low threshold are marked as invalid edges, and these invalid edges are deleted.

7. The method according to claim 1, characterized in that, The process of smoothing and optimizing the original outer contour of the steel structure cooling tower image using nonparametric smoothing techniques includes: Identify the non-smooth areas in the original outer contour of the image of the steel structure cooling tower; The non-smooth regions were smoothed using the loess function in R language through local weighted regression.

8. A drone inspection device for steel structure cooling towers, characterized in that, The device includes: The acquisition module is used to acquire overhead images of the steel structure cooling tower to be inspected. The segmentation module is used to segment the overhead image of the steel structure cooling tower using a segmentation model, wherein the segmentation model is used to segment the steel structure cooling tower area and the background area. The edge extraction module is used to extract edges from the segmented image using an edge detection algorithm to obtain the original outer contour of the steel structure cooling tower image; A smoothing optimization module is used to smooth and optimize the original outer contour of the steel structure cooling tower image using nonparametric smoothing technology; The inspection module is used to calculate the UAV's flight distance and waypoint information based on the smoothed and optimized outer contour of the steel structure cooling tower, combined with the UAV's on-board camera parameters, to determine the inspection route and control the UAV to perform inspection tasks according to the inspection route.

9. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the UAV inspection method for steel structure cooling towers as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the UAV inspection method for steel structure cooling towers as described in any one of claims 1 to 7.

Images