Unmanned aerial vehicle photographing method and system for wire inspection
Through drone capture and pre -processing wire environment image data, combined with the YOLOV5 algorithm recognition and marking wire defects, the problems that existing technology cannot achieve independent identification and markup defects can be solved, and accurate testing and evaluation of wire defects can be achieved.
Patent Information
- Application Number
- CN202411239621.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2025-01-28
AI Technical Summary
The prior art cannot independently identify and mark defect locations during wire inspection, resulting in the inability to detect and analyze defects in real time and automation.
Capture and process the wire environment image data through the pre -processing path according to the preset path, combine the YOLOV5 algorithm to identify the wires and potential defects in the image, to achieve accurate marks and classification of wire defects, and calculate defects according to the labeled defective area image. Severe.
The accurate labeling, classification and severity assessment of wire defects are realized, the accuracy of wire defect detection is improved, and the flying posture and data collection quality of drones are optimized.
Smart Images
Figure CN119380216A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a drone photography method and system for wire inspection. Background Art
[0002] The maintenance and monitoring of power transmission lines includes the inspection of high-voltage transmission lines, power towers and cables to ensure the safety and reliability of the lines. Inspection can identify defects such as fractures, corrosion, and wear, thereby preventing potential power outages or safety hazards. In the field of wire defect detection, traditional methods rely on manual inspection. The currently widely used manual inspection method requires technicians to perform careful image analysis and defect judgment. Since the inspection process of the wire requires the analysis of a large amount of data, this method is not only time-consuming, but also easily affected by human factors, resulting in inconsistencies and errors in the inspection results. At the same time, although drone technology can perform efficient wire inspections, without the support of effective defect detection algorithms, the images captured by drones often require a lot of post-processing and manual intervention. Existing drone systems are usually unable to autonomously identify and mark defect locations, resulting in the inability to perform defect identification and analysis in real time and automatically.
[0003] Although some wire inspection systems can effectively measure the distance between the drone and the high-voltage wire and the distance between the drone and the surrounding obstacles, they cannot directly detect the status of the high-voltage wire itself. Therefore, there is an urgent need for a drone photography method for wire inspection to detect wire defects. Summary of the invention
[0004] The present invention provides a drone photography method and system for conducting wire inspection, so as to solve the problems existing in the prior art.
[0005] To achieve the above object, the present invention provides a drone photography method for wire inspection, comprising the following steps:
[0006] S1: The UAV captures the wire environment image data according to the preset path and pre-processes the wire environment image data.
[0007] S2: The wires and potential defects in the wire environment image data are identified by the YOLOv5 algorithm to obtain an image of the marked wire defect area, including the wire defect location and the wire defect category.
[0008] S3: The severity of the defect is calculated based on the image of the marked wire defect area and rated based on the preset standard.
[0009] S4: Adjust the flight attitude of the UAV based on the position of the wire defect and capture the wire environment image data with the best flight attitude.
[0010] Preferably, the step S1 specifically comprises:
[0011] S11: The drone captures wire environment image data by using sensors in multiple spectral bands. The calculation formula of the spectral reflectance is specifically:
[0012]
[0013] Where R(λ) is the spectral reflectance at wavelength λ; I(λ) is the observed image intensity; I ref (λ) is the reference image intensity.
[0014] S12: Preprocessing the wire environment image data includes using a Gaussian filter to perform image denoising, adjusting image contrast and brightness, and converting the wire environment image data from a multispectral image to an RGB image through PCA.
[0015] Preferably, the step S2 is specifically:
[0016] S21: The input wire environment image data is forward propagated through the YOLOv5 algorithm to generate bounding box, category and confidence prediction.
[0017] S22: Remove redundant bounding boxes with high overlap through non-maximum suppression and retain the most likely detection results.
[0018] S23: Convert the prediction results to image coordinates and draw the bounding box and category label.
[0019] S24: Draw a boundary box of the detected wire defect area on the image to generate an image marking the wire defect area.
[0020] Preferably, the loss function of the YOLOv5 algorithm is specifically:
[0021] Loss T =Localuzation Loss+Classifucation Loss+Confidence Loss
[0022]
[0023] In the formula, Loss T is the loss function; Localization Loss is the positioning loss; λ coord is the positioning loss weight factor; MSE() is the mean square error function; pred i is the predicted bounding box coordinates; true i is the true bounding box coordinate; Classification Loss is the classification loss; λ clsis the classification loss weight factor; CrossEntropy() is the cross entropy function; Confidence Loss is the confidence loss; λ conf The confidence loss is serious; N is the number of samples.
[0024] Preferably, the step S3 is specifically:
[0025] The images of marked wire defect areas are classified into breakage, corrosion, wear, bending and dirt, and the severity of different defect types is calculated according to the set defect algorithm: for the wire defect type of breakage, edge detection and contour detection are used to accurately measure the crack length; for the wire defect types of corrosion and dirt, a threshold segmentation algorithm is used to separate the corrosion or dirt areas and calculate the wire corrosion or dirt area; for the wire defect type of wear, a stereo vision algorithm is used to measure the depth of wire surface wear; for the bent wire defect type, the Hough transform is used to measure the bending angle of the wire structure.
[0026] According to the obtained wire defect results, different wire defect types are quantified into slight, medium and severe levels based on a preset value range.
[0027] Preferably, the flight attitude of the UAV is adjusted based on the position of the wire defect in S4 as follows:
[0028] S41: Establish the drone coordinate system and convert the coordinates of the wire defect position in the image into three-dimensional coordinates (x t ,y t ,z t ).
[0029] S42: Calculate the pitch angle of the drone, that is, the tilt angle of the drone along the horizontal axis:
[0030]
[0031] Where pitch is the pitch angle of the drone; (x d ,y d ,z d ) are the coordinates of the drone.
[0032] S43: Calculate the yaw angle of the drone, that is, the rotation angle of the drone along the vertical axis:
[0033] yaw=arctan 2(y t -y d ,x t -x d )
[0034] Where yaw is the yaw angle of the drone.
[0035] On the other hand, the present invention provides a drone photography system for wire inspection, including a data acquisition and preprocessing module, a defect detection and identification module, a defect assessment module and a posture adjustment module.
[0036] The data acquisition and preprocessing module is used for the UAV to capture the wire environment image data according to the preset path and preprocess the wire environment image data.
[0037] The defect detection and identification module is used to identify the wires and potential defects in the wire environment image data through the YOLOv5 algorithm, and obtain an image with marked wire defect areas, including the wire defect location and wire defect category.
[0038] A defect assessment module is used to calculate the severity of the defect based on the image marking the defect area of the wire and rank it based on the preset criteria.
[0039] The attitude adjustment module is used to adjust the flight attitude of the UAV based on the position of the wire defect and capture the wire environment image data with the best flight attitude.
[0040] On the other hand, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the drone photography method for wire inspection as described in any embodiment of the present invention is implemented.
[0041] On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the drone photography method for wire inspection as described in any embodiment of the present invention is implemented.
[0042] Compared with the prior art, the present invention has the following technical effects:
[0043] The present invention uses a drone to capture and pre-process wire environment image data according to a preset path, and combines the YOLOv5 algorithm to identify the wires and potential defects in the image, thereby achieving accurate marking and classification of wire defects. Through the marked defect area image, the severity of the defect can be calculated and rated to evaluate the health status of the wire. Based on the evaluation results, the flight attitude of the drone is adjusted to further capture image data at the optimal angle. This process not only improves the accuracy of wire defect detection, but also optimizes the flight attitude and data acquisition quality of the drone, ensuring a comprehensive and accurate inspection of high-voltage wires. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is an overall flow chart of the drone photography method for wire inspection described in the present invention. DETAILED DESCRIPTION
[0045] In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in combination with specific embodiments of the present application and with reference to the accompanying drawings.
[0046] Embodiment 1
[0047] This embodiment provides a drone photography method for wire inspection, see Figure 1 As shown, the following steps are included:
[0048] S1: The UAV captures the wire environment image data according to the preset path and pre-processes the wire environment image data.
[0049] As a preferred implementation of this embodiment, step S1 is specifically as follows:
[0050] S11: The drone captures wire environment image data by using sensors with multiple spectral bands. The image of each band provides different information. The calculation formula of the spectral reflectance is specifically:
[0051]
[0052] Where R(λ) is the spectral reflectance at wavelength λ; I(λ) is the observed image intensity; I ref (λ) is the reference image intensity.
[0053] S12: Preprocessing the wire environment image data includes using a Gaussian filter to perform image denoising, adjusting image contrast and brightness through linear transformation to enhance image details, and converting the wire environment image data from a multispectral image to an RGB image through PCA for feature extraction and target detection.
[0054] Furthermore, the calculation formula of the Gaussian kernel function in the Gaussian filter is specifically:
[0055]
[0056] Where G(x,y) is the Gaussian function; (x,y) is the coordinate relative to the center of the filter; σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the filter.
[0057] The specific steps of converting the wire environment image data from multispectral image to RGB image through PCA are as follows: standardize each band of the multispectral image; calculate the covariance matrix of the standardized band data; calculate the eigenvalues and eigenvectors of the covariance matrix, where the eigenvalues represent the variance of each principal component and the eigenvectors represent the directions of the principal components; select the eigenvectors corresponding to the first N principal components; use the selected eigenvectors to convert the original multispectral image data to RGB space; and normalize the converted image to the range of [0,255] to adapt to RGB image display.
[0058] S2: The wires and potential defects in the wire environment image data are identified by the YOLOv5 algorithm to obtain an image of the marked wire defect area, including the wire defect location and the wire defect category.
[0059] As a preferred implementation of this embodiment, step S2 is specifically as follows:
[0060] S21: forward propagating the input wire environment image data through the YOLOv5 algorithm to generate a bounding box, category and confidence prediction. Specifically, the input wire environment image data size is usually 640x640 or other specified sizes. The bounding box includes center coordinates and width and height information.
[0061] S22: Remove redundant bounding boxes with too high overlap through non-maximum suppression (NMS) and retain the most likely detection results.
[0062] Furthermore, the implementation steps of NMS are as follows: sort all bounding boxes according to confidence; start with the bounding box with the highest confidence and keep it; calculate the overlap between the remaining bounding boxes and the currently selected bounding box (usually measured by IOU, Intersection Over Union), and if the overlap exceeds a preset threshold, delete these overlapping bounding boxes; repeat the above process for the remaining bounding boxes until all bounding boxes are processed.
[0063] S23: Convert the prediction results to image coordinates and draw the bounding box and category label. Specifically, the output coordinates of the YOLOv5 model are usually relative to the scale of the image (ie, normalized coordinates), and these normalized coordinates need to be converted back to the pixel coordinates on the actual image, that is, the actual coordinates can be obtained by multiplying the actual width and height of the image.
[0064] S24: Draw a boundary box of the detected wire defect area on the image to generate an image marking the wire defect area.
[0065] As a preferred implementation of this embodiment, the loss function of the YOLOv5 algorithm is specifically:
[0066] LossT =Localization Loss+Classification Loss+Confidence Loss
[0067]
[0068] In the formula, Loss T is the loss function; Localization Loss is the positioning loss; λ coord is the positioning loss weight factor, which is used to adjust the importance of positioning loss in the overall loss; MSE() is the mean square error function; pred i is the predicted bounding box coordinates; true i is the true bounding box coordinate; Classification Loss is the classification loss; λ cls is the classification loss weight factor; CrossEntropy() is the cross entropy function; Confidence Loss is the confidence loss; λ conf The confidence loss is serious; coord +λ cls +λ conf =1; N is the sample size.
[0069] S3: The severity of the defect is calculated based on the image of the marked wire defect area and rated based on the preset standard.
[0070] As a preferred implementation of this embodiment, step S3 is specifically as follows:
[0071] The images of marked wire defect areas are classified into fracture, corrosion, wear, bending and dirt, among which fracture is a defect caused by wire breakage, corrosion is corrosive damage on the wire surface, wear is wear of the wire due to long-term use, bending is bending of the wire due to external force, and dirt is dirt or other foreign matter attached to the wire surface. For different defect types, the severity is calculated according to the set defect algorithm: for the wire defect type of fracture, edge detection and contour detection are used to accurately measure the crack length; for the wire defect type of corrosion and dirt, the threshold segmentation algorithm is used to separate the corrosion or dirt area and calculate the wire corrosion or dirt area; for the wire defect type of wear, the stereo vision algorithm is used to measure the depth of wire surface wear; for the bent wire defect type, the Hough transform is used to measure the bending angle of the wire structure.
[0072] Furthermore, for the broken wire defect type, edge detection and contour detection are used to accurately measure the crack length. Specifically, the Canny algorithm is used to detect the crack edge, the contour is detected in the edge image, and the boundary of the crack is found; the detected contours are traversed and the length of each contour is calculated; for each contour, the Euclidean distance of the contour points is calculated to obtain the length of the crack.
[0073] For the wire defect types of corrosion and dirt, the threshold segmentation algorithm is used to separate the corrosion or dirt area and calculate the wire corrosion or dirt area. Specifically, corrosion usually appears as uneven spots or areas on the surface, which may have features such as rust and oxidation, while wear is usually a deposit covering the surface, which is usually irregular but relatively flat. Therefore, image segmentation based on color threshold is used to identify the area and calculate its area.
[0074] For the worn wire defect type, the depth of the wire surface wear is measured using a stereo vision algorithm as follows: a stereo image pair is acquired, and a disparity map is calculated using a stereo matching algorithm (such as block matching or semi-global matching); a depth map is calculated based on the disparity map and camera parameters, where each pixel value in the depth map represents the distance from the surface point to the camera; the worn area is identified and extracted in the depth map, and the depth value of the area is calculated by comparing the original surface depth with the depth of the worn area.
[0075] According to the obtained conductor defect results, different conductor defect types are quantified into mild, moderate, and severe levels based on the preset numerical range. Adjustments are made according to the specific transmission line type, environmental conditions, and maintenance requirements. For example, a break longer than 10 mm is rated as severe, a break between 5 mm and 10 mm is rated as moderate, and a break less than 5 mm is rated as mild; a corrosion area greater than 1 square centimeter is rated as severe, 0.5 to 1 square centimeter is rated as moderate, and less than 0.5 square centimeter is rated as mild; a wear depth greater than 1 mm is rated as severe, 0.5 to 1 mm is rated as moderate, and less than 0.5 mm is rated as mild; a bend angle greater than 30 degrees is rated as severe, 15 to 30 degrees is rated as moderate, and less than 15 degrees is rated as mild. A dirt area greater than 5 square centimeters is rated as severe, 2 to 5 square centimeters is rated as moderate, and less than 2 square centimeters is rated as mild. Further, the classification results and rating results are integrated into a report, including the type of defect, severity rating, and recommended maintenance or repair measures.
[0076] S4: Adjust the flight attitude of the drone based on the position of the wire defect, capture the wire environment image data with the best flight attitude, and maintain the best shooting angle.
[0077] As a preferred implementation of this embodiment, the flight attitude of the drone is adjusted based on the position of the wire defect in S4 as follows:
[0078] S41: Establish the drone coordinate system and convert the coordinates of the wire defect position in the image into three-dimensional coordinates (x t ,y t ,z t ).
[0079] S42: Calculate the pitch angle of the drone, that is, the tilt angle of the drone along the horizontal axis:
[0080]
[0081] Where pitch is the pitch angle of the drone; (x d ,y d ,z d ) are the coordinates of the drone.
[0082] S43: Calculate the yaw angle of the drone, that is, the rotation angle of the drone along the vertical axis:
[0083] yaw=arctan 2(y t -y d ,x t -x d )
[0084] Where yaw is the yaw angle of the drone.
[0085] In addition, drones are usually kept level, so the roll angle is usually adjusted in the horizontal plane, so the drone roll angle is usually set to 0.
[0086] Embodiment 2
[0087] Accordingly, this embodiment provides a drone photography system for wire inspection, including a data acquisition and preprocessing module, a defect detection and identification module, a defect assessment module and a posture adjustment module.
[0088] The data acquisition and preprocessing module is used for the drone to capture the wire environment image data according to the preset path and preprocess the wire environment image data. This module is used to implement the function of step S1 in the first embodiment and will not be described in detail here.
[0089] The defect detection and identification module is used to identify the wires and potential defects in the wire environment image data through the YOLOv5 algorithm, and obtain an image of the marked wire defect area, including the wire defect position and the wire defect category. This module is used to implement the function of step S2 in the first embodiment, which will not be repeated here.
[0090] The defect assessment module is used to calculate the severity of the defect based on the image of the marked wire defect area and to rate the defect based on a preset standard. This module is used to implement the function of step S3 in the first embodiment and will not be described in detail here.
[0091] The attitude adjustment module is used to adjust the flight attitude of the UAV based on the position of the wire defect and capture the wire environment image data with the best flight attitude. This module is used to implement the function of step S4 in embodiment 1 and will not be described in detail here.
[0092] Embodiment 3
[0093] This embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the drone photography method for wire inspection as described in any embodiment of the present invention is implemented.
[0094] Embodiment 4
[0095] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the drone photography method for wire inspection as described in any embodiment of the present invention is implemented.
[0096] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.
[0097] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0099] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.
[0100] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A drone photography method for wire inspection, characterized in that: The following steps are involved: S1: The drone captures wire environment image data according to the preset path and pre-processes the wire environment image data; S2: Identify the wires and potential defects in the wire environment image data using the YOLOv5 algorithm, and obtain an image of the marked wire defect area, including the wire defect location and wire defect category; S3: Calculate the severity of the defect based on the image of the marked wire defect area and rate it based on the preset standard; S4: Adjust the flight attitude of the UAV based on the position of the wire defect and capture the wire environment image data with the best flight attitude.
2. The drone photography method for wire inspection according to claim 1, characterized in that: The step S1 is specifically as follows: S11: The drone captures wire environment image data by using sensors in multiple spectral bands. The calculation formula of the spectral reflectance is specifically: Where R(λ) is the spectral reflectance at wavelength λ; I(λ) is the observed image intensity; I ref (λ) is the reference image intensity; S12: Preprocessing the wire environment image data includes using a Gaussian filter to perform image denoising, adjusting image contrast and brightness, and converting the wire environment image data from a multispectral image to an RGB image through PCA.
3. The drone photography method for wire inspection according to claim 1, characterized in that: The step S2 is specifically as follows: S21: forward propagating the input wire environment image data through the YOLOv5 algorithm to generate bounding box, category and confidence prediction; S22: Remove redundant bounding boxes with high overlap through non-maximum suppression and retain the most likely detection results; S23: Convert the prediction results to image coordinates and draw the bounding box and category label; S24: Draw a boundary box of the detected wire defect area on the image to generate an image marking the wire defect area.
4. The drone photography method for wire inspection according to claim 3 is characterized in that: The loss function of the YOLOv5 algorithm is specifically: Loss T =Localization Loss+Classification Loss+Confidence Loss In the formula, Loss T is the loss function; Localization Loss is the positioning loss; λ coord is the positioning loss weight factor; MSE() is the mean square error function; pred i is the predicted bounding box coordinates; true i is the true bounding box coordinate; Classification Loss is the classification loss; λ cls is the classification loss weight factor; CrossEntropy() is the cross entropy function; Confidence Loss is the confidence loss; λ conf The confidence loss is serious; N is the number of samples.
5. The drone photography method for wire inspection according to claim 1, characterized in that: The step S3 is specifically as follows: The images with marked wire defect areas are classified into fracture, corrosion, wear, bend and dirt. The severity of different defect types is calculated according to the set defect algorithm: for the wire defect type of fracture, edge detection and contour detection are used to accurately measure the crack length; for the wire defect type of corrosion and dirt, the threshold segmentation algorithm is used to separate the corrosion or dirt area and calculate the wire corrosion or dirt area; for the wire defect type of wear, the stereo vision algorithm is used to measure the depth of wire surface wear; for the wire defect type of bending, the Hough transform is used to measure the bending angle of the wire structure; According to the obtained wire defect results, different wire defect types are quantified into slight, medium and severe levels based on a preset value range.
6. The drone photography method for wire inspection according to claim 1, characterized in that: In S4, the flight attitude of the drone is adjusted based on the position of the wire defect as follows: S41: Establish the drone coordinate system and convert the coordinates of the wire defect position in the image into three-dimensional coordinates (x t ,y t ,z t ); S42: Calculate the pitch angle of the drone, that is, the tilt angle of the drone along the horizontal axis: Where pitch is the pitch angle of the drone; (x d ,y d ,z d ) are the coordinates of the drone; S43: Calculate the yaw angle of the drone, that is, the rotation angle of the drone along the vertical axis: yaw=arctan 2(y t -and d ,x t -x d ) Where yaw is the yaw angle of the drone.
7. A drone photography system for wire inspection, characterized in that: The system is configured to execute the drone photography method for wire inspection according to any one of claims 1 to 6, including a data acquisition and preprocessing module, a defect detection and identification module, a defect assessment module and a posture adjustment module; The data acquisition and preprocessing module is used for the drone to capture the wire environment image data according to the preset path and preprocess the wire environment image data; The defect detection and identification module is used to identify the wires and potential defects in the wire environment image data through the YOLOv5 algorithm, and obtain an image of the marked wire defect area, including the wire defect location and wire defect category; a defect assessment module for calculating the severity of the defect based on the image marking the defect area of the wire and rating it based on a preset standard; The attitude adjustment module is used to adjust the flight attitude of the UAV based on the position of the wire defect and capture the wire environment image data with the best flight attitude.
8. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the drone photography method for wire inspection as described in any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the drone photography method for wire inspection according to any one of claims 1 to 6 is implemented.