Power line visual detection method and system and attitude estimation method

Through the power line visual detection method and system, the background noise in the drone power line image is eliminated, the accuracy of power line detection is improved, and the accuracy of attitude estimation is ensured through the attitude estimation method, solving the problem that background noise affects power line detection.

CN118915085AInactive Publication Date: 2024-11-08NINGBO BEICHUANG HANGAO TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410970559.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

During the drone power line inspection, the background noise in the image has a great impact on the accuracy of power line detection, affecting the accuracy of detection.

Method used

Through the power line visual detection method and system, power line images are collected, edge features are extracted, candidate target sets are generated, and noise is eliminated through feature cleaning steps, power line segment diagram is obtained, and the power line segment diagram is superimposed with the original image to realize the position display of the power line, thereby Improve detection accuracy. The system includes UAV components and processing system, using a monocular camera and a single-line laser altimeter radar, combined with coarse selection and selection modules to eliminate fine noise and select power line features.

Benefits of technology

Effectively remove the influence of background noise, improve the accuracy of power line detection, and ensure the accuracy of attitude estimation through attitude estimation method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915085A_ABST
    Figure CN118915085A_ABST
Patent Text Reader

Abstract

The invention discloses a power line visual detection method and system and a posture estimation method. The power line visual detection method comprises the following steps: acquiring a power line image through image acquisition, acquiring an original image, extracting edge features from the image through feature extraction, and generating a candidate target set; the candidate target set is processed through feature cleaning, noise is eliminated, and a power line segment diagram is obtained; and finally, superposing the power line segment diagram with the original image to display the position of the power line, thereby completing the acquisition of the power line visual detection image, through the method, the influence of background noise can be effectively removed, the accuracy of power line detection is improved, attitude estimation is realized based on the visual detection system, and the accuracy of power line visual detection is improved. And the accuracy of attitude estimation can be ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of power line inspection, and in particular to a power line visual inspection method, system and posture estimation method. Background Art

[0002] As an important national infrastructure, power lines bear the important responsibility of power transportation, and power line inspection is an important guarantee for the smooth operation of the power system. With the rapid development of my country's power system, more and more power lines are extending from cities to complex geographical environments such as mountains and plains. The traditional manual inspection method is time-consuming and labor-intensive, and is affected by the natural environment and climate, which restricts the construction of my country's power system. Drone power line inspection is efficient, safe, and not affected by climate and terrain. It has become an important way to inspect power lines.

[0003] During the inspection of power lines by drones, low-altitude power line images can be taken with digital cameras carried by drones. These images contain the basic conditions of the power lines. By processing the low-altitude aerial power line images, abnormal conditions of the power lines can be discovered in time, so that they can be dealt with quickly.

[0004] However, when using drone inspections, the background noise in the image has a significant impact on the accuracy of power line detection, constantly interfering with the feature extraction of the power lines, and directly affecting the accuracy of detection.

[0005] In view of the above, this application is hereby filed. Summary of the invention

[0006] In response to the above problems, an embodiment of the present invention provides a visual detection method, system and posture estimation method for power lines. Through the visual detection method and system, the influence of background noise can be effectively removed, thereby improving the accuracy of power line detection, and posture estimation is implemented based on the visual detection system, which can ensure the accuracy of posture estimation.

[0007] The present invention is achieved through the following technical solutions:

[0008] First aspect

[0009] The embodiment of the present invention provides a power line visual detection method, comprising the following steps:

[0010] S1: Image acquisition: collect power line images and obtain original images;

[0011] S2: Feature extraction: extract edge features from the image and generate a candidate target set;

[0012] S3: Feature cleaning: Process the candidate target set, eliminate noise, and obtain the power line segment map;

[0013] S4: Image output: The power line segment diagram is superimposed on the original image to display the position of the power lines, thereby completing the power line visual detection image acquisition.

[0014] In this scheme, the power line visual detection method realizes the acquisition of power line images through image acquisition, and obtains the original image, and then extracts edge features from the image through feature extraction, and generates a candidate target set; and processes the candidate target set through feature cleaning, eliminates noise, and obtains a power line segment map; finally, the power line segment map is superimposed on the original image to realize the display of the position of the power line, thereby completing the power line visual detection image acquisition. Through this method, the influence of background noise can be effectively removed, thereby improving the accuracy of power line detection.

[0015] Furthermore, in step S3, the feature cleaning step specifically includes the following steps:

[0016] S31: Coarse selection: A coarse selector is used to eliminate small noise in the edge feature map;

[0017] S32: Selection: The edge feature map after eliminating small noise is selected to select the power lines hidden in all the remaining edge features.

[0018] Furthermore, the coarse selector is a coarse selector based on CNN.

[0019] Furthermore, in step S32, refinement is performed based on the ELSED line segment detection system.

[0020] Second aspect

[0021] The embodiment of the present invention further provides a power line visual detection system, which is used to implement the above-mentioned power line visual detection method, including:

[0022] A drone component, wherein the drone component is provided with a monocular camera and a single-line laser altimeter radar, wherein the measurement directions of the monocular camera and the single-line laser altimeter radar are both arranged vertically relative to the ground, and the monocular camera is used to collect power line images;

[0023] It also includes a processing system, which includes a feature extraction module, a feature cleaning module and an output module.

[0024] The feature extraction module is used to extract edge features from the image and generate a candidate target set;

[0025] The feature cleaning module is used to process the candidate target set, eliminate noise, and obtain the power line segment map

[0026] The output module is used to superimpose the power line segment diagram with the original image to display the position of the power line, thereby completing the acquisition of the power line visual detection image.

[0027] Furthermore, the feature cleaning module includes a rough selection module and a fine selection module, wherein the rough selection module eliminates fine noise in the edge feature map, and the fine selection module is used to finely select the edge feature map after eliminating the fine noise, and select the power lines hidden in all remaining edge features.

[0028] The third aspect

[0029] The embodiment of the present invention further provides a method for estimating the posture of a power line, which is based on the above-mentioned power line visual detection system and specifically includes the following steps:

[0030] Step 1: Obtain a power line model diagram through the power line visual detection system;

[0031] Step 2: construct a power line coordinate system based on the power line model diagram, and determine the attitude and position of the drone in the power line coordinate system;

[0032] Step 3: According to the attitude and position of the UAV, the attitude of the power line relative to the UAV is obtained to complete the attitude estimation.

[0033] Furthermore, the UAV component is also provided with a laser altimeter radar, and the measuring direction of the laser altimeter radar is arranged relative to the vertical ground; the attitude estimation is completed according to the homogeneous coordinates of the laser altimeter radar on the power line.

[0034] Furthermore, in step 3,

[0035] The default drone attitude is ξ b =[φ,θ,ψ] T , the position is P = [x, y, z] T , and define the coordinates of the center point of the power line [L] I =[A] I ×[B] I , where A and B are the homogeneous coordinates of the image points of the previous and next two frames on the detected power line respectively;

[0036] Reversely project a power line in the power line model diagram to a plane π in 3D space, where [π] C =K T [L] I , α u , α v are the product of the camera focal length and pixel density in the u and v directions of the camera imaging screen, respectively. o , v oCoordinates of the camera imaging plane;

[0037] The normal vector of the plane π is represented in the horizontal plane of the drone, and the yaw angle of the monocular camera relative to the power line and the complete rotation matrix are obtained through the normal vector of the plane π, where π is an imaging plane, π=[n,d] T , d is the distance from π to the origin of the coordinate system, and the normal vector n of plane π can be expressed in the horizontal plane of the drone:

[0038] In the formula, is the calibration value of the camera;

[0039] The yaw angle of the camera relative to the electric field is calculated by dividing the normal [n] VL Projection to X P Calculated above:

[0040] ψ=acos(α)

[0041] The complete rotation matrix of the camera relative to the direction of the electric field lines is:

[0042]

[0043] R(Z,ψ), R(Y,θ), and R(X,φ) are the rotation vectors of their respective axes;

[0044] Based on the complete rotation matrix, the homogeneous coordinates of the laser altimeter on the power line are defined, and the attitude estimation is completed; wherein:

[0045] It is defined as the homogeneous coordinates of a single-line laser radar point D on the power line can be expressed as:

[0046]

[0047] In the formula, and is the camera calibration value.

[0048] The attitude of the power line relative to the drone is calculated and can be expressed as follows:

[0049]

[0050] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0051] The present invention provides a power line visual detection method, system and posture estimation method. The power line visual detection method realizes the acquisition of power line images through image acquisition, and obtains the original image. Then, edge features are extracted from the image through feature extraction, and a candidate target set is generated; and the candidate target set is processed through feature cleaning to eliminate noise and obtain a power line segment diagram; finally, the power line segment diagram is superimposed on the original image to realize the display of the position of the power line, thereby completing the power line visual detection image acquisition. Through this method, the influence of background noise can be effectively removed, thereby improving the accuracy of power line detection, and posture estimation is realized based on the visual detection system, which can ensure the accuracy of posture estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative work:

[0053] Figure 1 A flowchart of a visual inspection method provided by an embodiment of the present invention;

[0054] Figure 2 A flowchart of a visual inspection method provided by another embodiment of the present invention;

[0055] Figure 3 A flowchart of a posture estimation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is apparent to one of ordinary skill in the art that these specific details are not necessarily employed to practice the present invention. In other instances, well-known structures, circuits, materials, or methods are not specifically described in order to avoid obscuring the present invention.

[0058] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment," "an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. In addition, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and / or sub-combination. In addition, it will be appreciated by those of ordinary skill in the art that the figures provided herein are for illustrative purposes and that the figures are not necessarily drawn to scale. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0059] In the description of the present invention, it should be understood that the terms "front", "rear", "left", "right", "up", "down", "vertical", "horizontal", "high", "low", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0060] Example

[0061] First aspect

[0062] The embodiment of the present invention provides a power line visual detection method, comprising the following steps:

[0063] S1: Image acquisition: collect power line images and obtain original images;

[0064] S2: Feature extraction: extract edge features from the image and generate a candidate target set;

[0065] S3: Feature cleaning: Process the candidate target set, eliminate noise, and obtain the power line segment map;

[0066] S4: Image output: The power line segment diagram is superimposed on the original image to display the position of the power lines, thereby completing the power line visual detection image acquisition.

[0067] In this scheme, the power line visual detection method realizes the acquisition of power line images through image acquisition, and obtains the original image, and then extracts edge features from the image through feature extraction, and generates a candidate target set; and processes the candidate target set through feature cleaning, eliminates noise, and obtains a power line segment map; finally, the power line segment map is superimposed on the original image to realize the display of the position of the power line, thereby completing the power line visual detection image acquisition. Through this method, the influence of background noise can be effectively removed, thereby improving the accuracy of power line detection.

[0068] In some embodiments, in step S3, the feature cleaning step specifically includes the following steps:

[0069] S31: Coarse selection: A coarse selector is used to eliminate small noise in the edge feature map;

[0070] S32: Selection: The edge feature map after eliminating small noise is selected to select the power lines hidden in all the remaining edge features.

[0071] Specifically, through the setting of coarse selection and fine selection steps, a detection method with a coarse-to-fine strategy is adopted. The coarse selection process eliminates most of the noise from the edge feature map, and only ignores the noise that shares high approximation features with the power lines; the fine selection process deliberately selects the power lines hidden in all the remaining edge features, and uses the weighted average method to calculate the distance H of the drone relative to the power lines, which is equivalent to giving the monocular camera depth information.

[0072] In some embodiments, the coarse selector is a CNN-based coarse selector.

[0073] In some embodiments, in step S32, refinement is performed based on the ELSED line segment detection system.

[0074] The embodiment of the present invention further provides a power line visual detection system, which is used to implement the above-mentioned power line visual detection method, including:

[0075] A drone component, wherein the drone component is provided with a monocular camera and a single-line laser altimeter radar, wherein the measurement directions of the monocular camera and the single-line laser altimeter radar are both arranged vertically relative to the ground, and the monocular camera is used to collect power line images;

[0076] It also includes a processing system, which includes a feature extraction module, a feature cleaning module and an output module.

[0077] The feature extraction module is used to extract edge features from the image and generate a candidate target set;

[0078] The feature cleaning module is used to process the candidate target set, eliminate noise, and obtain the power line segment map

[0079] The output module is used to superimpose the power line segment diagram with the original image to display the position of the power line, thereby completing the acquisition of the power line visual detection image.

[0080] In some embodiments, the feature cleaning module includes a rough selection module and a fine selection module, wherein the rough selection module eliminates fine noise in the edge feature map, and the fine selection module is used to finely select the edge feature map after eliminating the fine noise, and select the power lines hidden in all remaining edge features.

[0081] The embodiment of the present invention further provides a method for estimating the posture of a power line, which is based on the above-mentioned power line visual detection system and specifically includes the following steps:

[0082] Step 1: Obtain a power line model diagram through the power line visual detection system;

[0083] Step 2: construct a power line coordinate system based on the power line model diagram, and determine the attitude and position of the drone in the power line coordinate system;

[0084] Step 3: According to the attitude and position of the UAV, the attitude of the power line relative to the UAV is obtained to complete the attitude estimation.

[0085] Furthermore, the UAV component is also provided with a laser altimeter radar, and the measuring direction of the laser altimeter radar is arranged relative to the vertical ground; the attitude estimation is completed according to the homogeneous coordinates of the laser altimeter radar on the power line.

[0086] In some embodiments, in step 3,

[0087] The default drone attitude is ξ b =[φ,θ,ψ] T , the position is P = [x, y, z] T , and define the coordinates of the center point of the power line [L] I =[A] I ×[B] I , where A and B are the homogeneous coordinates of the image points of the previous and next two frames on the detected power line respectively;

[0088] Reversely project a power line in the power line model diagram to a plane π in 3D space, where [π] C =K T [L] I , α u , α v are the product of the camera focal length and pixel density in the u and v directions of the camera imaging screen, respectively. o , v oCoordinates of the camera imaging plane;

[0089] The normal vector of the plane π is represented in the horizontal plane of the drone, and the yaw angle of the monocular camera relative to the power line and the complete rotation matrix are obtained through the normal vector of the plane π, where π is an imaging plane, π=[n,d] T , d is the distance from π to the origin of the coordinate system, and the normal vector n of plane π can be expressed in the horizontal plane of the drone:

[0090] In the formula, is the calibration value of the camera;

[0091] The yaw angle of the camera relative to the electric field is calculated by dividing the normal [n] VL Projection to X P Calculated above:

[0092] ψ=acos(α)

[0093] The complete rotation matrix of the camera relative to the direction of the electric field lines is:

[0094]

[0095] R(Z,ψ), R(Y,θ), and R(X,φ) are the rotation vectors of their respective axes;

[0096] Based on the complete rotation matrix, the homogeneous coordinates of the laser altimeter on the power line are defined, and the attitude estimation is completed; wherein:

[0097] It is defined as the homogeneous coordinates of a single-line laser radar point D on the power line can be expressed as:

[0098]

[0099] In the formula, and is the camera calibration value.

[0100] The attitude of the power line relative to the drone is calculated and can be expressed as follows:

[0101]

[0102] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for visually detecting power lines, characterized in that: The steps include: S1: Image acquisition: collect power line images and obtain original images; S2: Feature extraction: extract edge features from the image and generate a candidate target set; S3: Feature cleaning: Process the candidate target set, eliminate noise, and obtain the power line segment map; S4: Image output: The power line segment diagram is superimposed on the original image to display the position of the power lines, thereby completing the power line visual detection image acquisition.

2. The power line visual detection method according to claim 1, characterized in that: In step S3, the feature cleaning step specifically includes the following steps: S31: Coarse selection: A coarse selector is used to eliminate small noise in the edge feature map; S32: Selection: The edge feature map after eliminating small noise is selected to select the power lines hidden in all the remaining edge features.

3. The power line visual detection method according to claim 2, characterized in that: The coarse selector is a coarse selector based on CNN.

4. The power line visual detection method according to claim 2, characterized in that: In step S32, refinement is performed based on the ELSED line segment detection system.

5. A power line visual inspection system, used to implement the power line visual inspection method according to any one of claims 1 to 4, characterized in that: include: A drone component, wherein the drone component is provided with a monocular camera and a single-line laser altimeter radar, wherein the measurement directions of the monocular camera and the single-line laser altimeter radar are both arranged vertically relative to the ground, and the monocular camera is used to collect power line images; It also includes a processing system, which includes a feature extraction module, a feature cleaning module and an output module. The feature extraction module is used to extract edge features from the image and generate a candidate target set; The feature cleaning module is used to process the candidate target set, eliminate noise, and obtain the power line segment map The output module is used to superimpose the power line segment diagram with the original image to display the position of the power line, thereby completing the acquisition of the power line visual detection image.

6. The power line visual inspection system according to claim 5, characterized in that: The feature cleaning module includes a rough selection module and a fine selection module, wherein the rough selection module eliminates fine noise in the edge feature map, and the fine selection module is used to finely select the edge feature map after the fine noise is eliminated, and select the power lines hidden in all the remaining edge features.

7. A method for estimating the posture of a power line, based on the power line visual detection system according to any one of claims 5-6, characterized in that: The specific steps include: Step 1: Obtain a power line model diagram through the power line visual detection system; Step 2: construct a power line coordinate system based on the power line model diagram, and determine the attitude and position of the drone in the power line coordinate system; Step 3: According to the attitude and position of the UAV, the attitude of the power line relative to the UAV is obtained to complete the attitude estimation.

8. The method for estimating the attitude of a power line according to claim 7, characterized in that: The drone component is also provided with a laser altimeter radar, and the measuring direction of the laser altimeter radar is arranged relative to the vertical ground; the attitude estimation is completed according to the homogeneous coordinates of the laser altimeter radar on the power line.

9. The method for estimating the attitude of a power line according to claim 8, characterized in that: In step 3, The default drone attitude is ξ b =[φ,θ,ψ] T , the position is P = [x, y, z] T , and define the coordinates of the center point of the power line [L] I =[A] I ×[B] I , where A and B are the homogeneous coordinates of the image points of the previous and next two frames on the detected power line respectively; Reversely project a power line in the power line model diagram to a plane π in 3D space, where [π] C =K T [L] I , α u , α v are the product of the camera focal length and pixel density in the u and v directions of the camera imaging screen, respectively. o , v o Coordinates of the camera imaging plane; The normal vector of the plane π is represented in the horizontal plane of the drone, and the yaw angle of the monocular camera relative to the power line and the complete rotation matrix are obtained through the normal vector of the plane π, where π is an imaging plane, π=[n,d] T , d is the distance from π to the origin of the coordinate system, and the normal vector n of plane π can be expressed in the horizontal plane of the drone: In the formula, is the calibration value of the camera; The yaw angle of the camera relative to the electric field is calculated by dividing the normal [n] VL Projection to X P Calculated above: ψ=acos(α) The complete rotation matrix of the camera relative to the direction of the electric field lines is: R(Z,ψ), R(Y,θ), and R(X,φ) are the rotation vectors of their respective axes; Based on the complete rotation matrix, the homogeneous coordinates of the laser altimeter on the power line are defined, and the attitude estimation is completed; wherein: It is defined as the homogeneous coordinates of a single-line laser radar point D on the power line can be expressed as: In the formula, and is the camera calibration value. The attitude of the power line relative to the drone is calculated and can be expressed as follows: