Unmanned aerial vehicle routing inspection line tracking processing method based on image processing

Abstract

The invention provides an unmanned aerial vehicle routing inspection line tracking processing method based on image processing, and belongs to the technical field of unmanned aerial vehicles. The distribution condition of the power transmission equipment in the inspection processing line is acquired, the tracking verification processing period of the unmanned aerial vehicle is determined according to the distribution condition and the historical operation data of different power transmission equipment, and image processing is performed on the monitoring image according to the tracking verification processing period; the vertical distance between the image center of the monitoring image and the vertical center of the power transmission line is obtained, and whether the unmanned aerial vehicle has inspection tracking deviation is determined according to the change condition of the vertical distance, so that the tracking processing accuracy of the power transmission line is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular to an unmanned aerial vehicle inspection route tracking processing method based on image processing. Background Art

[0002] As the scale of power grids continues to expand and the operating environment becomes increasingly complex, especially under extreme conditions such as bad weather, traditional manual inspection methods face severe challenges in efficiency and accuracy. Therefore, the intelligent inspection system using unmanned aerial vehicles is becoming an emerging technology in the field of power inspection due to its flexibility and wide coverage.

[0003] However, in the actual inspection process, the angle of the drone and the camera may change due to interference from external wind or tree branches, resulting in loss of the target transmission line, and tracking of adjacent transmission lines of other phases or other transmission lines, making it impossible to accurately implement the inspection of the transmission line.

[0004] In response to the above technical problems, the present invention provides a UAV inspection route tracking and processing method based on image processing. Summary of the invention

[0005] To achieve the purpose of the present invention, the present invention adopts the following technical solutions: In order to solve the above technical problems, the present invention provides a UAV inspection line tracking processing method based on image processing, which specifically includes: S1 acquires the monitoring image of the UAV in real time, and determines the change data of the monitoring image of the UAV in a preset time period according to the analysis result of the monitoring image. When the disturbance error probability of the UAV is determined by using the change data of the monitoring image to meet the requirement, proceed to the next step; S2 takes the inspection object of the drone as the inspection processing line, obtains the change data of the line direction of the inspection processing line during the inspection process, and determines that the tracking error probability of the drone meets the requirements based on the change data of the line direction and the control change data of the gimbal angle of the drone, and then proceeds to the next step; S3: obtaining the distribution of power transmission equipment in the inspection processing line, and determining the tracking and verification processing cycle of the UAV according to the distribution and historical operation data of different power transmission equipment; S4 performs image processing on the monitoring image in the tracking verification processing cycle to obtain the vertical distance between the image center of the monitoring image and the vertical center of the transmission line, and determines whether the UAV has an inspection tracking deviation based on the change of the vertical distance.

[0006] The beneficial effects of the present invention are: 1. In the present invention, the change data of the monitoring image is used to determine whether the disturbance error probability of the UAV meets the requirements, thereby realizing the identification of the disturbance situation of the UAV from the perspective of the change data of the monitoring image, avoiding the technical problem that the tracking accuracy of the transmission line does not meet the requirements due to excessive disturbance, and improving the accuracy of tracking and identification processing.

[0007] 2. In the present invention, the tracking and verification processing cycle of the UAV is determined based on the distribution situation and the historical operation data of different power transmission equipment, that is, the distribution density of the power transmission equipment in the transmission line is taken into account, and the determination of the suspected abnormal power transmission equipment in the transmission line is also taken into account from the perspective of historical operation data, thereby ensuring the efficiency of tracking and verification processing of transmission lines with a large number of power transmission equipment or a large number of suspected abnormal power transmission equipment, and also laying a foundation for the accuracy of the inspection processing of the transmission lines.

[0008] A further technical solution is that the monitoring image of the drone is determined based on image data uploaded by the gimbal of the drone.

[0009] A further technical solution is that the change data of the monitoring image is determined according to the change of pixels of the monitoring image between different adjacent moments, specifically including the number of pixels that have changed and the proportion of the number of pixels in the monitoring image.

[0010] A further technical solution is that when the tracking error probability of the UAV is greater than the preset tracking error probability threshold, it is determined that the tracking error probability of the UAV does not meet the requirement.

[0011] A further technical solution is that, when the tracking error probability of the UAV does not meet the requirements, the vertical distance between the image center of the monitoring image and the vertical center of the transmission line is directly determined, and whether the UAV has an inspection and tracking deviation is determined according to the change of the vertical distance.

[0012] A further technical solution is that when the historical operation data of the power transmission equipment is within a preset interval, the power transmission equipment is determined to be a power transmission equipment with potential fault hazards.

[0013] A further technical solution is that the method for determining the vertical distance between the center of the image and the vertical center of the transmission line is: The monitoring image is subjected to Gaussian filtering and non-maximum suppression to sharpen the edge using the Canny three-step edge detection method, then a double threshold method is used to distinguish weak edges from strong edges, and finally a two-dimensional edge map is obtained by eliminating weak edges that are not connected to strong edges; The vertical lines in the two-dimensional edge map are removed by convolution using the vertical filter kernel, and the transmission lines on the two-dimensional edge map are enhanced by convolution using the thickness filter kernel to obtain a feature-enhanced image: Using a first-order polynomial line as a reference model and performing a Hough transform to extract the power transmission line into the feature enhanced image; Calculate the vertical distance between the image center in the PTZ's monitoring image and the vertical center of the transmission line.

[0014] Other features and advantages will be described in the following description, and partly become apparent from the description, or understood by practicing the invention. The purpose and other advantages of the invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0017] Figure 1 It is a flow chart of a method for tracking and processing a UAV inspection route based on image processing; Figure 2 is a flow chart of a method for determining a tracking verification processing cycle of a UAV; Figure 3 Flow chart of a method for determining a vertical distance between an image center and a vertical center of a transmission line. DETAILED DESCRIPTION

[0018] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their detailed description will be omitted.

[0019] The terms "a", "an", "the", and "said" are used to indicate the presence of one or more elements / components / etc.; the terms "comprising" and "having" are used to express an open-ended inclusive meaning and mean that additional elements / components / etc. may be present in addition to the listed elements / components / etc.

[0020] To solve the above problems, according to one aspect of the present invention, Figure 1According to one aspect of the present invention, a method for tracking and processing a UAV inspection route based on image processing is provided, which specifically includes: S1 acquires the monitoring image of the UAV in real time, and determines the change data of the monitoring image of the UAV in a preset time period according to the analysis result of the monitoring image. When the disturbance error probability of the UAV is determined by using the change data of the monitoring image to meet the requirement, proceed to the next step; S2 takes the inspection object of the drone as the inspection processing line, obtains the change data of the line direction of the inspection processing line during the inspection process, and determines that the tracking error probability of the drone meets the requirements based on the change data of the line direction and the control change data of the gimbal angle of the drone, and then proceeds to the next step; S3: obtaining the distribution of power transmission equipment in the inspection processing line, and determining the tracking and verification processing cycle of the UAV according to the distribution and historical operation data of different power transmission equipment; S4 performs image processing on the monitoring image in the tracking verification processing cycle to obtain the vertical distance between the image center of the monitoring image and the vertical center of the transmission line, and determines whether the UAV has an inspection tracking deviation based on the change of the vertical distance.

[0021] Furthermore, the monitoring image of the drone is determined based on image data uploaded by the gimbal of the drone.

[0022] It is understandable that the change data of the monitoring image is determined according to the change of pixels of the monitoring image between different adjacent moments, specifically including the number of pixels that have changed and the proportion of the number of pixels in the monitoring image.

[0023] It should be noted that determining whether the disturbance error probability of the UAV meets the requirements specifically includes: Determine the pixels of the monitoring image that have changed between different adjacent moments using the change data of the monitoring image, and use them as the changed pixels, and determine the changed monitoring image according to the proportion of the number of pixels of the changed pixels in the monitoring image; Determining the disturbance error probability of the drone based on the proportion of the number of change monitoring images in a preset time period; Determine whether the disturbance error probability of the UAV meets the requirement according to a preset error probability threshold.

[0024] Furthermore, when the disturbance error probability of the UAV does not meet the requirements, the vertical distance between the image center of the monitoring image and the vertical center of the transmission line is directly determined, and whether the UAV has an inspection and tracking deviation is determined based on the change of the vertical distance.

[0025] It should also be noted that determining that the disturbance error probability of the UAV meets the requirements specifically includes steps S11-S12, specifically: S11 uses the change data of the monitoring image to determine the pixels of the monitoring image that have changed between different adjacent moments, and uses them as the changed pixels, and determines the changed monitoring image of the drone in a preset time period according to the proportion of the changed pixels in the monitoring image; S12 determines the disturbance error probability of the UAV based on the proportion of the number of change monitoring images in a preset time period, and determines whether the disturbance error probability of the UAV meets the requirement according to a preset error probability threshold.

[0026] Furthermore, the change data of the line direction of the inspection processing line during the inspection process is determined according to the layout of the towers of the inspection processing line during the inspection process, specifically including the distance between adjacent towers and the angle deviation between the connecting line between adjacent towers and the preset orientation.

[0027] It can be understood that determining that the tracking error probability of the UAV meets the requirements specifically includes: Determine the distance between adjacent towers and the angle deviation between the connecting line between adjacent towers and the preset orientation according to the change data of the line direction, and determine the line tracking deviation probability of the UAV between different adjacent towers by using the distance between adjacent towers and the angle deviation between the connecting line between adjacent towers and the preset orientation; Determining the number of control changes of the gimbal angle of the drone based on the control change data of the gimbal angle of the drone, and determining the error probability of the angle change of the drone using the number of control changes; The tracking error probability of the UAV is determined by using the angle variation error probability of the UAV and the line tracking deviation probability of the UAV between different adjacent towers, and whether the tracking error probability of the UAV meets the requirement is determined by using the preset tracking error probability threshold.

[0028] Further, when the tracking error probability of the UAV is greater than the preset tracking error probability threshold, it is determined that the tracking error probability of the UAV does not meet the requirement.

[0029] Specifically, when the tracking error probability of the UAV does not meet the requirements, the vertical distance between the image center of the monitoring image and the vertical center of the transmission line is directly determined, and whether the UAV has an inspection and tracking deviation is determined based on the change of the vertical distance.

[0030] It should also be noted that determining that the tracking error probability of the UAV meets the requirements specifically includes: Determining the number of control changes of the gimbal angle of the drone based on the control change data of the gimbal angle of the drone, and when the number of control changes of the gimbal angle of the drone does not meet the requirement, determining that the tracking error probability of the drone does not meet the requirement; When the number of control changes of the gimbal angle of the drone meets the requirements: Determine the angle deviation between the connection line between adjacent towers and the preset orientation according to the change data of the line direction, and use the adjacent towers whose angle deviation is within the preset angle range as angle-changing towers, and when the number of the angle-changing towers does not meet the requirement, determine that the tracking error probability of the UAV does not meet the requirement; When the number of angle-changing towers meets the requirements: The distances between the pole towers with different angle changes are obtained. When the distances between the pole towers with different angle changes do not meet the requirements, it is determined that the tracking error probability of the UAV does not meet the requirements. When the distance between towers is changed at different angles and meets the requirements: The line tracking deviation probability of the UAV between different adjacent towers is determined by using the distance between the adjacent towers and the angle deviation between the connecting line between the adjacent towers and the preset orientation, and when the number of adjacent towers whose line tracking deviation probability is not within the preset probability interval does not meet the requirement, it is determined that the tracking error probability of the UAV does not meet the requirement; When the number of adjacent towers that have a line tracking deviation probability that is not within the preset probability interval meets the requirements: The angle change error probability of the UAV is determined using the number of control changes, the tracking error probability of the UAV is determined using the angle change error probability of the UAV and the line tracking deviation probability of the UAV between different adjacent towers, and the preset tracking error probability threshold is used to determine whether the tracking error probability of the UAV meets the requirements.

[0031] Optionally, the above step S11 includes steps S111-S114, which are specifically: S111 uses the change data of the monitoring image to determine the pixels of the monitoring image that have changed between different adjacent moments, and uses them as the changed pixels. According to the proportion of the number of pixels of the changed pixels in the monitoring image, it is determined whether the drone has a changed monitoring image in the preset time period. If yes, proceed to step S112. If not, it is determined that the disturbance error probability of the drone meets the requirements. S112 determines the change amount of the monitoring image of the drone according to the pixel number ratio of the change pixels of different change monitoring images and the number of the change monitoring images, and judges whether the change amount of the monitoring image of the drone meets the requirement. If so, proceeds to step S113; if not, determines that the disturbance error probability of the drone does not meet the requirement; S113 takes the time corresponding to the change monitoring image as the reference time, and determines the disturbance sub-period based on the time length from the reference time, and judges whether the number of the disturbance sub-periods meets the requirement. If yes, proceed to step S114, if not, proceed to step S12; S114 determines the interference severity of different disturbance sub-periods according to the number of change monitoring images in different disturbance sub-periods and the pixel number ratio of change pixels in different change monitoring images, and judges whether there is a disturbance sub-period whose interference severity does not meet the requirements. If so, enters step S12; if not, determines that the disturbance error probability of the UAV meets the requirements.

[0032] Specifically, Figure 2 As shown, the method for determining the tracking verification processing cycle of the drone is: Determine the power transmission equipment with potential fault hazards according to the historical operation data of different power transmission equipment; Determine the number of power transmission equipment between different adjacent pole towers by using the distribution of the power transmission equipment, and determine the accuracy requirement value of the inspection process between different adjacent pole towers in combination with the number of power transmission equipment with potential faults between different adjacent pole towers; Dividing the transmission line into a plurality of transmission line sections according to the connection conditions between the pole towers and the preset transmission line length, and determining the inspection reliability requirements of different transmission line sections based on the accuracy requirements of the inspection processing between different adjacent pole towers in different transmission line sections; The tracking verification processing cycle of the UAV is determined according to the inspection reliability requirements of different transmission line sections.

[0033] Furthermore, the tracking verification processing cycle of the UAV is determined according to the inspection reliability requirements of different transmission line sections, specifically including: Determine the number of transmission line sections within different reliability requirement intervals according to inspection reliability requirements of different transmission line sections, and determine the preset periods corresponding to different reliability requirement intervals according to the number of transmission line sections within different reliability requirement intervals; The tracking verification processing period of the UAV is determined by using the minimum value of the preset period corresponding to different reliability requirement intervals.

[0034] It can be understood that when the historical operation data of the power transmission equipment is within a preset interval, it is determined that the power transmission equipment is a power transmission equipment with potential faults.

[0035] It should also be noted that the method for determining the tracking verification processing cycle of the drone is: Determine the power transmission equipment with potential faults according to the historical operation data of different power transmission equipment, and when the number of the power transmission equipment with potential faults does not meet the requirement, determine the tracking verification processing cycle of the UAV based on a preset verification processing cycle; When the number of power transmission equipment with potential faults meets the requirement: Obtain the number of faulty transmission equipment between different adjacent towers, and use the adjacent towers where the number of faulty transmission equipment is greater than the preset number of faulty transmission equipment as faulty equipment clustering towers; when the number of faulty equipment clustering towers is greater than the preset number of towers, determine the tracking verification processing cycle of the drone based on the preset verification processing cycle; When the number of towers where faulty equipment is concentrated is not greater than the preset number of towers: The number of power transmission equipment between different adjacent pole towers is determined by using the distribution of the power transmission equipment, and the accuracy requirement values ​​of the inspection processing between different adjacent pole towers are determined in combination with the number of power transmission equipment with potential faults among the power transmission equipment between different adjacent pole towers; when the number of adjacent pole towers whose accuracy requirement values ​​are greater than a preset requirement threshold is greater than a preset number of pole towers, the tracking verification processing cycle of the drone is determined based on a preset verification processing cycle; When the number of adjacent towers whose accuracy requirement value is greater than the preset requirement threshold is not greater than the preset number of towers: The transmission line is divided into a plurality of transmission line sections according to the connection conditions between the pole towers and the preset transmission line length, and the inspection reliability requirements of different transmission line sections are determined based on the accuracy requirements of the inspection processing between different adjacent pole towers in different transmission line sections. When there is a transmission line section with an inspection reliability requirement greater than the preset reliability requirement, the tracking verification processing cycle of the drone is determined based on the preset verification processing cycle; When there is no transmission line section where the inspection reliability requirement is greater than the preset reliability requirement: The tracking verification processing cycle of the UAV is determined according to the inspection reliability requirements of different transmission line sections.

[0036] Specifically, Figure 3 As shown, the method for determining the vertical distance between the center of the image and the vertical center of the transmission line is: The monitoring image is subjected to Gaussian filtering and non-maximum suppression to sharpen the edge using the Canny three-step edge detection method, then a double threshold method is used to distinguish weak edges from strong edges, and finally a two-dimensional edge map is obtained by eliminating weak edges that are not connected to strong edges; The vertical lines in the two-dimensional edge map are removed by convolution using the vertical filter kernel, and the transmission lines on the two-dimensional edge map are enhanced by convolution using the thickness filter kernel to obtain a feature-enhanced image: Using a first-order polynomial line as a reference model and performing a Hough transform to extract the power transmission line into the feature enhanced image; Calculate the vertical distance between the image center in the PTZ's monitoring image and the vertical center of the transmission line.

[0037] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0038] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0039] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A method for tracking and processing a drone inspection route based on image processing, characterized in that: Specifically include: Acquire the monitoring image of the UAV in real time, and determine the change data of the monitoring image of the UAV in a preset time period according to the analysis result of the monitoring image, and proceed to the next step when the disturbance error probability of the UAV is determined by using the change data of the monitoring image to meet the requirements; Taking the inspection object of the drone as the inspection processing line, obtaining the change data of the line direction of the inspection processing line during the inspection process, and determining that the tracking error probability of the drone meets the requirements based on the change data of the line direction and the control change data of the gimbal angle of the drone, proceed to the next step; Obtaining the distribution of power transmission equipment in the inspection processing line, and determining the tracking and verification processing cycle of the UAV according to the distribution and historical operation data of different power transmission equipment; The monitoring image is processed in the tracking verification processing cycle to obtain the vertical distance between the image center of the monitoring image and the vertical center of the transmission line, and whether the UAV has an inspection and tracking deviation is determined according to the change of the vertical distance.

2. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 1, characterized in that: The monitoring image of the drone is determined based on the image data uploaded by the gimbal of the drone.

3. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 1, characterized in that: The change data of the monitoring image is determined according to the change of pixels of the monitoring image between different adjacent moments, specifically including the number of pixels that have changed and the proportion of the number of pixels in the monitoring image.

4. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 1, characterized in that: Determining that the disturbance error probability of the UAV meets the requirements specifically includes: Determine the pixels of the monitoring image that have changed between different adjacent moments using the change data of the monitoring image, and use them as the changed pixels, and determine the changed monitoring image according to the proportion of the number of pixels of the changed pixels in the monitoring image; Determining the disturbance error probability of the drone based on the proportion of the number of change monitoring images in a preset time period; Determine whether the disturbance error probability of the UAV meets the requirement according to a preset error probability threshold.

5. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 4, characterized in that: When the tracking error probability of the UAV is greater than the preset tracking error probability threshold, it is determined that the tracking error probability of the UAV does not meet the requirement.

6. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 4, characterized in that: When the disturbance error probability of the UAV does not meet the requirements, the vertical distance between the image center of the monitoring image and the vertical center of the transmission line is directly determined, and whether the UAV has an inspection and tracking deviation is determined according to the change of the vertical distance.

7. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 1, characterized in that: The change data of the line direction of the inspection processing line during the inspection process is determined according to the arrangement of the towers of the inspection processing line during the inspection process, specifically including the distance between adjacent towers and the angle deviation between the connection line between adjacent towers and the preset orientation.

8. The method for tracking and processing the inspection route of an unmanned aerial vehicle based on image processing according to claim 1, characterized in that: When the historical operation data of the power transmission equipment is within a preset interval, it is determined that the power transmission equipment is a power transmission equipment with potential fault hazards.

Images