A method, device, drone and storage medium for self-inspection of power tower
By comparing the captured point image and reference image set above the power tower, attitude correction is performed, the problem of insufficient accuracy of routes and flight attitudes in the existing technology is solved, and more efficient and low-cost power tower inspection is achieved , suitable for multi-tower or multi-class tower scenarios.
Patent Information
- Application Number
- CN202210060466.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-19
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-01-19
AI Technical Summary
The existing drone autonomous power grid patrol technology lacks accuracy in route planning and flight attitude adjustment, resulting in low patrol efficiency and high cost, and lack of versatility, which cannot meet the patrol needs of multiple towers or multiple types of towers.
After the drone flies above the power tower, the gimbal camera uses the gimbal camera to compare the point image with the pre-created reference image set, determine the patrol attitude correction information, and perform the attitude correction between the drone and gimbal camera to achieve Accurate adjustment of routes and flight attitudes.
It improves the drone route and flight accuracy, reduces the dependence on prior information, has a certain degree of versatility, and is suitable for inspection scenarios of multiple towers or multiple types of towers, saving manpower and material resources, and reducing costs.
Smart Images

Figure CN114489102B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of unmanned aerial vehicle power grid inspection, and in particular to a method, device, unmanned aerial vehicle and storage medium for self-inspection of power towers. Background Art
[0002] In recent years, drone inspections have gradually been applied to power grid inspection services. The main mode of power grid inspection services is manual operation of drones. This mode of inspection has low efficiency, high labor costs, and the inspection results vary from person to person. Therefore, the demand for autonomous drone inspections has gradually emerged. In the prior art, autonomous drone inspection technologies also have differences in intelligence and automation, and the main difference lies in route planning. There are two main solutions with a lower degree of intelligence. One is to generate routes by manually teaching waypoints, and the other is to use geographic information system (GIS) point cloud map track planning. However, these two methods require a large amount of manpower and material resources for preliminary preparation, and require more prior information. Moreover, the prior information prepared each time can only be applied to one tower, which is not universal and cannot truly achieve intelligent and low-cost autonomous inspections. For highly intelligent fully autonomous inspection solutions, there are mainly visual perception, radar perception and multi-sensor fusion solutions. Although they can adapt to the inspection scenarios of multiple towers or multiple types of towers, they are not mature at the current stage. As a result, while the perception weight increases, the adjustment of the route and flight attitude is not accurate enough, which reduces the accuracy of route planning and flight control, and thus cannot meet the needs of inspection business. Therefore, improving the accuracy of route planning and flight accuracy has become a bottleneck of fully autonomous inspection technology. Summary of the invention
[0003] The present invention provides a method, device, unmanned aerial vehicle (UAV), and storage medium for self-inspection of power poles and towers, so as to achieve precise adjustment of flight routes and flight attitudes, improve the flight accuracy and flight accuracy of the UAV, and thus better meet the inspection business needs.
[0004] In a first aspect, an embodiment of the present invention provides a method for self-inspection of a power tower, the method being applied to a drone, the drone comprising a drone body and a pan / tilt camera, the method comprising:
[0005] After the drone body flies above the pole tower to be inspected, the drone body is controlled to fly according to a given inspection route;
[0006] When it is detected that the current flight position of the UAV body meets the capture condition, the pan / tilt camera is controlled to capture a point image relative to the pole tower to be inspected;
[0007] Determine the posture correction information of the drone inspection according to the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to the target inspection perspective;
[0008] The UAV body and the gimbal camera are inspected and corrected according to the attitude correction information, and the UAV returns to continue the point image capture operation during the continuous flight until the flight end is reached.
[0009] In a second aspect, an embodiment of the present invention further provides a power pole tower self-inspection device, which is applied to a drone, wherein the drone includes a drone body and a pan-tilt camera, and the device includes:
[0010] The route setting module is used to control the drone body to fly according to the given inspection route after the drone body flies above the tower to be inspected;
[0011] The first image capture module is used to control the pan / tilt camera to capture a point image relative to the pole tower to be inspected when detecting that the current flight position of the drone body meets the capture condition;
[0012] An information determination module, used to determine the posture correction information of the drone inspection based on the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to the target inspection perspective;
[0013] The second image capture module is used to perform inspection and attitude correction on the drone body and the gimbal camera according to the attitude correction information, and return to continue to perform point image capture operations during continuous flight until the flight end is reached.
[0014] In a third aspect, an embodiment of the present invention further provides a drone, comprising: a drone body, a pan / tilt camera fixedly arranged on the drone body, and: one or more controllers;
[0015] a memory for storing one or more programs,
[0016] When the one or more programs are executed by the one or more controllers, the one or more controllers implement the power pole tower self-inspection method as described in the first aspect.
[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the power tower self-inspection method as described in any one of the first aspects is implemented.
[0018] The above technical solution of the embodiment of the present invention controls the drone body to fly according to a given inspection route after the drone body flies above the pole tower to be inspected; when it is detected that the current flight position of the drone body meets the capture condition, the pan-tilt camera is controlled to capture the point image relative to the pole tower to be inspected; the attitude correction information of the drone inspection is determined according to the point image and the pre-created reference image set, and the inspection attitude correction of the drone body and the pan-tilt camera is performed according to the attitude correction information, and the point image capture operation is returned to continue in the continuous flight until the flight end is reached. In the embodiment of the present invention, the attitude correction information of the drone inspection is determined according to the point image and the pre-created reference image set, and the inspection attitude correction of the drone body and the pan-tilt camera is performed according to the attitude correction information, so as to achieve precise adjustment of the route and flight attitude, improve the route accuracy and flight accuracy of the drone, and thus better meet the inspection business needs. Compared with the prior art, the adopted power pole tower self-inspection method improves the accuracy of route planning and flight accuracy, has a certain degree of versatility, can be applied to the inspection scenes of multiple towers or multiple types of towers, and does not require a large amount of prior information, saving manpower and material resources, and reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 A flow chart of a power tower self-inspection method provided in Embodiment 1 of the present invention;
[0020] Figure 2 A flow chart of a power tower self-inspection method provided in Embodiment 2 of the present invention;
[0021] Figure 3 A schematic diagram of a flow chart of a power tower self-inspection method provided in Embodiment 2 of the present invention;
[0022] Figure 4 A schematic diagram of a flow chart of a precise adjustment phase of a UAV flight attitude provided by the second embodiment of the present invention;
[0023] Figure 5 A structural schematic diagram of a posture adjustment provided by Embodiment 2 of the present invention;
[0024] Figure 6 A schematic diagram of the structure of a power tower self-inspection device provided in Embodiment 3 of the present invention;
[0025] Figure 7 A schematic diagram of the structure of a drone provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0027] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.
[0028] The term “including” and its variations used in the present invention are open inclusions, that is, “including but not limited to.” The term “based on” means “based at least in part on.” The term “one embodiment” means “at least one embodiment.”
[0029] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0030] Embodiment 1
[0031] Figure 1 This is a flow chart of a power pole tower self-inspection method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation when a power pole tower is self-inspected. The method can be executed by a power pole tower self-inspection device, which can be implemented in hardware and / or software and can usually be configured in a drone device. The method is applied to a drone, which includes a drone body and a pan-tilt camera. The method specifically includes the following steps:
[0032] S110, after the drone body flies above the pole tower to be inspected, the drone body is controlled to fly according to a given inspection route.
[0033] The pole tower to be inspected can be understood as a power pole tower waiting for self-inspection. The given inspection route can be understood as a pre-given pole tower self-inspection route.
[0034] In this embodiment, the drone includes a drone body and a pan-tilt camera. The pan-tilt camera can be understood as a camera with a pan-tilt, which is a supporting device for installing and fixing the camera. The pan-tilt camera has a device for carrying the camera to rotate horizontally and vertically. Installing the camera on the pan-tilt can enable the camera to shoot key target points in the inspected tower from multiple angles, and the horizontal and vertical rotation angles can be adjusted by limit switches.
[0035] In this embodiment, when conducting a safety check and determining that the take-off point of the drone is far away from the tower and that the airspace above is unobstructed, the Global Positioning System (GPS) can be used for positioning, and the accurate GPS information of the tower to be inspected can be input or entered in advance on the handheld terminal, and then the relevant start-up instructions are sent to allow the drone to fly safely to directly above the tower to be inspected. After the drone body flies above the tower to be inspected, the drone body can be controlled to perform inspections according to a pre-given tower inspection route.
[0036] S120: When it is detected that the current flight position of the UAV body meets the capture condition, the pan / tilt camera is controlled to capture a point image corresponding to the pole tower to be inspected.
[0037] Among them, the current flight position meets the capture condition when the drone body flies above the pole tower to be inspected and then flies according to a given inspection route. When the drone body reaches the key inspection point according to the given inspection route, it can be considered that the current flight position of the drone body meets the capture condition.
[0038] In this embodiment, when it is detected that the current flight position of the drone body meets the capture conditions, the pan-tilt camera can be controlled to capture a point image relative to the pole tower to be inspected. The point image can be understood as a point image relative to the pole tower to be inspected captured by the pan-tilt camera when the capture conditions are met. The point image may include the relative position relationship of the drone relative to the target in the point image; it may also include the position information of the target; it may also include various parameter information of the pan-tilt camera during capture; this is not limited in this embodiment. Exemplarily, the target may be an insulator. The point image may include one target or multiple targets, which is not limited in this embodiment.
[0039] Specifically, the inspection key points of the drone can be determined based on the inspection route of the tower to be inspected. When the current flight position of the drone is detected to be at the inspection key point, it is considered that the image capture condition is met, and the gimbal camera can be controlled to capture the inspection image of the inspection key point at this time, which is used as the point image of the tower to be inspected. Among them, the inspection key point can be understood as the key inspection waypoint of the drone during the inspection of the tower to be inspected.
[0040] S130, determining attitude correction information for the drone inspection based on the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to a target inspection perspective.
[0041] In this embodiment, the pre-created reference image set can be understood as a pre-created database reference map. Before the autonomous inspection, the drone is flown to a precise waypoint according to the target inspection angle, and the gimbal camera is controlled to capture the inspection image of the key inspection point, and the image is stored in the pre-created reference image set as the reference image of the inspection tower, which is used for the relevant posture correction of the point images captured during the autonomous inspection of the drone. The reference image set contains at least one key point image captured according to the target inspection angle.
[0042] In an embodiment, the attitude correction information of the drone inspection can be determined based on the point image of the pole tower to be inspected captured by the PTZ camera and the pre-created reference image set. The attitude correction information can include the relevant position information, phase information and various parameter information of the attitude correction of the drone.
[0043] Specifically, the relevant inspection target frame containing the inspection key points can be extracted from the point image relative to the inspected pole tower captured by the pan-tilt camera, and the target key point image containing the inspection key points can be extracted from the pre-created reference image set. The current posture feature vector is determined by the relevant inspection target frame of the inspection key points, and the target posture feature vector is determined by the target key point image. Then, the current posture feature vector and the target posture feature vector are combined to determine the posture correction information of the drone inspection and perform posture correction.
[0044] S140, performing patrol attitude correction on the drone body and the gimbal camera according to the attitude correction information, and returning to continue the point image capture operation during continuous flight until reaching the flight end point.
[0045] In this embodiment, after determining the posture correction information of the drone inspection based on the point image and a pre-created reference image set, the inspection posture of the drone body and the gimbal camera is corrected according to the posture correction information, and the drone returns to continue the point image capture operation during continuous flight until the flight end is reached.
[0046] The above technical solution of the embodiment of the present invention controls the drone body to fly according to a given inspection route after the drone body flies above the pole tower to be inspected; when it is detected that the current flight position of the drone body meets the capture condition, the pan-tilt camera is controlled to capture the point image relative to the pole tower to be inspected; the attitude correction information of the drone inspection is determined according to the point image and the pre-created reference image set, and the inspection attitude correction of the drone body and the pan-tilt camera is performed according to the attitude correction information, and the point image capture operation is returned to continue in the continuous flight until the flight end is reached. In the embodiment of the present invention, the attitude correction information of the drone inspection is determined according to the point image and the pre-created reference image set, and the inspection attitude correction of the drone body and the pan-tilt camera is performed according to the attitude correction information, so as to achieve precise adjustment of the route and flight attitude, improve the route accuracy and flight accuracy of the drone, and thus better meet the inspection business needs. Compared with the prior art, the adopted power pole tower self-inspection method improves the accuracy of route planning and flight accuracy, has a certain degree of versatility, can be applied to the inspection scenes of multiple towers or multiple types of towers, and does not require a large amount of prior information, saving manpower and material resources, and reducing costs.
[0047] Optionally, the power tower self-inspection method further includes:
[0048] The posture of the drone body and the gimbal camera after posture adjustment is photographed and stored.
[0049] In this embodiment, when the posture of the drone body and the gimbal camera is corrected during the inspection according to the posture correction information, it is necessary to take pictures of the posture of the drone body and the gimbal camera after the posture adjustment and store them.
[0050] Optionally, the power tower self-inspection method further includes:
[0051] Based on the location information of the pole tower to be inspected carried in the inspection start command sent by the handheld terminal, the drone body is controlled to fly to directly above the pole tower to be inspected.
[0052] The position information of the tower to be inspected can be understood as the GPS information of the tower to be inspected. The handheld terminal can be understood as a portable data processing terminal with an operating system, memory, CPU, graphics card, screen, keyboard, and data transmission processing capabilities. The handheld terminal can communicate data with the drone, for example, the flight GPS information of the drone can be sent through the handheld terminal. The inspection start instruction can be understood as the inspection start instruction sent by the handheld terminal to the drone, and the inspection instruction contains the position information of the tower to be inspected.
[0053] In this embodiment, when conducting a safety check and determining that the drone's flight starting point is far away from the tower and the area above is unobstructed, the handheld terminal can send the drone the location information of the tower to be inspected carried in the inspection start command, and control the drone body to fly to directly above the tower to be inspected.
[0054] Embodiment 2
[0055] Figure 2 A flowchart of a method for self-inspection of power poles and towers provided in the second embodiment of the present invention. Based on the above embodiments, this embodiment further refines the method of controlling the pan-tilt camera to capture the point image of the pole tower to be inspected when the current flight position of the drone body meets the capture conditions, and determines the posture correction information of the drone inspection based on the point image and the pre-created reference image set. Specifically, it may include the following steps:
[0056] S210: After the drone body flies above the pole tower to be inspected, the drone body is controlled to fly according to a given inspection route.
[0057] In this embodiment, after the drone body flies above the pole tower to be inspected, the drone body can be controlled to fly according to a pre-given pole tower inspection route.
[0058] S220: Determine the next inspection key point of the UAV body according to the inspection route.
[0059] Among them, the next inspection key point can be understood as the next inspection key waypoint of the tower to be inspected. The next key point can be the first next key point in the cruise route, or the second next key point, and so on, until the key point of the end point of the inspection route of the tower to be inspected. The first next key point of the tower to be inspected can be the first next key point in the rough inspection path calculated according to the tower top posture. It should be noted that the first next key point of the tower to be inspected can be determined by the accurate position information of the tower to be inspected manually input or entered in advance by the handheld terminal, and then the second next key point in the cruise route and other next key points in the inspection route can be determined according to the determined first next key point. When each tower to be inspected determines the next inspection key point of the drone body according to the inspection route, the next inspection key point is determined in the above manner.
[0060] In this embodiment, after the drone body flies above the pole tower to be inspected, the drone body is controlled to fly according to a given inspection route, and the next inspection key waypoint of the drone body can be determined according to the inspection route.
[0061] S230: When it is detected that the current flight position of the drone body is at the next inspection key point, it is determined that the capture condition is met.
[0062] In this embodiment, when the current flight position of the drone body is detected to be at the next inspection key point, it can be considered that the capture condition is met, and the pan-tilt camera can be controlled to capture the inspection image containing the next inspection key point.
[0063] S240, controlling the pan / tilt camera to capture an inspection image containing the next inspection key point as a point image relative to the pole tower to be inspected.
[0064] In this embodiment, when it is determined that the capture condition is met, the PTZ camera can be controlled to capture an inspection image containing the next inspection key point, and the inspection image is used as a point image relative to the tower to be inspected.
[0065] S250, extracting an inspection target frame including the next inspection key point from the point image, and selecting a target key point image including the next inspection key point from the reference image set.
[0066] The inspection target frame can be understood as the inspection target frame of the target in the point image obtained by using the target detector. The target key point image can be understood as the target key point image of the next inspection key point contained in the preset reference image set.
[0067] In this embodiment, there are multiple inspection target frames in the point image, and the inspection target frames can be divided in multiple ways. For example, the inspection target frames can be divided according to the pixels of the captured point image. The point image may contain tens of thousands of pixels, and an inspection target frame can be formed every few pixels.
[0068] In this embodiment, after controlling the pan-tilt camera to capture the inspection image containing the next inspection key point and using it as the point image relative to the inspected pole tower, the inspection target frame of the target in the point image obtained by the target detector can be extracted from the point image, and the target key point image of the next inspection key point contained therein can be selected from the reference image set.
[0069] S260, determining the current posture feature vector of the next inspection key point in the inspection target frame, and the target posture feature vector of the next inspection key point in the target key point bit image.
[0070] The current posture feature vector may be determined by a point image captured by a PTZ camera, and the target posture feature vector may be determined by a target key point image in a preset reference image set.
[0071] In this embodiment, the current posture feature vector of the drone can be determined by the point image captured by the gimbal camera, and the target posture feature vector can be determined by the target key point image in the preset reference image set, so as to determine the posture correction information of the drone inspection based on the current posture feature vector of the drone and the target posture feature vector.
[0072] Specifically, the point image captured by the pan-tilt camera can be used with a target detector, such as ppyolo detection, to obtain an inspection target frame of the target in the point image, and then within the inspection target frame, the feature point information is searched for according to the set sliding window traversal to obtain all the feature point information, and the feature point information of the target key point image in the preset reference image set is also obtained to a certain extent, and the relative consistency of the feature point information in the point image captured by the pan-tilt camera and the feature point information of the target key point image in the preset reference image set is calculated.
[0073] Optionally, determining the current posture feature vector of the next inspection key point in the inspection target frame and the target posture feature vector of the next inspection key point in the target key point bit image includes:
[0074] According to the set sliding window, traverse the inspection target frame and the target key point image;
[0075] Obtaining first feature point information of each feature point included in the inspection target frame, and constructing a current posture feature vector based on each first feature point information;
[0076] The second feature point information of each feature point included in the target key point image is obtained, and a target posture feature vector is constructed based on each second feature point information.
[0077] Among them, the first feature point information can be understood as the feature point information composed of the sliding window of the inspection target frame of the point image captured by the pan-tilt camera. It should be noted that if the point image contains multiple targets, the first feature point information contains the feature point information of multiple targets. The first feature point information can include the position relationship of the drone relative to the target, position information, phase information, and the capture of various parameter information of the pan-tilt camera. The second feature point information can be understood as the feature point information of each feature point included in the target key point image in the preset reference image set. The second feature point target information corresponds to the first feature point information, and is used to determine the similarities and differences between the second feature point information and the first feature point information.
[0078] In this embodiment, the inspection target frame and the target key point image can be traversed according to the set sliding window to obtain the first feature point information of each feature point included in the inspection target frame of the point image captured by the pan-tilt camera, and the second feature point information of each feature point included in the target key point image in the preset reference image set, and the current posture feature vector is constructed based on the first feature point information, and the target posture feature vector is constructed based on the second feature point information.
[0079] S270: Determine the posture correction information of the UAV inspection according to the current posture feature vector and the target posture feature vector.
[0080] In this embodiment, the posture correction information of the drone inspection can be determined based on the current posture feature vector and the target posture feature vector. Specifically, the current posture feature vector can be formed based on the information of each first feature point, and the target posture feature vector can be formed based on the information of each second feature point. Then, the relative consistency value can be calculated based on the current posture feature vector and the target posture feature vector to determine the deviation feature point with the inspection posture deviation in the inspection target frame, and obtain the deviation feature point with the smallest deviation in the target key point position image in the preset reference image set, so as to adjust the posture according to the position information of each standard feature point and the external parameter matrix.
[0081] Optionally, determining the posture correction information of the drone inspection according to the current posture feature vector and the target posture feature vector includes:
[0082] Determine the relative consistency value between the current posture feature vector and the target posture feature vector;
[0083] According to the relative consistency value, the deviation feature points in the inspection target frame where the inspection posture deviation exists are determined;
[0084] Obtaining the standard feature point position information corresponding to each deviation feature point in the target key point image;
[0085] According to the position information of each standard feature point and the predetermined external parameter matrix, the target feature point position information of each deviation feature point is determined to form a posture adjustment matrix, which is recorded as the posture correction information of the UAV inspection.
[0086] The deviation feature point can be understood as a feature point where the current posture feature vector deviates from the target posture feature vector. The standard feature point position information can be understood as the feature point position information corresponding to each deviation feature point in the target key point image in the preset reference image set.
[0087] In this embodiment, the relative consistency value of the current posture feature vector and the target posture feature vector can be used to determine their relative consistency, and based on the relative consistency value, the deviation feature points in the inspection target frame where there is an inspection posture deviation can be determined, and the target feature point position information of each deviation feature point can be determined based on the standard feature point position information corresponding to each deviation feature point in the target key point image and a predetermined external parameter matrix, and a posture adjustment matrix can be constructed based on the target feature point position information to adjust the drone's posture.
[0088] S280, performing patrol attitude correction on the drone body and the gimbal camera according to the attitude correction information, and returning to continue the point image capture operation during continuous flight until the flight end is reached.
[0089] The embodiment of the present invention extracts an inspection target frame containing the next inspection key point from the point image, and selects a target key point image containing the next inspection key point from the reference image set; determines the current posture feature vector of the next inspection key point in the inspection target frame, and the target posture feature vector of the next inspection key point in the target key point image; determines the posture correction information of the drone inspection based on the current posture feature vector and the target posture feature vector, further realizes precise adjustment of the route and flight attitude, and improves the route accuracy and flight accuracy of the drone.
[0090] For example, to better understand the power tower self-inspection method, Figure 3 A schematic diagram of a flow chart of a power tower self-inspection method provided in the second embodiment of the present invention. Among them, key waypoints represent key points, such as Figure 3 As shown, the specific steps of the power tower self-inspection method are:
[0091] S301. The drone body takes off to the top of the target tower according to the GPS and flies according to the given inspection route.
[0092] In this embodiment, a safety check is first performed to determine that the take-off point of the drone is far away from the tower and that the airspace above is unobstructed. Based on the location information of the tower to be inspected carried in the inspection start command sent by the handheld terminal, the drone body is controlled to fly directly above the tower to be inspected. The drone body takes off to the top of the target tower according to the GPS and flies according to the given inspection route.
[0093] S302: When it is detected that the current flight position of the drone body meets the capture condition, the next inspection key point of the drone body flight is determined according to the inspection route.
[0094] In this embodiment, a rough inspection path and a rough key waypoint can be calculated according to the tower top posture, and the aircraft enters the first next key point of the rough inspection path on one side of the tower.
[0095] S303, controlling the pan / tilt camera to capture an inspection image containing the next inspection key point as a point image relative to the tower to be inspected.
[0096] S304: Compare the point image captured by the PTZ camera with the target key point image in the pre-created reference image set.
[0097] S305, determining the current posture feature vector of the next inspection key point in the inspection target frame, and the target posture feature vector of the next inspection key point in the target key point bit image.
[0098] S306 , outputting the current posture feature vector of the key point in the point image and the target posture feature vector of the inspection key point in the target key point image to obtain a relative position relationship.
[0099] S307: Determine the posture correction information of the UAV inspection according to the current posture feature vector and the target posture feature vector.
[0100] S308, performing inspection posture correction on the drone body and the gimbal camera according to the posture correction information, and taking photos of the drone body and the gimbal camera after the posture adjustment and storing them.
[0101] S309: During the continuous flight, return to continue the point image capture operation until the flight end point is reached.
[0102] S310, the drone returns.
[0103] In this embodiment, after the drone completes its mission, it rises to a safe altitude and returns.
[0104] It should be noted that when there is a deviation between the standard waypoint of the drone and the test flight waypoint of the drone, the flight attitude needs to be adjusted to the precise waypoint. In order to better understand the precise attitude adjustment stage of flight, Figure 4 The following is a flow chart of the precise adjustment phase of the flight attitude of a drone provided by the second embodiment of the present invention. Figure 4 As shown in the figure, the specific steps of the UAV aircraft precise attitude adjustment stage are:
[0105] S401. Input the point image of the pole tower to be inspected collected by the gimbal camera during the flight of the UAV.
[0106] S402: Use a target detector to perform target detection, and extract an inspection target frame containing the next inspection key point from the point image.
[0107] S403: In the inspection target frame, according to the set sliding window, the inspection target frame and the target key point image are traversed to find feature points, so as to obtain all feature point information.
[0108] S404: Calculate relative consistency for different feature point information.
[0109] S405 , based on the relative consistency value, filter out the feature points with the smallest deviation in the pre-created reference image set, and obtain the standard feature point position information corresponding to each deviation feature point in the target key point image.
[0110] S406, using the affine matrix and the intrinsic parameter matrix to inversely calculate the extrinsic parameter matrix.
[0111] S407: Use the feature point location information and the determined external parameter matrix to reversely calculate the GPS of the current location.
[0112] S408: Output the attitude adjustment matrix.
[0113] It should be noted that in order to better understand posture adjustment, Figure 5 A structural schematic diagram of attitude adjustment is provided in a second embodiment of the present invention, wherein the attitude adjustment schematic diagram includes: a target in a point image, such as an insulator, an actual flight waypoint of the UAV, a standard waypoint of the UAV in a pre-created reference image set, a current attitude feature vector corresponding to the actual flight waypoint of the UAV, a target attitude feature vector corresponding to the standard waypoint of the UAV in a pre-created reference image set, and an attitude adjustment vector.
[0114] Embodiment 3
[0115] Figure 6 This is a schematic diagram of the structure of a power pole tower self-inspection device provided in the third embodiment of the present invention. The power pole tower self-inspection device provided in this embodiment can be implemented by software and / or hardware, and can be configured in a server to implement a power pole tower self-inspection method in the embodiment of the present invention. Figure 6 As shown, the device is applied to a drone, which includes a drone body and a gimbal camera. The device may specifically include: a route setting module 610, a first image capturing module 620, an information determining module 630 and a second image capturing module 640.
[0116] The route setting module 610 is used to control the drone body to fly according to a given inspection route after the drone body flies above the tower to be inspected;
[0117] The first image capturing module 620 is used to control the pan / tilt camera to capture a point image relative to the pole tower to be inspected when detecting that the current flight position of the drone body meets the capturing condition;
[0118] An information determination module 630 is used to determine the posture correction information of the drone inspection based on the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to the target inspection perspective;
[0119] The second image capture module 640 is used to perform patrol attitude correction on the drone body and the gimbal camera according to the attitude correction information, and return to continue performing point image capture operations during continuous flight until the flight end is reached.
[0120] The above technical solution of the embodiment of the present invention controls the drone body to fly according to the given inspection route after the drone body flies above the pole tower to be inspected through the route setting module; when the first image capture module detects that the current flight position of the drone body meets the capture conditions, it controls the gimbal camera to capture the point image relative to the pole tower to be inspected; the information determination module determines the posture correction information of the drone inspection according to the point image and the pre-created reference image set, and the second image capture module performs inspection posture correction on the drone body and the gimbal camera according to the posture correction information, and returns to continue the point image capture operation during continuous flight until the flight end is reached. In the embodiment of the present invention, the information determination module determines the posture correction information of the drone inspection according to the point image and the pre-created reference image set, and performs inspection posture correction on the drone body and the gimbal camera according to the posture correction information, thereby achieving precise adjustment of the route and flight attitude, improving the drone route accuracy and flight accuracy, and better meeting the inspection business needs. Compared with the existing technology, the adopted power pole tower self-inspection method improves the accuracy of route planning and flight accuracy, has a certain versatility, can be applied to inspection scenarios of multiple towers or multiple types of towers, and does not require a large amount of prior information, saving manpower and material resources and reducing costs.
[0121] Optionally, the first image capturing module 620 includes:
[0122] A key point determination unit, used to determine the next inspection key point of the UAV body flight according to the inspection route;
[0123] A condition determination unit, used to determine that a capture condition is satisfied when detecting that the current flight position of the drone body is at the next inspection key point;
[0124] The image acquisition unit is used to control the pan-tilt camera to capture the inspection image containing the next inspection key point as a point image relative to the pole tower to be inspected.
[0125] Optionally, the information determination module 630 includes:
[0126] A target point selection unit, used for extracting an inspection target frame including the next inspection key point from the point image, and selecting a target key point image including the next inspection key point from the reference image set;
[0127] A feature vector determination unit, used to determine a current posture feature vector of a next inspection key point in the inspection target frame, and a target posture feature vector of a next inspection key point in the target key point bit image;
[0128] The information determination unit is used to determine the posture correction information of the unmanned aerial vehicle inspection according to the current posture feature vector and the target posture feature vector.
[0129] Optionally, the feature vector determination unit comprises:
[0130] An image traversal subunit, used to traverse the inspection target frame and the target key point image according to a set sliding window;
[0131] A current feature vector forming subunit is used to obtain first feature point information of each feature point included in the inspection target frame, and form a current posture feature vector based on each of the first feature point information;
[0132] The target feature vector forming subunit is used to obtain second feature point information of each feature point included in the target key point image, and form a target posture feature vector based on each second feature point information.
[0133] Optionally, the information determination unit comprises:
[0134] A consistency determination subunit, used to determine a relative consistency value between the current posture feature vector and the target posture feature vector;
[0135] A feature point determination subunit, used to determine, according to the relative consistency value, a deviation feature point having an inspection posture deviation in the inspection target frame;
[0136] A position information acquisition subunit, used to obtain the standard feature point position information corresponding to each of the deviation feature points in the target key point image;
[0137] The adjustment matrix constitutes a subunit, which is used to determine the target feature point position information of each deviation feature point according to the position information of each standard feature point and a predetermined external parameter matrix, to form a posture adjustment matrix, which is recorded as the posture correction information of the drone inspection.
[0138] Optionally, the power tower self-inspection device further includes:
[0139] The storage module is used to take pictures and store the postures of the drone body and the gimbal camera after posture adjustment.
[0140] Optionally, the power tower self-inspection device further includes:
[0141] The information acquisition module is used to control the drone body to fly to the top of the pole tower to be inspected based on the position information of the pole tower to be inspected carried in the inspection start command sent by the handheld terminal.
[0142] The road self-inspection device provided in the embodiment of the present invention can execute the road self-inspection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0143] Embodiment 4
[0144] Figure 7 A schematic diagram of the structure of a drone provided in the fourth embodiment of the present invention is shown in FIG. Figure 7 As shown, the drone includes a drone body 710, a pan-tilt camera 720, a controller 730, a memory 740, an input device 750, and an output device 760; the number of the controller 730 in the drone can be one or more. Figure 7 A controller 730 is taken as an example; the controller 730, the memory 740, the input device 750 and the output device 760 in the drone can be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.
[0145] The memory 740, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the road self-inspection method in the embodiment of the present invention (for example, the route setting module 610, the first image capture module 620, the information determination module 630 and the second image capture module 640 in the power pole self-inspection device). The controller 730 executes various functional applications and data processing of the drone by running the software programs, instructions and modules stored in the memory 740, that is, realizes the above-mentioned power pole self-inspection method.
[0146] The memory 740 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 740 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 740 may further include a memory remotely arranged relative to the controller 730, and these remote memories may be connected to the drone via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0147] The input device 750 may be used to receive input digital or character information and generate key signal input related to the user settings and function control of the drone. The output device 760 may include a display device such as a display screen.
[0148] Embodiment 5
[0149] Embodiment 5 of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute a power pole tower self-inspection method when executed by a computer processor, and the method is applied to a drone, wherein the drone includes a drone body and a pan-tilt camera, and the method includes:
[0150] After the drone body flies above the pole tower to be inspected, the drone body is controlled to fly according to a given inspection route;
[0151] When it is detected that the current flight position of the UAV body meets the capture condition, the pan / tilt camera is controlled to capture a point image relative to the pole tower to be inspected;
[0152] Determine the posture correction information of the drone inspection according to the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to the target inspection perspective;
[0153] The UAV body and the gimbal camera are inspected and corrected according to the attitude correction information, and the UAV returns to continue the point image capture operation during the continuous flight until the flight end is reached.
[0154] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided in an embodiment of the present invention are not limited to the method operations described above, and can also execute related operations in the power tower self-inspection method provided in any embodiment of the present invention.
[0155] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0156] It is worth noting that in the embodiment of the above-mentioned power pole tower self-inspection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0157] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for self-inspection of power towers. It is characterized in that Applied to a drone, the drone includes a drone body and a pan / tilt camera, the method includes: After the drone body flies above the pole tower to be inspected, the drone body is controlled to fly according to a given inspection route; When it is detected that the current flight position of the UAV body meets the capture condition, the pan / tilt camera is controlled to capture a point image relative to the pole tower to be inspected; Determine the posture correction information of the drone inspection according to the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to the target inspection perspective; Performing patrol attitude correction on the drone body and the gimbal camera according to the attitude correction information, and returning to continue performing point image capture operations during continuous flight until reaching the flight endpoint; When the current flight position of the drone body is detected to meet the capture condition, the pan / tilt camera is controlled to capture a point image relative to the pole tower to be inspected, including: Determine the next inspection key point of the UAV body flight according to the inspection route; When it is detected that the current flight position of the drone body is at the next inspection key point, it is determined that the capture condition is met; Controlling the pan / tilt camera to capture an inspection image containing the next inspection key point as a point image relative to the pole tower to be inspected; Among them, the next inspection key point is the next inspection key waypoint of the tower to be inspected; the next inspection key point includes the first next key point, the second next key point in the inspection route, and all next key points until the flight end point of the inspection route.
2. The method according to claim 1, It is characterized in that Determining the posture correction information of the UAV inspection according to the point image and the pre-created reference image set includes: Extracting an inspection target frame including the next inspection key point from the point image, and selecting a target key point image including the next inspection key point from the reference image set; Determine a current posture feature vector of a next inspection key point in the inspection target frame, and a target posture feature vector of a next inspection key point in the target key point bit image; According to the current posture feature vector and the target posture feature vector, posture correction information of the unmanned aerial vehicle inspection is determined.
3. The method according to claim 2, It is characterized in that The step of determining the current posture feature vector of the next inspection key point in the inspection target frame and the target posture feature vector of the next inspection key point in the target key point bit image comprises: According to the set sliding window, traverse the inspection target frame and the target key point image; Obtaining first feature point information of each feature point included in the inspection target frame, and constructing a current posture feature vector based on each of the first feature point information; Second feature point information of each feature point included in the target key point image is obtained, and a target posture feature vector is constructed based on each second feature point information.
4. The method according to claim 2, It is characterized in that The step of determining the posture correction information of the drone inspection according to the current posture feature vector and the target posture feature vector includes: Determining a relative consistency value between the current posture feature vector and the target posture feature vector; Determine, according to the relative consistency value, deviation feature points in the inspection target frame where there is an inspection posture deviation; Obtaining the standard feature point position information corresponding to each of the deviation feature points in the target key point image; According to the position information of each standard feature point and a predetermined external parameter matrix, the target feature point position information of each deviation feature point is determined to form a posture adjustment matrix, which is recorded as the posture correction information of the drone inspection.
5. The method according to claim 1, It is characterized in that Also includes: Use the drone body and gimbal camera after attitude adjustment to take pictures and store them.
6. The method according to any one of claims 1 to 5, It is characterized in that Also includes: Based on the location information of the pole tower to be inspected carried in the inspection start command sent by the handheld terminal, the drone body is controlled to fly to directly above the pole tower to be inspected.
7. A self-inspection device for power towers, It is characterized in that Applied to a drone, the drone includes a drone body and a pan / tilt camera, and the device includes: The route setting module is used to control the drone body to fly according to the given inspection route after the drone body flies above the tower to be inspected; The first image capture module is used to control the pan / tilt camera to capture a point image relative to the pole tower to be inspected when detecting that the current flight position of the drone body meets the capture condition; An information determination module, used to determine the posture correction information of the UAV inspection according to the point image and a pre-created reference image set, wherein the reference image set includes at least one key point image captured according to the target inspection perspective; The second image capture module is used to perform inspection and attitude correction on the drone body and the gimbal camera according to the attitude correction information, and return to continue to perform point image capture operations during continuous flight until the flight end is reached; Wherein, the first image capture module includes: a key point determination unit, used to determine the next inspection key point of the UAV body flight according to the inspection route; A condition determination unit, used to determine that a capture condition is satisfied when detecting that the current flight position of the drone body is at the next inspection key point; An image acquisition unit, used for controlling the pan / tilt camera to capture an inspection image containing the next inspection key point as a point image relative to the tower to be inspected; The next inspection key point is the next inspection key waypoint of the tower to be inspected; the next inspection key point includes the first next key point, the second next key point in the inspection route, and all key points until the flight end point of the inspection route.
8. A drone, include: The drone body, the pan / tilt camera fixedly mounted on the drone body, and further include: one or more controllers; a memory for storing one or more programs, When the one or more programs are executed by the one or more controllers, the one or more controllers implement the power pole tower self-inspection method as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the power tower self-inspection method as described in any one of claims 1 to 6 is implemented.
