Laser foreign matter removing method based on unmanned aerial vehicle

Abstract

A laser foreign matter removing method based on an unmanned aerial vehicle comprises the steps that 1, based on the position of a foreign matter mounted on a wire, the unmanned aerial vehicle is controlled to fly to approach the preset range of the foreign matter; 2, an airborne camera on the unmanned aerial vehicle is aligned with the position of the foreign matter and shoots a real-time video stream containing the foreign matter and the wire; 3, identifying the lead and foreign matter contours in the real-time video stream through a video identification technology, and calculating suggested cutting points in real time based on the lead and foreign matter contours; and 4, based on the suggested cutting point, the angle of a laser on the unmanned aerial vehicle is adjusted in real time through a holder, the laser is aligned with the suggested cutting point to emit laser, and foreign matter is removed through the laser. The unmanned aerial vehicle is used for carrying the laser, long-distance laser foreign matter removing can be achieved, and automatic tracking and removing of a long-distance target can be achieved by adopting an aiming mode based on the precise holder and the camera.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV)-mounted laser technology, and in particular to a laser foreign matter removal method based on an unmanned aerial vehicle (UAV). Background Art

[0002] Foreign objects hanging on overhead lines are one of the main reasons for unplanned line outages. Traditional treatment methods require power outages or live work, but line outages reduce the reliability of power supply, increase the risk of regional power outages, and may cause large economic losses and negative social impacts; live work treatment methods generally require personnel to enter high-voltage and strong electric fields to work, indirectly or directly contacting live objects, which poses a high risk of operation, and the operation is time-consuming and labor-intensive. Summary of the invention

[0003] In view of the technical defects and technical drawbacks in the prior art, the embodiment of the present invention provides a laser foreign body removal method based on a drone to overcome the above problems or at least partially solve the above problems. The specific scheme is as follows:

[0004] A laser foreign body removal method based on an unmanned aerial vehicle, the method comprising:

[0005] Step 1: Based on the position of the foreign object mounted on the wire, the drone is controlled to fly close to the preset range of the foreign object;

[0006] Step 2, aiming the drone's onboard camera at the location of the foreign object and capturing a real-time video stream containing the foreign object and the wire;

[0007] Step 3: Use video recognition technology to identify the outlines of the wires and foreign objects in the real-time video stream, and calculate the recommended cutting points in real time based on the outlines of the wires and foreign objects;

[0008] Step 4: Based on the recommended cutting point, the angle of the laser on the drone is adjusted in real time through the gimbal, so that the laser is aimed at the recommended cutting point to emit the laser and remove foreign matter through the laser.

[0009] Furthermore, step 1 also includes:

[0010] After the drone flies close to the preset range of the foreign object, the drone is controlled to perform a 360-degree inspection of the foreign object, and a 360-degree panoramic image containing the foreign object is captured by the onboard camera;

[0011] The image recognition technology is used to identify the outline of foreign objects in the 360-degree panoramic image, and the size of the foreign object outline at each angle is calculated. The corresponding drone shooting position A when the foreign object outline is the largest is obtained, and the drone is hovered at the corresponding position A.

[0012] Furthermore, in step 3, based on the contours of the wire and the foreign body, the recommended cutting points are calculated in real time, specifically including:

[0013] Based on the contours of the conductor and the foreign object, the vertical distance from a point on the foreign object to the conductor is calculated, and the point where the vertical distance from the foreign object to the conductor is equal to a first preset distance is used as a recommended cutting point, wherein the recommended cutting point is located below the conductor.

[0014] Furthermore, step 3 also includes calculating the position of the recommended cutting point in the image based on each frame image in the real-time video stream, assuming that the position of the recommended cutting point in the first frame image is B1, and calculating the position Bk of the recommended cutting point in the image once every k frames of images, comparing Bk with B1, and controlling the drone to perform relative displacement based on the position deviation between Bk and B1 until the position deviation between Bk and B1 is within a preset deviation range, wherein k is greater than or equal to 1.

[0015] Furthermore, step 4 also includes: when the laser is emitted at the recommended cutting point by aiming the laser, the position of the foreign object is tracked in real time, based on the change of the position of the foreign object, the recommended cutting point is recalculated and updated, and based on the new recommended cutting point, the angle of the laser on the drone is adjusted in real time through the gimbal.

[0016] Furthermore, real-time tracking of the position of foreign objects includes: based on the frame images in the video stream, identifying the position information of the foreign objects in the image, and extracting the feature information of the foreign objects; in subsequent frame images, comparing the image content with the feature information of the foreign objects; detecting the foreign objects in the subsequent frame images through an image matching algorithm; calculating the position and feature information of the foreign objects in the subsequent frames; and updating the foreign object feature information in real time based on the feature information of the foreign objects in the subsequent frame images.

[0017] Furthermore, the method also includes: based on the recommended cutting point, driving the gimbal to move, outputting the actual cutting point position of the foreign object, and calculating the deviation between the recommended cutting point and the actual cutting point, and automatically correcting the deviation of the gimbal based on the deviation to achieve automatic precise aiming and cutting.

[0018] Furthermore, the gimbal is a three-axis self-stabilizing gimbal.

[0019] Furthermore, step 4 also includes: identifying the material and type of the foreign matter, matching the light output power of the laser based on the material and type of the foreign matter, driving the laser to aim at the recommended cutting point to emit a laser of corresponding power to remove the foreign matter.

[0020] Furthermore, the material and type of the foreign matter are identified, and based on the material and type of the foreign matter, the output power of the laser is matched, specifically including:

[0021] Collect foreign objects of different types and materials, select different colors for each foreign object, obtain images of simulated foreign object hanging lines of different colors for each foreign object, and use the images of simulated foreign object hanging lines as training sets to train the neural network;

[0022] Laser ablation tests are performed on foreign bodies of different types and materials. In the ablation test, each sample is ablated using different laser output powers, and the ablation time, output power, and whether an open flame is generated are recorded, so as to obtain the fastest ablation output power for each foreign body without generating an open flame.

[0023] The trained neural network is used to identify the type and material of foreign objects in the real-time video stream, and based on the identified type and material, the fastest ablation light power that does not produce open flames is matched to the corresponding foreign object.

[0024] The present invention has the following beneficial effects:

[0025] 1. The present invention utilizes a laser mounted on an unmanned aerial vehicle to achieve long-distance laser foreign body removal, and adopts an aiming method based on a precision gimbal and a camera to achieve automatic tracking and removal of remote targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic flow chart of a laser foreign body removal method based on an unmanned aerial vehicle provided in an embodiment of the present invention.

[0027] Figure 2 A schematic diagram of a Gaussian template provided by an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of Hough line detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] See also Figure 1 As shown, an embodiment of the present invention provides a laser foreign body removal method based on an unmanned aerial vehicle, the method comprising:

[0031] Step 1: Based on the position of the foreign object mounted on the wire, the drone is controlled to fly close to the preset range of the foreign object;

[0032] The position of the foreign object is known in advance and can be obtained based on any existing technical means. The improvement of the present invention does not lie in how to obtain the position of the foreign object on the wire.

[0033] Step 2, aiming the drone's onboard camera at the location of the foreign object and capturing a real-time video stream containing the foreign object and the wire;

[0034] Step 3: Use video recognition technology to identify the outlines of the wires and foreign objects in the real-time video stream, and calculate the recommended cutting points in real time based on the outlines of the wires and foreign objects;

[0035] Step 4: Based on the recommended cutting point, the angle of the laser on the drone is adjusted in real time through the gimbal, so that the laser is aimed at the recommended cutting point to emit the laser and remove foreign matter through the laser.

[0036] The present invention utilizes a laser mounted on an unmanned aerial vehicle to achieve long-distance laser foreign body removal, and adopts an aiming method based on a precision pan / tilt and camera to achieve automatic tracking and removal of remote targets.

[0037] The gimbal is a three-axis self-stabilizing gimbal, which has high rigidity, high strength, high precision and high responsiveness. The unmanned hovering mode is realized, and the gimbal stabilization that resists wind and body self-vibration is enhanced, which provides the necessary conditions for achieving image stability and automatic tracking; optionally, by analyzing the underwing and ambient airflow, the disturbance frequency caused by the aiming and locking of the laser mounting host and the amplitude of the UAV's own flight vibration are analyzed and calculated, and a full-band shock absorbing mechanism is designed; by studying the electrical and mechanical characteristics of the brushless motor, three-axis frame, and liquid shock absorbing ball, combined with a multi-stage closed-loop stabilization control algorithm, a special airborne large-load high-precision three-axis self-stabilizing gimbal is developed, which has high rigidity, high strength, high precision and high responsiveness, and realizes the unmanned hovering mode, and the gimbal stabilization that resists wind and body self-vibration is enhanced, which provides the necessary conditions for achieving image stability and automatic tracking.

[0038] Wherein, the optical path of the laser is coaxial with the center of the camera field of view.

[0039] Preferably, step 1 further comprises:

[0040] After the drone flies close to the preset range of the foreign object, the drone is controlled to perform a 360-degree inspection of the foreign object, and a 360-degree panoramic image containing the foreign object is captured by the onboard camera;

[0041] The outline of foreign objects in the 360-degree panoramic image is identified through image recognition technology, and the size of the foreign object outline at each angle is calculated to obtain the drone shooting position A corresponding to when the foreign object outline is the largest, that is, the outline of the foreign object photographed by no one at the corresponding position and angle is the largest, and the drone is hovered at the corresponding position A.

[0042] In the above embodiment, a 360-degree panoramic image containing the foreign objects is obtained through a 360-degree inspection of the foreign objects, and based on the size analysis of the foreign object contour at each angle, the optimal position and angle for the initial hovering of the drone can be obtained.

[0043] Preferably, based on the contours of the wire and the foreign body, the real-time calculation of the recommended cutting points specifically includes:

[0044] Based on the contours of the conductor and the foreign object, the vertical distance from a point on the foreign object to the conductor is calculated, and the point where the vertical distance from the foreign object to the conductor is equal to a first preset distance is used as a recommended cutting point, wherein the recommended cutting point is located below the conductor.

[0045] In the above embodiment, the suggested cutting point may be one point or multiple points. When the suggested cutting point is multiple points, the actual cutting is one cutting line.

[0046] Preferably, step 3 also includes calculating the position of the recommended cutting point in the image based on each frame image in the real-time video stream, assuming that the position of the recommended cutting point in the first frame image is B1, calculating the position Bk of the recommended cutting point in the image once every k frames of images, comparing Bk with B1, and controlling the drone to perform relative displacement based on the position deviation between Bk and B1 until the position deviation between Bk and B1 is within a preset deviation range, wherein k is greater than or equal to 1.

[0047] In the above embodiment, the position of the drone is corrected with the suggested cutting point in the first frame image as the base point, and the position of the drone is corrected in real time. For example, if the suggested cutting point in the subsequent frame image is translated to the left by a certain distance relative to the suggested cutting point in the first frame image, the drone is controlled to translate to the right until the suggested cutting point in the subsequent frame image coincides with the suggested cutting point in the first frame image, or the position deviation is within a preset deviation range, thereby correcting the displacement change of the drone caused by the ambient wind.

[0048] Preferably, step 4 also includes: when emitting a laser aimed at the recommended cutting point, tracking the position of the foreign object in real time, recalculating and updating the recommended cutting point based on the change in the position of the foreign object, and adjusting the angle of the laser on the drone in real time through the gimbal based on the new recommended cutting point.

[0049] Among them, real-time tracking of the position of foreign objects includes: based on the frame images in the video stream, identifying the position information of the foreign objects in the image, and extracting the feature information of the foreign objects; in subsequent frame images, comparing the image content with the feature information of the foreign objects; detecting the foreign objects in the subsequent frame images through an image matching algorithm; calculating the position and feature information of the foreign objects in the subsequent frames; and updating the foreign object feature information in real time based on the feature information of the foreign objects in the subsequent frame images.

[0050] Traditional land-based laser automatic tracking and aiming algorithms usually use a wire recognition method, that is, the position of the aiming target is determined by identifying the wires or other specific markers in the laser irradiation area. The premise of this method is the clearance condition, and the image background is relatively simple. When the land-based laser removes foreign objects, the laser is emitted from the ground to the air, and the image background is the sky. Therefore, the automatic tracking and aiming method of wire recognition is very suitable for land-based lasers. However, in the automatic tracking and aiming of the laser foreign object removal device mounted on the drone, the emission background of the laser is usually more complex and changeable. The use of the wire recognition method cannot meet the needs of automatic tracking and aiming of the laser foreign object removal device mounted on the drone. Therefore, the automatic tracking and aiming of the present invention uses a template matching algorithm, with the first frame image or the clearest frame image as the target template. The algorithm matches the target template with the image in the current frame to determine the position and direction of the target, and then controls the laser to aim. The advantage of this algorithm is that it can adapt to different environments and backgrounds, has strong adaptability and accuracy, and the algorithm is relatively simple to implement and runs faster.

[0051] Preferably, the method further includes: based on the recommended cutting point, driving the pan-tilt head to move, outputting the actual cutting point position of the foreign object, and calculating the deviation between the recommended cutting point and the actual cutting point, performing automatic deviation correction, and realizing automatic precise aiming and cutting.

[0052] In the above embodiment, for target feature recognition, firstly, the frame image in the video stream is obtained, the position information of the target in the image is determined, and then the image preprocessing module (image graying, binarization, Gaussian filtering) is used to eliminate irrelevant information in the image, eliminate interference and noise, enhance the detectability of relevant information and simplify the data to the maximum extent and advance the feature information of the target; for the automatic tracking algorithm based on the video stream, after the target features in the image are obtained through the above process, real-time target tracking is required. In the subsequent video stream, the system will continuously read each frame image, compare the image content with the target features, detect the position of the target through the image matching algorithm and update the target feature information in real time; in addition to the real-time requirement, the reliability of target tracking is also very important. Due to the influence of factors such as illumination, target occlusion, and moving speed, the morphology and appearance characteristics of the target may change, resulting in changes in the target features. In order to adapt to this change, it is necessary to continuously update the target feature information so as to track the target more accurately. Gaussian filtering can be used to update the target feature information in real time, which can help to efficiently and accurately match the target features with the current frame image, thereby achieving more accurate and stable target tracking.

[0053] In image processing, commonly used filtering algorithms include mean filtering, median filtering, and Gaussian filtering. Mean filtering uses the average value of all pixels in the template to replace the grayscale value of the central pixel of the template. This method is easily interfered by noise and cannot completely eliminate noise, but can only relatively reduce noise; median filtering calculates the median of all pixels in the template and uses the calculated median to change the grayscale value of the central pixel of the template. This method is not so sensitive to noise and can effectively eliminate salt and pepper noise, but it is easy to cause image discontinuity. When Gaussian filtering smoothes pixels in the neighborhood of an image, pixels at different positions in the neighborhood are assigned different weights. While smoothing the image, it can also retain more of the overall grayscale distribution characteristics of the image.

[0054] In numerical image processing, Gaussian filtering can be implemented mainly using two methods. One is discretized window sliding window convolution, and the other method is through Fourier transform. The automatic tracking and aiming algorithm of the present invention is suitable for Gaussian filtering of the first method. Discretized window sliding window convolution mainly uses Gaussian kernel, which is generally a Gaussian template of an odd size.

[0055] Down Figure 2 In the figure, the left side is a 3x3 Gaussian template, and the right side is a 5x5 Gaussian template.

[0056] The Gaussian template is calculated by the following function

[0057]

[0058] From the Gaussian template formula, we can know that the square of x and the square of y represent the distance between other pixels in the neighborhood and the central pixel in the neighborhood respectively. σ represents the standard deviation. The smaller the standard deviation, the narrower the two-dimensional Gaussian image and the less obvious the smoothing effect. The larger the standard deviation, the shorter and wider the Gaussian image and the more obvious the filtering effect.

[0059] Selection of Canny edge detection operator threshold:

[0060] The edges of an image can point to different directions, so the classic Canny algorithm uses four gradient operators to calculate the gradients in the horizontal, vertical and diagonal directions respectively, but usually four gradient operators are not used to calculate the four directions separately. Commonly used edge difference operators (such as Rober, Prewitt, Sobel) calculate the horizontal and vertical differences Gx and Gy, so that the gradient magnitude and direction can be calculated as follows:

[0061]

[0062] 0 = atan2(Gy,Gx)

[0063] The gradient angle θ ranges from radians -π to π, and then it is approximated to four directions, representing horizontal, vertical and two diagonal directions (0°, 45°, 90°, 135°), which can be divided by ±iπ / 8 (i=1,3,5,7). The gradient angle falling in each area is given a specific value, representing one of the four directions.

[0064] The general edge detection algorithm uses a threshold to filter out small gradient values ​​caused by noise or color changes, while retaining large gradient values. The Canny algorithm uses a double threshold, a high threshold and a low threshold, to distinguish edge pixels. If the gradient value of the edge pixel is greater than

[0065] If the edge gradient value is less than the high threshold and greater than the low threshold, it is considered a strong edge point. If the edge gradient value is less than the high threshold and greater than the low threshold, it is marked as a weak edge point, and the point less than the low threshold is suppressed.

[0066] Condition optimization of Hough line detection:

[0067] The essence of wire detection is to make a final straight line judgment on the images processed in the above steps. At present, the Hough straight line detection algorithm is commonly used for straight line judgment.

[0068] For an image, it can be converted from the Cartesian coordinate system to the Hough space. For a straight line: in the Cartesian coordinate system, a straight line has two parameters: slope k and intercept b; in the Hough space, a straight line also has two parameters: distance d to the origin and angle θ, as follows Figure 3 shown.

[0069] The automatic tracking control algorithm based on template matching is implemented based on Gaussian filtering, Canny edge computing and Hough line detection technology. Through real-time detection and analysis, combined with changes in recognition results, the clearance status of the current aiming target is judged. If the characteristic value of the recognized target changes, the algorithm will generate a tracking instruction to drive the three-axis self-stabilizing gimbal to track the target.

[0070] Preferably, step 4 also includes: identifying the material and type of the foreign matter, matching the light output power of the laser based on the material and type of the foreign matter, driving the laser to aim at the recommended cutting point to emit a laser of corresponding power to remove the foreign matter.

[0071] Among them, identifying the material and type of the foreign matter, and matching the output power of the laser based on the material and type of the foreign matter specifically include:

[0072] Collect foreign objects of different types and materials, select different colors for each foreign object, obtain images simulating foreign objects hanging on the wire for each type of foreign object with different colors, and use the images simulating foreign objects hanging on the wire as training sets to train the neural network;

[0073] Laser ablation tests are performed on foreign bodies of different types and materials. In the ablation test, each sample is ablated using different laser output powers, and the ablation time, output power, and whether an open flame is generated are recorded, so as to obtain the fastest ablation output power for each foreign body without generating an open flame.

[0074] The trained neural network is used to identify the type and material of foreign objects in the real-time video stream, and based on the identified type and material, the fastest ablation light power that does not produce open flames is matched to the corresponding foreign object.

[0075] In the above implementation, based on the deep learning method and strategy for intelligent identification of foreign body materials, intelligent identification of the material of foreign bodies in the power grid and recommendation of fusing power are achieved, and a safety management method for fusing without fire is implemented during the laser foreign body removal process.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A laser foreign body removal method based on drone, characterized in that: The method comprises: Step 1: Based on the position of the foreign object mounted on the wire, the drone is controlled to fly close to the preset range of the foreign object; Step 2, aiming the drone's onboard camera at the location of the foreign object and capturing a real-time video stream containing the foreign object and the wire; Step 3, identifying the outlines of the wire and foreign objects in the real-time video stream, and calculating the recommended cutting points in real time based on the outlines of the wire and foreign objects; Step 4: Based on the recommended cutting point, the angle of the laser on the drone is adjusted in real time through the gimbal, so that the laser is aimed at the recommended cutting point to emit the laser and remove foreign matter through the laser.

2. The laser foreign body removal method based on drone according to claim 1 is characterized in that: Step 1 also includes: After the drone flies close to the preset range of the foreign object, the drone is controlled to perform a 360-degree inspection of the foreign object, and a 360-degree panoramic image containing the foreign object is captured by the onboard camera; The image recognition technology is used to identify the outline of foreign objects in the 360-degree panoramic image, and the size of the foreign object outline at each angle is calculated. The corresponding drone shooting position A when the foreign object outline is the largest is obtained, and the drone is hovered at the corresponding position A.

3. The laser foreign body removal method based on drone according to claim 1 is characterized in that: In step 3, based on the contours of the wire and foreign body, the recommended cutting points are calculated in real time, including: Based on the contours of the conductor and the foreign object, the vertical distance from a point on the foreign object to the conductor is calculated, and the point where the vertical distance from the foreign object to the conductor is equal to a first preset distance is used as a recommended cutting point, wherein the recommended cutting point is located below the conductor.

4. The laser foreign body removal method based on drone according to claim 1 is characterized in that: Step 3 also includes calculating the position of the recommended cutting point in the image based on each frame image in the real-time video stream, assuming that the position of the recommended cutting point in the first frame image is B1, calculating the position Bk of the recommended cutting point in the image once every k frames of images, comparing Bk with B1, and controlling the drone to perform relative displacement based on the position deviation between Bk and B1 until the position deviation between Bk and B1 is within a preset deviation range, where k is greater than or equal to 1.

5. The laser foreign body removal method based on drone according to claim 1 is characterized in that: Step 4 also includes: when the laser is aimed at the recommended cutting point and the laser is emitted, the position of the foreign object is tracked in real time, based on the change of the position of the foreign object, the recommended cutting point is recalculated and updated, and based on the new recommended cutting point, the angle of the laser on the drone is adjusted in real time through the gimbal.

6. The laser foreign body removal method based on drone according to claim 5 is characterized in that: Real-time tracking of the position of foreign objects includes: based on the frame images in the video stream, identifying the position information of the foreign objects in the image, and extracting the feature information of the foreign objects; in subsequent frame images, comparing the image content with the feature information of the foreign objects; detecting the foreign objects in the subsequent frame images through an image matching algorithm; calculating the position and feature information of the foreign objects in the subsequent frames; and updating the feature information of the foreign objects in real time based on the feature information of the foreign objects in the subsequent frame images.

7. The laser foreign body removal method based on drone according to claim 5 is characterized in that: The method also includes: based on the recommended cutting point, driving the gimbal to move, outputting the actual cutting point position of the foreign object, and calculating the deviation between the recommended cutting point and the actual cutting point, and automatically correcting the deviation of the gimbal based on the deviation to achieve automatic precise aiming and cutting.

8. The laser foreign body removal method based on drone according to claim 1 is characterized in that: The gimbal is a three-axis self-stabilizing gimbal.

9. The laser foreign body removal method based on drone according to claim 1, characterized in that: Step 4 also includes: identifying the material and type of the foreign matter, matching the light output power of the laser based on the material and type of the foreign matter, driving the laser to aim at the recommended cutting point to emit a laser of corresponding power to remove the foreign matter.

10. The laser foreign body removal method based on drone according to claim 9, characterized in that: Identify the material and type of foreign matter. Based on the material and type of foreign matter, match the output power of the laser, including: Collect foreign objects of different types and materials, select different colors for each foreign object, obtain images of simulated foreign object hanging lines of different colors for each foreign object, and use the images of simulated foreign object hanging lines as training sets to train the neural network; Laser ablation tests are performed on foreign bodies of different types and materials. In the ablation test, each sample is ablated using different laser output powers, and the ablation time, output power, and whether an open flame is generated are recorded, so as to obtain the fastest ablation output power for each foreign body without generating an open flame. The trained neural network is used to identify the type and material of foreign objects in the real-time video stream, and based on the identified type and material, the fastest ablation light power that does not produce open flames is matched to the corresponding foreign object.

Images