Laser foreign matter removing system based on unmanned aerial vehicle

Abstract

A laser foreign matter removing system based on an unmanned aerial vehicle comprises the unmanned aerial vehicle, a holder, a camera, a laser, a flight control module, a contour recognition module, a foreign matter type recognition module and a light emitting power control module, wherein the holder and the camera are hung on the unmanned aerial vehicle; the direction and the angle of the laser are controlled by the holder; the flight control module is used for controlling the unmanned aerial vehicle to fly to approach a preset range of the foreign matter based on the position of the foreign matter mounted on the wire; the camera is used for aligning the foreign matter position and shooting a real-time video stream containing the foreign matter and the wire; the contour recognition module is used for recognizing the lead and foreign matter contours in the real-time video stream, and calculating suggested cutting points in real time based on the lead and foreign matter contours; the holder is used for adjusting the angle of the laser on the unmanned aerial vehicle in real time based on the suggested cutting point, so that the laser aligns to the suggested cutting point; the foreign matter type identification module is used for identifying the material and the type of a foreign matter; the light output power control module is used for matching the light output power of the laser based on the material and the type of the foreign matter.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV)-mounted laser technology, and in particular to a laser foreign matter removal system based on an unmanned aerial vehicle (UAV). Background Art

[0002] Foreign objects hanging on overhead lines are one of the main reasons for unplanned line shutdowns. Traditional processing methods require power outages or live operations. There are also technical means to remove foreign objects using drones equipped with lasers. However, conventional drone-mounted laser foreign object removal methods simply use drones with lasers to approach foreign objects and then output lasers to remove foreign objects. Existing technologies do not consider the effects of foreign object types and materials on laser removal. Summary of the invention

[0003] In view of the technical defects and technical drawbacks in the prior art, the embodiment of the present invention provides a laser foreign body removal system based on a drone to overcome the above problems or at least partially solve the above problems. The specific solution is as follows:

[0004] A laser foreign body removal system based on a drone, comprising a drone, a gimbal and a camera mounted on the drone, a laser whose azimuth and angle are controlled by the gimbal, a flight control module, a contour recognition module, a foreign body type recognition module, and a light output power control module;

[0005] The flight control module is used to control the UAV to fly within a preset range of the foreign object based on the position of the foreign object mounted on the wire;

[0006] The camera is used to align the position of the foreign object and capture a real-time video stream containing the foreign object and the wire;

[0007] The contour recognition module is used to recognize the contours of wires and foreign objects in the real-time video stream, and calculate the recommended cutting points in real time based on the contours of wires and foreign objects;

[0008] The gimbal is used to adjust the angle of the laser on the drone in real time based on the recommended cutting point so that the laser is aligned with the recommended cutting point;

[0009] The foreign body type identification module is used to identify the material and type of the foreign body;

[0010] The output light power control module is used to match the output light power of the laser based on the material and type of the foreign matter, drive the laser to aim at the recommended cutting point and emit a laser with a corresponding power to remove the foreign matter.

[0011] Furthermore, the foreign object type recognition module is specifically used to: collect foreign objects of different types and materials, select various colors for each foreign object, obtain images of simulated foreign object hanging lines of different colors for each foreign object, use the images of simulated foreign object hanging lines as training sets to train a neural network, and identify the type and material of foreign objects in the real-time video stream through the trained neural network.

[0012] Furthermore, the light output power control module includes an ablation test module and a light output control module;

[0013] The ablation test module is used to perform laser ablation tests on foreign bodies of different types and materials. In the ablation test, each sample is ablated by using different laser output powers, and the time required for ablation, the output power, and whether an open flame is generated are recorded to obtain the fastest ablation output power for each foreign body without generating an open flame.

[0014] The light output control module is used to match the fastest ablation light output power of the corresponding foreign object without generating open flames based on the type and material of the foreign object identified by the foreign object type identification module and the fastest ablation light output power of each foreign object without generating open flames obtained by the ablation test module, and drive the laser to aim at the recommended cutting point to emit a laser of corresponding power to remove foreign objects.

[0015] Furthermore, the system also includes a hovering control module; the hovering control module is used to control the drone to perform a 360-degree inspection of the foreign object after the drone flies close to the preset range of the foreign object, and to capture and synthesize a 360-degree panoramic image containing the foreign object through an onboard camera; identify the contour of the foreign object in the 360-degree panoramic image through image recognition technology, and calculate the size of the foreign object contour at each angle, obtain the drone shooting position A corresponding to the maximum foreign object contour, and hover the drone at the corresponding position A.

[0016] Further,

[0017] Based on the contours of the conductor and the foreign object, the vertical distance from a point on the foreign object to the conductor is calculated, and the point where the vertical distance from the foreign object to the conductor is equal to the first preset distance is used as the recommended cutting point.

[0018] Furthermore, the system also includes a drone position correction module; the drone position correction module is used to calculate the position of the recommended cutting point in the image based on each frame image in the real-time video stream, and let the position of the recommended cutting point in the first frame image be B1. Every k frames of images, the position Bk of the recommended cutting point in the image is calculated once, and Bk is compared with B1. Based on the position deviation between Bk and B1, the drone is controlled to perform relative displacement until the position deviation between Bk and B1 is within a preset deviation range, where k is greater than or equal to 1.

[0019] Furthermore, the optical power control module is also used to: when the laser is emitted at the recommended cutting point through the laser, the position of the foreign object is tracked in real time, based on the change of the position of the foreign object, the recommended cutting point is recalculated and updated, and based on the new recommended cutting point, the angle of the laser on the drone is adjusted in real time through the gimbal.

[0020] Furthermore, real-time tracking of the position of foreign objects includes: based on the frame images in the video stream, identifying the position information of the foreign objects in the image, and extracting the feature information of the foreign objects; in subsequent frame images, comparing the image content with the feature information of the foreign objects; detecting the foreign objects in the subsequent frame images through an image matching algorithm; calculating the position and feature information of the foreign objects in the subsequent frames; and updating the foreign object feature information in real time based on the feature information of the foreign objects in the subsequent frame images.

[0021] Furthermore, the optical power control module is also used to drive the pan-tilt movement based on the recommended cutting point, output the actual cutting point position of the foreign object, and calculate the deviation between the recommended cutting point and the actual cutting point, and automatically correct the deviation of the pan-tilt based on the deviation to achieve automatic precise aiming and cutting.

[0022] Furthermore, the gimbal is a three-axis self-stabilizing gimbal.

[0023] The present invention has the following beneficial effects:

[0024] The present invention is based on a deep learning method and strategy for intelligent identification of foreign body materials, realizes intelligent identification of materials of foreign bodies in the power grid and recommendation of fusing power, and realizes a safety management and control method for fusing without fire during the laser foreign body removal process. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A structural diagram of a laser foreign body removal system based on an unmanned aerial vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] like Figure 1 As shown, an embodiment of the present invention provides a laser foreign body removal system based on a drone, characterized in that it includes a drone, a gimbal and a camera hung on the drone, a laser whose azimuth and angle are controlled by the gimbal, a flight control module, a contour recognition module, a foreign body type recognition module, and a light output power control module;

[0028] The flight control module is used to control the UAV to fly within a preset range of the foreign object based on the position of the foreign object mounted on the wire;

[0029] The camera is used to align the position of the foreign object and capture a real-time video stream containing the foreign object and the wire;

[0030] The contour recognition module is used to recognize the contours of wires and foreign objects in the real-time video stream, and calculate the recommended cutting points in real time based on the contours of wires and foreign objects;

[0031] The gimbal is used to adjust the angle of the laser on the drone in real time based on the recommended cutting point so that the laser is aligned with the recommended cutting point;

[0032] The foreign body type identification module is used to identify the material and type of the foreign body;

[0033] The output light power control module is used to match the output light power of the laser based on the material and type of the foreign matter, drive the laser to aim at the recommended cutting point and emit a laser with a corresponding power to remove the foreign matter.

[0034] Wherein, based on the contours of the conductor and the foreign object, the vertical distance from a point on the foreign object to the conductor is calculated, and the point where the vertical distance from the foreign object to the conductor is equal to a first preset distance is taken as a recommended cutting point. Optionally, the recommended cutting point is located below the conductor.

[0035] The gimbal is a three-axis self-stabilizing gimbal, the optical path of the laser is coaxial with the center of the camera's field of view, and the three-axis self-stabilizing gimbal has high rigidity, high strength, high precision, and high responsiveness. It realizes unmanned hovering mode, wind resistance, and body self-vibration resistance gimbal stabilization, providing the necessary conditions for achieving image stabilization and automatic tracking.

[0036] The present invention is based on a deep learning method and strategy for intelligent identification of foreign body materials, realizes intelligent identification of materials of foreign bodies in the power grid and recommendation of fusing power, and realizes a safety management and control method for fusing without fire during the laser foreign body removal process.

[0037] Preferably, the foreign object type recognition module is specifically used to: collect foreign objects of different types and materials, select various colors for each foreign object, obtain images of simulated foreign objects hanging on the line for each type of foreign object of different colors, use the images of simulated foreign objects hanging on the line as a training set to train a neural network, and identify the type and material of the foreign objects in each frame of the real-time video stream through the trained neural network.

[0038] In the above embodiment, the image of the foreign object hanging on the wire is the image of the foreign object hanging on the wire. The present invention realizes the intelligent identification of the material of the foreign object in the power grid by studying the deep learning method and strategy for the intelligent identification of the material of the foreign object.

[0039] Preferably, the light output power control module includes an ablation test module and a light output control module;

[0040] The ablation test module is used to perform laser ablation tests on foreign bodies of different types and materials. In the ablation test, each sample is ablated by using different laser output powers, and the time required for ablation, the output power, and whether an open flame is generated are recorded to obtain the fastest ablation output power for each foreign body without generating an open flame.

[0041] The light output control module is used to match the fastest ablation light output power of the corresponding foreign object without generating open flames based on the type and material of the foreign object identified by the foreign object type identification module and the fastest ablation light output power of each foreign object without generating open flames obtained by the ablation test module, and drive the laser to aim at the recommended cutting point to emit a laser of corresponding power to remove foreign objects.

[0042] The present invention studies the intelligent fusing power control technology based on image processing and deep learning. Through a large number of experiments, the fusing power density and combustion power density of common foreign materials under the action of laser are studied. The intelligent selection of appropriate laser power density is realized to remove foreign matter, so that the foreign matter is melted and removed without catching fire. When the selected power is used to remove foreign matter with fire, the power can be intelligently lowered to adapt to the on-site environment.

[0043] Preferably, the system also includes a hovering control module; the hovering control module is used to control the drone to perform a 360-degree inspection of the foreign object after the drone flies close to the preset range of the foreign object, and to shoot and synthesize a 360-degree panoramic image containing the foreign object through an onboard camera; identify the contour of the foreign object in the 360-degree panoramic image through image recognition technology, and calculate the size of the foreign object contour at each angle, obtain the drone shooting position A corresponding to the maximum foreign object contour, and hover the drone at the corresponding position A.

[0044] In the above embodiment, a 360-degree panoramic image containing the foreign objects is obtained through a 360-degree inspection of the foreign objects, and based on the size analysis of the foreign object contour at each angle, the optimal position and angle for the initial hovering of the drone can be obtained.

[0045] Among them, the system also includes a drone position correction module; the drone position correction module is used to calculate the position of the recommended cutting point in the image based on each frame image in the real-time video stream, and let the position of the recommended cutting point in the first frame image be B1. Every k frames of images, the position Bk of the recommended cutting point in the image is calculated once, and Bk is compared with B1. Based on the position deviation between Bk and B1, the drone is controlled to perform relative displacement until the position deviation between Bk and B1 is within a preset deviation range, wherein k is greater than or equal to 1.

[0046] In the above embodiment, the position of the drone is corrected with the suggested cutting point in the first frame image as the base point, and the position of the drone is corrected in real time. For example, if the suggested cutting point in the subsequent frame image is translated to the left by a certain distance relative to the suggested cutting point in the first frame image, the drone is controlled to translate to the right until the suggested cutting point in the subsequent frame image coincides with the suggested cutting point in the first frame image, or the position deviation is within a preset deviation range, thereby correcting the displacement change of the drone caused by the ambient wind.

[0047] Preferably, the light output power control module is also used to: track the position of the foreign object in real time when emitting a laser aimed at the recommended cutting point, recalculate and update the recommended cutting point based on the change in the position of the foreign object, and adjust the angle of the laser on the drone in real time through the gimbal based on the new recommended cutting point.

[0048] Among them, real-time tracking of the position of foreign objects includes: based on the frame images in the video stream, identifying the position information of the foreign objects in the image, and extracting the feature information of the foreign objects; in subsequent frame images, comparing the image content with the feature information of the foreign objects; detecting the foreign objects in the subsequent frame images through an image matching algorithm; calculating the position and feature information of the foreign objects in the subsequent frames; and updating the foreign object feature information in real time based on the feature information of the foreign objects in the subsequent frame images.

[0049] Traditional land-based laser automatic tracking and aiming algorithms usually use a wire recognition method, that is, the position of the aiming target is determined by identifying the wires or other specific markers in the laser irradiation area. The premise of this method is the clearance condition, and the image background is relatively simple. When the land-based laser removes foreign objects, the laser is emitted from the ground to the air, and the image background is the sky. Therefore, the automatic tracking and aiming method of wire recognition is very suitable for land-based lasers. However, in the automatic tracking and aiming of the laser foreign object removal device mounted on the drone, the emission background of the laser is usually more complex and changeable. The use of the wire recognition method cannot meet the needs of automatic tracking and aiming of the laser foreign object removal device mounted on the drone. Therefore, the automatic tracking and aiming of the present invention uses a template matching algorithm, with the first frame image or the clearest image as the target template. The algorithm matches the target template with the image in the current frame to determine the position and direction of the target, and then controls the laser to aim. The advantage of this algorithm is that it can adapt to different environments and backgrounds, has strong adaptability and accuracy, and the algorithm is relatively simple to implement and runs faster.

[0050] Preferably, the optical power control module is also used to drive the pan-tilt head to move based on the recommended cutting point, output the actual cutting point position of the foreign object, and calculate the deviation between the recommended cutting point and the actual cutting point, and automatically correct the deviation of the pan-tilt head based on the deviation to achieve automatic precise aiming and cutting.

[0051] In the above embodiment, the deviation of the gimbal is automatically corrected by the deviation, so that the gimbal can control the angle of the laser on the drone, so that the laser can be more accurately aligned with the recommended cutting point.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A laser foreign body removal system based on drone, characterized in that: It includes a drone, a gimbal and a camera mounted on the drone, a laser whose position and angle are controlled by the gimbal, a flight control module, a contour recognition module, a foreign object type recognition module, and an optical power control module; The flight control module is used to control the UAV to fly within a preset range of the foreign object based on the position of the foreign object mounted on the wire; The camera is used to align the position of the foreign object and capture a real-time video stream containing the foreign object and the wire; The contour recognition module is used to recognize the contours of wires and foreign objects in the real-time video stream, and calculate the recommended cutting points in real time based on the contours of wires and foreign objects; The gimbal is used to adjust the angle of the laser on the drone in real time based on the recommended cutting point so that the laser is aligned with the recommended cutting point; The foreign body type identification module is used to identify the material and type of the foreign body; The output light power control module is used to match the output light power of the laser based on the material and type of the foreign matter, drive the laser to aim at the recommended cutting point and emit a laser with a corresponding power to remove the foreign matter.

2. The laser foreign body removal system based on drone according to claim 1 is characterized in that: The foreign object type recognition module is specifically used to: collect foreign objects of different types and materials, select various colors for each foreign object, obtain images of simulated foreign object hanging lines of different colors for each foreign object, use the images of simulated foreign object hanging lines as training sets to train a neural network, and identify the type and material of foreign objects in a real-time video stream through the trained neural network.

3. The laser foreign body removal system based on drone according to claim 1 is characterized in that: The light output power control module includes an ablation test module and a light output control module; The ablation test module is used to perform laser ablation tests on foreign bodies of different types and materials. In the ablation test, each sample is ablated by using different laser output powers, and the time required for ablation, the output power, and whether an open flame is generated are recorded to obtain the fastest ablation output power for each foreign body without generating an open flame. The light output control module is used to match the fastest ablation light output power of the corresponding foreign object without generating open flames based on the type and material of the foreign object identified by the foreign object type identification module and the fastest ablation light output power of each foreign object without generating open flames obtained by the ablation test module, and drive the laser to aim at the recommended cutting point to emit a laser of corresponding power to remove foreign objects.

4. The laser foreign body removal system based on drone according to claim 1 is characterized in that: The system also includes a hovering control module; the hovering control module is used to control the drone to perform a 360-degree inspection of the foreign object after the drone flies close to the preset range of the foreign object, and to shoot and synthesize a 360-degree panoramic image containing the foreign object through an onboard camera; to identify the contour of the foreign object in the 360-degree panoramic image through image recognition technology, and to calculate the size of the foreign object contour at each angle, to obtain the drone shooting position A corresponding to the maximum foreign object contour, and to hover the drone at the corresponding position A.

5. The laser foreign body removal system based on drone according to claim 1 is characterized in that: Based on the contours of the conductor and the foreign object, the vertical distance from a point on the foreign object to the conductor is calculated, and the point where the vertical distance from the foreign object to the conductor is equal to the first preset distance is used as the recommended cutting point.

6. The laser foreign body removal system based on drone according to claim 1 is characterized in that: The system also includes a drone position correction module; the drone position correction module is used to calculate the position of the recommended cutting point in the image based on each frame image in the real-time video stream, and let the position of the recommended cutting point in the first frame image be B1. Every k frames of images, the position Bk of the recommended cutting point in the image is calculated once, and Bk is compared with B1. Based on the position deviation between Bk and B1, the drone is controlled to perform relative displacement until the position deviation between Bk and B1 is within a preset deviation range, where k is greater than or equal to 1.

7. The laser foreign body removal system based on drone according to claim 1 is characterized in that: The optical power control module is also used to: when emitting laser through the laser aimed at the recommended cutting point, track the position of the foreign object in real time, recalculate and update the recommended cutting point based on the change of the position of the foreign object, and adjust the angle of the laser on the drone in real time through the gimbal based on the new recommended cutting point.

8. The laser foreign body removal system based on drone according to claim 7, characterized in that: Real-time tracking of the position of foreign objects includes: based on the frame images in the video stream, identifying the position information of the foreign objects in the image, and extracting the feature information of the foreign objects; in subsequent frame images, comparing the image content with the feature information of the foreign objects; detecting the foreign objects in the subsequent frame images through an image matching algorithm; calculating the position and feature information of the foreign objects in the subsequent frames; and updating the feature information of the foreign objects in real time based on the feature information of the foreign objects in the subsequent frame images.

9. The laser foreign body removal system based on drone according to claim 7, characterized in that: The optical power control module is also used to drive the pan-tilt head to move based on the recommended cutting point, output the actual cutting point position of the foreign object, and calculate the deviation between the recommended cutting point and the actual cutting point, and automatically correct the deviation of the pan-tilt head based on the deviation to achieve automatic precise aiming and cutting.

10. The laser foreign body removal system based on drone according to claim 1, characterized in that: The gimbal is a three-axis self-stabilizing gimbal.

Images