Unmanned aerial vehicle power line inspection system

By combining voice and remote control methods, combined with depth cameras and sensors, a three-dimensional model and navigation path is generated, and automatic inspection and fault identification of the power line inspection system of the drone is realized, solving the complex operation and difficulty in trouble detection in the existing technology. The problem has improved the efficiency and safety of inspections.

CN118982880AInactive Publication Date: 2024-11-19QINGDAO UNIV

Patent Information

Application Number
CN202411447873.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-17
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing drone remote control control methods are complex and unintuitive, making it difficult to respond quickly in emergencies, and it is difficult to achieve the perfect fit between voice and remote control, resulting in low efficiency in power line inspection and difficulty in discovering faults.

Method used

A drone power line inspection system was designed, combining voice and remote control control, and generates three-dimensional models and inspection navigation paths through the handheld end and data processing center, and uses depth cameras and sensors to collect data in real time to realize automatic patrol. Check and trouble identification, equipped with obstacle avoidance module to deal with emergencies.

Benefits of technology

Reduced the professional skills requirements of operators, improved the response speed to emergencies, realized accurate and automatic inspection of the power line network, improved patrol efficiency, accuracy and comprehensiveness, and ensured the safe flight of drones. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982880A_ABST
    Figure CN118982880A_ABST
Patent Text Reader

Abstract

The invention discloses an unmanned aerial vehicle power line inspection system, and belongs to the technical field of unmanned aerial vehicles and intelligent control, the unmanned aerial vehicle power line inspection system comprises an unmanned aerial vehicle, a handheld terminal, a data processing center and a terminal device, the handheld terminal is used for an operator to input a control command of the unmanned aerial vehicle, and the unmanned aerial vehicle performs inspection flight along a power line network based on the control command; the unmanned aerial vehicle collects image data of a flight path and sends the image data to the data processing center, the data processing center establishes a three-dimensional model of a to-be-inspected power line network scene based on the collected image data and generates an inspection navigation path, and the unmanned aerial vehicle conducts inspection based on the inspection navigation path and collects real-time inspection image data. The data processing center completes inspection and troubleshooting of the power line network based on the real-time inspection image data, and the terminal equipment is used for displaying a three-dimensional model, a navigation path and inspection result information of a to-be-inspected power line network scene. The requirement for professional skills of operators is lowered, and the response speed to emergencies is increased.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles and intelligent control, and in particular relates to an unmanned aerial vehicle power line inspection system. Background Art

[0002] In order to ensure the safe and stable operation of power facilities, the power grid needs to be inspected and maintained regularly. Manual inspection is the most traditional inspection method, which has problems such as high labor cost, low efficiency, and high safety risks. With the rapid development of drone technology, its application in the field of power inspection is becoming more and more extensive. The operator controls the drone to fly along the power line network and confirms the fault point based on the drone data collected in real time. However, this method relies heavily on complex remote control operations, which requires high operator skills, and it is difficult to achieve intuitive and natural control, and it is difficult to respond quickly in an emergency. In addition, the fault point cannot be found in a timely and effective manner during the drone inspection process. To this end, the present invention provides a drone power line inspection system. Summary of the invention

[0003] The purpose of the present invention is to provide a UAV power line inspection system, which solves the problems that the existing UAV remote control means are complex to operate and the control is not intuitive, the voice control means have a single function, and it is difficult to perfectly match the voice and remote control means.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A drone power line inspection system comprises a drone, a handheld terminal, a data processing center and a terminal device, wherein the handheld terminal and the data processing center are both connected to the drone, and the terminal device is connected to the data processing center; the handheld terminal is used for an operator to input control commands of the drone, and the drone flies along the power line network for inspection based on the control commands, collects image data of the flight path, and sends the image data to the data processing center, and the data processing center establishes a three-dimensional model of the power line network scene to be inspected based on the collected image data and generates an inspection navigation path, and the drone inspects based on the inspection navigation path and collects real-time inspection image data, and the data processing center completes the inspection and troubleshooting of the power line network based on the real-time inspection image data, and the terminal device is used to display the three-dimensional model of the power line network scene to be inspected, the navigation path and the inspection result information.

[0006] The data processing center includes a UAV ground station control module, a patrol navigation path generation module, a fault detection module and a storage module. The patrol navigation path generation module, the fault detection module and the storage module are all connected to the UAV ground station control module. The patrol navigation path generation module establishes a three-dimensional model of the power line network scene to be inspected based on the image data collected during the handheld terminal control of the UAV patrol and generates a patrol navigation path. The fault detection module completes the inspection of the power line network based on the patrol navigation path and the image information transmitted in real time by the UAV, analyzes and confirms the fault location, and the storage module is used to store the image information and attitude information transmitted in real time by the UAV. The UAV ground station control module is a UAV ground station program used for parameter setting and real-time monitoring of the UAV.

[0007] The drone includes a drone body, a pixhawk4 flight controller, a wireless data transmission, an optical flow meter, a depth camera, a GPS module and a first battery. The drone body includes a frame, propeller blades, a motor bracket, a brushless motor and a V-shaped frame. A V-shaped frame is symmetrically arranged at both ends of the frame. A motor bracket is installed at each end of the V-shaped frame. The brushless motor is fixed on the motor bracket. The output end of the brushless motor is connected to the propeller blades. The GPS module is fixed on the upper side of the frame through a pole, the first battery is fixed on the bottom of the frame to provide power for the drone, the optical flowmeter is installed on the bottom of the frame to assist the depth camera in fusion positioning, and a depth camera is installed on the front side of the frame to collect RGB images. The pixhawk4 flight control has built-in gyroscopes, accelerometers, magnetometers, barometers and gyroscopes. Wireless data transmission, optical flowmeters, depth cameras, brushless motors, GPS modules, first batteries, gyroscopes, accelerometers, magnetometers, barometers and gyroscopes are all connected to the pixhawk4 flight control. The optical flowmeter, depth camera, GPS module, gyroscope, accelerometer, magnetometer, barometer and gyroscope are used to collect the attitude data of the drone during flight and send them to the pixhawk4 flight control. The pixhawk4 flight control is fixed on the frame to control the flight of the drone. The wireless data transmission is installed on one side of the frame to send the flight attitude data of the drone to the data processing center and the handheld terminal.

[0008] The handheld terminal includes a shell, a handheld terminal processor, a voice recognition chip, an NRF module, an antenna, a function button, a microphone and a second battery; the handheld terminal processor, the voice recognition chip and the NRF module are built into the shell, one end of the antenna is fixed on the shell, the antenna is connected to the handheld terminal processor through the NRF module, the function buttons embedded in the mounting holes on the shell surface are connected to the handheld terminal processor, the function buttons are used to input control commands to the handheld terminal processor, the microphone embedded in the shell surface is connected to the voice recognition chip, the voice recognition chip is connected to the handheld terminal processor, the microphone collects the voice input by the operator, the voice recognition chip is used to process the voice information collected by the microphone into corresponding control commands, and the handheld terminal processor sends the control commands to the drone through the NRF module and the antenna in sequence.

[0009] A method for using a drone power line inspection system is as follows: (1) Generate an inspection navigation path: The operator controls the drone to fly along the power line to be inspected through a handheld terminal, and the depth camera collects image data of the power network and its environment, and sends the image data to a data processing center. The data processing center uses a three-dimensional reconstruction module to generate a three-dimensional model of the power line scene to be inspected, calibrates the objects that need to be inspected in the three-dimensional model scene, and generates an inspection navigation path after the calibration is completed; (2) Inspection: Based on the assigned task, the corresponding inspection path is sent to the drone Pixhawk flight control, and the Pixhawk flight control The drone is controlled to fly according to the inspection path to inspect the power network to be inspected. During the process, the depth camera captures and collects image data in real time, and sends the real-time inspection image data and the flight status data of the drone to the data processing center. The data processing center determines the distance between the inspection target and the drone based on the image data, adjusts the flight attitude of the drone, and determines the power line fault and makes corresponding processing. Step (2) is specifically as follows: (201) Collect a fault image dataset and a normal image dataset, and mark the target category and target status in the fault image and the normal image respectively; (202) Use YOLO to v8 target detection network model architecture initializes the model, inputs the labeled fault image dataset and normal image dataset into the initialization model, calculates the loss through forward propagation, and updates the model weights through back propagation to minimize the loss, and obtains a trained fault recognition model. After training iterations, the model learns the feature representation in the dataset and generates a set of weight files. The weight files contain the model's recognition capabilities for faulty and normal targets; (203) Input the real-time collected image into the fault recognition model, and output the category, target state, position and confidence of each target in the image. The category of the target indicates the type of target identified, and the position indicates the type of target identified. The position indicates the position of the identified target relative to the drone, and the confidence indicates the accuracy of the identification. The staff further confirms the fault location by viewing the image through the terminal device; (204) If the distance between the target to be inspected and the drone is within the set range, no processing is performed. Otherwise, a corresponding command is sent to the Pixhawk flight control to control the flight attitude of the drone until the distance between the target to be inspected and the drone is within the set range. The flight attitude of the drone includes pitch angle, roll angle, yaw angle, flight altitude and flight speed. The ground station data processing center evaluates the flight status of the drone through the flight attitude data and detects flight faults in a timely manner.

[0010] The inspection navigation path includes a power line navigation path and a pole tower navigation path. The operator controls the drone to fly along the power line, collects the power line height and longitude and latitude in real time during the process, and records the video in real time, and sends the collected data to the data processing center. The data processing center uses a three-dimensional reconstruction module to generate a power line navigation path; the operator controls the drone to fly along the pole tower height, collects the pole tower height, longitude and latitude coordinates, images and other data, determines the pole tower type and line through the pole tower image, inputs the pole tower height, line layer height, longitude and latitude and type and other data coordinates into the three-dimensional reconstruction module to generate a pole tower model including the pole tower layer and the line layer, calibrates the locations that need to be photographed in the pole tower model, and then generates the pole tower navigation path according to the photographed locations.

[0011] The beneficial effects of the present invention are as follows: (1) During the first inspection, the two control modes of voice and remote control are combined, and the operator can flexibly choose according to needs, which reduces the requirements for the professional skills of the operator, and voice control can improve the response speed to emergencies; (2) The inspection navigation path and the inspection image data collected in real time by the depth camera are combined to complete the automatic inspection of the power network with inspection, accurately find the fault point, and improve the inspection efficiency, accuracy and comprehensiveness; (3) The drone has a built-in obstacle avoidance module, which can quickly respond to emergencies; (4) The drone is equipped with optical flow meters, depth cameras, GPS modules, gyroscopes, accelerometers, magnetometers, barometers and gyroscopes and other sensor devices, and the various devices can cooperate with each other to obtain accurate flight status data of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The present invention provides a principle block diagram of a UAV power line inspection system.

[0013] Figure 2 The present invention is a block diagram of the data processing center principle.

[0014] Figure 3 The present invention is a schematic diagram of the structure of the unmanned aerial vehicle.

[0015] Figure 4 This is a front structural diagram of the handheld terminal involved in the present invention.

[0016] Figure 5 This is a diagram showing the internal structure of the handheld terminal involved in the present invention.

[0017] Among them, 1 is a drone, 2 is a handheld terminal, 3 is a data processing center, 4 is a terminal device, 101 is a pixhawk4 flight control, 102 is a propeller, 103 is a wireless data transmission, 104 is an optical flow meter, 105 is a depth camera, 106 is a motor bracket, 107 is a brushless motor, 108 is a GPS module, 201 is a shell, 202 is a handheld processor, 203 is an NRF module, 204 is an antenna, 205 is a function button, 206 is a microphone, 207 is a second battery, 208 is an information display screen, 209 is a reserved IO port, 301 is a drone ground station control module, 302 is a patrol navigation path generation module, 303 is a troubleshooting module, and 304 is a storage module. DETAILED DESCRIPTION

[0018] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0019] Example

[0020] like Figure 1 As shown, the present embodiment involves a UAV power line inspection system, including a UAV 1, a handheld terminal 2, a data processing center 3 and a terminal device 4, the handheld terminal 2 and the data processing center 3 are both connected to the UAV 1, and the terminal device 4 is connected to the data processing center 3; the handheld terminal 2 is used for the operator to input the control command of the UAV 1, the UAV 1 flies along the power line network inspection based on the control command, collects image data of the flight path, and sends the image data to the data processing center 3, the data processing center 3 establishes a three-dimensional model of the power line network scene to be inspected based on the collected image data and generates an inspection navigation path, the UAV 1 inspects based on the inspection navigation path and collects real-time inspection image data, the data processing center 3 completes the inspection and troubleshooting of the power line network based on the real-time inspection image data, the terminal device 4 is used to display the three-dimensional model of the power line network scene to be inspected, the navigation path, the inspection results and other information, the terminal device 4 is a mobile phone, computer, tablet and other devices, the data processing center 3 sends real-time data to the terminal device 4 for on-site viewing by the UAV operator.

[0021] like Figure 2As shown, the data processing center 3 includes a UAV ground station control module 301, an inspection navigation path generation module 302, a fault detection module 303 and a storage module 304. The inspection navigation path generation module 302, the fault detection module 303 and the storage module 304 are all connected to the UAV ground station control module 301. The inspection navigation path generation module 302 establishes a three-dimensional model of the power line network scene to be inspected and generates an inspection navigation path based on the image data collected during the inspection process of the handheld terminal 2 controlling the UAV 1. The fault detection module 303 completes the inspection of the power line network based on the inspection navigation path and the image information transmitted in real time by the UAV 1, analyzes and confirms the fault location, and the storage module 304 is used to store the image information and posture information transmitted in real time by the UAV 1. The UAV ground station control module 301 is a UAV ground station program (such as QGroundControl) used for parameter setting and real-time monitoring of the UAV.

[0022] like Figure 3As shown, the drone 1 involved in this embodiment includes a drone body, a pixhawk4 flight control 101, a wireless data transmission 103, an optical flowmeter 104, a depth camera 105, a GPS module 108 and a first battery. The drone body includes a frame, blades 102, a motor bracket 106, a brushless motor 107 and a V-shaped frame. A V-shaped frame is symmetrically arranged at both ends of the frame, and a motor bracket 106 is installed at the end of the V-shaped frame. When landing, four motor brackets 106 touch the ground to ensure the stability of the landing of the drone 1. The brushless motor 107 is fixed on the motor bracket 106, and the output end of the brushless motor 107 is connected to the blade 102. The GPS module 108 is fixed to the upper side of the frame through a vertical pole. The first battery is fixed to the bottom of the frame to provide power for the drone. The optical flowmeter 104 is installed at the bottom of the frame to assist the depth camera 105 in fusion positioning. A depth camera 105 is installed on the front side of the frame to collect RGB images. Pixhawk4 flight controller 101 has built-in gyroscope, accelerometer, magnetometer, barometer and gyroscope. The gyroscope is used to measure the angular velocity of the drone and determine the attitude of the drone. The accelerometer is used to measure the acceleration of the drone in three axes and assist in attitude solution. The magnetometer is used to measure the direction of the geomagnetic field and help the drone determine the direction. The barometer is used to measure the atmospheric pressure and estimate the altitude of the drone. The wireless data transmission 103, the optical flowmeter 104, the depth camera 105, the brushless motor 107, the GPS module 108, the first battery, the gyroscope, the accelerometer, the magnetometer, the barometer and the gyroscope are all connected to the pixhawk4 flight control 101 to realize data exchange. The optical flowmeter 104, the depth camera 105, the GPS module 108, the gyroscope, the accelerometer, the magnetometer, the barometer and the gyroscope are used to collect the attitude data of the drone during flight and send it to the pixhawk4 flight control 101. The pixhawk4 flight control 101 is fixed on the frame to control the flight of the drone. The wireless data transmission 103 is installed on one side of the frame to send the flight attitude data of the drone 1 to the data processing center 3 and the handheld terminal 2.

[0023] It should be noted that the Pixhawk 4 flight controller 101 has a built-in Pixhawk Autopilot flight control program, including a flight control module, an emergency response module and a high-precision obstacle avoidance module. The flight control module controls the flight of the drone based on the control commands from the data processing center and the handheld terminal. The emergency response module controls the drone to enter the emergency mode based on the control commands from the handheld terminal. The high-precision obstacle avoidance module generates obstacle avoidance path planning based on the distance of obstacles collected by the drone in real time for emergency obstacle avoidance.

[0024] like Figure 4 and 5As shown, the handheld terminal 2 involved in this embodiment includes a shell 201, a handheld terminal processor 202, a voice recognition chip, an NRF module 203, an antenna 204, a function button 205, a microphone 206 and a second battery 207; the handheld terminal processor 202, the voice recognition chip and the NRF module 203 are built in the shell 201, one end of the antenna 204 is fixed on the shell 201, and the antenna 204 is connected to the handheld terminal processor 202 through the NRF module 203. The function buttons 205 embedded in the mounting holes on the surface of the shell 201 are all connected to the handheld terminal processor 202, and the function buttons 205 are used to input control commands to the handheld terminal processor 202. The microphone 206 embedded in the surface of the shell 201 is connected to the voice recognition chip, and the voice recognition chip is connected to the handheld terminal processor 202. The microphone 206 collects the voice input by the operator, and the voice recognition chip is used to process the voice information collected by the microphone 206 into a corresponding control command. The handheld terminal processor 202 sends the control command to the drone 1 through the NRF module 203 and the antenna 204 in sequence.

[0025] Specifically, the voice recognition chip involved in this embodiment adopts Anxinke VC-01, and VC-01 uses the voice chip US516P6 launched by Unisound, which is used to capture and analyze the operator's voice commands, and finally transmit them to the handheld processor 202 via serial communication.

[0026] Furthermore, the handheld terminal 2 also includes an information display screen 208 , which is connected to the handheld terminal processor 202 , and the information display screen 208 displays the command currently executed by the drone 1 and the posture information related to the drone 1 .

[0027] Furthermore, a reserved IO port 209 is provided on one side of the housing 201 of the handheld terminal 2, and the reserved IO port 209 is connected to the handheld terminal processor 202, and the reserved IO port 209 is used for later upgrades of the drone.

[0028] The function keys 205 involved in this embodiment include a menu switch key, a power switch key, a coordinate input key, a channel switch key, an up key, a down key, an emergency response mode, a yaw angle increase, a yaw angle decrease, etc.

[0029] A method for using a drone power line inspection system is as follows: (1) generating an inspection navigation path: an operator controls the drone to fly along the power line to be inspected through a handheld terminal, and a depth camera collects image data of the power network and its environment, and sends the image data to a data processing center. The data processing center uses a three-dimensional reconstruction module (such as Bundle Fusion) to generate a three-dimensional model of the power line scene to be inspected, and calibrates the objects that need to be inspected in the three-dimensional model scene, such as wire calibration, pole tower calibration, insulator calibration, etc. After the calibration is completed, an inspection navigation path is generated, and the inspection navigation path includes a wire navigation path and a pole tower navigation path; the image data includes a color image and depth information, and the value of each pixel point of the depth information represents the distance of the object relative to the depth camera; wire navigation path: an operator controls the drone to fly along the wire, and collects the height and longitude and latitude of the wire in real time during the process, and records the video in real time, and sends the collected data to the data processing center. The data processing center uses a three-dimensional reconstruction module (such as Bundle Fusion) to generate a three-dimensional model of the power line scene to be inspected, and calibrates the objects that need to be inspected in the three-dimensional model scene, such as wire calibration, pole tower calibration, insulator calibration, etc. After the calibration is completed, an inspection navigation path is generated, and the inspection navigation path includes a wire navigation path and a pole tower navigation path; the image data includes a color image and depth information, and the value of each pixel point of the depth information represents the distance of the object relative to the depth camera; wire navigation path: an operator controls the drone to fly along the wire, and collects the wire height and longitude and latitude in real time during the process, and records the video in real time, and sends the collected data to the data processing center. =Fusion) to generate the navigation path of the power line; Tower navigation path: The operator controls the UAV to fly along the height of the tower, collects the tower height, longitude and latitude coordinates, images and other data, determines the tower type and line through the tower image, inputs the tower height, line layer height, longitude and latitude and type and other data coordinates into the 3D reconstruction module (such as Bundle Fusion) to generate a tower model containing the tower layer and the line layer, calibrates the locations to be photographed in the tower model, such as insulators, and then generates the tower navigation path based on the photographed locations; (2) Inspection: Based on the assigned task, the corresponding inspection path is sent to the UAV Pixhawk flight control. The Pixhawk flight control controls the UAV to fly according to the inspection path and inspects the power network to be inspected. During the process, the depth camera captures and collects image data in real time, and sends the real-time inspection image data and the UAV's flight status data to the data processing center. The data processing center determines the distance between the inspection target and the UAV based on the image data, adjusts the UAV's flight attitude, and determines the power line fault and makes corresponding processing. The flight status data is data collected by the optical flow meter 104, the GPS module 108, the gyroscope, the accelerometer, the magnetometer, the barometer, and the gyroscope.

[0030] Step (2) is specifically as follows: (201) Collecting a dataset of fault images and a dataset of normal images, and marking the target categories and target states in the fault images and normal images, such as power line-normal, power line-damaged, pole tower-normal, pole tower-vegetation climbing, etc.; (202) Using the YOLO v8 target detection network model architecture to initialize the model, input the labeled fault image dataset and normal image dataset into the initialized model, calculate the loss through forward propagation, and update the model weights through back propagation to minimize the loss, and obtain a trained fault recognition model. After training iterations, the model learns the feature representation in the dataset and generates a set of weight files. The weight files contain the model's recognition capabilities for faulty and normal targets; (203) Inputting the real-time collected images into the fault recognition model, outputting the category, target state, location and confidence of each target in the image. The category of the target indicates the type of target recognized, such as power line, pole tower, insulator, etc. The position indicates the position of the identified target relative to the drone (depth camera), and the confidence level indicates the accuracy of the identification. The staff further confirms the fault location by viewing the image through the terminal device 4; (204) If the distance between the target to be inspected and the drone is within the set range, no processing is performed. Otherwise, a corresponding command is sent to the Pixhawk flight control to control the flight attitude of the drone until the distance between the target to be inspected and the drone is within the set range. The flight attitude of the drone includes but is not limited to the pitch angle, roll angle, yaw angle, flight altitude, flight speed, etc. The ground station data processing center evaluates the flight status of the drone through the flight attitude data and detects flight faults in a timely manner.

[0031] Basic operation mode: During the system startup phase, the operator first activates the drone and its supporting data processing center to ensure that the drone enters the standby state smoothly and is ready to perform inspection tasks at any time. The operator holds the handheld terminal and presses the menu switch key to select the input mode. There are two modes: voice input and remote control input. In the voice input mode, the operator's voice recognition chip converts the relevant serial port signal and transmits it to the handheld processor. For example, if the operator says "fly forward one meter", the voice recognition chip sends the parameter "39" to the handheld processor; in the remote control mode, the operator operates the function button to control the drone. As the core of the system inspection, the data processing center is responsible for summarizing and analyzing the relevant inspection images sent back from the depth camera 105, so that the operator can grasp the on-site situation in real time and ensure the comprehensiveness and accuracy of the inspection work.

[0032] Emergency response mode: In the face of emergencies that the drone may encounter during the inspection process, such as power line anomalies, sudden obstacles, etc., the operator quickly presses the emergency response mode to switch to the level 1 emergency response mode. In this mode, the operator issues commands such as "emergency climb" and "turn left to avoid obstacles" through voice. When the emergency response mode is pressed twice in succession, it switches to the level 2 emergency response mode. By pressing the digital combination key, it can perform extreme climb or descent, emergency landing, and return to the home position in response to serious system failures. After receiving the emergency control command, the drone will immediately start the built-in high-precision obstacle avoidance module and flight attitude adjustment mechanism. The implementation process is that the depth camera performs real-time environmental perception and obstacle detection, generates obstacle avoidance path planning based on obstacle location information and the current position of the drone, and automatically adjusts the flight attitude of the drone to avoid obstacles according to the obstacle avoidance path planning, ensuring that the inspection task is successfully completed under the premise of ensuring safety. The implementation of this mode not only reflects the rapid response capability of the present invention in responding to emergencies, but also further improves the overall safety and reliability of the drone power line inspection system.

[0033] To sum up, through the detailed elaboration of the above two specific implementation methods, the UAV power line inspection system based on the combination of voice and remote control implemented by the present invention has demonstrated significant advantages and values ​​in improving inspection efficiency, reducing operation difficulty and enhancing emergency response capabilities.

Claims

1. A UAV power line inspection system, characterized in that: It includes a drone, a handheld terminal, a data processing center and a terminal device. The handheld terminal and the data processing center are both connected to the drone, and the terminal device is connected to the data processing center. The handheld terminal is used for the operator to input control commands for the drone. The drone flies along the power line network based on the control commands, collects image data of the flight path, and sends the image data to the data processing center. The data processing center establishes a three-dimensional model of the power line network scene to be inspected based on the collected image data and generates an inspection navigation path. The drone inspects based on the inspection navigation path and collects real-time inspection image data. The data processing center completes the inspection and troubleshooting of the power line network based on the real-time inspection image data. The terminal device is used to display the three-dimensional model of the power line network scene to be inspected, the navigation path and the inspection result information.

2. The UAV power line inspection system according to claim 1, characterized in that: The data processing center includes a UAV ground station control module, a patrol navigation path generation module, a fault detection module and a storage module. The patrol navigation path generation module, the fault detection module and the storage module are all connected to the UAV ground station control module. The patrol navigation path generation module establishes a three-dimensional model of the power line network scene to be inspected based on the image data collected during the handheld terminal control of the UAV patrol and generates a patrol navigation path. The fault detection module completes the inspection of the power line network based on the patrol navigation path and the image information transmitted in real time by the UAV, analyzes and confirms the fault location, and the storage module is used to store the image information and attitude information transmitted in real time by the UAV. The UAV ground station control module is a UAV ground station program used for parameter setting and real-time monitoring of the UAV.

3. The UAV power line inspection system according to claim 2, characterized in that: The drone includes a drone body, a pixhawk4 flight controller, a wireless data transmission, an optical flow meter, a depth camera, a GPS module and a first battery. The drone body includes a frame, propeller blades, a motor bracket, a brushless motor and a V-shaped frame. A V-shaped frame is symmetrically arranged at both ends of the frame. A motor bracket is installed at each end of the V-shaped frame. The brushless motor is fixed on the motor bracket. The output end of the brushless motor is connected to the propeller blades. The GPS module is fixed on the upper side of the frame through a pole, the first battery is fixed on the bottom of the frame to provide power for the drone, the optical flowmeter is installed on the bottom of the frame to assist the depth camera in fusion positioning, and a depth camera is installed on the front side of the frame to collect RGB images. The pixhawk4 flight control has built-in gyroscopes, accelerometers, magnetometers, barometers and gyroscopes. Wireless data transmission, optical flowmeters, depth cameras, brushless motors, GPS modules, first batteries, gyroscopes, accelerometers, magnetometers, barometers and gyroscopes are all connected to the pixhawk4 flight control. The optical flowmeter, depth camera, GPS module, gyroscope, accelerometer, magnetometer, barometer and gyroscope are used to collect the attitude data of the drone during flight and send them to the pixhawk4 flight control. The pixhawk4 flight control is fixed on the frame to control the flight of the drone. The wireless data transmission is installed on one side of the frame to send the flight attitude data of the drone to the data processing center and the handheld terminal.

4. The UAV power line inspection system according to claim 3 is characterized in that: The handheld terminal includes a shell, a handheld terminal processor, a voice recognition chip, an NRF module, an antenna, a function button, a microphone and a second battery; the handheld terminal processor, the voice recognition chip and the NRF module are built into the shell, one end of the antenna is fixed on the shell, the antenna is connected to the handheld terminal processor through the NRF module, the function buttons embedded in the mounting holes on the shell surface are connected to the handheld terminal processor, the function buttons are used to input control commands to the handheld terminal processor, the microphone embedded in the shell surface is connected to the voice recognition chip, the voice recognition chip is connected to the handheld terminal processor, the microphone collects the voice input by the operator, the voice recognition chip is used to process the voice information collected by the microphone into corresponding control commands, and the handheld terminal processor sends the control commands to the drone through the NRF module and the antenna in sequence.

5. The UAV power line inspection system according to claim 1, characterized in that: How to use: (1) Generate inspection navigation path: The operator controls the drone to fly along the power line to be inspected through the handheld terminal. The depth camera collects image data of the power network and its environment and sends the image data to the data processing center. The data processing center uses the 3D reconstruction module to generate a 3D model of the power line scene to be inspected, calibrates the objects to be inspected in the 3D model scene, and generates the inspection navigation path after the calibration is completed; (2) Inspection: Based on the assigned tasks, the corresponding inspection path is sent to the UAV Pixhawk flight control. The Pixhawk flight control controls the flight of the UAV according to the inspection path to inspect the power network to be inspected. During the process, the depth camera captures and collects image data in real time, and sends the real-time inspection image data and the flight status data of the UAV to the data processing center. The data processing center determines the distance between the inspection target and the UAV based on the image data, adjusts the flight attitude of the UAV, and determines the power line fault and takes corresponding measures. Step (2) is as follows: (201) Collecting a fault image dataset and a normal image dataset, and marking the target categories and target states in the fault images and the normal images respectively; (202) The model is initialized using the YOLO v8 target detection network model architecture. The labeled fault image dataset and the normal image dataset are input into the initialization model. The loss is calculated through forward propagation, and the model weights are updated through back propagation to minimize the loss. A trained fault recognition model is obtained. After training iterations, the model learns the feature representation in the dataset and generates a set of weight files. The weight files contain the model's recognition capabilities for faults and normal targets. (203) The real-time collected image is input into the fault recognition model, and the category, target state, position and confidence of each target in the image are output. The category of the target indicates the type of the target recognized, the position indicates the position of the recognized target relative to the drone, and the confidence indicates the accuracy of the recognition. The staff further confirms the fault location by viewing the image through the terminal device; (204) If the distance between the inspection target and the UAV is within the set range, no processing is performed. Otherwise, a corresponding command is sent to the Pixhawk flight control to control the flight attitude of the UAV until the distance between the inspection target and the UAV is within the set range. The flight attitude of the UAV includes the pitch angle, roll angle, yaw angle, flight altitude and flight speed. The ground station data processing center evaluates the flight status of the UAV through the flight attitude data and detects flight faults in a timely manner.

6. The UAV power line inspection system according to claim 5, characterized in that: The inspection navigation path includes the power line navigation path and the pole tower navigation path. The operator controls the drone to fly along the power line. During the process, the power line height and longitude and latitude are collected in real time, and real-time video is recorded. The collected data is sent to the data processing center, and the data processing center uses a three-dimensional reconstruction module to generate the power line navigation path; the operator controls the drone to fly along the pole tower height, collects the pole tower height, longitude and latitude coordinates, and image data, determines the pole tower type and line through the pole tower image, and inputs the pole tower height, line layer height, longitude and latitude and type data coordinates into the three-dimensional reconstruction module to generate a pole tower model containing the pole tower layer and the line layer, calibrates the locations that need to be photographed in the pole tower model, and then generates the pole tower navigation path based on the shooting locations.

Citation Information

Patent Citations

  • Voice control remote controller for unmanned aerial vehicle

    CN105405273A

  • Control method of unmanned aerial vehicle, device and terminal

    CN108375986A

  • Cable trench and tunnel inspection unmanned aerial vehicle system

    CN109760837A

  • Pole tower model matching and visual navigation-based power unmanned aerial vehicle and inspection method

    CN110133440A

  • Transformer substation unmanned aerial vehicle inspection route planning method

    CN113625748A

Cited By

  • Unmanned aerial vehicle intelligent on-site disposal system and method

    CN119937610A