Vision-based accurate falling method and system for flying-away electric power inspection robot

Abstract

The invention provides a vision-based accurate landing method and system for a flying-away electric power inspection robot, and relates to the technical field of electric power inspection. An area to be inspected is determined, the inspection robot is controlled to fly to a specified flight height range, and then a target landing position is determined; according to the method, the target landing position and the three-dimensional coordinate position of the target landing position in the camera coordinate system are obtained, and then the inspection robot is controlled to land to the three-dimensional coordinate position of the target landing position based on a deep reinforcement learning algorithm, so that accurate landing of the inspection robot in a complex environment is realized through the deep reinforcement learning algorithm, and meanwhile, the accuracy of the inspection robot is improved. The invention further provides a vision-based fly-away electric power inspection robot accurate line falling system, the inspection robot and a ground base station are integrated, comprehensive monitoring and control over the inspection robot are achieved through remote data transmission, image receiving and remote operation, and the system has the advantages of being simple in structure, convenient and fast to operate, high in falling precision and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of power line inspection, and more specifically, to a precise line landing method and system for a flying power line inspection robot based on vision. Background Art

[0002] With the continuous expansion and complexity of the power grid, the inspection work of power lines has become increasingly important and onerous. Traditional inspection methods rely on manual labor, which is not only inefficient and costly, but also difficult to achieve comprehensive coverage of high-voltage and complex terrain areas. The emergence of flying power line inspection robots provides an innovative solution to this problem. These robots combine the abilities of flight and walking, can quickly fly and locate above the power line, and land on the power line for detailed inspection when needed, significantly improving the inspection efficiency and reducing the personnel risk, which is an important technological progress in the field of power line inspection.

[0003] Although flying power line inspection robots have significant technical advantages, their line landing accuracy and stability in practical applications still face many challenges. The line landing algorithms currently used by many drones often have high computational complexity and demanding requirements for hardware resources, resulting in poor real-time performance and difficulty in making accurate judgments quickly in complex and changing power line environments. At the same time, high computational complexity also means higher energy consumption, which limits the operation time and endurance of the inspection robot, thus affecting its practicality. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the purpose of the present invention is to propose a precise line landing method for a flying power line inspection robot based on vision, including:

[0005] Step 1: Determine the area to be inspected, and obtain the prior information of the overhead transmission line in the area to be inspected. The prior information at least includes line height information, and the line height information includes ground wire height, neutral wire height, live wire height, and lightning protection wire height; according to the ground wire height, determine the specified flight height range, and control the inspection robot to fly to the specified flight height range;

[0006] Step 2: After the inspection robot flies to the specified flight height range, collect the environmental image of the environment where the inspection robot is located through the vision sensor configured on the inspection robot, collect the motion state information of the inspection robot through the inertial sensor configured on the inspection robot, and process the environmental image and the motion state information based on the Visual-Inertial Odometry (VIO) algorithm to obtain the position and orientation of the inspection robot in the world coordinate system;

[0007] Step 3: Collect the power line image of the area to be landed in the area to be inspected through a vision sensor, and send the image into the trained power line recognition model to obtain a binary image, where the area where the power line is located in the binary image is white and other areas are black;

[0008] Step 4: Determine the target landing position according to the attribute information of the power line, the safety slope data and the binary image;

[0009] Step 6: According to the position of the target landing position in the binary image, use a binocular camera to obtain the three-dimensional coordinate position of the target landing position in the camera coordinate system;

[0010] Step 6: Based on the deep reinforcement learning algorithm, based on the position and orientation of the inspection robot in the world coordinate system, control the inspection robot to land at the three-dimensional coordinate position of the target landing position.

[0011] Optionally, the prior information in Step 1 further includes the geographical location of the overhead transmission line, the line structure characteristics and the environmental information where the overhead transmission line is located;

[0012] The line structure characteristics include the characteristics of the tower structure, the characteristics of the conductor type, and the characteristics of the insulator string;

[0013] The specified flight height range is the interval from one meter below the ground wire height to one meter above the ground wire height.

[0014] Optionally, the position and orientation in the world coordinate system in Step 2 include: the three-dimensional coordinates of the current position of the inspection robot, the three-dimensional velocity vector of the speed of the inspection robot, the attitude angle of the inspection robot, and the three-dimensional angular velocity vector of the angular velocity of the inspection robot;

[0015] Among them, the three-dimensional velocity vector includes the velocity along the X-axis, the velocity along the Y-axis, and the velocity along the Z-axis. The attitude angle represents the rotation angle of the inspection robot in three-dimensional space. The attitude angle includes the pitch angle, the yaw angle, and the roll angle; the three-dimensional angular velocity vector includes the rotation speed around the X-axis, the rotation speed around the Y-axis, and the rotation speed around the Z-axis.

[0016] Optionally, the power line recognition model in Step 3 is obtained through the following method:

[0017] Use a convolutional neural network model based on deep learning as the initial model, obtain multiple training samples, where the training samples include input samples and output samples, the input samples include historical power line images with multiple scales and multiple scenarios, and the output samples are binary images corresponding to the historical power line images. Input the input samples into the initial model, process the input samples through the CNN algorithm to obtain the binary images corresponding to the input samples, and then update the parameters of the initial model based on the binary images corresponding to the input samples and the output samples until the model converges to obtain a power line recognition model.

[0018] Optionally, step 4 specifically includes:

[0019] Step 4.1: Adjust the parameters in the power line coupling model according to the attribute information of the power line, the safety slope data, and the binary image to obtain the formula model of the power line;

[0020] Step 4.2: Randomly select a position on the power line, substitute the coordinates of this position and the gravity of the inspection robot into the formula model of the power line to obtain the power line slope;

[0021] Step 4.3: Determine whether the power line slope is greater than the safety slope. If the power line slope is not greater than the safety slope, use this position as the target landing position. If the power line slope is greater than the safety slope, return to execute step 4.2.

[0022] Optionally, step 6 specifically includes:

[0023] Step 6.1: Based on the target policy parameters, determine the policy network of the intelligent agent, and use the position and direction of the inspection robot in the world coordinate system as the current state;

[0024] Step 6.2: Input the current state into the determined policy network to obtain the target action. The target action includes the attitude adjustment angle and the speed adjustment instruction. The attitude adjustment angle includes the rotation angles around the X, Y, and Z axes, and the attitude adjustment angle is used to adjust the flight attitude of the inspection robot; the speed adjustment instruction includes the speed increase and decrease amounts along the X, Y, and Z axes, and the speed adjustment instruction is used to control the flight speed of the inspection robot;

[0025] Step 6.3: Convert the target action into an instruction and send it to the inspection robot, and the inspection robot executes the instruction;

[0026] Step 6.4: Obtain the position and direction of the inspection robot in the world coordinate system after executing the instruction, use it as the current state, and return to execute step 6.2 until the inspection robot lands at the three-dimensional coordinate position of the target landing position after executing the instruction.

[0027] Optionally, the target policy parameters described in step 6.1 are obtained specifically through the following methods:

[0028] Step A1: Simulate the real flight environment to obtain a simulated environment;

[0029] Step A2: Initialize the policy parameters of the agent's policy network, use the initialized policy network as the current policy network, determine the initial state and the termination state. The initial state includes the initial position and initial direction of the inspection robot in the world coordinate system, and the termination state includes the termination position of the inspection robot in the world coordinate system. Take the initial state as the first state, set the initial number of iterations, and use the initial number of iterations as the current number of iterations;

[0030] Step A3: In the current number of iterations, the agent selects an action according to the first state and the current policy network. In the simulated environment, convert the selected action into an instruction and send it to the inspection robot. The inspection robot executes the instruction, and the simulated environment returns the second state, a reward, and a judgment result. The judgment result indicates whether to terminate;

[0031] Step A4: Take the first state, the random action, the second state, the reward, and the judgment result as an experience, and store the experience;

[0032] Step A5: Take the second state as the new first state, and return to execute step A3;

[0033] Step A6: Sample from multiple experiences. For each sampled experience, use the current policy network to calculate the value of the second state, and then calculate the target value through the value and the reward; According to the target value and the value of the second state, calculate the loss value, and use the proximal policy optimization algorithm to minimize the loss function, thereby updating the current policy network. Use the updated policy network as the new current policy network, increment the current number of iterations by one, and return to execute step A3 until the loss value is less than a preset threshold or the current number of iterations reaches a preset threshold to obtain the target policy parameters.

[0034] A vision-based accurate wire landing system for a flying power inspection robot is used to implement a vision-based accurate wire landing method for a flying power inspection robot, and includes a power supply system, an inspection robot system, a walking mechanism, a flight mechanism, and a ground base station system;

[0035] The power supply system is used to provide power for the inspection robot system. The power supply system includes a battery pack and a power management module;

[0036] The inspection robot system includes an on-board computer, a vision sensor, an inertial sensor, a flight control system, a walking controller system, and a data transmission module; The data transmission module includes a data transmission module and a video transmission module;

[0037] The visual sensor is used to collect the environmental image of the environment where the inspection robot is located and the power line image of the landing area in the area to be inspected; the visual sensor is also used to transmit the collected data to the on-board computer;

[0038] The inertial sensor is used to collect the motion state information of the inspection robot and transmit the motion state information to the on-board computer;

[0039] The on-board computer receives the image data and motion state information collected by the visual sensor, and implements the steps in the precise line landing of the fly-away power inspection robot based on vision, and sends the image data collected by the visual sensor to the ground base station system through the data transmission module; it is also used to send the flight instruction to the flight control system, where the flight instruction includes an instruction to control the inspection robot to fly to a specified flight altitude range, and an instruction to control the inspection robot to land at the target landing position according to the line landing trajectory, and receives the flight state data sent by the flight control system; it is also used to send the walking instruction for walking on the power line to the walking control system and receive the walking state data sent by the walking control system;

[0040] The flight control system is used to receive the flight instruction sent by the on-board computer and send the flight instruction to the flight mechanism; it is also used to receive the flight state data sent by the flight mechanism and send the flight state data to the on-board computer;

[0041] The flight mechanism is used to receive the flight instruction and execute it, collect the flight state data, and send the flight state data to the flight control system;

[0042] The walking control system is used to receive the walking instruction sent by the on-board computer, send the walking instruction to the walking mechanism, receive the walking state data sent by the walking mechanism, and send the walking state data to the on-board computer;

[0043] The walking mechanism is used to receive the walking instruction sent by the walking control system and execute it, collect the walking state data, and send the walking state data to the walking control system;

[0044] The ground base station system includes a data transmission module, a WiFi module and an industrial computer;

[0045] The industrial computer is connected to the transmission module and the WiFi module through USB, and the industrial computer receives the image data sent by the on-board computer through the transmission module and the WiFi module.

[0046] The beneficial effects produced by adopting the above technical solutions are as follows:

[0047] The present invention determines the area to be inspected, obtains the prior information of the overhead transmission line in the area to be inspected, determines the specified flight altitude range according to the ground wire height, controls the inspection robot to fly to the specified flight altitude range, and then obtains the position and orientation of the inspection robot in the world coordinate system based on the Visual Inertial Odometry (VIO) algorithm. At the same time, through the visual sensor and the power line recognition model, a binary image of the power line image in the area to be landed is obtained. Then, according to the attribute information of the power line, the safety slope data, and the binary image, the target landing position is determined, and further the three-dimensional coordinate position of the target landing position in the camera coordinate system is determined. Furthermore, based on the depth reinforcement learning algorithm, based on the position and orientation of the inspection robot in the world coordinate system, the inspection robot is controlled to land at the three-dimensional coordinate position of the target landing position. Thus, through the depth reinforcement learning algorithm, the present invention makes the planning of the wire landing trajectory more intelligent and efficient, realizes the precise landing of the inspection robot in a complex environment. At the same time, the present invention also provides a precise wire landing system for a vision-based flying inspection robot for power grids, which integrates the inspection robot and the ground base station, and realizes the comprehensive monitoring and control of the inspection robot through remote data transmission, image reception, and remote operation. This system has the advantages of simple structure, convenient operation, high landing accuracy, etc., and can be widely applied in the field of power grid inspection to improve the inspection efficiency and safety. Through the implementation of the present invention, it is expected to solve the problems existing in the landing process of the existing power grid inspection robots, such as short cruising time and susceptibility to interference, and promote the further development of power grid inspection technology. Brief Description of the Drawings

[0048] Figure 1 It is a schematic flowchart of a precise wire landing method for a vision-based flying inspection robot for power grids in an embodiment of the present invention;

[0049] Figure 2 It is a schematic structural diagram of a precise wire landing system for a vision-based flying inspection robot for power grids in an embodiment of the present invention. Detailed Embodiments

[0050] The following combines the drawings and embodiments to further describe in detail the specific embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0051] In recent years, deep learning technology has made breakthrough progress in multiple fields with its powerful data processing and pattern recognition capabilities. For the problem of wire landing of flying power inspection robots, deep learning methods provide a new solution. By training deep learning models, robots can automatically learn the characteristics of power lines, the variation laws of environmental parameters, and the strategies for safe wire landing from a large amount of data, thus significantly reducing the complexity of algorithms and improving real-time performance. At the same time, deep learning models can effectively reduce energy consumption during the optimization of the calculation process and extend the operation time of inspection robots. Therefore, applying deep learning methods to the wire landing control of flying power inspection robots is expected to significantly improve their wire landing accuracy and operation efficiency, bringing a more reliable and efficient solution for power inspection.

[0052] Aiming at the problems existing in the prior art, the present invention proposes a method and system for accurate wire landing of a flying power inspection robot based on vision. This method realizes the accurate landing of the inspection robot in a complex environment through steps such as rapid takeoff, position and direction perception, power line recognition and segmentation, calculation of the position to be landed, three-dimensional positioning of the landing position, and planning and execution of the wire landing trajectory. Among them, the application of the deep reinforcement learning algorithm makes the planning of the wire landing trajectory more intelligent and efficient. At the same time, the present invention also provides a flying power inspection robot system, which integrates the inspection robot and the ground base station, and realizes the comprehensive monitoring and control of the inspection robot through remote data transmission, image reception, and remote operation. This system has the advantages of simple structure, convenient operation, and high landing accuracy, and can be widely applied in the field of power inspection to improve inspection efficiency and safety. Through the implementation of the present invention, it is expected to solve problems such as short cruising time and susceptibility to interference existing in the landing process of existing power inspection robots, and promote the further development of power inspection technology. Specifically, the present invention provides a method for accurate wire landing of a flying power inspection robot based on vision, combined with Figure 1 , which may include the following steps:

[0053] Step 1: Determine the area to be inspected, and obtain the prior information of the overhead transmission line in the area to be inspected. The prior information at least includes line height information, and the line height information includes ground wire height, neutral wire height, live wire height, and lightning protection wire height; according to the ground wire height, determine the specified flight height range, and control the inspection robot to fly to the specified flight height range;

[0054] Among them, the prior information also includes the geographical location of the overhead transmission line, such as longitude and latitude coordinates, line orientation, etc., and also includes line structure characteristics. The line structure characteristics include the characteristics of the tower structure, the characteristics of the conductor type, and the characteristics of the insulator string; it also includes the environmental information where the overhead transmission line is located, such as terrain, landform, vegetation coverage and other environmental information, which helps the robot better plan the flight and walking paths.

[0055] Among them, the prior information of the overhead transmission line can be obtained through various channels, including but not limited to:

[0056] Historical data: Information about the overhead transmission line extracted from past inspection records, maintenance logs, or databases.

[0057] Professional measurement: Using professional measurement equipment and technologies, such as GPS, radar, LiDAR, etc., to accurately measure and locate the overhead transmission line.

[0058] Remote sensing technology: Using remote sensing platforms such as remote sensing satellites or drones to obtain remote sensing image data of the overhead transmission line, and extracting relevant information through image processing and analysis technologies.

[0059] The specified flight altitude range is the interval from one meter below the ground wire height to one meter above the ground wire height.

[0060] In the specific implementation process, after receiving the wire-drop command, the inspection robot executes Step 1. When controlling the inspection robot to fly to the specified flight altitude range, adjust its flight altitude and speed to quickly reach the specified flight altitude range. At the same time, the flight control system of the inspection robot maintains height stability to ensure that it does not deviate from the predetermined trajectory during flight.

[0061] Step 2: After the inspection robot flies to the specified flight altitude range, collect the environmental image of the environment where the inspection robot is located through the visual sensor (such as a camera) configured on the inspection robot, and collect the motion state information of the inspection robot through the inertial sensor (such as an accelerometer, gyroscope) configured on the inspection robot. Based on the Visual Inertial Odometry (VIO) algorithm, process the environmental image and the motion state information to obtain the position and orientation of the inspection robot in the world coordinate system;

[0062] Among them, the position and orientation in the world coordinate system include: the three-dimensional coordinates of the current position of the inspection robot, the three-dimensional velocity vector of the speed of the inspection robot, the attitude angle of the inspection robot, and the three-dimensional angular velocity vector of the angular velocity of the inspection robot;

[0063] Among them, the three-dimensional velocity vector includes the velocity along the X-axis, the velocity along the Y-axis, and the velocity along the Z-axis. The attitude angle represents the rotation angle of the inspection robot in three-dimensional space, and the attitude angle includes the pitch angle, yaw angle, and roll angle; the three-dimensional angular velocity vector includes the rotational velocity around the X-axis, the rotational velocity around the Y-axis, and the rotational velocity around the Z-axis.

[0064] Step 3: Collect the power line image of the area to be landed in the area to be inspected through a vision sensor, and send the image into the trained power line recognition model to obtain a binary image, where the area where the power line is located in the binary image is white and other areas are black;

[0065] Among them, the power line recognition model is obtained through the following method:

[0066] Use a convolutional neural network model based on deep learning as the initial model, obtain multiple training samples, the training samples include input samples and output samples, the input samples include historical power line images of multiple scales and multiple scenarios, the output sample is the binary image corresponding to the historical power line image, input the input sample into the initial model, process the input sample through the CNN algorithm to obtain the binary image corresponding to the input sample, and then calculate the loss value based on the binary image corresponding to the input sample and the output sample, and update the parameters of the initial model. Specifically, use the backpropagation algorithm to update the weights of the model to reduce the difference between the prediction and the true label until the model converges. Specifically, after multiple iterations until the performance of the model on the validation set reaches an acceptable level, or the loss function no longer decreases significantly, the power line recognition model is obtained.

[0067] Step 4: Determine the target landing position according to the attribute information of the power line, the safety slope data, and the binary image;

[0068] Step 4.1: Adjust the parameters in the power line coupling model according to the attribute information of the power line (such as radius, material), the safety slope data, and the binary image to obtain the formula model of the power line;

[0069] Step 4.2: Randomly select a position on the power line, substitute the coordinates of this position and the gravity of the inspection robot into the formula model of the power line to obtain the power line slope;

[0070] Step 4.3: Judge whether the power line slope is greater than the safety slope. If the power line slope is not greater than the safety slope, use this position as the target landing position. If the power line slope is greater than the safety slope, return to execute Step 4.2.

[0071] Step 5: According to the position of the target landing position in the binary image, use a binocular camera to obtain the three-dimensional coordinate position of the target landing position in the camera coordinate system;

[0072] Step 6: Based on the deep reinforcement learning algorithm, based on the position and orientation of the inspection robot in the world coordinate system, control the inspection robot to land at the three-dimensional coordinate position of the target landing position.

[0073] Step 6.1: Based on the target policy parameters, determine the policy network of the agent, and use the position and orientation of the inspection robot in the world coordinate system as the current state;

[0074] Step 6.2: Input the current state into the determined policy network to obtain the target action. The target action includes the attitude adjustment angle and the speed adjustment instruction. The attitude adjustment angle includes the rotation angles around the X, Y, and Z axes, and is used to adjust the flight attitude of the inspection robot; the speed adjustment instruction includes the speed increment and decrement along the X, Y, and Z axes, and is used to control the flight speed of the inspection robot;

[0075] Step 6.3: Convert the target action into an instruction and send it to the inspection robot, and the inspection robot executes the instruction;

[0076] Step 6.4: Obtain the position and orientation of the inspection robot in the world coordinate system after executing the instruction, use it as the current state, and return to execute Step 6.2 until the inspection robot lands at the three-dimensional coordinate position of the target landing position after executing the instruction.

[0077] Among them, the target policy parameters are specifically obtained through the following methods:

[0078] Step A1: Simulate the real flight environment to obtain a simulated environment;

[0079] Step A2: Initialize the policy parameters of the agent's policy network, use the initialized policy network as the current policy network, determine the initial state and the termination state. The initial state includes the initial position and initial orientation of the inspection robot in the world coordinate system, and the termination state includes the termination position of the inspection robot in the world coordinate system. Use the initial state as the first state, set the initial iteration number, and use the initial iteration number as the current iteration number;

[0080] Step A3: In the current iteration number, the agent selects an action according to the first state and the current policy network. In the simulated environment, convert the selected action into an instruction and send it to the inspection robot. The inspection robot executes the instruction, and the simulated environment returns the second state, the reward, and the judgment result. The judgment result indicates whether to terminate;

[0081] Among them, the reward is obtained based on the reward function, and the reward function is defined according to the straight-line distance of the inspection robot from the position in the first state to the position in the termination state. If the robot is closer to the termination position, the reward is higher; if the robot deviates from the termination position or has unsafe behaviors such as collisions, the reward will be reduced or even become negative. The design purpose of the reward function is to encourage the robot to take actions that can make it closer to the target position and be safe and stable.

[0082] Step A4: Use the first state, random action, second state, reward, and judgment result as experience, and store the experience;

[0083] Step A5: Use the second state as the new first state, and return to execute Step A3;

[0084] Step A6: Sample from multiple experiences. For each sampled experience, use the current policy network to calculate the value of the second state, and then calculate the target value through the value and the reward; According to the target value and the value of the second state, calculate the loss value, and use the proximal policy optimization algorithm to minimize the loss function, so as to update the current policy network. Use the updated policy network as the new current policy network, increment the current iteration count by one, and return to execute Step A3 until the loss value is less than the preset threshold or the current iteration count reaches the preset threshold to obtain the target policy parameters.

[0085] Combined with Figure 2 , the present invention also provides a precise wire landing system for a flying power inspection robot based on vision, which is used to implement a precise wire landing method for a flying power inspection robot based on vision, and includes a power supply system, an inspection robot system, a walking mechanism, a flight mechanism, and a ground base station system;

[0086] The power supply system is used to provide power for the inspection robot system, and the power supply system includes a battery pack and a power management module;

[0087] The inspection robot system includes an on-board computer, a vision sensor, an inertial sensor, a flight control system, a walking controller system, and a data transmission module; The data transmission module includes a data transmission module and a video transmission module;

[0088] The vision sensor (binocular camera) is used to collect the environmental image of the environment where the inspection robot is located and the power line image of the landing area to be inspected in the area to be inspected; The vision sensor is also used to transmit the collected data to the on-board computer;

[0089] The inertial sensor is used to collect the motion state information of the inspection robot and transmit the motion state information to the on-board computer;

[0090] The airborne computer receives the image data and motion state information collected by the vision sensor, and implements the steps in the precise line landing of a fly-away power inspection robot based on vision. It sends the image data collected by the vision sensor to the ground base station system through the data transmission module. It is also used to send flight instructions to the flight control system. The flight instructions include instructions to control the inspection robot to fly to a specified flight altitude range, and instructions to control the inspection robot to land at the target landing position according to the line landing trajectory, and receive the flight state data sent by the flight control system. It is also used to send the walking instructions for walking on the power line to the walking control system and receive the walking state data sent by the walking control system.

[0091] The flight control system is used to receive the flight instructions sent by the airborne computer and send the flight instructions to the flight mechanism. It is also used to receive the flight state data sent by the flight mechanism and send the flight state data to the airborne computer.

[0092] The flight mechanism is used to receive and execute the flight instructions, collect the flight state data, and send the flight state data to the flight control system.

[0093] The walking control system is used to receive the walking instructions sent by the airborne computer, send the walking instructions to the walking mechanism, receive the walking state data sent by the walking mechanism, and send the walking state data to the airborne computer.

[0094] The walking mechanism is used to receive and execute the walking instructions sent by the walking control system, collect the walking state data, and send the walking state data to the walking control system.

[0095] The ground base station system includes a data transmission module, a WiFi module, and an industrial computer.

[0096] The industrial computer is connected to the transmission module and the WiFi module through USB. The industrial computer receives the image data sent by the airborne computer through the transmission module and the WiFi module.

[0097] The above description is only the preferred embodiments of the present disclosure and the description of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A vision-based flying power inspection robot precision line landing method, characterized in that: include: Step 1: Determine the area to be inspected, obtain the prior information of the overhead power transmission lines in the area to be inspected, the prior information at least includes the line height information, the line height information includes the ground line height, the neutral line height, the live line height and the lightning conductor height; determine the specified flight height range according to the ground line height, and control the inspection robot to fly to the specified flight height range; Step 2: After the inspection robot flies to the specified flight altitude range, the visual sensor configured by the inspection robot collects the environmental image of the inspection robot's environment, and the inertial sensor configured by the inspection robot collects the motion state information of the inspection robot. Based on the visual inertial odometer (VIO) algorithm, the environmental image and motion state information are processed to obtain the position and direction of the inspection robot in the world coordinate system; Step 3: Collect the power line image of the area to be landed in the area to be inspected by the visual sensor, and send the image to the trained power line recognition model to obtain a binary image, in which the area where the power line is located is white and the other areas are black; Step 4: Determine the target landing position based on the attribute information of the power line, the safety slope data and the binary image; Step 5: According to the position of the target landing position in the binary image, the three-dimensional coordinate position of the target landing position in the camera coordinate system is obtained using the binocular camera; Step 6: Based on the deep reinforcement learning algorithm, and based on the position and direction of the inspection robot in the world coordinate system, control the inspection robot to land at the three-dimensional coordinate position of the target landing position.

2. According to the vision-based flying power inspection robot precise line landing method of claim 1, it is characterized in that: The prior information in step 1 also includes the geographical location of the overhead transmission line, the line structure characteristics and the environmental information of the overhead transmission line; The line structure characteristics include characteristics of the tower structure, characteristics of the conductor type, and characteristics of the insulator string; The designated flight altitude range is from the ground level altitude minus one meter to the ground level altitude plus one meter.

3. According to the vision-based flying power inspection robot precise line landing method of claim 1, it is characterized in that: The position and direction in the world coordinate system in step 2 include: the three-dimensional coordinates of the current position of the inspection robot, the three-dimensional velocity vector of the velocity of the inspection robot, the attitude angle of the inspection robot, and the three-dimensional angular velocity vector of the angular velocity of the inspection robot; Among them, the three-dimensional velocity vector includes the velocity along the X-axis, the velocity along the Y-axis and the velocity along the Z-axis, the attitude angle represents the rotation angle of the inspection robot in the three-dimensional space, and the attitude angle includes the pitch angle, the yaw angle and the roll angle; the three-dimensional angular velocity vector includes the rotation speed around the X-axis, the rotation speed around the Y-axis and the rotation speed around the Z-axis.

4. According to the vision-based flying power inspection robot precise line landing method of claim 1, it is characterized in that: The power line identification model in step 3 is obtained by: A convolutional neural network model based on deep learning is used as an initial model, and multiple training samples are obtained, wherein the training samples include input samples and output samples, wherein the input samples include historical power line images of multiple scales and multiple scenes, and the output samples are binary images corresponding to the historical power line images. The input samples are input into the initial model, and the input samples are processed by a CNN algorithm to obtain binary images corresponding to the input samples. Then, based on the binary images corresponding to the input samples and the output samples, the parameters of the initial model are updated until the model converges to obtain a power line recognition model.

5. According to the vision-based flying power inspection robot precise line landing method of claim 1, it is characterized by: Step 4 specifically includes: Step 4.1: According to the attribute information of the power line, the safety slope data and the binary image, the parameters in the power line coupling model are adjusted to obtain the formula model of the power line; Step 4.2: Randomly select a position on the power line, and substitute the coordinates of the position and the gravity of the inspection robot into the formula model of the power line to obtain the slope of the power line; Step 4.3: Determine whether the slope of the power line is greater than the safety slope. If the slope of the power line is not greater than the safety slope, use the position as the target landing position. If the slope of the power line is greater than the safety slope, return to execute step 4.

2.

6. The method for accurately dropping a line by a flying electric inspection robot based on vision according to claim 1 is characterized in that: Step 6 specifically includes: Step 6.1: Based on the target policy parameters, determine the policy network of the agent, and take the position and direction of the inspection robot in the world coordinate system as the current state; Step 6.2: Input the current state into the determined strategy network to obtain the target action, which includes a posture adjustment angle and a speed adjustment instruction. The posture adjustment angle includes the rotation angle around the X, Y, and Z axes, and the posture adjustment angle is used to adjust the flight posture of the inspection robot; the speed adjustment instruction includes the speed increase or decrease along the X, Y, and Z axes, and the speed adjustment instruction is used to control the flight speed of the inspection robot; Step 6.3: Convert the target action into an instruction and send it to the inspection robot, which executes the instruction; Step 6.4: Get the position and direction of the inspection robot in the world coordinate system after executing the instruction, take it as the current state, and return to execute step 6.2 until the inspection robot lands at the three-dimensional coordinate position of the target landing position after executing the instruction.

7. The method for accurately placing a flying power inspection robot based on vision according to claim 1 is characterized in that: The target strategy parameters described in step 6.1 are obtained specifically in the following manner: Step A1: simulating a real flight environment to obtain a simulated environment; Step A2: Initialize the policy parameters of the policy network of the agent, use the initialized policy network as the current policy network, determine the initial state and the final state, the initial state includes the initial position and initial direction of the inspection robot in the world coordinate system, the final state includes the final position of the inspection robot in the world coordinate system, use the initial state as the first state, set the initial number of iterations, and use the initial number of iterations as the current number of iterations; Step A3: In the current iteration number, the agent selects an action according to the first state and the current policy network, and in the simulation environment, converts the selected action into an instruction, sends it to the inspection robot, and the inspection robot executes the instruction. The simulation environment returns the second state, the reward and the judgment result, and the judgment result indicates whether to terminate; Step A4: taking the first state, random action, second state, reward and judgment result as experience, and storing the experience; Step A5: take the second state as the new first state and return to execute step A3; Step A6: Sampling from multiple experiences, for each sampled experience, using the current policy network to calculate the value of the second state, and then calculating the target value through the value and reward; According to the target value and the value of the second state, the loss value is calculated, and the loss function is minimized through the proximal policy optimization algorithm to update the current policy network. The updated policy network is used as the new current policy network, the current number of iterations is increased by one, and the process returns to execute step A3 until the loss value is less than the preset threshold, or the current number of iterations reaches the preset threshold, and the target policy parameters are obtained.

8. A vision-based flying power inspection robot precision line-dropping system, used to implement a vision-based flying power inspection robot precision line-dropping method according to claim 1, characterized in that: Including power supply system, inspection robot system, walking mechanism, flying mechanism and ground base station system; The power supply system is used to provide power to the inspection robot system, and the power supply system includes a battery pack and a power management module; The inspection robot system includes an onboard computer, a visual sensor, an inertial sensor, a flight control system, a walking controller system and a data transmission module; the data transmission module includes a data transmission module and an image transmission module; The visual sensor is used to collect environmental images of the environment in which the inspection robot is located and power line images of the area to be landed in the area to be inspected; the visual sensor is also used to transmit the collected data to the onboard computer; The inertial sensor is used to collect the motion state information of the inspection robot and transmit the motion state information to the onboard computer; The onboard computer receives image data and motion state information collected by the visual sensor, and implements the steps of a vision-based flying power inspection robot accurately landing on the line, and sends the image data collected by the visual sensor to the ground base station system through the data transmission module; it is also used to send flight instructions to the flight control system, the flight instructions include instructions for controlling the inspection robot to fly to a specified flight altitude range, and instructions for controlling the inspection robot to land at a target landing position according to the landing line trajectory, and receive flight state data sent by the flight control system; it is also used to send walking instructions on the power line to the walking control system, and receive walking state data sent by the walking control system; The flight control system is used to receive the flight instructions sent by the onboard computer and send the flight instructions to the flight mechanism; it is also used to receive the flight status data sent by the flight mechanism and send the flight status data to the onboard computer; The flight mechanism is used to receive and execute flight instructions, collect flight status data, and send the flight status data to the flight control system; The walking control system is used to receive the walking instruction sent by the onboard computer, send the walking instruction to the walking mechanism, receive the walking status data sent by the walking mechanism, and send the walking status data to the onboard computer; The walking mechanism is used to receive and execute walking instructions sent by the walking control system, collect walking state data, and send the walking state data to the walking control system; The ground base station system includes a data transmission module, a WiFi module and an industrial computer; The industrial computer is connected to the transmission module and the WiFi module via USB, and the industrial computer receives image data sent by the onboard computer via the transmission module and the WiFi module.

Images