Power transmission line unmanned aerial vehicle line-imitating flight navigation control method and system

Abstract

The invention belongs to the technical field of unmanned aerial vehicle line simulation, and provides a power transmission line unmanned aerial vehicle line simulation flight navigation control method and system. The method comprises the following steps: pre-planning a reference flight path according to a power transmission line coordinate of a to-be-simulated line of an unmanned aerial vehicle; on the reference flight path, determining a detection frame coordinate center position of the power transmission line according to the infrared image and the visible light image of the power transmission line, and dynamically predicting the position of the unmanned aerial vehicle according to a change trend of the detection frame coordinate center position of the power transmission line and a slope change trend of a fitting straight line of the power transmission line; extracting temperature gradient field characteristics in a detection frame of the power transmission line, calculating a main direction angle of the power transmission line according to the temperature gradient field characteristics, and calculating a direction angle of the power transmission line by combining with a slope of a fitting straight line of the power transmission line; and controlling the flight speed on each coordinate axis of the unmanned aerial vehicle coordinate system according to the direction angle of the power transmission line, the dynamically predicted unmanned aerial vehicle position and the preset flight speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) line simulation, and in particular relates to a flight navigation control method and system for a UAV line simulation of a power transmission line. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Drones have begun to be used to conduct photo inspections of power transmission line conductors, identifying defects such as broken strands, rust, and burns, thereby preventing problems such as wire breakage or increased resistance and heat generation. Line-following flight differs significantly from traditional, refined inspections. Traditional refined inspections utilize a waypoint flight mode, where the drone flies to a certain waypoint, adjusts the gimbal, adjusts the nose angle, takes photos, and then flies to the next waypoint and repeats the process. Line-following flight, on the other hand, requires the drone's camera to be in close proximity to the conductor, capturing continuous images along the entire length of the conductor as it flies. Because power transmission lines are long and exhibit sag, using the same route planning methods used for traditional refined inspections for line-following flight would complicate route planning, significantly reducing the efficiency of line-following flight.

[0004] Currently, during the route planning process for line-mimicking flight, a starting point and an end point are usually planned. During the flight from the starting point to the end point, the drone continuously takes photos at a fixed nose angle and interval. However, when photographing transmission lines with sag in this way, the transmission lines can easily exceed the camera's shooting range. To avoid the situation where the wires exceed the camera's shooting range, the existing technology adopts the method of planning multiple routes, but this increases complexity and may even cause the position of the transmission lines in the image to be uncertain, affecting the subsequent recognition effect. Summary of the Invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a transmission line UAV line-mimicking flight navigation control method and system, which obtains UAV flight control information by fusing visible light images and infrared images, thereby autonomously controlling the UAV to fly and shoot at the same time to complete the inspection of transmission lines.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A first aspect of the present invention provides a transmission line UAV line-mimicking flight navigation control method.

[0007] A transmission line UAV flight navigation control method, comprising: Pre-plan the reference flight path based on the coordinates of the transmission line to be simulated by the UAV; On the reference flight path, the center of the transmission line detection frame coordinates is determined based on the infrared and visible light images of the transmission line. The drone's position is then dynamically predicted based on the changing trend of the center of the transmission line detection frame coordinates and the slope of the transmission line fitting line. The temperature gradient field characteristics within the detection frame of the transmission line are extracted, and the main direction angle of the transmission line is calculated based on this. Then, the direction angle of the transmission line is calculated by combining the slope of the transmission line fitting line. According to the direction angle of the transmission line, the dynamically predicted UAV position and the preset flight speed, the flight speed on each coordinate axis of the UAV coordinate system is controlled.

[0008] As an implementation method, the flight speed value on the X-axis of the drone coordinate system is: The preset flight speed value and the safety distance value between the UAV and the reference flight path are weighted summed with the corresponding adjustment gains.

[0009] As an implementation method, the flight speed value on the Y axis of the drone coordinate system is: ; in, is the flight speed value on the Y axis of the drone coordinate system; is the lateral deviation of the transmission line on the image; is the left and right translation component of the direction angle; 、 、 、 They are the proportional gain reference, deviation adaptive coefficient, direction angle compensation coefficient and differential gain on the Y axis respectively.

[0010] As an implementation method, the flight speed value on the Z axis of the drone coordinate system is: ; in, The flight speed value on the Z axis of the drone coordinate system; is the longitudinal deviation of the transmission line on the image; is the left and right translation component of the direction angle; 、 、 、 They are the proportional gain reference, deviation adaptive coefficient, direction angle compensation coefficient and differential gain on the Z axis respectively.

[0011] As an implementation method, the process of determining the coordinate center position of the detection frame of the transmission line is as follows: Decompose the infrared image and visible light image of the transmission line into corresponding base layer image and detail layer image; fusing the base layer images of the infrared image and the visible light image of the power transmission line using base layer weights to obtain a base layer fused image; The detail layer weights are used to fuse the detail layer images of the infrared image and the visible light image of the transmission line to obtain a detail layer fused image; Directly add the base layer fusion image and the detail layer fusion image to obtain a fusion image; The fused image is processed using a pre-trained target recognition model to obtain the center position of the detection frame coordinates of the transmission line and its confidence.

[0012] As an implementation method, the base layer weight is determined by lighting conditions.

[0013] As an implementation method, the detail layer images of the infrared image and the visible light image of the transmission line are fused based on the detail layer weight and weighted average strategy. The detail layer weights of the infrared image and the visible light image of the transmission line are respectively:

[0014]

[0015]

[0016] in, i =1,2; is the intermediate parameter; is the mean filter; is the median filter; is the detail layer weight of the infrared image of the transmission line; is the detail layer weight of the visible light image of the transmission line; are the pixel coordinates in the image.

[0017] A second aspect of the present invention provides a transmission line UAV line-mimicking flight navigation and control system.

[0018] A transmission line UAV flight navigation control system, comprising: A reference flight path planning module, which is used to pre-plan a reference flight path based on the coordinates of the transmission line to be simulated by the UAV; The UAV position dynamic prediction module is used to determine the center position of the transmission line detection frame coordinate based on the infrared image and visible light image of the transmission line on the reference flight path, and then dynamically predict the UAV position based on the change trend of the transmission line detection frame coordinate center position and the slope change trend of the transmission line fitting line; The transmission line direction angle calculation module is used to extract the temperature gradient field characteristics within the detection frame of the transmission line, calculate the main direction angle of the transmission line based on this, and then calculate the direction angle of the transmission line by combining it with the slope of the transmission line fitting line; The flight speed control module is used to control the flight speed on each coordinate axis of the UAV coordinate system according to the direction angle of the transmission line, the dynamically predicted UAV position and the preset flight speed.

[0019] A third aspect of the present invention provides a computer-readable storage medium.

[0020] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned transmission line UAV line-mimicking flight navigation control method.

[0021] A fourth aspect of the present invention provides a drone.

[0022] A drone comprises a drone body, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the above-described transmission line drone flight navigation control method are implemented.

[0023] Compared with the prior art, the present invention has the following beneficial effects: The present invention innovatively proposes a coupling technology of image fusion and transmission line UAV line-mimicking flight navigation control, which solves the current problem that transmission lines are easily out of the camera shooting range during line-mimicking flight and the recognition effect of planning multiple routes is poor. On the basis of pre-planned benchmark flight path, the infrared image and visible light image of the transmission line are fused to determine the center position of the detection frame coordinate of the transmission line and dynamically predict the position of the UAV, and the direction angle of the transmission line is combined to control the flight speed of each coordinate axis of the UAV coordinate system, realizing autonomous control of the UAV to fly and shoot, and improving the image recognition effect during the UAV line-mimicking flight.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a flow chart of a transmission line UAV line-mimicking flight navigation control method according to an embodiment of the present invention; Figure 2This is a diagram of a transmission line UAV flight navigation control process according to an embodiment of the present invention; Figure 3 The present invention is a schematic structural diagram of a transmission line UAV line-mimicking flight navigation control system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0030] Currently, there are two main methods for power line tracking in the market: pure visual flight and LiDAR-based flight. Pure visual flight relies solely on the drone's visible light camera, utilizing zoom and acquired video frame data, combined with visible light image recognition, to determine the relative position of the conductor and the drone, ensuring that the conductor is centered in the captured image and controlling the drone's flight along the conductor. However, this solution has significant drawbacks: the conductors are mostly silver-gray or black, which matches the ground color in the wild. They are also thin, making them difficult to identify, and are particularly unsuitable at night. While LiDAR-based solutions can address the difficulty of conductor identification, they are expensive, require a certain payload capacity for drones, are difficult for small and light drones to handle, and significantly reduce flight time, resulting in poor economic benefits.

[0031] Example 1 Combine Figure 1 and Figure 2 The embodiment of the present invention provides a transmission line UAV flight navigation control method, which includes: S101: Pre-plan a reference flight path based on the coordinates of the transmission line to be simulated by the UAV.

[0032] Pre-planning a baseline flight path ensures the drone won't stray off the target or collide with obstacles. In sufficient lighting, binocular ranging can be used to measure distance and calculate spatial coordinates. In less-than-ideal conditions, manual calibration of the flight path's starting and ending positions is necessary.

[0033] Under adequate lighting conditions, the drone photographs and identifies the wire end attachment point. The drone then retreats a distance of L meters and photographs and identifies the wire end attachment point again. The captured image coordinates are combined with the identification results to establish three-dimensional spatial coordinates. Finally, the other end attachment point is photographed and identified to determine the spatial coordinates of both ends of the wire. Finally, a certain displacement is added to the spatial coordinates of the wire end attachment point to determine the drone's flight start and end positions.

[0034] In the case of insufficient lighting, use manual positioning to manually set the drone's flight head and tail positions.

[0035] Assume that the wire end hanging points are A(x1,y1,z1) and B(x2,y2,z2); Assume that the Z axis of the three-dimensional coordinate system is vertically upward and positive, and the X axis and Y axis are coordinate axes perpendicular to each other and parallel to the ground.

[0036] The parabola equation of the local coordinate system of the conductor is: , ; in, is the horizontal spacing, ; is the height difference, ; is sag; Global coordinate transformation: local coordinates ->Global coordinates , the global coordinate formula is:

[0037] It should be noted that the local coordinate system is a temporary coordinate system constructed at the hanging point A, whose XZ plane happens to contain the wire path. The global coordinate system is a three-dimensional coordinate system constructed based on the earth.

[0038] S102: On the reference flight path, based on the infrared image and visible light image of the transmission line, determine the center position of the detection frame coordinate of the transmission line, and then dynamically predict the position of the UAV based on the change trend of the center position of the detection frame coordinate of the transmission line and the change trend of the slope of the transmission line fitting line.

[0039] When flying along a line, the image is taken of only a few tenths of the end of the wire. It can be approximately considered that the wire fits the branch line in the image. Using the straight line equation Perform fitting and take the midpoint of the fitting line segment As a reference point for comparison with the actual flight position of the drone, the change in k (slope) value is used to determine the direction of the drone's flight.

[0040] In practice, infrared and visible light images of power transmission lines are captured directly by a dual-light camera mounted on a drone. Video frame data is directly captured via video streams. In infrared images, conductors appear as continuous bright lines due to current heating, while the ambient background (sky / vegetation) appears dark due to its cooler temperature. Visible light images are used for later recognition and spatial coordinate calculation.

[0041] After acquiring the infrared and visible light images of the transmission lines, preprocessing operations are performed, including dynamic temperature normalization, normalization calculation, and mask-guided CLAHE enhancement.

[0042] The process of dynamic temperature normalization is as follows: During the flight, the first frame is processed: the Otsu threshold method is used to preliminarily segment the conductor area and obtain the temperature range of the transmission line.

[0043] Dynamic interception formula: , . T wire This is the pixel temperature set in the transmission line area; the temperature range is updated every 5 frames to adapt to environmental changes.

[0044] Normalized calculation: , the transmission line temperature is linearly mapped to the [0,1] interval, and the background area is compressed to a low value.

[0045] The process of mask-guided CLAHE enhancement is: Generate binary masks based on transmission line areas for infrared and visible light images , applying restricted contrast adaptive histogram equalization only within the masked region.

[0046] In the specific implementation process, the process of determining the coordinate center position of the detection frame of the transmission line is as follows: Step a: Decompose the infrared image and visible light image of the transmission line into corresponding base layer images and detail layer images.

[0047] The pre-processed infrared image and visible light images , decomposed into base layer and detail layer using mean filter.

[0048]

[0049]

[0050] 、 is the base layer image, is the calculation formula of the mean filter. After obtaining the base layer, the detail layer image is obtained by combining the original image and the base layer image.

[0051]

[0052]

[0053] 、 Detail layer image.

[0054] Step b: using the base layer weights to fuse the base layer images of the infrared image and the visible light image of the power transmission line to obtain a base layer fused image.

[0055] ; Wherein, the base layer weight is related to the lighting conditions;

[0056] Among them, clamp() is a universal function, its meaning is: , It is a weight, which is used to determine the proportion of infrared and visible light base layers later; is the average value of the lighting conditions, and They are respectively the conditions of sufficient light and darkness.

[0057] Step c: Using detail layer weights to fuse the detail layer images of the infrared image and the visible light image of the transmission line to obtain a detail layer fused image.

[0058] Among them, the detail layer images of the infrared image and the visible light image of the transmission line are fused based on the detail layer weight and weighted average strategy. The detail layer weights of the infrared image and the visible light image of the transmission line are:

[0059]

[0060]

[0061] in, i =1,2; is the intermediate parameter; is the mean filter; is the median filter; is the detail layer weight of the infrared image of the transmission line; is the detail layer weight of the visible light image of the transmission line; are the pixel coordinates in the image.

[0062] ; Step d: directly add the base layer fusion image and the detail layer fusion image to obtain a fusion image; .

[0063] Step e: Use the pre-trained target recognition model to process the fused image to obtain the detection frame coordinate center position of the transmission line and its confidence.

[0064] For example, YOLOv8n is used for lightweight recognition to output the center position of the wire detection frame coordinates. and confidence.

[0065] S103: extracting the temperature gradient field characteristics within the detection frame of the transmission line, calculating the main direction angle of the transmission line based on the temperature gradient field characteristics, and then calculating the direction angle of the transmission line in combination with the slope of the transmission line fitting line.

[0066] Calculate the temperature gradient field within the detection frame: , .

[0067] Calculation of the main direction angle of the conductor: ; The temperature is highest at the center of the conductor and decreases radially toward the edge.

[0068] S104: Control the flight speed of each coordinate axis of the UAV coordinate system according to the direction angle of the transmission line, the dynamically predicted UAV position and the preset flight speed.

[0069] Table 1 UAV coordinate system definition:

[0070] Identify the average of the wire direction angles of the infrared image and the fused image: , get a real-time changing wire direction angle .

[0071] The left and right translation components of the direction angle are: ; The up and down translation components of the direction angle are: ; In step S104, the flight speed value on the X-axis of the drone coordinate system is: The preset flight speed value and the safety distance value between the UAV and the reference flight path are weighted summed with the corresponding adjustment gains.

[0072] For example, X-axis control (moving forward and backward): ; in, is the preset flight speed; The safe distance between the drone and the planned reference route; 、 Adjustment gain for speed and distance.

[0073] Y-axis control (left and right translation): The flight speed value on the Y-axis of the drone coordinate system is: ; in, is the flight speed value on the Y axis of the drone coordinate system; is the lateral deviation of the transmission line on the image; is the left and right translation component of the direction angle; 、 、 、 They are the proportional gain reference, deviation adaptive coefficient, direction angle compensation coefficient and differential gain on the Y axis respectively.

[0074] Z-axis control (up and down translation): The flight speed value on the Z axis of the drone coordinate system is: ; in, The flight speed value on the Z axis of the drone coordinate system; is the longitudinal deviation of the transmission line on the image, that is: ; is the left and right translation component of the direction angle; 、 、 、 They are the proportional gain reference, deviation adaptive coefficient, direction angle compensation coefficient and differential gain on the Z axis respectively.

[0075] In this embodiment, the X-axis control relies on real-time positioning of the preset path and position to ensure a safe distance between the drone and the tower; the YZ-axis control of the drone uses the image center deviation as the main control variable, and the wire direction angle provides auxiliary correction.

[0076] Example 2 like Figure 3 As shown, this embodiment provides a transmission line UAV line-mimicking flight navigation control system, including: A reference flight path planning module 301 is used to pre-plan a reference flight path based on the transmission line coordinates of the UAV to be simulated; The UAV position dynamic prediction module 302 is used to determine the center position of the transmission line detection frame coordinates based on the infrared image and visible light image of the transmission line on the reference flight path, and then dynamically predict the UAV position based on the change trend of the transmission line detection frame coordinate center position and the change trend of the slope of the transmission line fitting line; The transmission line direction angle calculation module 303 is used to extract the temperature gradient field characteristics within the detection frame of the transmission line, calculate the main direction angle of the transmission line based on the temperature gradient field characteristics, and then calculate the direction angle of the transmission line based on the slope of the transmission line fitting line; The flight speed control module 304 is used to control the flight speed of each coordinate axis of the UAV coordinate system according to the direction angle of the transmission line, the dynamically predicted UAV position and the preset flight speed.

[0077] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0078] Example 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the transmission line UAV line-mimicking flight navigation control method as described above are implemented.

[0079] Example 4 An embodiment of the present invention provides a drone, including a drone body, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the transmission line drone flight navigation control method described above are implemented.

[0080] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams and the combination of processes and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A transmission line UAV flight navigation control method, characterized in that: include: Pre-plan the reference flight path based on the coordinates of the transmission line to be simulated by the UAV; On the reference flight path, the center of the transmission line's detection frame coordinates is determined based on the infrared and visible light images of the transmission line. The drone's position is then dynamically predicted based on the changing trend of the center of the transmission line's detection frame coordinates and the slope of the transmission line's fitted line. The temperature gradient field characteristics within the detection frame of the transmission line are extracted, and the main direction angle of the transmission line is calculated based on this. Then, the direction angle of the transmission line is calculated by combining the slope of the transmission line fitting line. According to the direction angle of the transmission line, the dynamically predicted UAV position and the preset flight speed, the flight speed on each coordinate axis of the UAV coordinate system is controlled.

2. The transmission line UAV flight navigation control method according to claim 1, characterized in that: The flight speed value on the X-axis of the drone coordinate system is: The preset flight speed value and the safety distance value between the UAV and the reference flight path are weighted summed with the corresponding adjustment gains.

3. The transmission line UAV flight navigation control method according to claim 1, characterized in that: The flight speed value on the Y axis of the drone coordinate system is: ； in, is the flight speed value on the Y axis of the drone coordinate system; is the lateral deviation of the transmission line on the image; is the left and right translation component of the direction angle; 、 、 、 They are the proportional gain reference, deviation adaptive coefficient, direction angle compensation coefficient and differential gain on the Y axis respectively.

4. The transmission line UAV flight navigation control method according to claim 1, characterized in that: The flight speed value on the Z axis of the drone coordinate system is: ； in, The flight speed value on the Z axis of the drone coordinate system; is the longitudinal deviation of the transmission line on the image; is the left and right translation component of the direction angle; 、 、 、 They are the proportional gain reference, deviation adaptive coefficient, direction angle compensation coefficient and differential gain on the Z axis respectively.

5. The transmission line UAV flight navigation control method according to claim 1, characterized in that: The process of determining the center position of the detection frame coordinate of the transmission line is as follows: Decompose the infrared image and visible light image of the transmission line into corresponding base layer image and detail layer image; fusing the base layer images of the infrared image and the visible light image of the power transmission line using base layer weights to obtain a base layer fused image; The detail layer weights are used to fuse the detail layer images of the infrared image and the visible light image of the transmission line to obtain a detail layer fused image; Directly add the base layer fusion image and the detail layer fusion image to obtain a fusion image; The fused image is processed using a pre-trained target recognition model to obtain the center position of the detection frame coordinates of the transmission line and its confidence.

6. The transmission line UAV flight navigation control method according to claim 5, characterized in that: The base layer weight is determined by lighting conditions.

7. The transmission line UAV flight navigation control method according to claim 5, characterized in that: The detail layer images of the infrared image and visible light image of the transmission line are fused based on the detail layer weight and weighted average strategy. The detail layer weights of the infrared image and visible light image of the transmission line are: in, i =1,2; is the intermediate parameter; is the mean filter; is the median filter; is the detail layer weight of the infrared image of the transmission line; is the detail layer weight of the visible light image of the transmission line; are the pixel coordinates in the image.

8. A transmission line UAV flight navigation control system, characterized in that: include: A reference flight path planning module, which is used to pre-plan a reference flight path based on the coordinates of the transmission line to be simulated by the UAV; The UAV position dynamic prediction module is used to determine the center position of the transmission line detection frame coordinate based on the infrared image and visible light image of the transmission line on the reference flight path, and then dynamically predict the UAV position based on the change trend of the transmission line detection frame coordinate center position and the slope change trend of the transmission line fitting line; The transmission line direction angle calculation module is used to extract the temperature gradient field characteristics within the detection frame of the transmission line, calculate the main direction angle of the transmission line based on this, and then calculate the direction angle of the transmission line by combining it with the slope of the transmission line fitting line; The flight speed control module is used to control the flight speed on each coordinate axis of the UAV coordinate system according to the direction angle of the transmission line, the dynamically predicted UAV position and the preset flight speed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the transmission line UAV line-mimicking flight navigation control method as described in any one of claims 1 to 7 are implemented.

10. A drone comprising a drone body, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the transmission line UAV line-mimicking flight navigation control method according to any one of claims 1 to 7 are implemented.

Images