A distribution network line drone intelligent inspection method

Abstract

The present invention relates to the technical field of intelligent operation and maintenance of power lines, and specifically to a method for intelligent inspection by drones of distribution network lines, comprising the following steps: a drone loads an inspection route and performs autonomous inspection by the drone. During the inspection process, the drone video stream is pushed to a streaming media server, an AI box is loaded, image recognition is performed on the video stream, the position of the shooting target is calculated, the offset angle result is fed back to the drone control end, the drone lens is controlled to rotate, the shooting angle is adjusted, the shooting target is centered, and shooting is performed. The present invention can make the shooting target subject more centered in the photo, reduce the waste rate of shooting, improve the efficiency of autonomous inspection, provide more reliable samples for later image processing, and improve the use value of the picture.

Description

Technical Field

[0001] The invention relates to a distribution network line unmanned aerial vehicle intelligent inspection method, belonging to the technical field of intelligent operation and maintenance of power lines. Background Art

[0002] With the rapid development of drone technology, drone inspections have been widely used and have gradually replaced conventional manual inspections. The line inspection efficiency of manually operated drones is more than 40 times higher than that of traditional manual inspections, effectively reducing labor intensity and improving line maintenance efficiency. For example:

[0003] Chinese patent document CN112731960A discloses a UAV remote transmission line intelligent inspection system and method, which consists of a remote control center, a UAV nest and a UAV. The remote control center sends the formulated inspection task GPS inspection route to the flight control module, and the UAV makes preparations before takeoff; reads the battery status of the UAV, and formulates a landing point and charging plan according to the GPS inspection route; the UAV takes off remotely and autonomously according to the inspection task; the image acquisition module uploads the collected images to the cloud through the wireless communication module, and performs intelligent identification and analysis of inspection image defects; the UAV flight control module generates an allowed flight route, and the remote control center sends a termination command to terminate the mission, and performs landing and return operations. After the inspection, the UAV flies to the target nest.

[0004] Chinese patent document CN106873627B discloses a multi-rotor UAV and method for automatically inspecting power transmission lines. The UAV includes a machine vision unit, a flight control unit, a line shooting unit and a battery power supply unit; the machine vision unit and the line shooting unit are both connected to the flight control unit; the machine vision unit captures image information of the power transmission line in real time, and calculates the position and direction of each power transmission line in the image and the relative distance of each power transmission line in the image based on the captured image information, and then sends this data to the flight control module to stabilize the position, direction and height of the UAV relative to the transmission line, so that the UAV can fly parallel to the line; the line shooting unit is used to obtain image information of the transmission line when the UAV flies parallel to the line.

[0005] In summary, the GPS module of the drone in the prior art is limited by external environmental factors, such as strong winds, air humidity, etc., which leads to slight positioning deviations. The position of the photographed target object in the photo will have some deviations, and in severe cases, the target will not be fully displayed, resulting in wasted film, time-consuming and labor-intensive. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention discloses a distribution network line intelligent inspection method using a drone.

[0007] The detailed technical scheme of the present invention is as follows:

[0008] A distribution network line drone intelligent inspection method, characterized in that the method comprises the following steps:

[0009] The drone autonomously flies according to the inspection route to inspect the inspection target, wherein the inspection route includes the location information of the target waypoint, for example, the location information of the tower to be inspected; and also includes the corresponding gimbal angle information when the drone reaches the target waypoint; the drone uses the lens to collect video of the inspection object, and pushes the video stream to the streaming media server through the drone control terminal;

[0010] According to the target waypoint ID, the inspection recognition model in the AI ​​box is retrieved and loaded into the drone control terminal. Target waypoints at different locations have independent IDs. Different power transmission objects in the target waypoint can also be distinguished by ID according to the application scenario. For example, the target waypoint includes a pole tower, and different IDs are set according to the location of the pole tower or the sensor. According to the target waypoint ID, the inspection recognition model is retrieved from the AI ​​box. The AI ​​box can write the recognition models of all working scenarios in advance.

[0011] S1. The drone performs autonomous inspection according to the inspection route: when the drone arrives at the target waypoint, the drone control terminal sends the corresponding target waypoint ID to the AI ​​box, and the AI ​​box automatically loads the corresponding inspection recognition model according to the target waypoint ID. The AI ​​boxes used in the present invention are all intelligent modules that can be purchased in this field, and the inspection recognition modules loaded therein are recognition programs preset in advance according to different application scenarios. Therefore, the inspection recognition modules loaded in the AI ​​box are not the technical content to be protected by the present invention;

[0012] S2, the video stream collected by the drone in real time is pushed to the streaming media server through the RTMP protocol by the drone control terminal;

[0013] S3, the streaming media server inputs the video data loaded with the target waypoint ID into the AI ​​box, and the inspection recognition model is used to select the corresponding target waypoint to form an identification frame, and calculate the distance from the center point of the identification frame to the center point of the video, and then obtain the offset angle between the center point of the identification frame and the center point of the video;

[0014] S4. The AI ​​box returns the obtained offset angle to the drone control end according to the target waypoint ID to further control the adjustment of the drone lens to pitch and yaw. The pitch is rotation around the X-axis, also called the pitch angle; the yaw is rotation around the Y-axis, also called the yaw angle; the AI ​​box also sends the recognition result to the drone control end according to the target waypoint ID. The recognition result is correspondingly identified by the built-in inspection recognition model.

[0015] Preferably, according to the present invention, in step S3, a minimum threshold distance from the center point of the recognition frame to the center point of the video is pre-set in the AI ​​box;

[0016] The streaming media server updates the video screen in real time after the lens rotates, and pushes the video to the AI ​​box every 500ms to calculate the distance from the recognition frame to the center point of the video:

[0017] When the distance is less than or equal to the minimum threshold, it is considered that the identified target is already in the center of the image, and the AI ​​box sends a photo command to the drone control terminal, so that the drone starts formal shooting, and adds the target waypoint ID to the formally shot video data and stores it in the streaming media server and / or the drone control terminal;

[0018] When the distance is greater than the minimum threshold, it is considered that the identified target deviates from the center of the image, and the AI ​​box sends the obtained offset angle to the drone control end to further control the drone lens to adjust the pitch and yaw until the formal shooting video data corresponding to the current target waypoint is collected.

[0019] Preferably, according to the present invention, steps S1-S3 are repeated until all target waypoints on the inspection route are photographed.

[0020] Preferably, the drone control terminal and the AI ​​box communicate via the MavLink protocol. The MavLink protocol used in the present invention is a communication protocol for small unmanned vehicles, which has extremely powerful functions, strong scalability, convenient and flexible expansion, and reliable protocol design.

[0021] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0022] (1) The present invention can improve the efficiency of autonomous inspection of power grid lines, make the shooting target position more centered, and reduce the impact of external factors such as GPS positioning longitude and latitude deviation, barometer measurement altitude deviation, and drone shaking caused by strong wind, which leads to the deviation of drone shooting targets.

[0023] (2) The present invention can reduce the waste rate of film shooting and improve the use value of pictures. Pictures with clear targets are convenient for later big data analysis, model training, data sorting, etc.

[0024] (3) The present invention uses an AI box to realize multi-model recognition to achieve but not limited to distribution network power inspection, transmission power inspection and substation power inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the process of the method of the present invention;

[0026] Figure 2 The drone takes images before correcting the deviation of the hanging point at the end of the horizontal tower;

[0027] Figure 3 The drone takes images after performing deviation correction on the hanging point at the end of the horizontal tower.

[0028] Figure 4 The drone takes images before correcting the deviation of the hanging point at the end of the cross-string wire;

[0029] Figure 5 The drone takes images after correcting the deviation of the hanging point at the end of the cross-string wire;

[0030] Figure 6 It is the image taken by the drone before the weight correction is performed;

[0031] Figure 7 The image is taken after the drone uses a heavy hammer to correct the deviation. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the embodiments and the accompanying drawings, but is not limited thereto.

[0033] Embodiment 1,

[0034] This embodiment takes the specific steps of using a drone to perform refined intelligent inspection of a power grid line as an example. After the pilot arrives at the target area, he turns on the drone, turns on the drone control terminal, and checks whether the drone is in good working condition. The drone working condition includes whether the WiFi signal, GPS signal, flight sensor, and drone obstacle avoidance system are normal.

[0035] like Figure 1 shown.

[0036] A method for intelligent inspection of distribution network lines by drones, the method comprising the following steps:

[0037] The drone autonomously flies according to the inspection route to inspect the inspection target, wherein the inspection route includes the location information of the target waypoint, for example, the location information of the tower to be inspected; and also includes the corresponding gimbal angle information when the drone reaches the target waypoint; the drone uses the lens to collect video of the inspection object, and pushes the video stream to the streaming media server through the drone control terminal;

[0038] According to the target waypoint ID, the inspection recognition model in the AI ​​box is retrieved and loaded into the drone control terminal. Target waypoints at different locations have independent IDs. Different power transmission objects in the target waypoint can also be distinguished by ID according to the application scenario. For example, the target waypoint includes a pole tower, and different IDs are set according to the location of the pole tower or the sensor. According to the target waypoint ID, the inspection recognition model is retrieved from the AI ​​box. The AI ​​box can write the recognition models of all working scenarios in advance.

[0039] The drone control terminal establishes a connection with the AI ​​box through a socket, and uploads the drone status information and current position coordinate information to the AI ​​box through the Socket heartbeat. The pilot selects the inspection route. Each waypoint in the inspection route should include the inspection route name, coordinate information, tower information, inspection point ID, waypoint action, flight speed and other information. The minimum parameter waypoint information that meets the basic function of adjusting the angle is as shown in Table 1:

[0040] Table 1:

[0041] name Waypoint Labels Waypoint information Coordinate information coordinates 117.31189234044592，36.70498750695165，158.4859 Head angle Heading -8(degree) Flight speed Speed 10(m / s) Shooting area targetShoot 1001 PTZ Angle GimbalPitch 41 (degrees) Tower information Info 500KV Hong Kong-Kowloon Line 10001 Tower

[0042] Inspection point ID, that is, the target waypoint ID information is mapped to different IDs according to different inspection locations to facilitate model training, as shown in Table 2.

[0043] Table 2

[0044] Serial number Inspection ID Inspection Name 1 10001 I string cross arm end hanging point 2 10002 I string insulator string 3 10003 I String wire end hanging point 4 10004 V string left cross arm end hanging point 5 10005 V string left insulator string 6 10006 V string conductor end hanging point 7 10007 V string right cross arm end hanging point 8 10008 Tension string cross arm end hanging point 9 10009 Tension string 10 10010 Tension conductor end hanging point … … …

[0045] S1. The drone performs autonomous inspection according to the inspection route: when the drone arrives at the target waypoint, the drone control terminal sends the corresponding target waypoint ID to the AI ​​box, and the AI ​​box automatically loads the corresponding inspection recognition model according to the target waypoint ID. The AI ​​boxes used in the present invention are all intelligent modules that can be purchased in this field, and the inspection recognition modules loaded therein are recognition programs preset in advance according to different application scenarios. Therefore, the inspection recognition modules loaded in the AI ​​box are not the technical content to be protected by the present invention;

[0046] S2, the video stream collected by the drone in real time is pushed to the streaming media server through the RTMP protocol by the drone control terminal;

[0047] S3, the streaming media server inputs the video data loaded with the target waypoint ID into the AI ​​box, and the inspection recognition model is used to select the corresponding target waypoint to form an identification frame, and calculate the distance from the center point of the identification frame to the center point of the video, and then obtain the offset angle between the center point of the identification frame and the center point of the video;

[0048] In this embodiment, taking the DJI Mavic 2 Advanced Edition drone as an example, the horizontal field of view of the drone lens is horizontal_angle=71.56214772f, the vertical field of view is vertical_angle=56.7860388f, the drone video resolution is 1920*1080, viewWidth=1920, viewHeight=1080, the streaming resolution is 1920*1080, and the plane two-dimensional coordinate system is established with the coordinate origin (0, 0) at the center of the screen;

[0049] S4. The AI ​​box returns the obtained offset angle to the drone control end according to the target waypoint ID to further control the adjustment of the drone lens to pitch and yaw. The pitch is rotation around the X-axis, also called the pitch angle; the yaw is rotation around the Y-axis, also called the yaw angle; the AI ​​box also sends the recognition result to the drone control end according to the target waypoint ID. The recognition result is correspondingly identified by the built-in inspection recognition model.

[0050] In this embodiment, in the AI ​​box: the center coordinates of the framed target are (300, -200), then the distance from the center coordinates of the framed target to the center of the image (0, 0) is calculated, x=300, y=-200, then the horizontal angle of the drone movement is degreeHor=x / viewWidth*horizontal_angle, the vertical angle of the drone lens movement is degreeVer=y / viewHeight*vertical_angle, after calculating the horizontal angle and vertical angle of the lens deflection, they are transmitted to the drone control end through the mavLink protocol, and after receiving the drone control end, it makes an adjustment in the opposite direction of the angle, the drone gimbal lens yaw adjustment angle is -degreeHor, and the drone gimbal lens pitch adjustment angle is -degreeVer.

[0051] Embodiment 2,

[0052] As described in the method of Example 1, in step S3, a minimum threshold distance from the center point of the recognition box to the center point of the video is pre-set in the AI ​​box;

[0053] The streaming media server updates the video screen in real time after the lens rotates, and pushes the video to the AI ​​box every 500ms to calculate the distance from the recognition frame to the center point of the video:

[0054] When the distance is less than or equal to the minimum threshold, it is considered that the identified target is already in the center of the image, and the AI ​​box sends a photo command to the drone control terminal, so that the drone starts formal shooting, and adds the target waypoint ID to the formally shot video data and stores it in the streaming media server and / or the drone control terminal;

[0055] When the distance is greater than the minimum threshold, it is considered that the identified target deviates from the center of the image, and the AI ​​box sends the obtained offset angle to the drone control end to further control the drone lens to adjust the pitch and yaw until the formal shooting video data corresponding to the current target waypoint is collected.

[0056] After adjustment, the AI ​​box determines whether the target is centered. After confirming that the target is centered, the AI ​​box stops sending adjustment commands and sends a formal shooting command. When it is confirmed that the target is not centered, it continues to adjust the angle. After 5 unsuccessful adjustments, the AI ​​box stops sending angle correction commands and gives up correcting the angle of the current target video data acquisition.

[0057] Repeat steps S1-S3 until all the target waypoints on the inspection route are photographed. The drone goes to the next waypoint and continues to identify, correct deviations, and take photos until the autonomous inspection route is completed.

[0058] The drone control terminal and the AI ​​box communicate via the MavLink protocol.

[0059] like Figure 2 , 3 Due to the difference in GPS, the target in the picture of the horizontal string tower end is biased to the right. After the lens is corrected by the method of the present invention, the position of the horizontal string tower end is centered.

[0060] like Figure 4 , 5 Due to the error in the height caused by the barometer, the target object in the picture of the horizontal wire end is biased upward. After S5, the lens is corrected to make the position of the horizontal wire end in the picture centered;

[0061] like Figure 6 , 7 Due to the influence of wind factors, the position of the drone moved slightly when taking pictures after arriving, and the target object in the picture of the heavy hammer was on the left. The method of the present invention was used to adjust the heavy hammer so that it was centered before taking pictures.

[0062] In summary, it can be seen that the present invention uses drones for autonomous inspection flights. Since DJI drones currently account for a huge proportion in the power field, secondary development and creation with the help of DJI MSDK can give play to the advantages of drone inspections and reduce development time. DJI MSDK supports RTMP protocol streaming, and drone videos can be pushed to streaming media servers for analysis and identification. The streaming effect is stable and the delay can be reduced to within 200ms. The algorithm model is deployed in the AI ​​box to analyze, identify and calculate the accessed RTMP video stream, and the recognition results are fed back to the drone control end to make micro-adjustments to the drone gimbal angle, so that the shooting target is placed in the center of the image, reducing external factors such as wind, temperature and humidity, poor GPS signals, and inaccurate barometer height measurement, causing the drone position to shift, etc., which will cause deviations in the position of the target in the captured picture. Thereby improving the efficiency of inspection shooting and reducing the scrap rate. At the same time, taking more accurate pictures is helpful for subsequent image recognition training, etc., and improving the professional value of pictures.

Claims

1. A distribution network line drone intelligent inspection method, characterized in that: The method comprises the following steps: S1. The drone performs autonomous inspection according to the inspection route: when the drone reaches the target waypoint, the drone control terminal sends the corresponding target waypoint ID to the AI ​​box, and the AI ​​box automatically loads the corresponding inspection recognition model according to the target waypoint ID; S2, the video stream collected by the drone in real time is pushed to the streaming media server by the drone control terminal; S3, the streaming media server inputs the video data loaded with the target waypoint ID into the AI ​​box, and the inspection recognition model is used to select the corresponding target waypoint to form an identification frame, and calculate the distance from the center point of the identification frame to the center point of the video, and then obtain the offset angle between the center point of the identification frame and the center point of the video; S4. The AI ​​box returns the obtained offset angle to the drone control end according to the target waypoint ID to further control the adjustment of the drone lens to pitch and yaw; the AI ​​box also sends the recognition result to the drone control end according to the target waypoint ID.

2. According to claim 1, a distribution network line drone intelligent inspection method is characterized in that: In step S3, a minimum threshold distance from the center point of the recognition frame to the center point of the video is pre-set in the AI ​​box; The streaming media server updates the video image in real time after the lens rotates, and pushes it to the AI ​​box to calculate the distance from the recognition frame to the center point of the video: When the distance is less than or equal to the minimum threshold, the AI ​​box sends a photo command to the drone control terminal, so that the drone starts formally shooting, and adds the target waypoint ID to the formally shot video data and stores it in the streaming media server and / or the drone control terminal; When the distance is greater than the minimum threshold, the AI ​​box sends the obtained offset angle to the drone control end to further control the drone lens to adjust the pitch and yaw until the formal shooting video data corresponding to the current target waypoint is collected.

3. According to claim 1, a distribution network line drone intelligent inspection method is characterized in that: Repeat steps S1-S3 until all target waypoints on the inspection route are photographed.

4. According to claim 1, a distribution network line drone intelligent inspection method is characterized in that: The drone control terminal and the AI ​​box communicate via the MavLink protocol.

Images