Operation area surrounding route planning method and system based on automatic search

Abstract

The invention discloses an operation area surrounding route planning method and system based on automatic search, and the method comprises the steps: carrying out the large-range area search under the combination of different pitch angles and yaw angles through controlling the yaw angle and the pan-tilt angle of an unmanned aerial vehicle, extracting a video image key frame, carrying out the target recognition, correcting the actual position of an operation site, and carrying out the large-range area search through the control of the yaw angle and the pan-tilt angle of the unmanned aerial vehicle. The method overcomes the problem that the unmanned aerial vehicle is difficult to supervise autonomous flight due to relatively dispersed operation sites, complex environment, uncertainty, inaccurate operation target position and the like, is especially suitable for unmanned aerial vehicle supervision scenes of electric power operation construction, can ensure that the unmanned aerial vehicle arrives at a real operation site position and plans surrounding flight according to the position, and improves the unmanned aerial vehicle supervision efficiency. The unmanned aerial vehicle supervising autonomous flight of the working site in a scene with an inaccurate working target position is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicle route planning, and particularly relates to an environment self-adaptive operation area surrounding route planning method and system based on automatic search. BACKGROUND

[0002] At present, the unmanned aerial vehicle technology is gradually popularized and integrated into various industries in life, especially in the power industry. For the operation and maintenance of the built power network, unmanned aerial vehicle inspection has become one of the mainstream auxiliary means. Before actual operation, the staff will plan a fixed flight path (usually a point cloud model is established first, and then a flight point is set based on the point cloud model, and a flight path is planned) according to the preset position and direction of the power transmission line, power distribution tower and other equipment. The unmanned aerial vehicle flies along the path to check possible faults of the equipment in long-term operation. Since the facility position and line direction of the built power network are fixed, the flight path and detection focus of the unmanned aerial vehicle are stable for a long time.

[0003] With the popularization of unmanned aerial vehicle technology, the unmanned aerial vehicle for power operation construction supervision is gradually put on the agenda. The power operation construction supervision refers to the supervision and inspection of whether there is violation, illegal operation, and operation safety hazards in the construction site. The location of the power operation construction often has no fixed position, and it is impossible to perform line sampling and establish a laser point cloud in advance. Moreover, even if a laser point cloud is established, since the power construction operation site is relatively dispersed, the environment is complex and has uncertainty, the facility state and operation position in the same construction area are in dynamic change, and the error between the operation plan position point and the actual site operation often occurs, that is, the operation target position is inaccurate, so that the traditional unmanned aerial vehicle inspection mode of power facilities cannot completely meet the unmanned aerial vehicle supervision autonomous flight demand of the power operation construction scene. SUMMARY

[0004] The purpose of the present application is to provide an operation area surrounding route planning method based on automatic search, which can solve the problems of relatively dispersed power operation site, complex and uncertain environment, inaccurate operation target position, and dynamic change of on-site operation environment.

[0005] Another purpose of the present application is to provide an operation area surrounding route planning system based on automatic search.

[0006] An operation area surrounding route planning method based on automatic search, the method comprises: S1. controlling an unmanned aerial vehicle to fly to a preset target point and hover above the target point; S2. adjusting a gimbal to a first preset depression angle, and extracting any key frame in the first preset depression angle state; S3. performing image recognition on the key frame to determine whether a specified target exists; If the specified target is identified, step S5 is performed, otherwise step S4 is performed; S4. Adjust the gimbal to a second preset depression angle, and control the UAV to make adjustment in a preset yaw angle interval along the circumference at the hovering position; Extract any key frame at each yaw direction position; Perform image recognition on the extracted key frame to determine whether the specified target exists; If the specified target is identified, step S5 is performed; S5. Calculate the latitude and longitude of the specified target according to the key frame in which the specified target is identified; S6. Take the latitude and longitude of the specified target as the center, and determine a pre-circumference starting point according to the circumradius corresponding to the current operation; S7. Control the UAV to fly to any point on the vertical line of the pre-circumference starting point, and adjust to the corresponding height according to the circumradius corresponding to the current operation to determine the initial circumnavigation point; S8. Adjust the UAV lens to always face the specified target, and perform the flight task of circumnavigating the specified target according to the circumradius, taking the initial circumnavigation point as the starting point.

[0007] In the above automatic search-based operation area circumnavigation route planning method, step S4 is: S421. Adjust the gimbal to a second preset depression angle as the first yaw direction position; S422. Extract any key frame at the current yaw direction position, and perform image recognition on the key frame to determine whether the specified target exists; S423. If the specified target is not identified, adjust the preset yaw angle along the circumference and repeat steps S422 and S423 until N preset yaw angle adjustments are completed or the specified target is identified; , , the preset yaw angle; S424. If the specified target is identified, step S5 is performed.

[0008] In the above automatic search-based operation area circumnavigation route planning method, in step S4, if the specified target is not identified after N preset yaw angle adjustments are completed, take the preset target point as the point of interest, and perform circumnavigation flight according to the circumradius and the point of interest.

[0009] In the above automatic search-based operation area circumnavigation route planning method, the first preset depression angle is (90±5)°; The second preset depression angle is (45±5)°.

[0010] In the automatic search-based operation area surrounding flight path planning method, the preset yaw angle is determined by the following relationship: ; ; ; is a horizontal field of view angle of the UAV lens; is a preset yaw angle.

[0011] In the automatic search-based operation area surrounding flight path planning method, in step S5, the latitude and longitude of the specified target is calculated by the following method: extracting a key frame in which the specified target is recognized as a reference picture; recognizing a pixel position of the specified target in the reference picture; calculating an actual distance L of the specified target from the current position according to the pixel position: calculating the latitude and longitude position of the specified target according to the yaw angle β of the current UAV and the actual distance L.

[0012] In the automatic search-based operation area surrounding flight path planning method, the actual distance L is calculated by the following method: L=L1+L2; L1 represents an actual distance of the specified target from the center of the picture; L1=△L×λ,△L is a pixel deviation of the specified target from the center of the picture, and λ is a picture scale of the reference picture; L2 represents a horizontal position deviation caused by lens pitch; L2=H×tanα, H is a relative height of the UAV, and α is a pitch angle of the current lens, when α is 90°, L2=0.

[0013] In the automatic search-based operation area surrounding flight path planning method, in step S5, based on the latitude and longitude of the specified target, the UAV is controlled to fly to the vicinity of the specified target and then step S6 is executed; In step S8, the specified target is set as a point of interest so that the UAV lens always faces the specified target.

[0014] In the automatic search-based operation area surrounding flight path planning method, in step S8, the specified target is set as a point of interest by the following method: using the latitude and longitude and a preset elevation reference height as the latitude and longitude and elevation of the specified target to determine an initial point of interest; setting an initial orientation of the UAV lens based on the initial point of interest; With the initial orientation as the reference, the gimbal pitch angle is adjusted within a set angle range, and laser ranging sampling is performed at the corresponding angle, so as to correct the altitude of the specified target; The corrected longitude and latitude are used to determine the point of interest; With the initial surrounding point as the starting point, the surrounding flight is performed according to the surrounding radius and the point of interest.

[0015] An automatic search-based operation area surrounding flight path planning system, comprising a UAV and a cloud service, wherein the cloud service is used to execute the method.

[0016] The present application has the advantages that: the present scheme avoids the data storage pressure caused by laser point cloud planning flight path and the manpower consumption of collecting laser point cloud in advance; at the same time, it overcomes the problems of relatively dispersed operation site, complex and uncertain environment, inaccurate operation target position, etc., which lead to the difficulty of autonomous flight of the UAV for inspection, especially in the scene of UAV inspection for power operation construction, which can ensure that the UAV reaches the real operation site position and plans the surrounding flight based on it, realizing the autonomous flight of the UAV for inspection at the operation site in the scene of inaccurate operation target position. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The operation area search flowchart of the automatic search-based operation area surrounding flight path planning method of the present application; Figure 2 The surrounding planning flowchart of the automatic search-based operation area surrounding flight path planning method of the present application; Figure 3 The target identification flowchart of the automatic search-based operation area surrounding flight path planning method of the present application. DETAILED DESCRIPTION

[0018] The present embodiment provides an automatic search-based operation area surrounding flight path planning method, mainly in automatic search of operation area and autonomous surrounding flight path planning combining the functions of laser radar, laser ranging and obstacle avoidance of the UAV itself after searching the operation area.

[0019] As shown in the figure, Figure 1 The present embodiment takes 90° as the first preset depression angle, 45° as the second preset depression angle, and the power operation construction scene as an example to introduce the operation area search method in detail: Firstly, the approximate longitude and latitude of the construction site are provided by the construction party, and the construction site usually uses a control ball, which can be used to obtain the approximate longitude and latitude, and the longitude and latitude are taken as the preset target point, and the UAV is controlled to hover above the preset target point.

[0020] Adjust the tilt angle of the gimbal to 90°, and switch the lens from wide angle to 5x zoom. In this state, extract the key frame, I-frame, of the captured video stream.

[0021] Determine whether the specified target exists in the key frame through image recognition technology.

[0022] In the power operation construction scene, the specified target can be the features that often appear in the power operation construction site, such as towers, large machinery, fences, multiple safety helmets, etc.

[0023] If the specified target is not identified, adjust the tilt angle of the gimbal to 45° as the first yaw direction position. Extract any key frame at the current yaw direction position, and perform image recognition on the key frame to determine whether the specified target exists. If the specified target is not identified, adjust the preset yaw angle and repeat the above process until the cumulative preset yaw angle adjustment is completed N times or the specified target is identified. , The preset yaw angle is 60°.

[0024] This embodiment takes 60° yaw angle as an example, and 6 key frames can be extracted around 360°. In actual use, it is not limited to this angle. In general, the preset yaw angle is determined as follows: ; ; ; The horizontal field of view of the UAV lens is 60°. The preset yaw angle is 60°, which requires that after N preset yaw angle adjustments, the captured image can cover the area below the UAV in the circumferential direction.

[0025] If the specified target is identified at any of the above nodes, such as the 90° tilt angle state or any yaw direction position, further processing is performed on the specified target to obtain the latitude and longitude of the specified target.

[0026] If the specified target is not identified at any of the above nodes, the preset target point is taken as the specified target.

[0027] When the specified target is not identified at any of the above nodes, the circumnavigation route is planned as follows: Take the latitude and longitude of the preset target point as the center, and determine a pre-circumnavigation starting point according to the circumnavigation radius corresponding to the current operation.

[0028] The unmanned aerial vehicle is controlled to fly above the pre-circulation starting point, and is adjusted to a corresponding height according to the corresponding circulation height of the current operation, so as to determine the initial circulation point. In another embodiment, the unmanned aerial vehicle can fly to the pre-circulation starting point, or can fly below the pre-circulation starting point, and is not limited in this way.

[0029] The preset target point is taken as a latitude and longitude, and the latitude and longitude is taken as a latitude and longitude height of the unmanned aerial vehicle. The unmanned aerial vehicle takes the initial circulation point as a starting point, and performs a flight task of circling the preset target according to the circulation radius required by the current operation and the interest point determined in the foregoing manner.

[0030] The preset altitude reference is generally an altitude of an unmanned aerial vehicle airport.

[0031] As shown in FIG. 1, if the specified target is identified in the foregoing manner, a circling route is planned in the following manner to make the unmanned aerial vehicle capture a picture in cooperation with an actual height of a work site: Figure 2 The latitude and longitude of the specified target is obtained. A pre-circulation starting point is determined according to the corresponding circulation radius of the current operation, with the latitude and longitude of the specified target as a center. The unmanned aerial vehicle is controlled to fly above the pre-circulation starting point, and is adjusted to a corresponding height according to the corresponding circulation height of the current operation, so as to determine the initial circulation point. In another embodiment, the unmanned aerial vehicle can fly to the pre-circulation starting point, or can fly below the pre-circulation starting point, and is not limited in this way.

[0032] The latitude and longitude of the specified target and the preset altitude reference are taken as a latitude and longitude height of the specified target, so as to determine an initial interest point. The initial direction of the lens of the unmanned aerial vehicle is set based on the initial interest point. The gimbal pitch angle is adjusted within a set angle range, and laser ranging sampling is performed at a corresponding angle, so as to obtain the altitude of the specified target by ranging. The altitude data obtained at multiple angles is taken, and a mean value is taken, so as to correct the altitude of the specified target.

[0033] The preset altitude reference is represented by H. The relative distance in the vertical direction between the unmanned aerial vehicle and the specified target is represented by h. The laser ranging value is represented by L. The lens pitch angle corresponding to the i th laser ranging value is represented by i. The height of the unmanned aerial vehicle relative to the preset altitude reference is represented by h. ​​ represents the number of ranging times.

[0034] The angle and the number of laser ranging sampling times are determined by those skilled in the art according to requirements, and preferably the angle ranging is adjusted 3-5 times within a range of 20°.

[0035] The corrected longitude and latitude are used to determine the point of interest. The initial surrounding point is taken as the starting point, and the surrounding flight is performed according to the surrounding radius and the point of interest.

[0036] In the above-mentioned surrounding flight path planning, the laser ranging function of the unmanned aerial vehicle is used to adjust the gimbal pitch angle within a certain angle range and perform laser ranging at the corresponding angle based on the preset altitude as the reference, so as to correct the error caused by the non-uniformity of the target point altitude and the preset altitude reference height, so that the operation target is better presented in the center of the video picture during the surrounding flight, thereby ensuring the visual effect of the picture presentation.

[0037] Specifically, as shown in Figure 3 the longitude and latitude of the specified target are calculated by the following method: extracting the key frame in which the specified target is recognized as a reference picture; recognizing the pixel position of the specified target in the reference picture; calculating the actual distance L between the specified target and the current position according to the pixel position: calculating the longitude and latitude position of the specified target according to the yaw angle β of the current unmanned aerial vehicle and the actual distance L.

[0038] Specifically, the actual distance L is calculated by the following method: L=L1+L2; L1 represents the actual distance of the specified target from the center of the picture; L1=△L×λ,△L is the pixel deviation of the specified target from the center of the picture, which is calculated by image recognition, and λ is the picture scale of the reference picture; L2 represents the horizontal position deviation caused by the lens pitch; L2=H×tanα, H is the height of the unmanned aerial vehicle relative to the preset altitude reference, and α is the pitch angle of the current lens, when α is 90°, L2=0.

[0039] λ=(H×s) / (f×k) H is the height of the unmanned aerial vehicle relative to the preset altitude reference; s is the physical size of the image sensor in the corresponding direction f is the focal length of the lens k is the number of pixels of the picture in the corresponding direction.

[0040] According to the current UAV yaw angle β (since the lens is set to be consistent with the nose, here the yaw angle of the UAV is consistent with the yaw angle of the lens) and the actual distance L, the latitude and longitude position of the specified target is calculated, and the formula is as follows: Starting point: longitude lon0 (unit: degree), latitude lat0 (unit: degree); Earth radius R; Convert angle to radian: lat0_rad=lat0*π / 180 lon0_rad=lon0*π / 180 β_rad=β*π / 180 Calculate the angular distance (earth center angle): θ=L / R Calculate the new latitude: lat1_rad=arcsin(sin(lat0_rad)*cos(θ)+cos(lat0_rad)*sin(θ)*cos(β_rad)) Calculate the new longitude: lon1_rad=lon0_rad+arctan2(sin(β_rad)*sin(θ)*cos(lat0_rad),cos(θ)-sin(lat0_rad)*sin(lat1_rad)) Convert radian to angle: lat1=lat1_rad*180 / π lon1=lon1_rad*180 / π Longitude range correction (ensure between -180° and 180°): lon1=(lon1+540)%360-180.

[0041] Above, by controlling the yaw angle of the UAV and the angle of the gimbal, a large range of area search is performed under different combinations of pitch angle and yaw angle, while extracting video image key frames for target recognition, correcting the actual position of the work site, overcoming the problem of relatively dispersed power operation site, complex and uncertain environment, and realizing autonomous flight of UAV supervision in power operation construction scene.

[0042] Further, the embodiment also provides an operation area surrounding flight path planning system based on automatic search, including a UAV and a cloud service, the cloud service is used for executing the tree climbing method to plan the operation area surrounding flight path.

[0043] The above takes the power operation construction scene as an example, when put into use, it can also be used for other scenarios of surrounding flight path planning based on automatic search of operation area, such as environmental protection processing scene, highway construction scene, etc.

[0044] The specific embodiments described herein are merely illustrative of the spirit of the application. Various modifications or changes in addition or substitution to the described specific embodiments can be made by those skilled in the art without departing from the spirit of the application or exceeding the scope of the appended claims.

[0045] Although the terms preset target point, gimbal, first preset depression angle, second preset depression angle, key frame, designated target, preset yaw angle, pre-surround start point, initial surround point, and point of interest are used more frequently herein, the possibility of using other terms is not excluded. The use of these terms is merely for the convenience of describing and explaining the essence of the application; any interpretation of them as an additional limitation is contrary to the spirit of the application.

Claims

1. An automatic search-based job area wrap-around route planning method, characterized by, The method comprises: S1. controlling the unmanned aerial vehicle to fly to a preset target point and hover above the target point; S2. adjusting a holder to a first preset depression angle, and extracting any key frame in the first preset depression angle state; S3. performing image recognition on the key frame to determine whether a specified target exists; if the specified target is recognized, step S5 is performed, otherwise, step S4 is performed; S4. adjusting the holder to a second preset depression angle, and controlling the unmanned aerial vehicle to adjust at a preset yaw angle interval in a circumferential direction while hovering; extracting any key frame at each yaw direction position; performing image recognition on the extracted key frame to determine whether the specified target exists; if the specified target is recognized, step S5 is performed; S5. calculating the latitude and longitude of the specified target according to the key frame in which the specified target is recognized; S6. taking the latitude and longitude of the specified target as the center, and determining a pre-circulation starting point according to a circumferential radius corresponding to the current operation; S7. controlling the unmanned aerial vehicle to fly to any point on the vertical line of the pre-circulation starting point, and adjusting to the corresponding height according to the circumferential height corresponding to the current operation to determine an initial circumferential point; S8. adjusting the lens of the unmanned aerial vehicle to always face the specified target, taking the initial circumferential point as the starting point, and performing a flight task of circling the specified target according to the circumferential radius.

2. The automatic search-based mission area loop planning method according to claim 1, wherein, Step S4 is: S421. adjusting the holder to the second preset depression angle as the first yaw direction position; S422. extracting any key frame at the current yaw direction position, and performing image recognition on the key frame to determine whether the specified target exists; S423. if the specified target is not recognized, adjusting at a preset yaw angle in a circumferential direction and repeating steps S422 and S423 until N times of preset yaw angle adjustment are completed or the specified target is recognized; , is a preset yaw angle; S424. if the specified target is recognized, step S5 is performed.

3. The automatic search-based mission area loop planning method according to claim 2, wherein, In step S4, if the specified target is not recognized after N times of preset yaw angle adjustment, taking the preset target point as a point of interest, and performing a circumferential flight according to the circumferential radius and the point of interest.

4. The automatic search-based mission area loop planning method according to claim 1, wherein, The first preset depression angle is (90±5)°; The second preset depression angle is (45±5)°.

5. The automatic search-based mission area loop planning method of claim 1, wherein, The preset yaw angle is determined by the following relationship: ； ； ； is the horizontal field of view angle of the UAV lens; is a preset yaw angle.

6. The automatic search-based mission area loop planning method according to claim 1, wherein, In step S5, the latitude and longitude of the specified target are calculated by the following method: extracting a key frame in which the specified target is recognized as a reference picture; recognizing the pixel position of the specified target in the reference picture; calculating the actual distance L of the specified target from the current position according to the pixel position; calculating the latitude and longitude position of the specified target according to the yaw angle β of the current unmanned aerial vehicle and the actual distance L.

7. The automatic search-based mission area loop planning method according to claim 6, wherein, The actual distance L is calculated by the following method: L=L1+L2; L1 represents the actual distance of the specified target from the center of the picture; L1=△L×λ,△L is the pixel deviation of the specified target from the center of the picture, and λ is the picture scale of the reference picture; L2 represents the horizontal position deviation caused by the lens depression; L2=H×tanα, H is the relative height of the unmanned aerial vehicle, and α is the depression angle of the current lens, when α is 90°, L2=0.

8. The automatic search-based mission area loop planning method of claim 1, wherein, In step S5, based on the longitude and latitude of the specified target, the UAV is controlled to fly to the vicinity of the specified target and then step S6 is executed; In step S8, the specified target is set as a point of interest so that the UAV camera is always directed towards the specified target.

9. The automatic search-based mission area loop planning method according to claim 8, wherein, In step S8, the specified target is set as a point of interest by the following method: The longitude and latitude and the preset altitude reference height are used as the longitude and latitude and altitude of the specified target to determine an initial point of interest; Based on the initial point of interest, the initial direction of the UAV camera is set; Based on the initial direction, within a set angle range, the gimbal pitch angle is adjusted and laser ranging sampling is performed at the corresponding angle, so that the altitude of the specified target is corrected; The corrected longitude and latitude and altitude are used to determine the point of interest; Based on the initial orbit point and the orbit radius and the point of interest, orbit flight is performed.

10. An automatic search-based job area circumnavigation planning system comprising a UAV and a cloud service, characterized in that, The cloud service is used to execute the method of any one of claims 1-9.

Images