Job area surrounding route planning method and system based on automatic search

Abstract

The application discloses a kind of based on automatic search's job area surround flight path planning method and system, by controlling the yaw angle of unmanned aerial vehicle and the angle of pan-tilt, in the combination of different pitch angle and yaw angle, large-scale area search is carried out, video image key frame is extracted simultaneously to carry out target identification, corrects actual position of work site, overcomes the problem that unmanned aerial vehicle supervision autonomous flight is difficult due to the relative dispersion of work site, complex and having uncertainty of environment, inaccuracy of work target position etc., especially applicable to unmanned aerial vehicle supervision scene of electric power operation construction, can guarantee that unmanned aerial vehicle reaches real work site position and plans surround flight based on it, realizes the unmanned aerial vehicle supervision autonomous flight of work site in the scene of inaccuracy of work target position.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) route planning technology, and in particular relates to an environmentally adaptive, automatically search-based method and system for planning operational area circumnavigation routes. Background Technology

[0002] Currently, drone technology is gradually becoming widespread and integrated into various industries, especially in the power sector. For the operation and maintenance of existing power grids, drone inspections have become one of the mainstream auxiliary methods. Before actual operation, staff plan a fixed flight path based on the preset locations and orientations of transmission lines, distribution towers, and other equipment (usually by first establishing a point cloud model, then setting flight points based on the point cloud model, and finally planning the flight path). The drone flies along this path to check for potential faults that may occur in the equipment during long-term operation. Because the locations and routes of facilities in existing power grids are fixed, the drone's flight path and inspection focus remain stable over a long period.

[0003] With the popularization of drone technology, the use of drones for power construction supervision is gradually being put on the agenda. Power construction supervision refers to monitoring and inspecting whether there are any violations, irregularities, or safety hazards at the construction site. Power construction sites often lack fixed locations, making it impossible to collect line data and establish laser point clouds in advance. Moreover, even if laser point clouds are established, the relatively dispersed locations, complex and uncertain environments of power construction sites, and the dynamic changes in facility status and work positions within the same construction area often lead to discrepancies between the planned work location and the actual on-site work location. This means the target location is inaccurate, making traditional drone inspection methods for power facilities unable to fully meet the autonomous flight requirements for drone supervision in power construction scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a method for planning a work area around the work area based on automatic search, which can address the problems of relatively dispersed power work locations, complex and uncertain environments, inaccurate work target locations, and dynamic changes in the on-site work environment.

[0005] Another objective of this invention is to propose a work area circumnavigation route planning system based on automatic search.

[0006] A method for planning a work area circumnavigation route based on automatic search, the method comprising:

[0007] S1. Control the drone to fly to the preset target point and hover;

[0008] S2. Adjust the gimbal to the first preset tilt angle and extract any keyframe under the first preset tilt angle state;

[0009] S3. Image recognition is performed on the key frame to determine whether the specified target exists;

[0010] If the specified target is recognized, step S5 is performed, otherwise step S4 is performed;

[0011] S4. The gimbal is adjusted to a second preset depression angle, and the UAV is controlled to make adjustment at a preset yaw angle interval along the circumferential direction at the hovering position;

[0012] Any key frame is extracted at each yaw direction position;

[0013] Image recognition is performed on the extracted key frame to determine whether the specified target exists;

[0014] If the specified target is recognized, step S5 is performed;

[0015] S5. The latitude and longitude of the specified target are calculated according to the key frame in which the specified target is recognized;

[0016] S6. A pre-circulation starting point is determined according to the circumferential radius corresponding to the current operation, with the latitude and longitude of the specified target as the center;

[0017] S7. The UAV is controlled to fly to any point on the vertical line of the pre-circulation starting point, and is adjusted to the corresponding height according to the circumferential height corresponding to the current operation, to determine an initial circumferential point;

[0018] S8. The UAV lens is adjusted to always face the specified target, and a flight task of circling the specified target is performed according to the circumferential radius, with the initial circumferential point as the starting point.

[0019] In the above automatic search-based operation area circumferential flight path planning method, step S4 is:

[0020] S421. The gimbal is adjusted to a second preset depression angle as a first yaw direction position;

[0021] S422. Any key frame at the current yaw direction position is extracted, and image recognition is performed on the key frame to determine whether the specified target exists;

[0022] S423. If the specified target is not recognized, the preset yaw angle is adjusted along the circumferential direction, and steps S422 and S423 are repeated until N times of preset yaw angle adjustment are completed or the specified target is recognized;

[0023] , is the preset yaw angle;

[0024] S424. If the specified target is recognized, step S5 is performed.

[0025] In the automatic search-based operation area surrounding flight path planning method, if the preset target is not identified after N preset yaw angle adjustments are completed in step S4, a preset target point is taken as a point of interest, and a surrounding flight is performed according to the surrounding radius and the point of interest.

[0026] In the automatic search-based operation area surrounding flight path planning method, the first preset depression angle is (90±5)°.

[0027] The second preset depression angle is (45±5)°.

[0028] In the automatic search-based operation area surrounding flight path planning method, the preset yaw angle is determined by the following relationship:

[0029] ;

[0030] ;

[0031] ;

[0032] is a horizontal field of view angle of a lens of the UAV;

[0033] is a preset yaw angle.

[0034] In the automatic search-based operation area surrounding flight path planning method, in step S5, the latitude and longitude of the specified target is calculated by the following method:

[0035] A key frame in which the specified target is identified is extracted as a reference picture;

[0036] A pixel position of the specified target in the reference picture is identified;

[0037] An actual distance L of the specified target from a current position is calculated according to the pixel position:

[0038] A latitude and longitude position of the specified target is calculated according to a yaw angle β of the current UAV and the actual distance L.

[0039] In the automatic search-based operation area surrounding flight path planning method, the actual distance L is calculated by the following method:

[0040] L=L1+L2;

[0041] L1 represents an actual distance of the specified target from a center of the picture;

[0042] L1=△L×λ, △L is a pixel deviation of the specified target from the center of the picture, and λ is a picture scale of the reference picture.

[0043] L2 represents the position deviation in the horizontal direction caused by the lens tilt;

[0044] L2=H x tan alpha, H is the relative height of the unmanned aerial vehicle, alpha is the tilt angle of the current lens, when alpha is 90 degrees, L2=0.

[0045] In the above-mentioned automatic search-based work area loop flight planning method, in step S5, based on the latitude and longitude of the specified target, the unmanned aerial vehicle is controlled to fly to the vicinity of the specified target, and then step S6 is executed;

[0046] In step S8, the specified target is set as a point of interest so that the lens of the unmanned aerial vehicle always points to the specified target.

[0047] In the above-mentioned automatic search-based work area loop flight planning method, in step S8, the specified target is set as a point of interest by the following method:

[0048] The latitude and longitude and the preset elevation reference height are used as the latitude and longitude of the specified target to determine the initial point of interest;

[0049] Based on the initial point of interest, the initial orientation of the unmanned aerial vehicle lens is set;

[0050] Based on the initial orientation, within a set angle range, the gimbal tilt angle is adjusted and laser ranging sampling is performed at the corresponding angle, thereby correcting the elevation of the specified target;

[0051] The corrected latitude and longitude are used to determine the point of interest;

[0052] Starting from the initial loop point, the loop flight is performed according to the loop radius and the point of interest.

[0053] An automatic search-based work area loop flight planning system, comprising an unmanned aerial vehicle and a cloud service, the cloud service being used to execute the method.

[0054] The advantages of the present application are that the present solution avoids the data storage pressure caused by laser point cloud planning of flight path and the consumption of manpower for collecting laser point cloud in advance, and overcomes the problems of difficulty in autonomous flight of unmanned aerial vehicle for inspection due to relatively dispersed work sites, complex and uncertain environment, and inaccurate work target position, etc., especially suitable for unmanned aerial vehicle inspection scene of power work construction, and can ensure that the unmanned aerial vehicle reaches the real work site position and plans loop flight based thereon, realizing autonomous flight of unmanned aerial vehicle for inspection at work site in the scene of inaccurate work target position. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The work area search flowchart of the automatic search-based work area loop flight planning method of the present application;

[0056] Figure 2 The surrounding planning flowchart of the automatic search-based job area surrounding flight path planning method of the present application;

[0057] Figure 3 The target recognition flowchart of the automatic search-based job area surrounding flight path planning method of the present application. DETAILED DESCRIPTION

[0058] The embodiment provides an automatic search-based job area surrounding flight path planning method, mainly in automatically searching a job area and autonomously planning a surrounding flight path by combining a laser radar, laser ranging and obstacle avoidance function of a UAV after searching the job area.

[0059] As shown in the figure, Figure 1 The embodiment takes a first preset depression angle of 90°, a second preset depression angle of 45° and a power operation construction scene as an example to introduce the job area searching method in detail:

[0060] Firstly, the approximate longitude and latitude of the construction site are provided by the construction party, and the construction site usually uses a control ball, which can be used to obtain the approximate longitude and latitude, and the longitude and latitude are taken as a preset target point to control the UAV to hover above the preset target point.

[0061] The gimbal depression angle is adjusted to 90°, and the lens is switched from wide-angle to 5 times zoom, and the key frame-I frame of the captured video stream is extracted in this state.

[0062] The key frame is judged by image recognition technology whether the specified target exists.

[0063] In the power operation construction scene, the specified target can be the features that often appear in the power operation construction site, such as towers, large machinery, fences, multiple safety helmets, etc.

[0064] If the specified target is not identified, the gimbal depression angle is adjusted to 45° as the first yaw direction position;

[0065] Any key frame of the current yaw direction position is extracted, and image recognition is performed on the key frame to determine whether the specified target exists;

[0066] If the specified target is not identified, the preset yaw angle is adjusted circumferentially and the above process is repeated until N times of preset yaw angle adjustment are completed or the specified target is identified;

[0067] , The preset yaw angle.

[0068] This embodiment takes 60° yaw angle as an example, and 6 key frames can be extracted around 360°. In use, it is not limited to this angle. Generally, the preset yaw angle is determined as follows:

[0069] ;

[0070] ;

[0071] ;

[0072] is the horizontal field of view angle of the UAV lens;

[0073] is the preset yaw angle, which requires that the shooting picture can cover the area below the UAV after N times of preset yaw angle adjustment.

[0074] If any of the above nodes, such as 90° pitch angle state, or any yaw direction position, recognizes the specified target, further processing is performed on the specified target to obtain the latitude and longitude of the specified target.

[0075] If none of the above nodes recognizes the specified target, the preset target point is taken as the specified target.

[0076] When none of the above nodes recognizes the specified target, the surrounding flight path is planned as follows:

[0077] Taking the latitude and longitude of the preset target point as the center, a pre-surrounding starting point is determined according to the surrounding radius corresponding to the current operation.

[0078] The UAV is controlled to fly above the pre-surrounding starting point, and is adjusted to the corresponding height according to the surrounding height corresponding to the current operation to determine the initial surrounding point. In another embodiment, the UAV can fly to the pre-surrounding starting point, or can fly below the pre-surrounding starting point, which is not limited here.

[0079] Taking the preset altitude reference height as the height and the latitude and longitude of the preset target point as the latitude and longitude, the latitude and longitude height is set as the interest point of the UAV. The UAV takes the initial surrounding point as the starting point, and performs the flight task of surrounding the preset target according to the surrounding radius required by the current operation and the interest point determined in the foregoing.

[0080] The preset altitude reference is generally the altitude of the UAV airport.

[0081] As shown in Figure 2 , if the specified target is recognized by the foregoing method, the surrounding flight path is planned as follows to make the UAV shooting picture cooperate with the actual height of the operation site:

[0082] The latitude and longitude of the specified target are obtained;

[0083] Determine a pre-circulation starting point according to the current operation corresponding to the radius of the surrounding center with the specified target latitude and longitude as the center;

[0084] Control the unmanned aerial vehicle to fly above the pre-circulation starting point, and adjust to the corresponding height according to the current operation corresponding to the surrounding height, to determine the initial surrounding point. In another embodiment, the unmanned aerial vehicle can fly to the pre-circulation starting point, or fly below the pre-circulation starting point, which is not limited here.

[0085] Use the latitude and longitude of the specified target and the preset altitude reference height as the latitude and longitude of the specified target to determine the initial interest point;

[0086] Set the initial orientation of the unmanned aerial vehicle lens based on the initial interest point;

[0087] With the initial orientation as the reference, adjust the gimbal pitch angle within the set angle range and perform laser ranging sampling at the corresponding angle, obtain the altitude of the specified target by ranging, take the altitude data measured at multiple angles, take the mean value, and thus correct the altitude of the specified target , The specific method is as follows:

[0088]

[0089] Indicates the preset altitude reference height;

[0090] Indicates the relative distance between the unmanned aerial vehicle and the specified target in the vertical direction;

[0091] Indicates the laser ranging value;

[0092] Indicates the lens pitch angle corresponding to the i-th laser ranging value;

[0093] Indicates the height of the unmanned aerial vehicle relative to the preset altitude reference;

[0094] Indicates the ranging times.

[0095] The set angle and the number of laser ranging sampling times are determined by those skilled in the art according to the needs, and preferably 3-5 times of angle ranging within 20°.

[0096] Use the corrected latitude and longitude to determine the interest point;

[0097] Take the initial surrounding point as the starting point, and perform a surrounding flight according to the surrounding radius and the interest point.

[0098] In the above-mentioned flight path planning, the laser ranging function of the unmanned aerial vehicle is used to set the preset altitude as a reference, adjust the gimbal pitch angle within a certain angle range, and perform laser ranging at the corresponding angle to correct the error caused by the inconsistency between the target point altitude and the preset altitude reference height, so that the operation target is better presented in the center of the video screen during the surrounding flight, thereby ensuring the visual effect of the picture.

[0099] Specifically, as shown in Figure 3 the longitude and latitude of the specified target are calculated by the following method:

[0100] The key frame in which the specified target is recognized is extracted as a reference picture;

[0101] The pixel position of the specified target in the reference picture is identified;

[0102] The actual distance L between the specified target and the current position is calculated according to the pixel position:

[0103] The longitude and latitude of the specified target are calculated according to the yaw angle β of the current unmanned aerial vehicle and the actual distance L.

[0104] Specifically, the actual distance L is calculated by the following method:

[0105] L=L1+L2;

[0106] L1 represents the actual distance of the specified target from the center of the picture;

[0107] L1=△L×λ,△L is the pixel deviation of the specified target from the center of the picture, which is calculated by image recognition, and λ is the picture scale of the reference picture;

[0108] L2 represents the horizontal position deviation caused by the lens pitch;

[0109] L2=H×tanα, H is the height of the unmanned aerial vehicle relative to the preset altitude reference, and α is the pitch angle of the current lens, when α is 90°, L2=0.

[0110] λ=(H×s) / (f×k)

[0111] H is the height of the unmanned aerial vehicle relative to the preset altitude reference;

[0112] s is the physical size of the image sensor in the corresponding direction

[0113] f is the focal length of the lens

[0114] k is the number of pixels of the picture in the corresponding direction.

[0115] According to the current UAV yaw angle β (since the lens is set to be consistent with the nose, here the yaw angle of the UAV is consistent with the yaw angle of the lens) and the actual distance L, the latitude and longitude position of the specified target is calculated, the formula is as follows:

[0116] Starting point: longitude lon0 (unit: degree), latitude lat0 (unit: degree);

[0117] Earth radius R;

[0118] Convert angle to radian:

[0119] lat0_rad=lat0*π / 180

[0120] lon0_rad=lon0*π / 180

[0121] β_rad=β*π / 180

[0122] Calculate the angular distance (central angle of the earth): θ=L / R

[0123] Calculate the new latitude:

[0124] lat1_rad=arcsin(sin(lat0_rad)*cos(θ)+cos(lat0_rad)*sin(θ)*cos(β_rad))

[0125] Calculate the new longitude:

[0126] lon1_rad=lon0_rad+arctan2(sin(β_rad)*sin(θ)*cos(lat0_rad),cos(θ)-sin(lat0_rad)*sin(lat1_rad))

[0127] Convert radian to angle:

[0128] lat1=lat1_rad*180 / π

[0129] lon1=lon1_rad*180 / π

[0130] Longitude range correction (ensure between -180° and 180°):

[0131] lon1=(lon1+540)%360-180.

[0132] By controlling the yaw angle of the unmanned aerial vehicle and the angle of the holder, the large-area region search is performed under different combinations of the pitch angle and the yaw angle, the video image key frame is extracted for target identification, and the actual position of the work site is corrected, thereby overcoming the problems of relatively dispersed power work sites, complex environment and uncertainty, and realizing autonomous flight of the unmanned aerial vehicle for supervision in the power work construction scene.

[0133] Further, the embodiment also provides an automatic-search-based work region surrounding flight path planning system, which comprises an unmanned aerial vehicle and a cloud service, and the cloud service is configured to execute the tree climbing method to plan a surrounding flight path of a work region.

[0134] The above takes the power work construction scene as an example, and when put into use, can also be used for automatic-search-based surrounding flight path planning of work regions in other scenes, such as environmental protection processing scenes, highway construction scenes, etc.

[0135] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, without deviating from the spirit of the present application or exceeding the scope defined by the appended claims.

[0136] Although the terms such as preset target point, holder, first preset pitch angle, second preset pitch angle, key frame, specified target, preset yaw angle, preset surrounding starting point, initial surrounding point, and interest point are used more frequently herein, the possibility of using other terms is not excluded. The use of these terms is only for more convenient description and explanation of the essence of the present application; any kind of additional limitation by interpreting them is contrary to the spirit of the present application.

Claims

1. A method for planning a work area circumnavigation route based on automatic search, characterized in that, The method includes: S1. Control the drone to fly to the preset target point and hover; S2. Adjust the gimbal to the first preset tilt angle and extract any keyframe under the first preset tilt angle state; S3. Perform image recognition on the keyframes to determine whether a specified target exists; If the specified target is identified, proceed to step S5; otherwise, proceed to step S4. S4. Includes the following process: S421. Adjust the gimbal to the second preset depression angle, which will serve as the first yaw direction position; S422. Extract any keyframe at the current yaw direction position, perform image recognition on the keyframe to determine whether the specified target exists; S423. If the specified target is not identified, adjust the preset yaw angle along the circumference and repeat steps S422 and S423 until the preset yaw angle adjustment is completed N times or the specified target is identified. , Preset yaw angle; S424. If the specified target is identified, proceed to step S5; S5. Calculate the latitude and longitude of the specified target based on the keyframes in which the specified target is identified: Extract the keyframes from which the specified target is identified as reference images; Identify the pixel position of the specified target in the reference image; Calculate the actual distance L between the specified target and the current position based on the pixel position: The latitude and longitude position of the designated target is calculated based on the current yaw angle β of the UAV and the actual distance L; The actual distance L is calculated as follows: L = L1 + L2; L1 represents the actual distance of the specified target from the center of the image; L1 = △L × λ, where △L is the pixel deviation between the specified target and the center of the image, and λ is the image scale of the reference image; L2 represents the horizontal positional deviation caused by the lens tilt; L2 = H × tanα, where H is the relative altitude of the drone and α is the current pitch angle of the camera. When α is 90°, L2 = 0. S6. Using the latitude and longitude of the specified target as the center, determine a pre-circling starting point based on the current orbital radius; S7. Control the drone to fly to any point on the vertical line of the pre-circling starting point, and adjust it to the corresponding height according to the current orbiting height to determine the initial orbiting point; S8. Adjust the drone's camera to always be pointing towards the designated target, and starting from the initial orbiting point, execute a flight mission around the designated target according to the orbiting radius.

2. The method for planning a work area circumnavigation route based on automatic search according to claim 1, characterized in that, In step S4, if the specified target is not identified after N preset yaw angle adjustments, the preset target point is used as the point of interest, and orbital flight is performed based on the orbital radius and the point of interest.

3. The method for planning a work area circumnavigation route based on automatic search according to claim 1, characterized in that, The first preset depression angle is (90±5)°; The second preset depression angle is (45±5)°.

4. The method for planning a work area around a route based on automatic search according to claim 1, characterized in that, The preset yaw angle is determined by the following formula: ； ； ； This refers to the horizontal field of view of the drone's camera. Preset yaw angle.

5. The method for planning a work area circumnavigation route based on automatic search according to claim 1, characterized in that, In step S5, based on the latitude and longitude of the designated target, the UAV is controlled to fly to the vicinity of the designated target before step S6 is executed; In step S8, the designated target is set as a point of interest so that the drone camera is always pointed at the designated target.

6. The method for planning a work area circumnavigation route based on automatic search according to claim 5, characterized in that, In step S8, the specified target is set as a point of interest in the following manner: The latitude and longitude and the preset altitude reference height are used as the latitude and longitude of the specified target to determine the initial point of interest; Set the initial orientation of the drone camera based on the initial point of interest; Based on the initial orientation, within a set angle range, the gimbal pitch angle is adjusted and laser ranging sampling is performed at the corresponding angle, thereby correcting the altitude of the specified target; Use the corrected latitude, longitude, and altitude to determine the points of interest; Starting from the initial orbiting point, an orbital flight is performed based on the orbiting radius and the point of interest.

7. A work area circumnavigation route planning system based on automatic search, comprising unmanned aerial vehicles (UAVs) and cloud services, characterized in that, The cloud service is used to perform the method described in any one of claims 1 to 6.

Images