A method and device for planning an unmanned aerial vehicle survey path

Abstract

The present invention provides a method and device for planning an unmanned aerial vehicle survey path, which are applied to an unmanned aerial vehicle controller, and shooting points are set on a preset survey path. During the flight of the unmanned aerial vehicle, temporary shooting points are set according to obstacles obtained after obstacle detection. A small number of images are shot at the preset shooting points, and additional photos are only taken at the temporary shooting points, thereby shortening the overall flight and shooting time of the unmanned aerial vehicle, reducing the number of images shot by the unmanned aerial vehicle as a whole, thereby shortening the later three-dimensional modeling time, and increasing the local modeling accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and more specifically, to a method and device for planning an unmanned aerial vehicle survey path. Background Art

[0002] In the early stage of the construction of a photovoltaic power station, it is necessary to conduct a topographic survey of the photovoltaic power station area, including terrain area survey and obstacle survey, so as to determine the area where photovoltaic modules can be installed based on the survey results, and thus determine the installation capacity.

[0003] At present, terrain surveys are generally conducted manually, which is time-consuming, labor-intensive, and very inefficient. With the continuous development of drone technology, drones have begun to be used in terrain surveys. Drones generally fly along a pre-planned fixed survey path, set shooting points in the fixed survey path, and take images at each shooting point to generate a three-dimensional model. Since drones take indiscriminate photos at each shooting point in the survey path, and in order to ensure accuracy, a large number of images need to be taken at each shooting point, resulting in a long flight time for drones, and the subsequent modeling time using the images taken by drones is also long. Summary of the invention

[0004] In view of this, the present invention provides a method and device for UAV survey path planning, which shortens the overall flight and photography time of the UAV, reduces the number of images taken by the UAV as a whole, thereby shortening the later three-dimensional modeling time and increasing the local modeling accuracy.

[0005] In order to achieve the above-mentioned invention object, the specific technical solution provided by the present invention is as follows:

[0006] A method for planning a UAV survey path is applied to a UAV controller, and the method comprises:

[0007] Control the drone to fly along a pre-set survey path and take images at pre-set shooting points;

[0008] Perform obstacle detection on the captured image;

[0009] Set temporary shooting points based on detected obstacles;

[0010] Images are captured at the temporary capturing point, and the number of images captured at the temporary capturing point is greater than the number of images captured at the capturing point.

[0011] Optionally, performing obstacle detection on the captured image includes:

[0012] Perform shadow recognition on the captured image;

[0013] When a shadow area is identified, estimating the size of the shadow area;

[0014] Make sure the obstacle is detected.

[0015] Optionally, setting a temporary shooting point according to the detected obstacle includes:

[0016] marking said obstruction;

[0017] Determining whether the obstacle is repeatedly marked;

[0018] If the obstacle is not repeatedly marked, estimating the height and width of the obstacle according to the sun altitude angle and the size of the shadow area;

[0019] When the height of the obstacle is greater than a threshold, the obstacle is set as the temporary shooting point.

[0020] Optionally, marking the obstacle comprises:

[0021] Calculate the centroid of the shadow area according to the size of the shadow area;

[0022] Calculating the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point;

[0023] The obstacle is marked based on the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point.

[0024] Optionally, capturing an image at the temporary capturing point includes:

[0025] Determine a temporary flight path of the UAV at the temporary shooting point according to the height and width of the obstacle;

[0026] The drone is controlled to fly around the obstacle according to the temporary flight path, and images are captured when a preset condition is triggered during the flight of the drone.

[0027] A UAV survey path planning device, applied to a UAV controller, comprising:

[0028] The first shooting unit is used to control the UAV to fly along a preset survey path and to shoot images at preset shooting points;

[0029] An obstacle detection unit, used to perform obstacle detection on the captured image;

[0030] A temporary shooting point setting unit, used to set a temporary shooting point according to the detected obstacle;

[0031] The second shooting unit is used to shoot images at the temporary shooting point, and the number of images shot at the temporary shooting point is greater than the number of images shot at the shooting point.

[0032] Optionally, the obstacle detection unit is specifically used to:

[0033] Perform shadow recognition on the captured image;

[0034] When a shadow area is identified, estimating the size of the shadow area;

[0035] Make sure the obstacle is detected.

[0036] Optionally, the temporary shooting point setting unit includes:

[0037] An obstacle marking subunit, used for marking the obstacle;

[0038] A repeated marking judgment subunit, used to judge whether the obstacle is repeatedly marked;

[0039] A shadow height estimation subunit, configured to estimate the height and width of the obstacle according to the sun altitude angle and the size of the shadow area if the obstacle is not repeatedly marked;

[0040] The temporary shooting point setting subunit is used to set the obstacle as the temporary shooting point when the height of the obstacle is greater than a threshold.

[0041] Optionally, the obstacle marking subunit is specifically used to:

[0042] Calculate the centroid of the shadow area according to the size of the shadow area;

[0043] Calculating the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point;

[0044] The obstacle is marked based on the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point.

[0045] Optionally, the second shooting unit is specifically used to:

[0046] Determine a temporary flight path of the UAV at the temporary shooting point according to the height and width of the obstacle;

[0047] The drone is controlled to fly around the obstacle according to the temporary flight path, and images are captured when a preset condition is triggered during the flight of the drone.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] The present invention discloses a method and device for planning a UAV survey path, which sets shooting points on a preset survey path, sets temporary shooting points according to obstacles obtained after obstacle detection during the flight of the UAV, shoots a small number of images at the preset shooting points, and only takes additional photos at the temporary shooting points, thereby shortening the overall flight and shooting time of the UAV, reducing the number of images taken by the UAV as a whole, thereby shortening the later three-dimensional modeling time, and increasing the local modeling accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 A schematic diagram of a flow chart of a method for planning a UAV survey path disclosed in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a UAV survey path disclosed in an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a partial method flow of a method for planning a UAV survey path disclosed in an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of an obstacle shadow disclosed in an embodiment of the present invention;

[0055] Figure 5 It is a schematic diagram of the height of the shadow area disclosed in the embodiment of the present invention;

[0056] Figure 6 The present invention is a schematic structural diagram of a UAV survey path planning device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The present invention provides a method and device for planning a UAV survey path, which is applied to a UAV controller, which can be arranged inside or outside the UAV, and communicates with the UAV through wireless communication. The present invention adopts a combination of coarse shooting at a pre-set shooting point and fine shooting of obstacles, thereby shortening the overall flight shooting time of the UAV, reducing the number of images taken by the UAV as a whole, thereby shortening the time for later three-dimensional modeling, and increasing the local modeling accuracy.

[0059] For details, please refer to Figure 1 This embodiment discloses a method for planning a UAV survey path, comprising the following steps:

[0060] S101: Control the UAV to fly along a preset survey path and take images at preset shooting points;

[0061] The survey path of the drone is pre-set. Taking rooftop photovoltaic as an example, since the shading of the photovoltaic components on the roof is mainly blocked by objects in the south, the starting point of the drone's survey path is usually set on the south side of the roof. The default path is an S-shaped path, an M-shaped path, or a Z-shaped path. The present invention is not limited to this. A shooting point is set at every preset distance. An optional implementation method of the drone's survey path is as follows: Figure 2 shown.

[0062] The number of images captured at a preset shooting point is less than the number of images captured by a traditional UAV at a shooting point. For example, one image may be captured at a preset shooting point.

[0063] S102: performing obstacle detection on the captured image;

[0064] Specifically, shadow recognition is performed on the captured image, and when a shadow area is recognized, it is determined that an obstacle is detected.

[0065] This embodiment uses RGB color space to quickly identify shadows. RGB color space is a commonly used space and is the most widely used color space. When using RGB color space to check shadows, RGB is first converted to C1C2C3.

[0066]

[0067]

[0068]

[0069] In the shadow area, the grayscale of the blue light channel decreases the least, so in the C3 component of the C1C2C3 color space, the shadow area mainly occupies high pixel values. The preliminary shadow area is obtained by using the threshold segmentation method on the C3 component map. Only the area above a certain threshold in the C3 component and below a certain threshold in the B component is detected as a shadow area, so that the size of the detected shadow area can be estimated.

[0070] S103: Setting a temporary shooting point according to the detected obstacle;

[0071] It should be noted that only obstacles with a certain height need to be focused on, and the same obstacle cannot be photographed repeatedly. On this basis, in order to improve the setting accuracy of temporary shooting points, this embodiment sets obstacles that are not repeatedly marked and whose height is greater than a threshold as temporary shooting points.

[0072] See also Figure 3 This embodiment provides an optional implementation of S103, including:

[0073] S201: Mark obstacles;

[0074] Specifically, the size of the shadow area has been estimated in the above shadow recognition process, so the centroid of the shadow area can be calculated according to the size of the shadow area, and the straight-line distance S1 between the centroid and the current shooting point and the straight-line distance S2 between the centroid and the previous shooting point can be calculated.

[0075] Finally, based on the straight-line distance S1 between the centroid and the current shooting point and the straight-line distance S2 between the centroid and the previous shooting point, mark the obstacle. The obstacle can be marked as (Pn, m, S1, S2), where Pn is the nth row of flight trajectory in the east-west direction, and m is the mth shooting point on Pn.

[0076] S202: Determine whether the obstacle is repeatedly marked;

[0077] Specifically, check in the recorded obstacle mark records (Pn, m-1, ...) whether S1 is equal to S2 in (Pn, m, S1, S2). If so, it means that the obstacle is repeated, so (Pn, m, S1, S2) is marked as repeated.

[0078] If the obstacle is repeatedly marked, then S203 is executed: the obstacle is not set as a temporary shooting point;

[0079] If the obstacle is not repeatedly marked, execute S204: estimate the height and width of the obstacle according to the solar altitude angle and the size of the shadow area;

[0080] The size of the shadow area has been estimated in the above shadow recognition process. Therefore, the height and width of the obstacle can be estimated here based on the solar altitude angle and the size of the shadow area. It should be noted that the height and width of the obstacle mentioned here are actually the height and width of the obstacle outline seen by the drone from the north.

[0081] Specifically, the solar azimuth and altitude angles are calculated based on the drone's aerial photography time and location coordinates. Based on optical principles and the size of the shadow area, the height and width of the obstacle outline are inferred. The specific calculation method is an existing technology and will not be described in detail. Figure 4 , the height of p1, that is, the height of the obstacle contour, can be deduced from p1`, and the width of the obstacle contour, p1~p2, can be deduced from the lengths of p1` and p2`.

[0082] S205: When the height of the obstacle is greater than the threshold, the obstacle is set as a temporary shooting point.

[0083] S104: photographing images at the temporary photographing point, wherein the number of images photographed at the temporary photographing point is greater than the number of images photographed at the photographing point.

[0084] Specifically, the temporary flight path of the UAV at the temporary shooting point is determined based on the estimated height and width of the obstacle, wherein the radius of the temporary flight path around the obstacle is determined based on the width of the obstacle, and the flight altitude of the UAV is determined based on the height of the obstacle. It should be noted that the UAV cannot fly too high, as this will result in unclear images being taken, and the UAV cannot fly too low, as this will not allow for clear images of the shadow of the obstacle. Therefore, the flight altitude of the UAV needs to be determined based on the obstacle height, the sun altitude angle, and the parameters of the UAV image acquisition device, such as Figure 5 The h in the figure is an example of the flying altitude of the drone.

[0085] Control the drone to fly around the obstacle along a temporary flight path, and take images when the preset conditions are triggered during the flight of the drone.

[0086] The preset conditions may be an 80% heading overlap rate and a 70% lateral overlap rate, that is, during the flight of the drone, if it is detected that the current captured image has an 80% heading overlap rate and a 70% lateral overlap rate with the previous captured image, the preset conditions are determined to be triggered and the shooting is automatically triggered.

[0087] After finishing shooting at the temporary shooting point, continue to control the drone to continue flying and shooting along the pre-set survey path.

[0088] It can be seen that the present embodiment discloses a method for planning a UAV survey path, which sets shooting points on a pre-set survey path, sets temporary shooting points according to obstacles obtained after obstacle detection during the flight of the UAV, shoots a small number of images at pre-set shooting points, and only takes additional photos at temporary shooting points, thereby shortening the overall flight and shooting time of the UAV, reducing the number of images taken by the UAV as a whole, and thus shortening the later three-dimensional modeling time, while increasing the local modeling accuracy.

[0089] Based on the UAV survey path planning method disclosed in the above embodiment, this embodiment correspondingly discloses a UAV survey path planning device, which is applied to the UAV controller, see Figure 6 , the device comprises:

[0090] The first shooting unit 100 is used to control the UAV to fly along a preset survey path and to shoot images at preset shooting points;

[0091] The obstacle detection unit 200 is used to perform obstacle detection on the captured image;

[0092] A temporary shooting point setting unit 300, used to set a temporary shooting point according to the detected obstacle;

[0093] The second shooting unit 400 is used to shoot images at the temporary shooting point, and the number of images shot at the temporary shooting point is greater than the number of images shot at the shooting point.

[0094] Optionally, the obstacle detection unit 200 is specifically used to:

[0095] Perform shadow recognition on the captured image;

[0096] When a shadow area is identified, estimating the size of the shadow area;

[0097] Make sure the obstacle is detected.

[0098] Optionally, the temporary shooting point setting unit 300 includes:

[0099] An obstacle marking subunit, used for marking the obstacle;

[0100] A repeated marking judgment subunit, used to judge whether the obstacle is repeatedly marked;

[0101] A shadow height estimation subunit, configured to estimate the height and width of the obstacle according to the sun altitude angle and the size of the shadow area if the obstacle is not repeatedly marked;

[0102] The temporary shooting point setting subunit is used to set the obstacle as the temporary shooting point when the height of the obstacle is greater than a threshold.

[0103] Optionally, the obstacle marking subunit is specifically used to:

[0104] Calculate the centroid of the shadow area according to the size of the shadow area;

[0105] Calculating the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point;

[0106] The obstacle is marked based on the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point.

[0107] Optionally, the second shooting unit 400 is specifically used to:

[0108] Determine a temporary flight path of the UAV at the temporary shooting point according to the height and width of the obstacle;

[0109] The drone is controlled to fly around the obstacle according to the temporary flight path, and images are captured when a preset condition is triggered during the flight of the drone.

[0110] The present embodiment discloses a UAV survey path planning device, which sets shooting points on a preset survey path, sets temporary shooting points according to obstacles obtained after obstacle detection during the flight of the UAV, shoots a small number of images at the preset shooting points, and only takes additional photos at the temporary shooting points, thereby shortening the overall flight and shooting time of the UAV, reducing the number of images taken by the UAV as a whole, thereby shortening the later three-dimensional modeling time, and increasing the local modeling accuracy.

[0111] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0112] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0113] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0114] The above-mentioned embodiments may be combined arbitrarily. For the above description of the disclosed embodiments, the features recorded in the embodiments in this specification may be replaced or combined with each other, so that professional and technical personnel in this field can implement or use the present application.

[0115] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for planning a UAV survey path, characterized in that: Applied to a drone controller, the method comprises: Control the drone to fly along a pre-set survey path and take images at pre-set shooting points; Perform obstacle detection on the captured image; Setting a temporary shooting point according to the detected obstacle; wherein an obstacle that has not been repeatedly marked and whose height is greater than a threshold is set as the temporary shooting point; Images are captured at the temporary capturing point, and the number of images captured at the temporary capturing point is greater than the number of images captured at the capturing point.

2. The method according to claim 1, characterized in that The performing obstacle detection on the captured image includes: Perform shadow recognition on the captured image; When a shadow area is identified, estimating the size of the shadow area; Make sure the obstacle is detected.

3. The method according to claim 2, characterized in that The step of setting a temporary shooting point according to the detected obstacle includes: marking said obstruction; Determining whether the obstacle is repeatedly marked; If the obstacle is not repeatedly marked, estimating the height and width of the obstacle according to the sun altitude angle and the size of the shadow area; When the height of the obstacle is greater than a threshold, the obstacle is set as the temporary shooting point.

4. The method according to claim 3, characterized in that Marking of said obstacles, including: Calculate the centroid of the shadow area according to the size of the shadow area; Calculating the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point; The obstacle is marked based on the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point.

5. The method according to claim 3, characterized in that: Taking an image at the temporary taking point includes: Determine a temporary flight path of the UAV at the temporary shooting point according to the height and width of the obstacle; The drone is controlled to fly around the obstacle according to the temporary flight path, and images are captured when a preset condition is triggered during the flight of the drone.

6. A UAV survey path planning device, characterized in that: Applied to a drone controller, the device comprises: The first shooting unit is used to control the UAV to fly along a preset survey path and to shoot images at preset shooting points; An obstacle detection unit, used to perform obstacle detection on the captured image; A temporary shooting point setting unit, used to set a temporary shooting point according to the detected obstacle; wherein an obstacle that is not repeatedly marked and has a height greater than a threshold is set as the temporary shooting point; The second shooting unit is used to shoot images at the temporary shooting point, and the number of images shot at the temporary shooting point is greater than the number of images shot at the shooting point.

7. The device according to claim 6, characterized in that The obstacle detection unit is specifically used for: Perform shadow recognition on the captured image; When a shadow area is identified, estimating the size of the shadow area; Make sure the obstacle is detected.

8. The device according to claim 7, characterized in that The temporary shooting point setting unit comprises: An obstacle marking subunit, used for marking the obstacle; A repeated marking judgment subunit, used to judge whether the obstacle is repeatedly marked; A shadow height estimation subunit, configured to estimate the height and width of the obstacle according to the sun altitude angle and the size of the shadow area if the obstacle is not repeatedly marked; The temporary shooting point setting subunit is used to set the obstacle as the temporary shooting point when the height of the obstacle is greater than a threshold.

9. The device according to claim 8, characterized in that The obstacle marking subunit is specifically used for: Calculate the centroid of the shadow area according to the size of the shadow area; Calculating the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point; The obstacle is marked based on the straight-line distance between the centroid and the current shooting point and the straight-line distance between the centroid and the previous shooting point.

10. The device according to claim 8, characterized in that The second shooting unit is specifically used for: Determine a temporary flight path of the UAV at the temporary shooting point according to the height and width of the obstacle; The drone is controlled to fly around the obstacle according to the temporary flight path, and images are captured when a preset condition is triggered during the flight of the drone.

Images