A multi-uav cooperative shooting system and method for natural driving data collection

Abstract

The present application relates to a kind of multi-unmanned aerial vehicle cooperative shooting system and method for natural driving data acquisition, comprising the steps S1, the shooting limit of single unmanned aerial vehicle is set;Step S2, according to the lane type of selected shooting section, the setting strategy corresponding to lane type is executed, until the total shooting range of unmanned aerial vehicle can completely cover selected shooting section and the required number of unmanned aerial vehicle is minimum, while generating the shooting end point of each unmanned aerial vehicle;Step S3, with the shortest total track as possible, with the smallest risk as possible, for each unmanned aerial vehicle, the shooting end point is assigned, and the optimal flight path of each unmanned aerial vehicle to reach corresponding shooting end point is planned;Step S4, unmanned aerial vehicle flies to shooting end point with the optimal flight path and shoots.The present application provides more abundant data support for automatic driving simulation test.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle shooting, in particular to a multi-unmanned aerial vehicle cooperative shooting system and method for natural driving data collection. BACKGROUND

[0002] At present, with the rapid development of automatic driving technology, it is an essential link to carry out simulation test for automatic driving cars before they go on the road. Scene-based automatic driving simulation test is currently a mainstream simulation test method, and the extraction of simulation test scenes highly depends on the integrity and accuracy of natural driving data. Using unmanned aerial vehicles to take aerial photos of traffic sections to obtain traffic images and extract natural driving data from them is one of the main means of collecting natural driving data. However, due to the limitation of the maximum flight height of unmanned aerial vehicles in many major traffic sections by national laws, the range of unmanned aerial vehicle shooting is narrow, which leads to the short shooting time of vehicles and the inability to obtain the relevant information required for extracting natural driving data. However, it is difficult to use multiple unmanned aerial vehicles to shoot at the same time, and it is necessary to ensure that the position of each unmanned aerial vehicle is reasonable and the flight path of the unmanned aerial vehicle is reasonable without collision. At the same time, since the purpose of video collection is to extract natural driving data, only the videos shot by all unmanned aerial vehicles at the same time are effective, which is also a problem to be solved for the joint shooting of multiple unmanned aerial vehicles serving natural driving data collection. SUMMARY

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a multi-unmanned aerial vehicle cooperative shooting system and method for natural driving data collection, which is used to solve the technical problems of small shooting range of a single unmanned aerial vehicle and unreasonable shooting position and path planning of multiple unmanned aerial vehicles.

[0004] To achieve the above-mentioned purpose, the present application provides a multi-unmanned aerial vehicle cooperative shooting method for natural driving data collection, comprising:

[0005] Step S1, setting the shooting limit of a single unmanned aerial vehicle;

[0006] Step S2, according to the lane type of the selected shooting section, executing the setting strategy corresponding to the lane type until the total shooting range of the unmanned aerial vehicles can completely cover the selected shooting section and the required number of unmanned aerial vehicles is minimized, and generating the shooting endpoints of each unmanned aerial vehicle;

[0007] Step S3, assigning the shooting endpoint to each unmanned aerial vehicle and planning the optimal flight path of each unmanned aerial vehicle to reach the corresponding shooting endpoint with the shortest total flight path and the smallest risk degree as the target;

[0008] Step S4, the unmanned aerial vehicle flies to the shooting endpoint with the optimal flight path to shoot.

[0009] The present invention also provides a multi-UAV collaborative shooting system for natural driving data acquisition, including UAVs and a ground terminal. The ground terminal obtains the shooting endpoint and the optimal flight path according to the method described above, and transmits the shooting endpoint and the optimal flight path to the UAV. After the UAV flies to the shooting endpoint according to the optimal path, it shoots according to the instructions of the ground terminal and transmits the shooting data to the ground terminal.

[0010] The beneficial effects of this invention are as follows: Compared to using a single drone to collect traffic videos from high altitude, this invention, by employing multiple drones, can obtain a wider field of view, increase the appearance time of the studied vehicle in the video, and allow researchers to extract a greater amount of natural driving data. This enables the discovery of more autonomous driving simulation test scenarios and provides richer data support for autonomous driving simulation testing. This invention adapts to the actual characteristics of different traffic segments, allowing it to be applied in various locations while ensuring data accuracy and avoiding the use of inaccurate data to extract natural driving data. Step S2 aims to cover the required traffic segments with as few drones as possible, reducing the difficulty and cost of filming. Step S3 enables the drone path planning method to shorten data collection time, reduce drone power consumption, and improve the efficiency of natural driving data collection. Attached Figure Description

[0011] Figure 1 This is a simplified flowchart of the steps in Example 1;

[0012] Figure 2 This is a schematic diagram illustrating the calculation method for the actual length corresponding to the shooting range at any shooting height in Example 1.

[0013] Figure 3 This is a flowchart illustrating the steps involved in generating the endpoint image in Example 1.

[0014] Figure 4 This is a schematic diagram of the initial positioning results generated by the drone when the road segment being photographed only includes straight sections, as shown in Example 1.

[0015] Figure 5 This is a schematic diagram of the initial positioning result generated by the drone when the road segment being photographed in Example 1 only includes straight sections.

[0016] Figure 6 This is a schematic diagram of the initial positioning result generated by the drone when the road section being photographed only includes curves, as shown in Example 1.

[0017] Figure 7 This is a schematic diagram of another initial positioning result generated by the drone when the road section being photographed in Example 1 only includes curves;

[0018] Figure 8 FIG. 1 is a schematic diagram of the results of the generation of a UAV shooting initial point when a shooting road section includes both a straight road and a curved road according to an embodiment of the present application;

[0019] Figure 9 FIG. 2 is a flowchart of a path planning algorithm according to an embodiment of the present application;

[0020] Figure 10 FIG. 3 is a flowchart of a method according to an embodiment of the present application;

[0021] Figure 11 FIG. 4 is a schematic diagram of a system structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application is described herein with reference to particular embodiments for purposes of description, but the present application can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these described embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. In the drawings, like reference numerals refer to like elements throughout the several views.

[0023] As shown in FIG. 1, the present embodiment provides a multi-UAV cooperative shooting method for natural driving data collection, including: Figures 1-11

[0024] Step S1, setting a shooting limit for a single UAV;

[0025] Step S2, according to the lane type of the selected shooting road section, performing a setting strategy corresponding to the lane type until the total shooting range of the UAVs can completely cover the selected shooting road section and the number of UAVs required is minimized, while generating shooting endpoints for each UAV;

[0026] Step S3, assigning a shooting endpoint to each UAV with the goal of the shortest total flight path and the smallest risk, and planning an optimal flight path for each UAV to reach the corresponding shooting endpoint;

[0027] Step S4, the UAV flies to the shooting endpoint according to the optimal flight path to perform shooting.

[0028] ​Compared with using a single unmanned aerial vehicle to collect traffic video from high altitude, the present application can obtain a larger field of view, improve the appearance time of the vehicle in the video, and enable researchers to extract a larger amount of natural driving data, so as to mine more automatic driving simulation test scenarios and provide more abundant data support for automatic driving simulation test. In the present application, the actual characteristics of different traffic sections are adapted, so that the application can be applied in different places while ensuring the accuracy of the data and avoiding the use of unrealistic data to extract natural driving data. When the unmanned aerial vehicle finishes shooting the initial point, the unmanned aerial vehicle path planning method of step S4 can shorten the data collection time, reduce the power consumption of the unmanned aerial vehicle, and improve the efficiency of natural driving data collection.

[0029] In this example, the ground terminal and multiple unmanned aerial vehicles are wirelessly connected for unmanned aerial vehicle control, the ground terminal accesses the Internet, the unmanned aerial vehicle has a camera, and the unmanned aerial vehicle collects traffic information through the camera.

[0030] Before step S1, the ground terminal calls a third-party satellite map platform interface to obtain road composition and coordinate information of the road section to be shot, and an operator frames the rectangular traffic section range to be shot. On the basis of having selected the traffic section range to be shot, the operator further selects the lane to be shot, i.e., the shooting road section, and sets the lane type as one of a single-lane straight lane, a multi-lane straight lane, a single-lane curved lane, a multi-lane curved lane, or a mixed lane containing a straight lane and a curved lane. How to frame the traffic section range, the shooting road section, and the judgment of the lane type can be implemented by using existing technology.

[0031] The shooting restrictions of the unmanned aerial vehicle include the maximum shooting height, the maximum shooting range, the relationship between the shooting range and the shooting height of the unmanned aerial vehicle, and the positional relationship between the unmanned aerial vehicle and the shooting range.

[0032] In this example, the shooting range is a rectangle, and the intersection of the diagonal lines of the shooting range coincides with the orthographic projection of the unmanned aerial vehicle, i.e., the unmanned aerial vehicle is located directly above the shooting range. The lower the shooting height of the unmanned aerial vehicle, the smaller the shooting range. Specifically, as shown in Figure 2 The shooting range of the unmanned aerial vehicle at any shooting height is calculated as follows:

[0033] First, the longer side of the unmanned aerial vehicle at the highest height is long, and the shorter side is wide. A rectangle with the long and wide as the shooting range of the unmanned aerial vehicle at the highest height on the ground is taken, and the four vertices of the rectangle (shooting range) are denoted as A, B, C, and D (as shown in Figure 2

[0034] ​Secondly, the intersection O of the diagonal line of the rectangle is the projection point of the unmanned aerial vehicle on the ground at this shooting height, a straight line perpendicular to the ground is drawn at the O point, and a point F above the straight line is taken at the maximum shooting height of the unmanned aerial vehicle, F is the position of the unmanned aerial vehicle at the maximum shooting height;

[0035] Then, the line FO connecting the F point and the O point is taken as the high line of the shooting range of the unmanned aerial vehicle, and the length thereof represents the maximum shooting height H FO .

[0036] Next, the F point is connected with the four vertices A, B, C and D of the rectangle, and the shooting range of the unmanned aerial vehicle at the maximum shooting height is obtained.

[0037] Finally, at the position of the maximum shooting height of the unmanned aerial vehicle, an arbitrary point G is taken along the high line downwards, FG is taken as the high line at an arbitrary shooting height, and FG is ensured to be greater than 0 and less than the maximum shooting height of the unmanned aerial vehicle. The length of FG is taken as an arbitrary shooting height H FG . A new plane parallel to the plane where the O point is located is drawn through the G point, intersects FA at point A1, intersects FB at point B1, intersects FC at point C1, and intersects FD at point D1. The new plane formed by connecting A1, B1, C1 and D1 is taken as the ground at an arbitrary shooting height, and the shooting range at an arbitrary shooting height H FG is obtained.

[0038] The longer side of the shooting range of the unmanned aerial vehicle is M, and the shorter side is N. The relationship between the shooting range and the shooting height is as follows:

[0039]

[0040] Wherein, M FG is the longer side of the shooting range of the unmanned aerial vehicle at an arbitrary shooting height, N FG is the shorter side, M FO is the longer side of the shooting range of the unmanned aerial vehicle at the maximum shooting height, N FO is the shorter side of the shooting range of the unmanned aerial vehicle at the maximum shooting height, H FG is the height of the unmanned aerial vehicle at an arbitrary shooting height, H FO is the maximum shooting height of the unmanned aerial vehicle.

[0041] According to the pre-set maximum shooting height and maximum shooting range of the unmanned aerial vehicle, the shooting range of the unmanned aerial vehicle at an arbitrary shooting height can be calculated. In other words, the length and width of the shooting range at an arbitrary height are obtained, and the shooting range of the unmanned aerial vehicle at an arbitrary shooting height is obtained, which prepares for the subsequent calculation of the continuous shooting range of multiple unmanned aerial vehicles.

[0042] The lane types of the required shooting section include straight lanes, curved lanes and both straight lanes and curved lanes, and the corresponding setting strategies include:

[0043] 1) The lane type of the photographed road section is a straight lane: when the width of the photographed road section is less than the width (shorter side) of the preset UAV photographing range, the long side of the UAV photographing range is perpendicular to the wide side of the lane, as shown in FIG. 1A; when the width of the photographed road section is greater than the width of the preset UAV photographing range but less than the length (longer side) of the UAV photographing range and has a surplus, the long side of the UAV photographing range is perpendicular to the wide side of the lane, that is, all the positions of the UAVs are horizontally rotated by 90 degrees relative to the above case, as shown in FIG. 1B. Figure 4 Figure 5

[0044] Then, based on the above rules, the photographing starting point of the first UAV is set on the center line of the total width of the lane, the symmetric axis of the UAV photographing range is parallel to the center line of the lane and coincides with the center line of the lane, and the left side of the photographed road section is located in the photographing range of the first UAV, that is, the edges of the left part of the lane range are all located in the photographing range of the first UAV, and then the UAVs are sequentially added from left to right until the total photographing range of the multiple UAVs completely covers the range of the road section to be photographed, and the photographing ranges of adjacent two UAVs have an overlapping part, as shown in FIGS. 1C and 1D. In this example, the ending point of the UAV is moved left along the center line of the lane or the center line of the lane width, so that it can photograph a range with a specified width outside the left edge, that is, the left side of the photographing range of the first UAV exceeds the left side of the photographed road section by a specified width, and this way of reserving a surplus photographing range ensures that the left part of the lane range can be completely photographed. In this example, the specified width is 1 meter. Figure 4 Figure 5

[0045] Whenever the next UAV ending point is set, it is determined whether the photographing range of the current UAV has completely covered the photographed road section, that is, whether it can photograph the selected lane and a range of 1 meter outside the lane. If it can completely cover, the ending point is generated; if it cannot completely cover, the next UAV ending point is generated.

[0046] The flight ending point of the next UAV is generated at the flight ending point of the previous UAV, and then is moved right along the center line of the lane or the center line of the lane width until the overlapping part of the next UAV and the previous UAV has a specified width, and then the movement is stopped, and the stopping point is taken as the photographing starting point of the next UAV. In this example, the specified width is 1 meter.

[0047] The subsequent UAVs generate the ending points in the same way until the ending points are completely generated.

[0048] ​​​​2) The lane type of the shooting section is a curve: the center point of the UAV shooting range is located on the center line of the shooting section, and the point where the center point of the UAV shooting range coincides with the center line of the shooting section is set as the base point. When the width of the shooting section is less than the preset width of the UAV shooting range, the long side of the shooting range is parallel to the tangent line at the base point of the center line of the shooting section, as shown in FIG. 8; when the width of the shooting section is greater than the preset width of the UAV shooting range, but less than the length of the UAV shooting range, and there is a surplus, the wide side of the shooting range is parallel to the tangent line at the base point of the center line of the shooting section, as shown in FIG. 9. Figure 6 Figure 7

[0049] Then, based on the above rules, the shooting initial point of the first UAV is set on the center line of the total width of the shooting section, and one end of the shooting section is located in the shooting range of the first UAV. Then, the UAV is added, and the newly added UAV takes the position of the previous UAV as the starting point and moves along the center line of the shooting section to the other end of the shooting section, while the UAV rotates to ensure that the long side / wide side of the UAV shooting range is parallel to the tangent line at the base point of the center line of the shooting section, until the shooting ranges of the multiple UAVs completely cover the range of the shooting section to be shot. As shown in FIGS. 10 and 11. Figure 6 Figure 7

[0050] Before the next UAV end point is set, it is judged whether the current UAV shooting range has completely covered the shooting section, that is, whether the selected lane and the range of 1 meter outside the lane can be shot. If it can be completely covered, the end point generation is completed; if it cannot be completely covered, the next UAV end point generation is performed.

[0051] After the newly added UAV moves, the overlapping part with the previous UAV ensures that the shooting section can be shot. In this example, when the total shooting range boundary distance of the newly added UAV (the next UAV) and the previous UAV is a specified width, the movement is stopped, and the stopping point is taken as the shooting initial point of the newly added UAV. The specified width is at least 1 meter.

[0052] The subsequent UAV generates the end point according to the same rule until the end point generation is completed.

[0053] 3) The shooting section includes both straight and curved lanes: the range covered by the multiple UAVs is set as the circumscribed rectangular range of the shooting section, and the multiple UAVs are added from left to right and from front to back until the shooting range of the UAV covers the shooting section. As shown in FIGS. 12 and 13. Figure 8 ​​​​As shown, in this example, the wider side of the drone's shooting range is parallel to the wider side of the rectangular shooting area, and the longer side of the drone's shooting range is parallel to the longer side of the rectangular shooting area. Alternatively, the wider side of the drone's shooting range can be parallel to the longer side of the rectangular shooting area, and vice versa.

[0054] It should be noted that all initial shooting points are on the same horizontal plane. Front, back, left, and right are relative to the horizontal plane where the initial shooting points are located. The front, back, left, and right directions are not limited to specific directions.

[0055] First, the initial shooting points are generated from left to right. In this example, if... Figure 6 As shown, the first initial shooting point is generated at the top left corner of the rectangular area of ​​the traffic segment to be photographed. This initial shooting point is moved backward until the front boundary of the drone's shooting range is 1 meter away from the rectangular area of ​​the traffic segment to be photographed, at which point the movement stops. Then, this initial shooting point is moved to the right until the left boundary of the drone's shooting range is 1 meter away from the rectangular area of ​​the traffic segment to be photographed, at which point the movement stops. The second initial shooting point is generated from the position of the first initial shooting point and moved to the right until the shooting range of the second drone overlaps with the shooting range of the first drone by a width of 1 meter, at which point the movement stops. Subsequent initial shooting points are generated and moved in accordance with this rule. Before generating each new initial shooting point, it is determined whether the right boundary of the shooting range corresponding to the last generated initial drone shooting point exceeds the rectangular range of the traffic segment to be shot by 1 meter. If it does not exceed the right boundary, the generation of initial shooting points continues according to the above rules. If it does exceed the right boundary, it is further determined whether the rear boundary of the shooting range corresponding to the last generated initial drone shooting point exceeds the rectangular range of the traffic segment to be shot by 1 meter. If its rear boundary also exceeds the rectangular range of the traffic segment to be shot by 1 meter, the generation of initial shooting points ends. If its rear boundary does not exceed the rectangular range of the traffic segment to be shot by 1 meter, the rectangular range of the traffic segment to be shot is not fully covered, and it is necessary to continue generating initial drone shooting points. At this time, the generation of initial drone shooting points from front to back begins. Specifically, multiple initial drone shooting points are generated simultaneously and the initial shooting points are moved simultaneously. The detailed method is as follows:

[0056] The left-to-right generated shooting initial point is recorded as the first row of unmanned aerial vehicle shooting initial point (with multiple unmanned aerial vehicle shooting initial points), and one unmanned aerial vehicle shooting initial point is generated at the first row of unmanned aerial vehicle shooting initial point, that is, multiple unmanned aerial vehicle shooting initial points are generated at the same time. The newly generated unmanned aerial vehicle shooting initial point is recorded as the second row of unmanned aerial vehicle shooting initial point, and the second row of unmanned aerial vehicle shooting initial point moves backward until the second row of unmanned aerial vehicle shooting range and the first row of unmanned aerial vehicle shooting range overlap by 1 meter. Then, it is judged whether the rear boundary of the second row of unmanned aerial vehicle shooting range exceeds the rectangular range of the traffic section to be shot by 1 meter. If it exceeds, the shooting initial point generation is completed; if it does not exceed, the third row of unmanned aerial vehicle shooting initial point is generated in the manner of generating the second row of unmanned aerial vehicle shooting initial point, until the rear boundary of the shooting range exceeds the rectangular range of the shooting traffic section by 1 meter. The unmanned aerial vehicle shooting initial point of the next row is generated at the unmanned aerial vehicle shooting initial point of the previous row. Of course, the unmanned aerial vehicle shooting initial point of the next row can also be generated at any unmanned aerial vehicle shooting initial point of the relatively front row.

[0057] It should be noted that the unmanned aerial vehicle corresponding to the first row of unmanned aerial vehicle shooting initial point is the first row of unmanned aerial vehicle, the unmanned aerial vehicle corresponding to the second row of unmanned aerial vehicle shooting initial point is the second row of unmanned aerial vehicle, and so on. There can be a third row of unmanned aerial vehicle, a fourth row of unmanned aerial vehicle, and an Nth row of unmanned aerial vehicle.

[0058] In this example, the shooting limit of the unmanned aerial vehicle also includes setting the maximum number of unmanned aerial vehicles to be dispatched. In the setting strategy, when the number of newly added unmanned aerial vehicles is greater than the maximum number of unmanned aerial vehicles, the shooting initial point generation fails, and step S2 is re-executed. In other words, during the generation of the unmanned aerial vehicle shooting initial point, if the number of unmanned aerial vehicles reaches the upper limit and still cannot completely cover the rectangular range of the required shooting section, that is, the condition of the initial shooting initial point generation is not met, the shooting initial point generation fails.

[0059] The corresponding setting strategy is executed, the shooting range of the unmanned aerial vehicle covers the shooting section, the shooting initial point is generated, then the shooting height of the unmanned aerial vehicle is lowered, and the number of shooting initial points is used as the number of unmanned aerial vehicles to be dispatched. The corresponding setting strategy is executed again until after the height is lowered for a certain time, the total shooting range cannot cover the shooting section, the iteration stops, and the shooting initial point generated in the second last time is taken as the shooting terminal point.

[0060] Specifically, if the preliminary generation of the initial shooting point of the UAV can be completed after the UAV reduces the height, the height is continuously reduced, and the corresponding setting strategy of the preliminary generation of the initial shooting point of the UAV is repeated until the preliminary generation of the initial shooting point of the UAV cannot be completed after the UAV reduces the flight height once using the updated number of UAVs, that is, the corresponding number of UAVs cannot completely cover the shooting section, the iteration stops, and the distribution of the initial shooting point of the UAV before the last time the flight height of the UAV is reduced is taken as the final shooting endpoint coordinates of the UAV with the least number of UAVs and the highest shooting accuracy, that is, the shooting endpoint is formed after the initial shooting point is adjusted. Under the condition that the same number of UAVs can completely cover the shooting section, the height of the UAV is reduced, the shooting range of each UAV is reduced, and the flight endpoint of the UAV is adjusted twice to improve the shooting accuracy.

[0061] wherein the step length of the shooting height of the UAV can be flexibly set according to requirements.

[0062] Before the path of the UAV is optimized, the pairing of the starting point and the shooting endpoint (hereinafter referred to as the endpoint) needs to be determined first, that is, the starting point and the shooting endpoint of the UAV are paired and combined, and then the optimal flight path is obtained by joint optimization based on the wolf swarm algorithm under the condition that the UAVs do not collide.

[0063] Specifically, all possible starting point-endpoint combination schemes are obtained according to permutation and combination, the length d of the straight line segment from the starting point to the endpoint of each combination scheme is calculated s , and then the average absolute deviation d s of d a of each combination scheme is calculated.

[0064]

[0065] wherein d is the length of the straight line segment from the i-th starting point to the endpoint of the combination scheme, is the average value of the lengths of the straight line segments from all starting points to the endpoint of the combination scheme, and the starting point-endpoint combination scheme with the smallest d a is selected as the starting point and the endpoint of the UAV path optimization, and D represents D tracks.

[0066] The flight path of the UAV is optimized. First, the parameters are initialized, the starting point of the UAV is set as according to the pairing scheme of the starting point and the endpoint, the endpoint of the UAV is set as , and N DThe path combination scheme is regarded as a smart group, and the single path combination scheme is regarded as a smart individual. Each path combination scheme includes D flight paths from a specified starting point to a specified ending point according to the starting point and ending point pairing result. After connecting the starting point and the ending point of each flight path, n+1 equal divisions are made in the x-axis direction, each division is a flight path segment, n is artificially set, the equal division points are x pid (p represents the pth smart individual, p = 1, 2, …, N D ; i represents the ith flight path, i = 1, 2, …, D; d represents the d-dimensional solution space), the corresponding flight path point p pid has coordinates (x pid , y pid , z pid ), and the requirements to be met are that two flight paths will not cause a UAV collision during flight, that is, the distance between any flight path point on a flight path and any flight path point on another flight path is greater than 2R D , and R D is a safety flight radius set according to the specific model of the UAV.

[0067] In the path optimization process, the flight path is optimized to maximize the fitness value Y of the smart individual, and the expression of Y is:

[0068]

[0069] wherein W e , W s , and W l are weight coefficients corresponding to the factors, used to balance the importance of different factors; e ij is the length of the jth flight path segment of the ith flight path of the smart individual, s ij is the risk degree of the jth flight path segment of the ith flight path of the smart individual, and the risk degree of a flight path segment is the reciprocal of the minimum distance of the flight path segment to all other flight path segments; l a is the average absolute deviation of all flight paths of a single smart individual:

[0070]

[0071] wherein l i is the average length of all flight paths of a single smart individual, and l is the total length of the ith flight path:

[0072]

[0073] As shown in FIG. 1, the optimized path includes: Figure 9

[0074] Step S301, smart group parameter initialization;

[0075] ​Step S302: Calculate the fitness value of each intelligent individual's location, and select the intelligent individual with the highest fitness value as the leader, followed by the individual with the next highest fitness value. n Each intelligent individual acts as a navigator; the leader initiates a summoning action, gathering all intelligent individuals except the leader and navigators to its current location; the navigators are responsible for dynamic searching, continuously exploring forward until the fitness of a navigator surpasses that of the leader, or the maximum number of navigator searches T is reached. max ;

[0076] T n The value can be [N] D / (α+1),N D A random integer between N and α], N D Let α be the total number of intelligent individuals, and let α be the proportion factor of navigators, which is the proportion of navigators among intelligent individuals.

[0077] During the dynamic search process, the fitness value Y of the navigator q's location is... q Y q Fitness score Y of the leader leader For comparison, if fitness Y q >Y leader Then we can obtain Y leader =Y q That is, the navigator takes the leader's place and sends a summons signal to intelligent individuals ranked lower in fitness, thus the navigator becomes the leader; if fitness Y q <Y leader The navigator will move in the h direction with a fixed step size. If the navigator moves forward one step, and takes its current position as the starting point, then the navigator's position after moving along the z direction (z = 1, 2, ..., h) is... Will be updated to:

[0078]

[0079] x is the navigator's search movement step size. qid This represents the position the navigator was in before moving.

[0080] Step S303: The intelligent individual m with the lower fitness value responds to the leader's call and moves towards the leader's location by a step size. Rapid movement. Then, the position of the intelligent individual m, ranked lower in fitness, in d-dimensional space during the (k+1)th iteration. Will be updated to:

[0081]

[0082] in, denotes the position of the leader after the kth iteration in the d-dimensional space, denotes the position of the intelligent individual m with the worst fitness in the d-dimensional space, denotes that the intelligent individual m with the worst fitness gradually approaches the leader, is the moving step of the UAV individual with the worst ranking in response to the calling behavior of the navigator.

[0083] If the fitness of the intelligent individual m with the worst fitness is better than that of the leader during the fast moving process, the intelligent individual m with the worst fitness initiates the calling behavior instead of the leader, otherwise, the intelligent individual m with the worst fitness continues to respond to the calling signal until the distance d ms between the intelligent individual m with the worst fitness and the leader is less than the distance determination value d near , all intelligent individuals with the worst fitness start to change from the behavior of responding to the calling signal to the behavior of centralized search, otherwise, the intelligent individuals with the worst fitness continue to move towards the optimal solution;

[0084] The distance determination factor calculation formula is as follows:

[0085]

[0086] wherein w is the distance determination factor, the size of which determines the convergence speed of the algorithm, under the same conditions, the greater the value of the determination factor, the faster the convergence speed of the algorithm. However, if w is too large, the centralized search behavior of the intelligent group will be affected, resulting in the inability to enter the centralized search state; D represents the number of spatial dimensions (also the total number of flight paths).

[0087] Min d and Max d respectively represent the minimum value and the maximum value of the d-dimensional space to be optimized.

[0088] In the centralized search behavior, the position of the leader closest to the optimal solution is represented as the position of the optimal solution, and for the kth generation of UAV individuals, the position of the optimal solution in the d-dimensional space can be represented as Therefore, when the kth iteration is performed, the position of the intelligent individual m in the d-dimensional space after the centralized search can be represented as:

[0089]

[0090] wherein λ is a random constant between [-1, 1], is the moving step of the intelligent individual with the worst fitness in initiating the centralized search behavior. When the intelligent individual m with the worst fitness detects that the fitness is greater than the fitness of the original position during the centralized search, the position of the intelligent individual m is updated, otherwise, the position of the intelligent individual m with the worst fitness does not change.

[0091] There is a certain relationship between the steps of the three intelligent behaviors in the V-space, as shown in the following formula:

[0092]

[0093] wherein, is the search step of the navigator, is the movement step of the drone individual ranked last in response to the call behavior of the navigator, is the movement step of the intelligent individual ranked last in fitness to initiate the centralized search behavior, and L represents the step factor of the intelligent behavior, which affects the refinement degree of the search endpoint.

[0094] In step S304, the position of the leader is selected as the position of the optimal solution, the intelligent individuals with poor fitness values perform centralized search on the optimal solution, the positions of the intelligent individuals participating in the centralized search are updated, and if the updated position x mid of the intelligent individual exceeds the range of the solution space, the position is set as the boundary value.

[0095] In step S305, the position of the leader is updated according to the leader generation rule, and then the intelligent group is updated, R intelligent individuals with the worst fitness are deleted, and R intelligent individuals are randomly generated.

[0096] Specifically, the R intelligent individuals with low fitness will be eliminated due to the competition mechanism, and R intelligent individuals are randomly generated. When generating a new intelligent individual, the change value of the optimal fitness of this iteration is determined. If the change value of the optimal fitness is greater than Y u (Y u is the fitness change threshold of unknown combination exploration, which is artificially set), when generating a new intelligent individual, the pairing combination of the start point and the end point that has not been generated is generated first. If the change value of the optimal fitness is less than Y u , when generating a new intelligent individual, the pairing combination of the start point and the end point that has been generated is generated first. The value range of R is [N D / 2β, N D / β], and β is the intelligent group update proportion factor.

[0097] In step S306, it is determined whether the optimization accuracy requirement is met or the iteration number meets the maximum number K max . If yes, the position of the leader is output, that is, the optimal flight path obtained finally, and if no, the process returns to step S402.

[0098] In the above process, it is determined whether the position x pid of the intelligent individual meets the condition that the unmanned aerial vehicle will not collide during flight. If yes, x pidXi p i d is the position of the pth intelligent individual, the ith path, the dth dimension in the solution space. Otherwise, a point satisfying the condition is randomly selected in the solution space as the position of the pth intelligent individual, the ith path.

[0099] The optimal flight path of the unmanned aerial vehicle, the two-dimensional coordinates of the shooting endpoint, the shooting endpoint height, and the attitude adjustment requirement are transmitted to the unmanned aerial vehicle through the data communication module of the ground terminal.

[0100] As shown in Figure 10 Step S4 includes:

[0101] Step S401, after each unmanned aerial vehicle receives the corresponding flight data, a takeoff ready signal is returned to the data communication module of the ground terminal.

[0102] Step S402, the data communication module of the ground terminal receives the takeoff ready signal from the unmanned aerial vehicle and transmits it to the data processing module. After the signal is parsed and processed by the data processing module, it is transmitted to the user operation module and displayed on the user operation interface, prompting the user about the readiness of the unmanned aerial vehicle before takeoff.

[0103] The user can autonomously select one-key takeoff or specify the takeoff of the unmanned aerial vehicle according to the readiness of the unmanned aerial vehicle. However, since the system serves natural driving data collection, considering the consistency and synchronization of data, only the data simultaneously captured by each unmanned aerial vehicle is valid data. Therefore, the optimal case is to select one-key takeoff, which can ensure the full application of the path planning algorithm and enable the unmanned aerial vehicle group to reach the shooting endpoint in the shortest time and with the least energy consumption.

[0104] Step S403, when the unmanned aerial vehicle reaches the set shooting endpoint, attitude adjustment is performed according to the attitude adjustment requirement received before takeoff. When the attitude adjustment is complete, a shooting ready signal is sent to the data communication module of the ground terminal.

[0105] Step S404, the data communication module of the ground terminal receives the shooting ready signal from the unmanned aerial vehicle and transmits it to the data processing module. After the signal is parsed and processed by the data processing module, it is transmitted to the user operation module and displayed on the user operation interface, prompting the user about the readiness of the unmanned aerial vehicle before shooting.

[0106] The user can autonomously select one-key shooting or specify the shooting of the unmanned aerial vehicle according to the shooting readiness of the unmanned aerial vehicle. However, since the system serves natural driving data collection, considering the consistency and synchronization of data, only the data simultaneously captured by each unmanned aerial vehicle is valid data. Therefore, the optimal case is to select one-key shooting, which can ensure the efficiency of the data captured by the unmanned aerial vehicle.

[0107] In step S405, after the user selects a specific drone through the user interface, they issue a start shooting command. The user operation module sends the start shooting command and the selected drone information to the data processing module. The data processing module processes and integrates the data before sending it to the data communication module. The data communication module then transmits the start shooting command to the specified drone based on the received data. Upon receiving the start shooting command, the specified drone immediately begins shooting.

[0108] Users can issue a one-click start shooting command through the user interface. The user operation module then sends this command to the data processing module, which processes it and forwards it to the data communication module. The data communication module then transmits the start shooting command to all drones based on the received data. Once all drones receive the start shooting command, they immediately begin shooting.

[0109] In step S406, the user can also choose to pause filming with a single click or pause filming for a specific drone through the user interface. Specifically: After selecting a specific drone through the user interface, the user issues a pause command. The user operation module sends the pause command and the selected drone information to the data processing module. The data processing module processes and integrates the data before sending it to the data communication module. The data communication module then transmits the pause command to the specified drone based on the received data. The specified drone immediately pauses filming upon receiving the pause command. Alternatively, the user can issue a one-click pause command through the user interface. The user operation module sends the one-click pause command to the data processing module, which processes it before sending it to the data communication module. The data communication module then transmits the pause command to all drones based on the received data. All drones immediately pause filming upon receiving the pause command.

[0110] During filming, the drone transmits its status to the ground terminal's data communication module in real time. After receiving the data, the data communication module transmits it to the data processing module. The data processing module receives the data from the data communication module, parses and processes the data, and then transmits it to the user operation module. The user operation module displays the drone's status data on the user interface for user monitoring.

[0111] Step S407, the data processing module monitors the power of all drones in real time, and calculates the power required for return. When it is found that there is an early risk of a drone being unable to return, the user is warned, prompting the user to issue a return instruction. The user can issue a one-key return instruction or specify the drone to return. However, since the system serves natural driving data collection, considering the consistency and synchronization of data, only the data taken by each drone at the same time is valid data. Therefore, the optimal situation is to choose one-key return, which can also ensure the efficiency of the data taken by the drone, reduce invalid data, and reduce the difficulty of subsequent data processing.

[0112] After selecting the specified drone through the user operation interface, the return instruction is issued. The user operation module sends the return instruction and the selected drone information to the data processing module, which processes and integrates the data and sends it to the data communication module. The data communication module transmits the return instruction to the specified drone according to the received data. The specified drone returns immediately after receiving the return instruction.

[0113] Alternatively, a one-key return instruction is issued through the user operation interface. The user operation module sends the one-key return instruction to the data processing module, which processes and sends it to the data communication module. The data communication module transmits the return instruction to all drones according to the received data. All drones return immediately after receiving the return instruction.

[0114] Step S408, when the data processing module finds that a drone is about to be in an unreturnable state without receiving a return instruction from the user operation module, the data processing module automatically issues a one-key return instruction to the data communication module. The data communication module transmits the return instruction to all drones, ensuring that the drones can return safely and protecting the drones without endangering the traffic at the shooting location. In this invention, the ground terminal automatically controls the return of the drone, which can avoid the risk of the drone crashing due to power depletion caused by manually issuing a return instruction, reducing the impact on traffic and potential economic losses when using drones for natural driving data collection.

[0115] Embodiment Two

[0116] This embodiment provides a multi-drone cooperative shooting system for natural driving data collection, including drones and a ground terminal. The ground terminal obtains the shooting endpoint and path according to the method described in Embodiment One, and controls the drones to fly to the shooting endpoint according to the path for shooting. Figure 11As shown, the unmanned aerial vehicle of the present application has at least two; the ground terminal obtains the shooting traffic section, generates the shooting end point and plans the optimal flight path of the unmanned aerial vehicle to the shooting end point and transmits it to the unmanned aerial vehicle. The ground terminal includes a data communication module, a user operation module and a data processing module, sends the obtained and preset information to the data processing module, calculates the shooting range of the unmanned aerial vehicle at the maximum shooting height and at any height, generates the shooting end point and plans the optimal path of each unmanned aerial vehicle, transmits it to the unmanned aerial vehicle through the data communication module, and the unmanned aerial vehicle flies to the shooting end point according to the planned path. The unmanned aerial vehicle executes the shooting task and returns according to the signal of the ground terminal.

[0117] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought disclosed by the present application should be covered by the claims of the present application.

Claims

1. A multi-UAV collaborative shooting method for natural driving data acquisition, characterized in that, include: Step S1: Set shooting limits for a single drone; Step S2: Based on the lane type of the selected shooting section, execute the corresponding lane type setting strategy until the total shooting range of the drone can completely cover the selected shooting section and the number of drones required is minimized, and at the same time generate the shooting endpoint of each drone. The limitations of drone photography include maximum shooting height, maximum shooting range, the relationship between shooting range and drone shooting height, and the positional relationship between the drone and the shooting range; The required road segment for filming includes straight sections, curves, and sections that combine both. The corresponding setup strategies include: 1) The lane type of the shooting section is a straight road: When the width of the shooting section is less than the width of the preset drone shooting range, the long side of the drone shooting range is perpendicular to the wide side of the lane; when the width of the shooting section is greater than the width of the preset drone shooting range, but less than the length of the drone shooting range, the long side of the drone shooting range is perpendicular to the wide side of the lane, that is, all drones are rotated horizontally by 90 degrees relative to the positions of the above situations. Based on the above rules, the initial shooting point of the first drone is set on the center line of the total width of the lane, and the drone's shooting range is parallel to the axis of symmetry of the lane center line and coincides with the lane center line. The left side of the shooting section is within the shooting range of the first drone. Then, drones are added from left to right until the total shooting range of multiple drones completely covers the shooting section. 2) The lane type of the shooting section is a curve: The center point of the drone's shooting range is located on the center line of the shooting section. The point where the center point of the drone's shooting range coincides with the center line of the shooting section is set as the base point. When the width of the shooting section is less than the preset width of the drone's shooting range, the long side of the shooting range is parallel to the tangent at the base point of the center line of the shooting section. When the width of the shooting section is greater than the preset width of the drone's shooting range, but less than the length of the drone's shooting range, the wide side of the shooting range is parallel to the tangent at the base point of the center line of the shooting section. Based on the above rules, the initial shooting point of the first drone is set on the center line of the total width of the shooting section, and one end of the shooting section is within the shooting range of the first drone. Then, more drones are added. The new drones take the position of the previous drone as the starting point and move along the center line of the shooting section to the other end of the shooting section. At the same time, the drones rotate to ensure that the long side / wide side of the drone's shooting range is parallel to the tangent at the base point of the center line of the shooting section, until the shooting range of multiple drones completely covers the range of the road section to be shot. 3) The shooting section includes both straight and curved roads: The shooting range of multiple drones is set to the bounding rectangle of the shooting section. Multiple drones are added sequentially from left to right and from front to back until the shooting range of the drones covers the shooting section. Execute the corresponding settings strategy, the drone's shooting range covers the shooting section, generate the initial shooting point, then reduce the drone's shooting altitude, and use the number of initial shooting points as the number of drones to be dispatched. Execute the corresponding settings strategy again until the total shooting range can no longer cover the shooting section after a certain altitude reduction, the iteration stops, and the second to last generated initial shooting point is used as the shooting endpoint. Step S3: With the goal of minimizing the total flight path and minimizing the risk, assign a shooting destination to each drone and plan the optimal flight path for each drone to reach the corresponding shooting destination. Step S4: The drone flies to the shooting destination along the optimal flight path to take pictures.

2. The method according to claim 1, characterized in that, The limitations of drone shooting also include the maximum number of drones that can be deployed. In the setting strategy, if the number of newly added drones exceeds the maximum number of drones, the initial shooting point generation will fail, and step S2 will be executed again.

3. The method according to claim 1, characterized in that, The shooting area is rectangular, and the intersection of the diagonals of the shooting area coincides with the orthographic projection of the drone; The relationships between the length and width of the shooting area and the shooting height are as follows: ， ， in, Let be the longer side of the drone's shooting range at any shooting altitude. For the shorter side, This is the longer side of the drone's shooting range at its maximum shooting altitude. This represents the shorter side of the drone's shooting range at its maximum shooting altitude. The altitude at which the drone initially sets up the shooting point at any given time. This is the maximum shooting altitude for the drone.

4. The method according to claim 1, characterized in that, The left side of the first drone's shooting range extends beyond the left side of the shooting road segment by a specified width. In the first strategy setting, the shooting endpoint of the next drone is generated at the flight endpoint of the previous drone, and then it moves to the right along the lane centerline or the lane width's center line until the overlap between the next and previous drones reaches a specified width. At this point, it stops moving, and the stopping point is used as the initial shooting point for the next drone. In the second strategy setting, when the boundary of the total shooting range formed by the new drone and the previous drone is a specified width away from the lane, it stops moving, and the stopping point is used as the initial shooting point for the new drone.

5. The method according to claim 1, characterized in that, To minimize the total path length of the drone, the starting point and the shooting endpoint of the drone are paired and combined. Then, based on the wolf pack algorithm, the optimal flight path is obtained through joint optimization while ensuring that the drones do not collide.

6. The method according to claim 5, characterized in that, First, obtain all possible combinations of start and end points using permutations and combinations, and then calculate the length of the straight line segment from the start to the end point for each combination. Then calculate the value of each combination scheme. Mean absolute deviation : ， in, For the first combination scheme The length of the straight line segment from the start point to the end point of the group. Choose the average length of all straight segments from the starting point to the ending point in this combination scheme. The minimum combination of start and end points is used as the start and end points for UAV path optimization. express A flight path.

7. The method according to claim 6, characterized in that, In the wolf pack algorithm, individual path combinations are considered as intelligent individuals, with the goal of maximizing the fitness value of these intelligent individuals. Optimize flight path, The expression is: ， in, These are the weighting coefficients for the corresponding factors, used to balance the importance of different factors; For intelligent individuals The first of the flight paths The length of the flight path segment, For intelligent individuals The first of the flight paths Danger level of each flight path segment; The average absolute deviation of all tracks of a single intelligent individual: ， in The average length of all tracks of a single intelligent individual. For the first Total length of the track: 。 8. The method according to claim 7, characterized in that, Step S3 includes the following steps: Step S301, Initialize intelligent population parameters; Step S302: Calculate the fitness value of each intelligent individual's location, select the intelligent individual with the highest fitness value as the leader, and then select the intelligent individual with the next highest fitness value as the leader. Each intelligent individual acts as a navigator; the leader initiates a summoning action, gathering all intelligent individuals except the leader and navigators to its current location; the navigators are responsible for dynamic searching, continuously exploring forward until the fitness of a navigator surpasses that of the leader, or the maximum number of navigator searches is reached. ; Step S303, intelligent individuals with lower fitness values Responding to the leader's call, move quickly towards the optimal solution. If, during the movement, an intelligent individual with a lower fitness value... If an intelligent individual's fitness level is superior to that of the leader, it will initiate a summoning action in place of the leader; otherwise, the intelligent individual will not. Continue responding to the call signal until a low-fitness intelligent individual is reached. Distance from the leader Less than the distance judgment value When the behavior of responding to the call signal is changed to focused search behavior, all intelligent individuals with lower fitness values ​​will start to change their behavior to focused search behavior; otherwise, intelligent individuals with lower fitness values ​​will continue to move towards the optimal solution. Step S304: Select the leader's position as the optimal solution's position. Intelligent individuals with lower fitness values ​​perform a centralized search for the optimal solution, updating the positions of the intelligent individuals participating in the centralized search. If the updated intelligent individual positions... If the value exceeds the known spatial range, it is set as a boundary value; Step S305: According to the leader generation rules, update the leader position, then update the intelligent swarm, and delete the leader with the worst fitness. A number of intelligent individuals are generated randomly at the same time. An intelligent individual; Step S306: Determine whether the optimization accuracy requirement is met, or whether the number of iterations meets the maximum number. If yes, output the leader's position, which is the final optimal flight path; otherwise, return to step S302.

9. A multi-UAV collaborative shooting system for natural driving data acquisition, characterized in that, The system includes a drone and a ground terminal. The ground terminal obtains the shooting endpoint and the optimal flight path according to the method described in any one of claims 1-8, and transmits the shooting endpoint and the optimal flight path to the drone. After the drone flies to the shooting endpoint according to the optimal flight path, it takes pictures according to the instructions of the ground terminal and transmits the shooting data to the ground terminal.

Images