Multi-unmanned aerial vehicle cooperative shooting system and method for natural driving data acquisition

Abstract

The invention relates to a multi-unmanned aerial vehicle cooperative shooting system and method for natural driving data collection. The method comprises the steps that S1, shooting limitation of a single unmanned aerial vehicle is set; s2, according to the lane type of the selected shooting road section, executing a setting strategy corresponding to the lane type until the total shooting range of the unmanned aerial vehicles can completely cover the selected shooting road section and the number of the required unmanned aerial vehicles is minimum, and generating shooting end points of the unmanned aerial vehicles at the same time; s3, by taking a total flight path as short as possible and a risk degree as small as possible as targets, distributing a shooting terminal point for each unmanned aerial vehicle, and planning an optimal flight path for each unmanned aerial vehicle to reach the corresponding shooting terminal point; and S4, the unmanned aerial vehicle flies to the shooting terminal point according to the optimal flight path for shooting. According to the invention, richer data support is provided for automatic driving simulation test.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) photography, and in particular to a multi-UAV collaborative photography system and method for natural driving data collection. Background Art

[0002] With the rapid development of autonomous driving technology, it is essential to conduct simulation tests on autonomous driving vehicles before they are put on the road. Scenario-based autonomous driving simulation testing is currently a mainstream simulation testing method, and the extraction of simulation test scenarios is highly dependent on the integrity and accuracy of natural driving data. Using drones to obtain traffic images from high-altitude aerial traffic sections and extract natural driving data from them is one of the mainstream methods for collecting natural driving data. However, due to national laws that limit the maximum flight altitude of drones in many major traffic sections, the range of drone shooting is narrow, which in turn causes the vehicle to be shot for too short a time to obtain the relevant information needed to extract natural driving data. However, since it is difficult to use multiple drones to shoot at the same time, it is necessary to ensure that the position of each drone is reasonable and the flight path of the drone is reasonable and does not collide. At the same time, since the purpose of video acquisition is to extract natural driving data, only videos shot by all drones at the same time are valid, which is also a problem that needs to be solved by multi-drone joint shooting to serve the collection of natural driving data. Summary of the invention

[0003] In view of the shortcomings of the prior art mentioned above, the object of the present invention is to provide a multi-UAV collaborative shooting system and method for natural driving data collection, which is used to solve technical problems such as the small shooting range of a single UAV and unreasonable shooting positions and path planning of multiple UAVs.

[0004] To achieve the above object, the present invention provides a multi-UAV collaborative shooting method for natural driving data collection, comprising:

[0005] Step S1, setting shooting restrictions for a single drone;

[0006] Step S2, according to the lane type of the selected shooting section, executing the setting strategy corresponding to the lane type, until the total shooting range of the drone can completely cover the selected shooting section and the number of drones required is minimized, and at the same time generating the shooting endpoints of each drone;

[0007] Step S3, with the goal of making the total flight path as short as possible and the risk as low as possible, assign a shooting end point to each drone, and plan the optimal flight path for each drone to reach the corresponding shooting end point;

[0008] Step S4: The drone flies to the shooting destination along the optimal flight path to shoot.

[0009] The present invention also provides a multi-UAV collaborative shooting system for natural driving data collection, including a UAV and a ground terminal. The ground terminal obtains the shooting end point and the optimal flight path according to the method as described above, and transmits the shooting end point and the optimal flight path to the UAV. After the UAV flies to the shooting end point according to the optimal path, it shoots according to the instructions of the ground terminal and transmits the shooting data to the ground terminal.

[0010] The beneficial effect of the present invention is that compared with using a single drone to collect traffic videos from a high altitude, the present invention uses multiple drones, which can obtain a wider field of view and increase the appearance time of the studied vehicle in the video, so that researchers can extract more natural driving data from it, thereby digging out more autonomous driving simulation test scenarios, providing richer data support for autonomous driving simulation tests. In the present invention, the actual characteristics of different traffic sections are adapted, so that the invention can be applied in different locations while ensuring the accuracy of the data and avoiding the use of unreal data to extract natural driving data. Step S2 is to use as few drones as possible to cover the traffic sections that need to be filmed, reducing the difficulty and cost of filming. Through step S3, the drone path planning method can shorten the data collection time, reduce the power consumption of the drone, and improve the efficiency of collecting natural driving data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a simplified flowchart of the steps in Example 1;

[0012] Figure 2 It is a schematic diagram of a method for calculating the actual length corresponding to the shooting range at any shooting height in Example 1;

[0013] Figure 3 A flowchart of the steps for generating a shooting endpoint in Embodiment 1;

[0014] Figure 4 This is a schematic diagram of the generation result of the initial fixed point shot by the drone when the shooting section in Example 1 includes only straight roads;

[0015] Figure 5 This is a schematic diagram of another method of generating a result of an initial fixed point when the shooting section of the first embodiment only includes a straight road;

[0016] Figure 6 This is a schematic diagram of the initial fixed point generation result of the drone when the shooting road section includes only curves in the first embodiment;

[0017] Figure 7 This is a schematic diagram of another method of generating the initial fixed point of a drone when the road section only includes a curve in the first embodiment;

[0018] Figure 8 This is a schematic diagram of the generation result of the initial fixed point shot by the drone when the road section shot in the first embodiment includes both straight roads and curves;

[0019] Fig. 9 This is a flow chart of a path planning algorithm according to Embodiment 1;

[0020] Fig.10 is a flow chart of an embodiment method;

[0021] Fig.11 Schematic diagram of the system structure of the embodiment. DETAILED DESCRIPTION

[0022] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0023] like Figure 1-11 As shown, this embodiment provides a multi-UAV collaborative shooting method for natural driving data collection, including:

[0024] Step S1, setting shooting restrictions for a single drone;

[0025] Step S2, according to the lane type of the selected shooting section, executing the setting strategy corresponding to the lane type, until the total shooting range of the drone can completely cover the selected shooting section and the number of drones required is minimized, and at the same time generating the shooting endpoints of each drone;

[0026] Step S3, with the goal of making the total flight path as short as possible and the risk as low as possible, assign a shooting end point to each drone, and plan the optimal flight path for each drone to reach the corresponding shooting end point;

[0027] Step S4: The drone flies to the shooting destination along the optimal flight path to shoot.

[0028] Compared with using a single drone to collect traffic videos from a high altitude, the present invention uses multiple drones, which can obtain a wider field of view and increase the appearance time of the studied vehicle in the video, so that researchers can extract more natural driving data from it, thereby digging out more autonomous driving simulation test scenarios, providing richer data support for autonomous driving simulation tests. In the present invention, the actual characteristics of different traffic sections are adapted, so that the invention can be applied in different locations while ensuring the accuracy of the data and avoiding the use of unreal data to extract natural driving data. When the generation of the initial fixed point of the drone shooting is completed. Through step S4, the method of drone path planning can shorten the data collection time, reduce the power consumption of the drone, and improve the collection efficiency of natural driving data.

[0029] In this example, a ground terminal is used to wirelessly connect multiple drones to control the drones. At the same time, the ground terminal is connected to the Internet. The drones have their own cameras, and the drones collect traffic information through the cameras.

[0030] Before step S1, the ground terminal calls the third-party satellite map platform interface to obtain the road structure and coordinate information of the road section to be photographed, and the operator selects the rectangular traffic section range to be photographed. On the basis of the selected traffic section range to be photographed, the operator further selects the lane to be photographed, that is, the shooting section, and sets the lane type to be one of a single lane straight road, a multi-lane straight road, a single lane curve, a multi-lane curve, or a mixed lane containing both straight roads and curves. How to select the traffic section range, the shooting section, and the judgment of the lane type can all be achieved using existing technologies.

[0031] The shooting limitations of drones include the maximum shooting altitude, the maximum shooting range, the relationship between the shooting range and the drone’s shooting altitude, and the positional relationship between the drone and the shooting range.

[0032] In this example, the shooting range is a rectangle, and the intersection of the diagonal lines of the shooting range coincides with the orthographic projection of the drone, which means that the drone is directly above the shooting range. The lower the shooting altitude of the drone, the smaller the shooting range. Figure 2 As shown, the shooting range of the drone at any shooting height is calculated as follows:

[0033] First, the longer side of the drone's photo taken at the highest altitude is the length, and the shorter side is the width. The length and width are used to make a rectangle as the drone's shooting range on the ground at the highest altitude. The four vertices of the rectangle (shooting range) are recorded as A, B, C, and D (such as Figure 2 shown);

[0034] Secondly, take the intersection point O of the diagonal lines of the rectangle as the projection point of the drone on the ground at this shooting height, draw a straight line perpendicular to the ground at point O, and take a point F at the maximum shooting height of the drone above the straight line. F is the position of the drone at the maximum shooting height.

[0035] Then, the line connecting point F and point O is used as the height line FO of the drone's shooting range, and its length represents the maximum shooting height H. FO .

[0036] Next, connect point F with the four vertices A, B, C, and D of the rectangle to obtain the shooting range of the drone at the maximum shooting height.

[0037] Finally, at the position of the drone's highest shooting height, intercept any point G along the height line downward, that is, FG is recorded as the height line at any shooting height, ensuring that FG is greater than 0 and less than the drone's maximum shooting height. The length of FG is recorded as any shooting height as H FG Draw a new plane through point G parallel to the plane where point O is located and intersects FA at point A 1 , intersecting FB at point B 1 , intersecting FC at point C 1 , intersecting FD at point D 1 , with A 1 , B 1 , C 1 , D 1 The new plane formed by the connection is taken as the ground at any shooting height, that is, the arbitrary shooting height H is obtained. FG The shooting range below.

[0038] The longer side of the drone's shooting range is length M, and the shorter side is width N. The relationship between the shooting range and the shooting height change is:

[0039]

[0040] Among them, M FG is the longer side of the drone’s shooting range at any shooting height, N FG is the shorter side, M FO N is the longer side of the drone’s shooting range at the maximum shooting altitude. FO H is the shorter side of the drone’s shooting range at the maximum shooting altitude. FG H is the height of the drone at any initial shooting point (any shooting point), FO The maximum shooting altitude of the drone.

[0041] According to the pre-set maximum shooting height and maximum shooting range of the drone, the shooting range of the drone at any shooting height can be calculated. In other words, the length and width of the shooting range at any height are obtained, that is, the shooting range of the drone at any shooting height is obtained, which prepares for the subsequent continuous calculation of the shooting ranges of multiple drones.

[0042] The lane types of the road sections to be photographed include straight roads, curves, and both straight roads and curves. The corresponding setting strategies include:

[0043] 1) The lane type of the shooting section is a straight road: When the width of the shooting section is smaller than the preset width of the drone shooting range (the shorter side), the long side of the drone shooting range is perpendicular to the wide side of the lane, such as Figure 4 As shown in the figure; when the width of the shooting section is larger than the preset width of the drone shooting range, but smaller than the length (longer side) of the drone shooting range, and there is a margin, the long side of the drone shooting range is perpendicular to the wide side of the lane, that is, all drones are horizontally rotated 90 degrees relative to the above situation, as shown in the figure. Figure 5 shown.

[0044] Then, based on the above rules, the first drone's initial shooting point is set on the center line of the lane's total width, and the drone's shooting range is parallel to the lane's center line and coincides with the lane's center line. The left side of the shooting section is within the first drone's shooting range, that is, the left edge of the lane range is ensured to be within the first drone's shooting range. Then, drones are added from left to right until the total shooting range of multiple drones completely covers the range of the road section to be shot, and the shooting ranges of two adjacent drones have overlapping parts, such as Figure 4 and Figure 5 In this example, the end point of the drone is moved to the left along the center line of the lane or the middle line of the lane width, so that it can capture the range of the specified width outside the left edge, that is, the left side of the first drone's shooting range exceeds the specified width on the left side of the shooting section range. This way of retaining a surplus shooting range ensures that the left side of the lane range can be fully captured. In this example, the specified width here is 1 meter.

[0045] Before setting the next drone destination, determine whether the current drone shooting range has completely covered the shooting section, that is, whether it can shoot the selected lane and the range of 1 meter outside the lane. If it can be completely covered, the destination is generated; if not, the next drone destination is generated.

[0046] The flight endpoint of the next drone is generated at the flight endpoint of the previous drone, and then it moves to the right along the center line of the lane or the center line of the lane width until the width of the overlap between the next drone and the previous drone reaches the specified width, then stops moving and uses the stop point as the initial shooting point of the next drone. In this example, the specified width is 1 meter.

[0047] Subsequent drones generate the endpoint using the same rules until the endpoint is generated.

[0048] 2) The lane type of the shooting section is a curve: the center point of the drone shooting range is located on the center line of the shooting section, and the point where the center point of the drone shooting range coincides with the center line of the shooting section is set as the base point. When the width of the shooting section is smaller than the preset width of the drone shooting range, the long side of the shooting range is parallel to the tangent line at the base point of the center line of the shooting section, such as Figure 6 As shown; when the width of the shooting section is larger than the preset width of the drone shooting range, but smaller than the length of the drone shooting range, and there is a margin, the wide side of the shooting range is parallel to the tangent line at the base point of the midline of the shooting section, as shown Figure 7 shown.

[0049] Then, based on the above rules, the initial shooting point of the first drone is set on the midline of the total width of the shooting section, and one end of the shooting section is located within the shooting range of the first drone. Then, drones are added, starting from the position of the previous drone, and moving along the midline of the shooting section to the other end of the shooting section. At the same time, the drones are rotated to ensure that the long side / wide side of the drone's shooting range is parallel to the tangent at the base point of the midline of the shooting section, until the shooting range of multiple drones completely covers the range of the section to be shot. Figure 6 and Figure 7 shown.

[0050] Before setting the next drone endpoint, determine whether the current drone's shooting range has completely covered the shooting section, that is, whether it can capture the selected lane and the range of 1 meter outside the lane. If it can be fully covered, the endpoint is generated; if it cannot be fully covered, the next drone endpoint is generated.

[0051] After the newly added drone moves, it overlaps with the previous drone to ensure that all shooting sections can be photographed. In this example, when the total shooting range boundary formed by the newly added drone (the next drone) and the previous drone is the specified width from the lane, it stops moving and the stop point is used as the initial shooting point of the newly added drone. The specified width here is at least 1 meter.

[0052] Subsequent drones generate the endpoint using the same rules until the endpoint is generated.

[0053] 3) The shooting section includes both straight roads and curves: The shooting range of multiple drones is set to the circumscribed rectangular range of the shooting section, and multiple drones are added from left to right and from front to back until the shooting range of the drones covers the shooting section. Figure 8 As shown, in this example, the wide side of the drone shooting range is parallel to the wide side of the shooting section rectangular range, and the long side of the drone shooting range is parallel to the long side of the shooting section rectangular range. In addition, the wide side of the drone shooting range can also be parallel to the long side of the shooting section rectangular range, and the long side of the drone shooting range can be parallel to the wide side of the shooting section rectangular range.

[0054] It should be noted that all the initial shooting points are on the same horizontal plane, and front, back, left, and right are relative to the horizontal plane where the initial shooting points are located. The front, back, left, and right directions are not limited to specific directions.

[0055] First, generate the initial shooting point from left to right. In this example, Figure 6 As shown, the first initial shooting point is generated at the upper left corner of the rectangular range of the traffic section to be photographed, and the initial shooting point is moved backward until the front boundary of the drone shooting range is 1 meter away from the rectangular range of the traffic section to be photographed, then the movement stops, and then the initial shooting point is moved to the right until the left boundary of the drone shooting range is 1 meter away from the rectangular range of the traffic section to be photographed, then the movement stops. The second initial shooting point is generated at the first initial shooting point and moves to the right until the second drone shooting range overlaps with the first drone shooting range by 1 meter, then the movement stops, and the subsequent initial shooting points are generated and moved according to this rule. Before generating a new initial shooting point each time, determine whether the right boundary of the shooting range corresponding to the last generated initial shooting point of the drone exceeds the rectangular range of the traffic section to be photographed by 1 meter: if not, continue to generate the initial shooting point according to the above rules; if it exceeds, further determine whether the rear boundary of the shooting range corresponding to the last generated initial shooting point of the drone exceeds the rectangular range of the traffic section to be photographed by 1 meter. If its rear boundary also exceeds the rectangular range of the traffic section to be photographed by 1 meter, the generation of the initial shooting point ends; if its rear boundary does not exceed the rectangular range of the traffic section to be photographed by 1 meter, the rectangular range of the section to be photographed is not fully covered, and it is necessary to continue to generate initial shooting points of the drone. At this time, start to generate initial shooting points of the drone from front to back, specifically, generate multiple initial shooting points of the drone at the same time and move the initial shooting points at the same time. The detailed method is as follows:

[0056] The initial points generated from left to right are recorded as the first row of drone initial points (with multiple drone initial points). A drone initial point is generated at each of the first row of drone initial points, that is, multiple drone initial points are generated at the same time. The newly generated drone initial points are recorded as the second row of drone initial points. The second row of drone initial points move backward until the second row of drone shooting range overlaps with the first row of drone shooting range by 1 meter and stops moving. Then, it is determined whether the rear boundary of the second row of drone shooting range exceeds the rectangular range of the traffic section to be photographed by 1 meter. If it exceeds, the generation of the initial points is completed; if it does not exceed, the third row of drone initial points is generated in the same way as the second row of drone initial points until the rear boundary of the shooting range exceeds the rectangular range of the traffic section by 1 meter. The drone initial points of the next row are generated at the drone initial points of the previous row. Of course, the drone initial points of the next row can also be generated at any relatively forward row of drone initial points.

[0057] It should be noted that the drones corresponding to the initial fixed points taken by the first row of drones are the first row of drones, and the drones corresponding to the initial fixed points taken by the second row of drones are the second row of drones, and so on. There may be a third row of drones, a fourth row of drones, and so on and so forth. There may be Nth row of drones.

[0058] In this example, the drone shooting restriction also includes setting the maximum number of drones to be dispatched. In the setting strategy, when the number of newly added drones is greater than the maximum number of drones, the generation of the initial shooting points fails, and step S2 is re-executed. In other words, during the generation of the initial shooting points of the drones, if the number of drones reaches the upper limit and still cannot completely cover the rectangular range of the required shooting section, that is, the condition for the initial generation of the initial shooting points cannot be met, the generation of the initial shooting points fails.

[0059] Execute the corresponding setting strategy, the shooting range of the drone covers the shooting section, generate the initial shooting points, then lower the shooting height of the drone, and use the number of initial shooting points as the number of drones dispatched, and execute the corresponding setting strategy again until the total shooting range cannot cover the shooting section after a certain height reduction, the iteration stops, and the second to last generated initial shooting point is used as the shooting end point.

[0060] Specifically, if the initial generation of the drone shooting initial fixed point can be completed after the drone shooting initial fixed point is lowered, the altitude will continue to be lowered, and the corresponding setting strategy for the generation of the drone shooting initial fixed point will be repeated until the initial generation of the drone shooting initial fixed point cannot be completed using the updated number of drones after the drone flight altitude is lowered once, that is, the corresponding number of drones cannot completely cover the shooting section, and the iteration stops, and the distribution of the drone shooting initial fixed point before the last reduction of the drone flight altitude is used as the drone shooting end point coordinates with the least drones and the highest shooting accuracy, that is, the shooting end point is formed after the initial fixed point is adjusted. Under the condition that the same number of drones can completely cover the shooting section, the altitude of the drone is lowered, the shooting range of each drone is narrowed, and the flight end point of the drone is adjusted twice to improve the shooting accuracy.

[0061] Among them, the step length for lowering the shooting height of the drone can be flexibly set according to needs.

[0062] Before optimizing the drone path, we first need to minimize the total drone path and determine the pairing of the starting point and the shooting end point (hereinafter referred to as the end point), that is, pair the starting point and the shooting end point of the drone, and then based on the wolf pack algorithm, jointly optimize to obtain the optimal flight path while ensuring that the drones do not collide.

[0063] Specifically, all possible combinations of the starting point and the ending point are obtained according to the permutations and combinations, and the length d of the straight line segment from the starting point to the ending point of each combination is calculated. s , and then calculate the d of each combination solution s The mean absolute deviation d a :

[0064]

[0065] in, is the length of the straight line segment from the starting point to the end point of the i-th group of the combination scheme, Select d as the average length of all straight line segments from the start point to the end point of this combination. a The smallest start-end combination solution is used as the start-end and end-point of the UAV path optimization, and D represents D tracks.

[0066] To optimize the flight path of the drone, first, the parameters are initialized. According to the pairing scheme of the starting point and the end point, the starting point of the drone is set to Set the drone's endpoint to Randomly generate N DThe path combination schemes are regarded as intelligent groups, and the single path combination scheme is regarded as intelligent individuals. Each path combination scheme includes D tracks from the specified starting point to the specified end point according to the start-end pairing results. After connecting the starting point and the end point of each track, divide it into n+1 equal parts along the x-axis direction. Each equal part is a track segment. n is set manually, and the equal division point is x. pid (p represents the pth intelligent individual, p=1,2,……,N D ; i represents the i-th track, i = 1, 2, ..., D; d represents the d-dimensional solution space), the corresponding track point p pid The coordinates are (x pid ,y pid ,z pid ), the requirement that should be met is that the two tracks will not cause UAV collision during the flight, that is, the distance between any track point on one track and any track point on the other track is greater than 2R D , R D It is a safe flight radius set according to the specific model of the drone.

[0067] During the path optimization process, the flight path is optimized by maximizing the fitness value Y of the intelligent individual. The expression of Y is:

[0068]

[0069] Among them, W e , W s , W l is the weight coefficient of the corresponding factor, which is used to balance the importance of different factors; e ij is the length of the jth track segment of the ith track of the intelligent individual, s ij is the hazard level of the jth track segment of the ith track of the intelligent individual. The hazard level of a track segment is the reciprocal of the minimum distance between the track segment and all other track segments; l a is the average absolute deviation of all the tracks of a single intelligent individual:

[0070]

[0071] in is the average length of all the tracks of a single intelligent individual, l i is the total length of the ith track:

[0072]

[0073] like Fig. 9 As shown, the optimization path includes:

[0074] Step S301, initializing intelligent group parameters;

[0075] Step S302, calculate the fitness value of each intelligent individual at the location, select the intelligent individual with the largest fitness value as the leader, and the T with the fitness value lower than the leader n The leader initiates the call to call all intelligent individuals except the leader and navigator to their location; the navigator is responsible for dynamic search and continues to explore until the fitness of a navigator is better than the fitness of the leader, or the maximum number of navigator searches is reached. max ;

[0076] T n The value of is [N D / (α+1),N D / α], a random integer between D is the total number of all intelligent individuals, and α is the proportion factor of navigators, that is, the proportion of navigators in the intelligent individuals.

[0077] During the dynamic search process, the fitness value Y of the location of the navigator q is q , Y q and the leader’s fitness value Y leader For comparison, if the fitness Y q >Y leader , then we can get Y leader =Y q , that is, the navigator replaces the leader and sends a call signal to the intelligent individuals with lower fitness rankings, that is, the navigator becomes the leader; if the fitness Y q <Y leader , the navigator will move in the direction h with a fixed step size Swim forward one step, and take the navigator's current position as the starting point, then the navigator's position after moving along the z (z = 1, 2, ..., h) direction Will be updated to:

[0078]

[0079] is the search step length of the navigator, x qid The position of the navigator before moving.

[0080] Step S303: The intelligent individual m with the lowest fitness value responds to the leader's call and moves towards the leader's position with a step length Move quickly. Then when the intelligent individual m with the lower fitness ranking experiences the k+1th iteration, its position in the d-dimensional space is Will be updated to:

[0081]

[0082] in, represents the position of the leader after the kth iteration in the d-dimensional space, Indicates the spatial position of the intelligent individual m with the lowest fitness at this time, It means that the intelligent individual m with lower fitness gradually approaches the leader. It is the moving step length of the UAVs ranked later in response to the navigator’s call.

[0083] If the fitness of the intelligent individual m with a lower fitness value is better than that of the leader during the fast movement, it will initiate the call behavior instead of the leader. Otherwise, the intelligent individual m will continue to respond to the call signal until the distance d between the intelligent individual m with a lower fitness value and the leader is ms Less than the distance judgment value d near When , all intelligent individuals with lower fitness values ​​begin to switch from responding to the call signal to concentrated search behavior, otherwise, the intelligent individuals with lower fitness values ​​continue to move towards the optimal solution;

[0084] The distance determination factor calculation formula is as follows:

[0085]

[0086] Among them, w is the distance determination factor, and its size determines the convergence speed of the algorithm. Under the same conditions, the larger the value of the determination factor, the faster the convergence speed of the algorithm. However, if w is too large, it will affect the centralized search behavior of the intelligent group and make it impossible to enter the centralized search state; D represents the number of spatial dimensions (also the total number of tracks).

[0087] Min d Max d They represent the minimum and maximum values ​​of the d-th dimension space to be optimized.

[0088] In the centralized search behavior, the leader position closest to the optimal solution is represented as the position of the optimal solution. For the k-th generation drone individual, the position of the optimal solution in the d-dimensional space can be expressed as Then, in the kth iteration, the position of intelligent individual m in the dth dimensional space after performing concentrated search can be expressed as:

[0089]

[0090] Among them, λ is a random constant between [-1,1], The moving step length of the intelligent individuals with lower fitness rankings to initiate centralized search behavior. When the fitness of the intelligent individual m with lower fitness rankings is detected to be greater than the fitness of its original position during centralized search, the position of the intelligent individual m is updated. Otherwise, the position of the intelligent individual m with lower fitness rankings does not change.

[0091] There is a certain connection between the corresponding step sizes of the three intelligent behaviors in the dimensional space, as shown in the following formula:

[0092]

[0093] in, is the search step length of the navigator, is the moving step length of the UAVs at the back of the list in response to the navigator’s call, It is the moving step length of the concentrated search behavior initiated by the intelligent individuals with lower fitness ranking. L represents the step length factor of the intelligent behavior, and the value affects the degree of refinement of the search endpoint.

[0094] Step S304: select the leader's position as the optimal solution position, and the intelligent individuals with lower fitness values ​​conduct centralized search for the optimal solution, and update the positions of the intelligent individuals participating in the centralized search. If the updated intelligent individual position x mid If it exceeds the solution space range, it is set to the boundary value.

[0095] Step S305, according to the leader generation rule, the leader position is updated, and then the intelligent group is updated, the R intelligent individuals with the worst fitness are deleted, and R intelligent individuals are randomly generated.

[0096] Specifically, R intelligent individuals with lower fitness will be eliminated due to the competition mechanism, and R intelligent individuals will be randomly generated. When a new intelligent individual is generated, the change value of the optimal fitness of this iteration is judged. If the change value of the optimal fitness is greater than Y u (Y u is the fitness change threshold for exploring unknown combinations, which is set manually). When generating new intelligent individuals, the pairing combination of the starting point and the end point that has not been generated before is generated first. If the optimal fitness change value is less than Y u , when generating new intelligent individuals, the pairing combination of the starting point and the end point that have been generated is generated first. The value range of R is [N D / 2β,N D / β], β is the intelligent group update scaling factor.

[0097] Step S306: determine whether the optimization accuracy requirement is met or the number of iterations meets the maximum number K. max If yes, then output the position of the leader, that is, the final optimal flight path; if no, return to step S402.

[0098] In the above process, the position x of the intelligent individual is determined pid Whether the drone will not collide during flight. If so, take x pidis the position of the p-th intelligent individual, the i-th track, and the d-dimensional space. Otherwise, a point that satisfies the conditions is randomly selected in the solution space as the position of the i-th track of the p-th intelligent individual.

[0099] The optimal flight path, two-dimensional coordinates of the shooting endpoint, and the height of the shooting endpoint of the drone are obtained, and the attitude adjustment requirements are transmitted to the drone through the data communication module of the ground terminal.

[0100] like Fig.10 As shown, step S4 includes:

[0101] Step S401: After receiving the corresponding flight data, each UAV returns a take-off ready signal to the data communication module of the ground terminal.

[0102] In step S402, the data communication module of the ground terminal receives the takeoff-ready signal from the drone and transmits it to the data processing module. The data processing module parses and processes the signal and transmits it to the user operation module, which displays it on the user operation interface to prompt the user of the drone's readiness before takeoff.

[0103] Users can choose to take off with one button or designate a drone to take off based on the drone's readiness. However, since the system serves the collection of natural driving data, considering the consistency and synchronization of the data, only the data captured by each drone at the same time is valid data. Therefore, the optimal situation is to choose one-button takeoff, which can ensure the full application of the path planning algorithm and allow the drone group to reach the shooting destination in the shortest time and with the lowest energy consumption.

[0104] Step S403, when the drone reaches the set shooting end point, it will adjust its attitude according to the attitude adjustment requirement received before takeoff, and when the attitude adjustment is completed, it will send a shooting ready signal to the data communication module of the ground terminal.

[0105] Step S404: The data communication module of the ground terminal receives the shooting-ready signal from the drone and transmits it to the data processing module. The data processing module parses and processes the signal and transmits it to the user operation module, which displays it on the user operation interface to prompt the user of the drone's readiness before shooting.

[0106] Users can choose to start shooting with one click or specify a drone to start shooting based on the drone's shooting readiness. However, since the system serves the collection of natural driving data, considering the consistency and synchronization of the data, only the data captured by each drone at the same time is valid data. Therefore, the best situation is to choose to start shooting with one click, which can ensure the effectiveness of the data captured by the drone.

[0107] Step S405: After the user selects a designated drone through the user operation interface, he issues a start shooting command. The user operation module sends the start shooting command and the selected drone information to the data processing module, which processes and integrates the information and sends it to the data communication module. The data communication module transmits the start shooting command to the designated drone based on the received data. The designated drone starts shooting immediately after receiving the start shooting command.

[0108] The one-key start shooting command is issued through the user operation interface. The user operation module will send the one-key start shooting command to the data processing module, which will process it and send it to the data communication module. The data communication module will transmit the start shooting command to all drones based on the received data. All drones will start shooting immediately after receiving the start shooting command.

[0109] In step S406, the user can also independently choose to pause shooting with one key or to specify a drone to pause shooting through the user operation interface. Specifically, after the user selects a specified drone through the user operation interface, a pause shooting instruction is issued. The user operation module will send the pause shooting instruction and the selected drone information to the data processing module, which will be processed and integrated by the data processing module and sent to the data communication module. The data communication module will transmit the pause shooting instruction to the specified drone according to the received data. The specified drone will immediately pause shooting after receiving the pause shooting instruction. Alternatively, a one-key pause shooting instruction is issued through the user operation interface. The user operation module will send the one-key pause shooting instruction to the data processing module, which will be processed by the data processing module and sent to the data communication module. The data communication module will transmit the pause shooting instruction to all drones according to the received data. All drones will immediately pause shooting after receiving the pause shooting instruction.

[0110] During the shooting process, the drone transmits its status to the data communication module of the ground terminal in real time. After receiving the data, the data communication module transmits it to the data processing module. After receiving the data from the data communication module, the data processing module parses and processes the data and transmits it to the user operation module. The user operation module displays the drone status data on the user interface for user monitoring.

[0111] Step S407, the data processing module monitors the power of all drones in real time and calculates the power required for their return. When it is found that a drone is at risk of not being able to return, it will alert the user and prompt the user to issue a return command. The user can issue a one-key return command or specify the drone to return. However, since the system serves the collection of natural driving data, considering the consistency and synchronization of the data, only the data captured by each drone at the same time is valid data. Therefore, the optimal situation is to choose a one-key return, which can also ensure the efficiency of the data captured by the drone, reduce invalid data, and reduce the difficulty of subsequent data processing.

[0112] After selecting a designated drone through the user operation interface, a return command is issued. The user operation module sends the return command and the selected drone information to the data processing module, which processes and integrates the information and sends it to the data communication module. The data communication module transmits the return command to the designated drone based on the received data. The designated drone returns immediately after receiving the return command.

[0113] Alternatively, a one-key return command is issued through the user operation interface, and the user operation module sends the one-key return command to the data processing module, which processes it and sends it to the data communication module. The data communication module transmits the return command to all drones based on the received data. All drones return immediately after receiving the return command.

[0114] Step S408: When the data processing module finds that a drone is about to be unable to return and has not received a return command from the user operation module, the data processing module will automatically issue a one-key return command to the data communication module, and the data communication module will transmit the return command to all drones, thereby ensuring that the drones can return safely, protecting the drones while not endangering the traffic at the shooting location. In the present invention, the return of the drone is automatically controlled by the ground terminal, which can avoid the risk of traffic accidents caused by the drone running out of power due to the manual issuance of the return command, and reduce the impact on traffic and the economic losses that may be caused when using drones to collect natural driving data.

[0115] Embodiment 2

[0116] This embodiment provides a multi-UAV collaborative shooting system for natural driving data collection, including UAVs and ground terminals. The ground terminal obtains the shooting destination and path according to the method described in the first embodiment, and controls the UAVs to fly to the shooting destination along the path for shooting. Fig.11As shown, there are at least two drones in the present invention; the ground terminal acquires the traffic section for shooting, generates the shooting end point, and plans the optimal flight path for the drone to reach the shooting end point and transmits it to the drone. The ground terminal includes a data communication module, a user operation module, and a data processing module, and sends the acquired and preset information to the data processing module. The data processing module calculates the shooting range of the drone at the maximum shooting height and any height, generates the shooting end point, and plans the optimal path for each drone, which is transmitted to the drone through the data communication module. The drone flies to the shooting end point according to the planned path, and the drone performs the shooting task and returns according to the signal of the ground terminal.

[0117] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A multi-UAV collaborative shooting method for natural driving data collection, characterized in that: include: Step S1, setting shooting restrictions for a single drone; Step S2, according to the lane type of the selected shooting section, executing the setting strategy corresponding to the lane type, until the total shooting range of the drone can completely cover the selected shooting section and the number of drones required is minimized, and at the same time generating the shooting endpoints of each drone; Step S3, with the goal of making the total flight path as short as possible and the risk as low as possible, assign a shooting end point to each drone, and plan the optimal flight path for each drone to reach the corresponding shooting end point; Step S4: The drone flies to the shooting destination along the optimal flight path to shoot.

2. The method according to claim 1, characterized in that The shooting restrictions of drones include the maximum shooting altitude, the maximum shooting range, the relationship between the shooting range and the drone’s shooting altitude, and the positional relationship between the drone and the shooting range; The lane types of the road sections to be photographed include straight roads, curves, and both straight roads and curves. The corresponding setting strategies include: 1) The lane type of the shooting section is a straight road: when the width of the shooting section is smaller than the preset width of the drone shooting range, the long side of the drone shooting range is perpendicular to the wide side of the lane; when the width of the shooting section is larger than the preset width of the drone shooting range but smaller than the length of the drone shooting range, the long side of the drone shooting range is perpendicular to the wide side of the lane, that is, all drones are horizontally rotated 90 degrees relative to the above situation; Then, based on the above rules, the first drone's initial shooting point is set on the center line of the lane's total width, and the drone's shooting range is parallel to the lane's center line and the symmetry axis coincides with the lane's center line, and the left side of the shooting section is within the first drone's shooting range. Then, drones are added from left to right until the total shooting range of multiple drones completely covers the shooting section. 2) The lane type of the shooting section is a curve: the center point of the drone shooting range is located on the center line of the shooting section, and the point where the center point of the drone shooting range coincides with the center line of the shooting section is set as the base point. When the width of the shooting section is smaller than the preset width of the drone shooting range, the long side of the shooting range is parallel to the tangent line at the base point of the center line of the shooting section; when the width of the shooting section is larger than the preset width of the drone shooting range but smaller than the length of the drone shooting range, the wide side of the shooting range is parallel to the tangent line at the base point of the center line of the shooting section; Then, based on the above rules, the initial shooting point of the first drone is set on the midline of the total width of the shooting section, and one end of the shooting section is located within the shooting range of the first drone. Then, drones are added, and the newly added drones take the position of the previous drone as the starting point and move along the midline of the shooting section to the other end of the shooting section. At the same time, the drones are rotated to ensure that the long side / wide side of the drone shooting range is parallel to the tangent at the base point of the midline of the shooting section, until the shooting range of multiple drones completely covers the range of the section to be shot; 3) The shooting section includes both straight roads and curves: the shooting range of multiple drones is set to the circumscribed rectangular range of the shooting section, and multiple drones are added from left to right and from front to back until the shooting range of the drones covers the shooting section; Execute the corresponding setting strategy, the shooting range of the drone covers the shooting section, generate the initial shooting points, then lower the shooting height of the drone, and use the number of initial shooting points as the number of drones dispatched, and execute the corresponding setting strategy again until the total shooting range cannot cover the shooting section after a certain height reduction, the iteration stops, and the second to last generated initial shooting point is used as the shooting end point.

3. The method according to claim 2, characterized in that The shooting restrictions of the drone also include the maximum number of drones that can be dispatched. In the setting strategy, when the number of newly added drones is greater than the maximum number of drones, the generation of the initial shooting point fails, and step S2 is executed again.

4. The method according to claim 2, characterized in that: The shooting range is a rectangle, and the intersection of the diagonals of the shooting range coincides with the orthographic projection of the drone; The relationship between the length and width of the shooting range and the change in shooting height is: Among them, M FG is the longer side of the drone’s shooting range at any shooting height, N FG is the shorter side, M FO N is the longer side of the drone’s shooting range at the maximum shooting altitude. FO H is the shorter side of the drone’s shooting range at the maximum shooting altitude. FG H is the height of the drone at any initial shooting point. FO The maximum shooting altitude of the drone.

5. The method according to claim 2, characterized in that: The left side of the first drone's shooting range exceeds the specified width on the left side of the shooting section range; when the first strategy is set, the shooting endpoint of the next drone is generated at the flight endpoint of the previous drone, and then moves to the right along the center line of the lane or the middle line of the lane width until the overlapping part of the next drone and the previous drone is the specified width, then stops moving, and uses the stopping point as the initial shooting point of the next drone; when the second strategy is set, when the boundary of the total shooting range composed of the newly added drone and the previous drone is the specified width away from the lane, stops moving, and uses the stopping point as the initial shooting point of the newly added drone.

6. The method according to claim 1, characterized in that In order to minimize the total path of the drones, the starting point and the shooting end point of the drones are paired and combined, and then based on the wolf pack algorithm, the optimal flight path is obtained by joint optimization while ensuring that the drones do not collide.

7. The method according to claim 6, characterized in that First, all possible combinations of start and end points are obtained based on the permutations and combinations, and the length d of the straight line segment from the start point to the end point of each combination is calculated. s , and then calculate the d of each combination solution s The mean absolute deviation d a : in, is the length of the straight line segment from the starting point to the end point of the i-th group of the combination scheme, Select d as the average length of all straight line segments from the start point to the end point of this combination. a The smallest start-end combination solution is used as the start-end and end-point of the UAV path optimization, and D represents D tracks.

8. The method according to claim 7, characterized in that In the wolf pack algorithm, a single path combination scheme is used as an intelligent individual to optimize the flight path by maximizing the fitness value Y of the intelligent individual. The expression of Y is: Among them, W e , W s , W l is the weight coefficient of the corresponding factor, which is used to balance the importance of different factors; e ij is the length of the jth track segment of the ith track of the intelligent individual, s ij is the risk level of the jth track segment of the ith track of the intelligent individual; l a is the average absolute deviation of all the tracks of a single intelligent individual: in is the average length of all the tracks of a single intelligent individual, l i is the total length of the ith track:

9. The method according to claim 8, characterized in that Step S3 includes the following steps: Step S301, initializing intelligent group parameters; Step S302, calculate the fitness value of each intelligent individual at the location, select the intelligent individual with the largest fitness value as the leader, and the T with the fitness value lower than the leader n The leader initiates the call to call all intelligent individuals except the leader and navigator to their location; the navigator is responsible for dynamic search and continues to explore until the fitness of a navigator is better than the fitness of the leader, or the maximum number of navigator searches is reached. max ; Step S303, the intelligent individual m with a lower fitness value responds to the leader's call and moves quickly towards the direction of the optimal solution. If the fitness of the intelligent individual m with a lower fitness value is better than that of the leader during the movement, it will initiate the call behavior on behalf of the leader. Otherwise, the intelligent individual m will continue to respond to the call signal until the distance d between the intelligent individual m with a lower fitness value and the leader is ms Less than the distance judgment value d near When , all intelligent individuals with lower fitness values ​​begin to switch from responding to the call signal to concentrated search behavior, otherwise, the intelligent individuals with lower fitness values ​​continue to move towards the optimal solution; Step S304: select the leader's position as the optimal solution position, and the intelligent individuals with lower fitness values ​​conduct centralized search for the optimal solution, and update the positions of the intelligent individuals participating in the centralized search. If the updated intelligent individual position x mid If it exceeds the solution space range, it is set as the boundary value; Step S305, according to the leader generation rule, the leader position is updated, and then the intelligent group is updated, the R intelligent individuals with the worst fitness are deleted, and R intelligent individuals are randomly generated; Step S306: determine whether the optimization accuracy requirement is met or the number of iterations meets the maximum number K. max If yes, then output the position of the leader, that is, the final optimal flight path; if no, return to step S302.

10. A multi-UAV collaborative shooting system for natural driving data collection, characterized in that: It includes a drone and a ground terminal. The ground terminal obtains a shooting end point and an optimal flight path according to the method described in any one of claims 1 to 9, and transmits the shooting end point and the optimal flight path to the drone. After the drone flies to the shooting end point according to the optimal path, it shoots according to the instructions of the ground terminal and transmits the shooting data to the ground terminal.

Images