Coordination control method and system for multiple unmanned aerial vehicles

Abstract

The invention relates to the technical field of unmanned aerial vehicle control, and particularly discloses a coordinated control method and system for multiple unmanned aerial vehicles, and the method comprises the steps: carrying out the partitioning of a scene map according to the pedestrian flow at each coordinate; determining a motion path of the unmanned aerial vehicle according to the partitioned scene map, synchronously generating a control instruction based on the motion path, and sending the control instruction to the unmanned aerial vehicle; according to the invention, the total pedestrian volume in the monitoring area of each unmanned aerial vehicle is similar, when the total pedestrian volume in the monitoring area of the unmanned aerial vehicle is small, the unmanned aerial vehicle can monitor more areas, and when the total pedestrian volume in the monitoring area of the unmanned aerial vehicle is large, the unmanned aerial vehicle can monitor less areas. Finally, the unmanned aerial vehicles monitoring less areas can return to the main end more frequently, and data feedback is carried out; besides, the working range of the unmanned aerial vehicle is very easy to determine, can be quickly determined by computer equipment, can be quickly changed along with the change of actual conditions, and is relatively strong in dynamic property and high in integrating degree with the actual conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and in particular to a coordinated control method and system for multiple unmanned aerial vehicles. Background Art

[0002] Natural scenic spots refer to natural environment areas with natural landscapes as the main body and have not undergone large-scale artificial transformation or development. In existing natural scenic spots, such as forests and mountainous areas, cameras are only installed at important intersections due to cost issues, and the entire scene cannot be monitored. The comprehensiveness of the information acquisition process is slightly insufficient. Therefore, it is necessary to introduce a supplementary collection process based on drones.

[0003] However, the existing drone-based supplementary collection processes are mostly fixed collection solutions. For example, the scene is divided into multiple areas, and different drones are responsible for different partitions. Then a comprehensive path is pre-determined to collect information from the partitions at regular intervals. The logic of this method is relatively simple, but it assumes that all partitions are the same. In fact, the distribution of personnel is different, and different partitions themselves are different. The fixed collection solution ignores this difference and is not very consistent with the actual situation. Summary of the invention

[0004] The purpose of the present invention is to provide a coordinated control method and system for multiple UAVs to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A coordinated control method and system for multiple unmanned aerial vehicles, the method comprising:

[0007] Query the image collector registered in the scene, and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle;

[0008] Mapping the image into the scene map according to the acquisition parameters, performing personnel recognition on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes the position and orientation of the person;

[0009] Partition the scene map according to the flow of people at each coordinate;

[0010] Determine the motion path of the UAV based on the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV;

[0011] The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

[0012] As a further solution of the present invention: the steps of mapping the image into the scene map according to the acquisition parameters, identifying people on the scene map containing the image, obtaining people information, and calculating the flow of people at each coordinate in the scene map according to the people information include:

[0013] Read the scene map of the scene;

[0014] Determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates containing a time tag;

[0015] Create a map grid according to the preset grid length, and use each grid unit as the membership range of the grid node at the upper left endpoint;

[0016] Determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the belonging range in the latest mapping area, and insert it into the belonging range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than the preset time threshold;

[0017] After each grid node is processed, the latest scene map is obtained;

[0018] Perform personnel identification on the latest scene map, obtain personnel information, and calculate the flow of people at each coordinate in the scene map according to the personnel information;

[0019] When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

[0020] As a further solution of the present invention: the steps of identifying personnel on the latest scene map, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include:

[0021] Based on the preset personnel feature table, the scene map is traversed to locate personnel and obtain their position and orientation.

[0022] Determine the instantaneous flow of people at each coordinate in the scene map according to the positions and orientations of all people;

[0023] Record the instantaneous flow of people at each coordinate corresponding to the scene map at each moment within a preset time period, and calculate the flow of people at each coordinate;

[0024] The calculation process of the instantaneous flow of people is:

[0025] As a further solution of the present invention: the step of partitioning the scene map according to the flow of people at each coordinate includes:

[0026] For any two coordinates, calculate the spatial distance;

[0027] Calculate the difference in the flow of people at the two coordinates as the flow distance;

[0028] Clustering all coordinates according to the spatial distance and the flow distance, each type of coordinates forming a sub-area;

[0029] Among them, the clustering process is:.

[0030] As a further solution of the present invention: the step of determining the motion path of the UAV according to the partitioned scene map, and synchronously generating a control instruction based on the motion path and sending it to the UAV comprises:

[0031] Read the partitioned scene map and calculate the sum of the flow of people at all coordinates in each sub-area as the feature value of the sub-area;

[0032] Select UAVs in turn based on the characteristic value of each sub-area, and simultaneously determine the movement path of the UAVs;

[0033] Count the selected UAVs and their motion paths as a monitoring plan; wherein the data structure of the monitoring plan is a table, including a sequence number column and a motion path column; the sequence number in the sequence number column represents the sequence number of the selected UAV, and the motion path in the motion path column represents the motion path corresponding to each UAV;

[0034] The best solution is selected from all monitoring solutions, and the control instructions for each drone are determined based on the best solution and sent to the drones.

[0035] As a further solution of the present invention: the step of selecting drones in sequence based on the characteristic value of each sub-area and synchronously determining the motion path of the drones comprises:

[0036] Randomly select a sub-area in the scene map as the starting area;

[0037] Selecting the next area in a chain based on the starting area as the passing area;

[0038] Calculate the total eigenvalues ​​of all passed areas, and stop the chain selection process when the total eigenvalue reaches the preset threshold;

[0039] A new UAV is selected, and a motion path of the UAV is determined in the selected passing area as the motion path of the UAV; the condition for determining the motion path is that the union of the collection areas of the UAV on the motion path is not less than the passing area, and the motion path includes at least an initial segment, which is a segment of the path from the storage place of the UAV to the starting area;

[0040] In the scene map, the areas that have been used as passing areas are eliminated, and the above process is recursively performed until all passing areas correspond to at least one motion path.

[0041] As a further solution of the present invention: the step of chain-selecting the next area based on the starting area as the passing area includes:

[0042] Calculate the eigenvalue differences of adjacent regions and select the region with the smallest eigenvalue difference as the next region.

[0043] As a further solution of the present invention: the step of selecting the best solution from all monitoring solutions, determining the control instructions of each drone based on the best solution, and sending the control instructions to the drones includes:

[0044] For any monitoring scheme, the sum of the path lengths of all motion paths is calculated, and the monitoring scheme with the smallest sum is selected as the optimal scheme.

[0045] The technical solution of the present invention also provides a coordinated control system for multiple UAVs, the system comprising:

[0046] An image receiving module is used to query the image collector registered in the scene and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle;

[0047] A personnel identification module is used to map the image into a scene map according to the acquisition parameters, identify personnel on the scene map containing the image, obtain personnel information, and calculate the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes personnel position and personnel orientation;

[0048] Map partitioning module, used to partition the scene map according to the flow of people at each coordinate;

[0049] A control instruction generation module is used to determine the motion path of the UAV according to the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV;

[0050] The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

[0051] As a further solution of the present invention: the image receiving module includes:

[0052] A map reading unit, used for reading a scene map of a scene;

[0053] A mapping area determination unit, used to determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a boundary coordinate set containing a time tag;

[0054] A grid building unit is used to create a map grid according to a preset grid length, and each grid unit is used as the belonging range of the grid node at the upper left end point;

[0055] The interception and insertion unit is used to determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the belonging range in the latest mapping area, and insert it into the belonging range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold;

[0056] A map output unit is used to obtain the latest scene map after each grid node has been processed;

[0057] A crowd flow calculation unit, used to identify people on the latest scene map, obtain people information, and calculate the crowd flow at each coordinate in the scene map according to the people information;

[0058] When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The present invention determines the distribution of people in the entire scene according to the existing monitoring system, and determines the working range of different drones according to the distribution of people, so that the total flow of people in the monitoring area of ​​each drone is similar. When the total flow of people in the monitoring area of ​​the drone is small, it will monitor more areas. When the total flow of people in the monitoring area of ​​the drone is large, it will monitor fewer areas. Finally, the drones monitoring fewer areas will return to the main end more frequently to provide data feedback. In addition, the process of determining the working range of the drone is very easy, and can be quickly determined by computer equipment, and can be quickly changed as the actual situation changes. It is highly dynamic and highly consistent with the actual situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0062] Figure 1The overall flow chart of the coordinated control method of multiple UAVs is shown.

[0063] Figure 2 The structural diagram of the coordinated control system of multiple UAVs is shown. DETAILED DESCRIPTION

[0064] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] Figure 1 The figure is a general flow chart of a coordinated control method and system for multiple UAVs. In an embodiment of the present invention, a coordinated control method for multiple UAVs includes:

[0066] Step S100: querying the image collector registered in the scene, and receiving the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle;

[0067] The image collector in this application is slightly different from the traditional image collector. The traditional image collector is a camera installed in the scene. These cameras are generally set at key intersections to obtain videos in the scene. When the area of ​​the scene is large, the monitoring area of ​​the camera generally does not completely cover the entire scenic area; while the image collector in this application includes smart devices of people in the scene, such as mobile phones, etc. Smart devices can also take pictures. If a connection channel is established between the people in the scene and the execution subject of this method, the people in the scene can upload some images through the smart devices, which has a similar function to the camera.

[0068] It is worth mentioning that the camera of this application is also different from the traditional camera in that the video it obtains does not need to be uploaded in real time, but only needs to send some image frames at regular intervals.

[0069] Step S200: mapping the image into a scene map according to the acquisition parameters, performing personnel recognition on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes personnel position and personnel orientation;

[0070] When the image collector acquires an image, it will simultaneously record the acquisition parameters. The acquisition parameters indicate when and where the image was acquired, the direction of acquisition, and the wide angle of the image collector. They are represented by the four parameters of acquisition time, acquisition coordinates, acquisition direction, and acquisition wide angle. The acquired image is a plane image. Combined with the acquisition parameters, the plane image can be mapped to the scene map. The process of converting the plane image to a horizontal plane based on the acquisition parameters belongs to the prior art and will not be discussed in this application. After mapping the image to the scene map, personnel identification is performed on the scene map to obtain the distribution of personnel in the scene, which is called personnel information. The personnel information is analyzed to calculate the flow of people at each coordinate in the scene map. It should be noted that the personnel information in this application includes the position and orientation of each person, which means that the calculation process of the flow of people is not only related to the position of the person, but also to the orientation of the person.

[0071] Step S300: partitioning the scene map according to the flow of people at each coordinate;

[0072] After the flow of people at each coordinate in the scene map is calculated, the scene map can be partitioned based on the flow of people. The partitioning rule is to classify two coordinates with similar flow of people into one area. Similar flow of people means that the difference in flow of people is less than the preset difference threshold.

[0073] Step S400: determining the motion path of the UAV according to the partitioned scene map, synchronously generating a control instruction based on the motion path, and sending it to the UAV;

[0074] After the scene map is partitioned, a UAV is selected according to the partitioned scene map, and the motion path of the UAV is determined. A control command is generated based on the motion path and sent to the selected UAV.

[0075] Specifically, regarding the drone selection process, the drone selection process of the present application is a sequential selection process. For each sub-area in the scene map (the small area obtained by partitioning is called a sub-area), a string of sub-areas are selected as the monitoring objects of a drone. After the selection is completed, the string of sub-areas is removed from the original scene map, and from the remaining sub-areas, another string of sub-areas is selected as the monitoring objects of the next drone. The above process is continuously executed. Each time a string of sub-areas is selected, a drone is equipped for detection until all sub-areas are monitored by at least one drone.

[0076] Specifically, the process of selecting a string of sub-areas is to first randomly select an initial sub-area, then select the next sub-area from the adjacent sub-areas, and then further select the next sub-area from the adjacent sub-area of ​​the next sub-area, and so on, to obtain a string of sub-areas; the length of the sub-area is determined by a preset pedestrian flow condition, for example, the sum of all pedestrian flows in the sub-area reaches a preset pedestrian flow threshold.

[0077] In addition, there is an implicit condition in the process of determining the movement path of the drone in a string of sub-areas, that is, the sum of the collection areas of the drone on the movement path needs to completely cover the corresponding string of sub-areas.

[0078] Furthermore, regarding the significance of the above-mentioned drone control process, it can be understood that the function of the drone is actually to obtain video, and the camera of the present application can also obtain video. If a wireless transmission channel is built, the execution subject of the present method (the master control end) can obtain the situation in the scene in real time, and the real-time performance is higher. For this, the selection of drones for video acquisition seems to be a bit superfluous, but in fact it is not the case. First, the scene for which the present application is generally large, such as forest areas or mountainous areas, and the camera will only be installed at important intersections, not globally. Its comprehensiveness is slightly insufficient. By using drones, images of all locations in the scene can be obtained, which is more comprehensive. Second, the camera uploads the video in real time, which consumes a lot of resources, and the amount of data of the video itself is very large, and the transmission process will be very slow. By using drones to obtain images of all locations, when it returns to the master control end, it is very possible to directly use the card reader to obtain the collected data, which consumes very little resources and has a faster transmission speed. Therefore, in summary, the monitoring architecture of the present application reduces the real-time performance, but improves the comprehensiveness of the data, reduces the difficulty of data acquisition, and reduces the cost, including the layout cost of the camera (the number can be reduced) and the resource cost required for real-time data upload.

[0079] It should be noted that regarding the movement paths of all drones, the process of determining the movement paths of drones is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold. When monitoring is performed on the corresponding movement paths of all drones, the union of the monitoring ranges completely covers the scene.

[0080] Regarding step S200, the steps of mapping the image into the scene map according to the acquisition parameters, identifying personnel on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include:

[0081] Read the scene map of the scene;

[0082] Determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates containing a time tag;

[0083] Create a map grid according to the preset grid length, and use each grid unit as the membership range of the grid node at the upper left endpoint;

[0084] Determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the belonging range in the latest mapping area, and insert it into the belonging range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than the preset time threshold;

[0085] After each grid node is processed, the latest scene map is obtained;

[0086] Personnel identification is performed on the latest scene map to obtain personnel information, and the flow of people at each coordinate in the scene map is calculated based on the personnel information.

[0087] In an example of the technical solution of the present invention, the recognition and analysis process of the scene map is described. The scene map of each scene is known data. The scene map of the scene is read, and the mapping area is determined in the scene map according to the acquisition parameters. The mapping area is mainly a two-dimensional outline, that is, a range in the scene map; a map grid is superimposed on the scene map, and the map grid is the grid function in many existing software. Each grid unit is a rectangle, and each grid unit is used as the belonging range of the grid node of the upper left endpoint. In other words, each grid unit belongs to the range of its upper left endpoint; for each grid node, the latest mapping area corresponding to it is queried (the mapping area corresponding to the image collected at the most recent time). The area of ​​the latest mapping area is very large, and the range corresponding to the grid node is intercepted and retained; when each grid node is operated once, the latest scene map is obtained; the purpose of this point-by-point interception and splicing scheme is to coordinate the conflict problem of multiple mapping areas, because the acquired images are likely to overlap in area, and which one to select from the overlapping parts is actually a small problem. The present application directly splits the scene map into multiple small blocks (grid units) and processes each small block separately, thereby solving the overlapping selection problem.

[0088] Finally, personnel identification is performed on the latest scene map to obtain personnel information, and the flow of people at each coordinate in the scene map is calculated based on the personnel information.

[0089] It should be noted that when there is no mapping area at a certain grid node, the data within the subordinate range will be cleared. The meaning of this process is that when each grid unit queries the latest mapping area, it only queries the mapping area within a preset time range, such as the mapping area within one day. If the mapping area at the most recent moment exceeds one day, the original map will be retained, that is, the data within the subordinate range will be cleared.

[0090] Specifically, the steps of identifying personnel on the latest scene map, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include:

[0091] Based on the preset personnel feature table, the scene map is traversed to locate personnel and obtain their position and orientation.

[0092] Determine the instantaneous flow of people at each coordinate in the scene map according to the positions and orientations of all people;

[0093] Record the instantaneous flow of people at each coordinate corresponding to the scene map at each moment within a preset time period, and calculate the flow of people at each coordinate.

[0094] The above content explains the calculation process of pedestrian flow. Human flow is a very common parameter in the prior art. The conventional human flow is the number of people per unit area. On this basis, this application defines the parameter of pedestrian flow. First, based on the preset personnel feature table, the scene map is traversed to locate the personnel, and the personnel position and personnel orientation are obtained at the same time. The instantaneous human flow at each location is determined by combining the two parameters of personnel position and personnel orientation. The same operation is performed on the scene map at each moment. At this time, the instantaneous human flow at different times can be obtained at each coordinate, and the final human flow can be calculated based on the instantaneous human flow at different times; wherein, the final human flow can be the sum of the instantaneous human flow or the mean of the instantaneous human flow.

[0095] Specifically, the calculation process of the instantaneous flow of people is as follows:

[0096]

[0097]

[0098] In the formula, S(x, y) represents the instantaneous flow of people at the position (x, y), N represents the total number of people located, and T i represents the projection of the ith person at position (x, y), x i Represents the horizontal coordinate of the i-th person, y i represents the vertical coordinate of the i-th person; a i It is the direction vector in coordinate form of the person orientation of the i-th person.

[0099] The above calculation process is explained as follows: for a certain position, the influence of all people at that position needs to be considered. For any person, the coordinates of the person are obtained, and the vector between it and the position is calculated based on the coordinates of the person as the relative position vector. Then, the person's orientation is obtained. The person's orientation also uses a vector in the form of coordinates. The projection of the person's orientation on the relative position vector is calculated. Finally, the sum of the projections corresponding to all people is calculated to obtain the instantaneous flow of people with direction.

[0100] The above process introduces the concept of personnel orientation on the basis of tradition. If a certain position is in the personnel orientation of multiple persons, it means that the position is the personnel's focus position. Even if the personnel have not reached the position, the position is considered to have human traffic. In actual scenarios, there are many places that personnel cannot reach. After the above processing, the positions that personnel cannot reach can also be counted.

[0101] It should be noted that in the process of calculating the projection amount, when a person's direction is opposite to the position to be analyzed and the projection amount is non-positive, the projection amount is set to zero, which means that the concept of negative influence is not introduced. Because as the person moves, the person's position changes. Even if he was facing a certain position before, as the position changes, he will eventually turn his back to that position. If a negative influence is introduced, the projection amount previously facing that position is actually eliminated. Therefore, this application does not consider negative values.

[0102] In addition, when the projection amount is positive, The term is a coefficient, and the denominator represents the distance. The greater the distance, the smaller the projection amount. The term represents the projection of the personnel's orientation on the relative position vector, which belongs to the projection calculation formula.

[0103] It is worth mentioning that in the calculation process of the projection amount, Introduce α into the term i term, as the numerator, α i The item is a weight coefficient item, which is used to indicate the importance of different personnel. For example, VIP personnel can choose a larger weight coefficient item. This needs to be specifically defined according to the actual situation and will not be elaborated in this application.

[0104] Regarding step S300, the step of partitioning the scene map according to the flow of people at each coordinate includes:

[0105] For any two coordinates, calculate the spatial distance;

[0106] Calculate the difference in the flow of people at the two coordinates as the flow distance;

[0107] All coordinates are clustered according to the spatial distance and the human flow distance, and each type of coordinates constitutes a sub-area.

[0108] In an example of the technical solution of the present invention, the partitioning process of the scene map is explained. The partitioning process is very simple, which is to classify similar coordinates into one category. The focus of this application is that similar coordinates require similar pedestrian flow in addition to the traditional distance. Specifically, for any two coordinates, the spatial distance and the pedestrian flow distance are calculated, and all coordinates are clustered together according to the spatial distance and the pedestrian flow distance.

[0109] Among them, the clustering process is:.

[0110] Regarding step S400, the steps of determining the motion path of the drone according to the partitioned scene map, and synchronously generating a control instruction based on the motion path and sending it to the drone include:

[0111] Read the partitioned scene map and calculate the sum of the flow of people at all coordinates in each sub-area as the feature value of the sub-area;

[0112] Select UAVs in turn based on the characteristic value of each sub-area, and simultaneously determine the movement path of the UAVs;

[0113] Count the selected UAVs and their motion paths as a monitoring plan; wherein the data structure of the monitoring plan is a table, including a sequence number column and a motion path column; the sequence number in the sequence number column represents the sequence number of the selected UAV, and the motion path in the motion path column represents the motion path corresponding to each UAV;

[0114] The best solution is selected from all monitoring solutions, and the control instructions for each drone are determined based on the best solution and sent to the drones.

[0115] In an example of the technical solution of the present invention, the partitioned scene map is read, and the sum of the pedestrian flow of all coordinates in each sub-area is calculated as the characteristic value of the sub-area. The larger the characteristic value, the larger the pedestrian flow. Based on the characteristic value of each sub-area, drones are selected in turn, and the movement path of the drones is determined synchronously. There are many ways to select drones and determine the movement path of drones, each of which is called a monitoring plan. The best plan is selected from the multiple determined monitoring plans as the final plan. Based on the final plan, the control instructions for each drone are determined and sent to the drones.

[0116] Specifically, the step of selecting drones in sequence based on the characteristic value of each sub-area and synchronously determining the motion path of the drones includes:

[0117] Randomly select a sub-area in the scene map as the starting area;

[0118] Selecting the next area in a chain based on the starting area as the passing area;

[0119] Calculate the total eigenvalues ​​of all passed areas, and stop the chain selection process when the total eigenvalue reaches the preset threshold;

[0120] A new UAV is selected, and a motion path of the UAV is determined in the selected passing area as the motion path of the UAV; the condition for determining the motion path is that the union of the collection areas of the UAV on the motion path is not less than the passing area, and the motion path includes at least an initial segment, which is a segment of the path from the storage place of the UAV to the starting area;

[0121] In the scene map, the areas that have been used as passing areas are eliminated, and the above process is recursively performed until all passing areas correspond to at least one motion path.

[0122] For the process of selecting drones in sequence based on the characteristic values ​​of each sub-area and simultaneously determining the motion path of the drone, the process of selecting drones is to select them in sequence, and the motion path of the drone is determined by the characteristic values ​​of each sub-area. Specifically:

[0123] The process of selecting drones in this application is a sequential selection process, first selecting a drone, determining its movement path, then selecting the next drone, determining its movement path, and continuously executing this process in a loop until the monitoring areas of all drones on the movement paths completely cover the entire scene.

[0124] A sub-area is randomly selected in the scene map as the starting area, and the next area is chain-selected based on the starting area as the passing area. The meaning of chain selection is that the selected area must be adjacent to the previous area, and the obtained passing area is a "string of areas". When the process of chain selection of the next area stops is determined by the total value of the characteristic values ​​of all passing areas. When the total value of the characteristic values ​​reaches a preset threshold, the chain selection process is stopped; at this time, the selected passing area is used as a monitoring area of ​​a drone, and the movement path of the drone is determined in the selected passing area as the movement path of the drone.

[0125] After the above processing, a drone and its motion path are obtained. On this basis, the areas that have been passed through are eliminated in the scene map, and the above process is recursively executed to obtain multiple drones and their motion paths. When all passed through areas correspond to at least one motion path, it means that all areas in the entire scene have at least one drone for image collection. At this time, the selected drones and their motion paths are statistically selected as a monitoring plan.

[0126] Since the initially selected area, that is, the starting area, is random, and the next area selected in the chain selection process is also random, the movement path of each drone is not fixed. By executing the monitoring plan generation process multiple times, multiple monitoring plans can be obtained.

[0127] It is worth mentioning that regarding the generation process of the motion path of each drone, when the passing area is known, the generation process of the motion path is a plan. Its requirement is that the union of the collection area of ​​the drone on the motion path is not less than the passing area, and the motion path includes at least an initial segment, which is a section of the path from the drone storage location to the starting area, that is, the drone needs to reach the starting area first and then completely cover all the passing areas; a conventional motion path generation scheme is to obtain the single monitoring radius of the drone, divide the passing area into multiple strip areas by the single monitoring radius, and connect the center lines of each strip area to obtain the final motion path.

[0128] As an example of the technical solution of the present invention, the step of chain-selecting the next area based on the starting area as the passing area includes:

[0129] Calculate the eigenvalue differences of adjacent regions and select the region with the smallest eigenvalue difference as the next region.

[0130] In the above content, the selection process of the starting area is random, and the next area selected in the chain selection process is also random. The randomness of this method is too high. In order to reduce a certain degree of randomness, this application limits the process of selecting the next area in the chain selection process. For a certain area, when selecting the next area, the difference between the feature values ​​of the adjacent area and the current area is calculated, and the next area with the smallest difference is selected as the selection result.

[0131] As a preferred embodiment of the technical solution of the present invention, the steps of selecting the best solution from all monitoring solutions, determining the control instructions of each drone based on the best solution, and sending the control instructions to the drone include:

[0132] For any monitoring scheme, the sum of the path lengths of all motion paths is calculated, and the monitoring scheme with the smallest sum is selected as the optimal scheme.

[0133] In an example of the technical solution of the present invention, the process of selecting the optimal solution is described. For any monitoring solution, the sum of the path lengths of all motion paths is calculated, and the monitoring solution with the smallest sum is selected as the optimal solution.

[0134] It is worth mentioning that there is another selection scheme, which is to use each passing area as a benchmark, query the number of motion paths corresponding to each passing area, and calculate the total number. The total number indicates how many drones will pass through the passing area during the movement process (drones generally pass through other passing areas in the initial stage). The monitoring scheme with the least total number is selected as the optimal scheme.

[0135] Figure 2The structure diagram of the coordinated control system of multiple UAVs is shown. In a preferred embodiment of the technical solution of the present invention, a coordinated control system of multiple UAVs is also provided. The system 10 includes:

[0136] The image receiving module 11 is used to query the image collector registered in the scene and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle;

[0137] A personnel identification module 12 is used to map the image into a scene map according to the acquisition parameters, identify personnel on the scene map containing the image, obtain personnel information, and calculate the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes the position and orientation of the personnel;

[0138] A map partitioning module 13 is used to partition the scene map according to the flow of people at each coordinate;

[0139] A control instruction generation module 14 is used to determine the motion path of the UAV according to the partitioned scene map, and simultaneously generate a control instruction based on the motion path and send it to the UAV;

[0140] The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

[0141] Furthermore, the image receiving module 11 includes:

[0142] A map reading unit, used for reading a scene map of a scene;

[0143] A mapping area determination unit, used to determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a boundary coordinate set containing a time tag;

[0144] A grid building unit is used to create a map grid according to a preset grid length, and each grid unit is used as the belonging range of the grid node at the upper left end point;

[0145] The interception and insertion unit is used to determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the belonging range in the latest mapping area, and insert it into the belonging range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold;

[0146] A map output unit is used to obtain the latest scene map after each grid node has been processed;

[0147] A crowd flow calculation unit, used to identify people on the latest scene map, obtain people information, and calculate the crowd flow at each coordinate in the scene map according to the people information;

[0148] When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

[0149] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A coordinated control method for multiple UAVs, characterized in that: The method comprises: Query the image collector registered in the scene, and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle; Mapping the image into the scene map according to the acquisition parameters, performing personnel recognition on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes the position and orientation of the person; Partition the scene map according to the flow of people at each coordinate; Determine the motion path of the UAV based on the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV; The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

2. The coordinated control method of multiple UAVs according to claim 1, characterized in that: The steps of mapping the image into the scene map according to the acquisition parameters, performing personnel recognition on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include: Read the scene map of the scene; Determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates containing a time tag; Create a map grid according to the preset grid length, and use each grid unit as the membership range of the grid node at the upper left endpoint; Determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the belonging range in the latest mapping area, and insert it into the belonging range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than the preset time threshold; After each grid node is processed, the latest scene map is obtained; Perform personnel identification on the latest scene map, obtain personnel information, and calculate the flow of people at each coordinate in the scene map according to the personnel information; When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

3. The coordinated control method of multiple UAVs according to claim 2, characterized in that: The steps of identifying personnel on the latest scene map, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include: Based on the preset personnel feature table, the scene map is traversed to locate personnel and obtain their position and orientation. Determine the instantaneous flow of people at each coordinate in the scene map according to the positions and orientations of all people; Record the instantaneous flow of people at each coordinate corresponding to the scene map at each moment within a preset time period, and calculate the flow of people at each coordinate; The calculation process of the instantaneous flow of people is:

4. The coordinated control method of multiple UAVs according to claim 1, characterized in that: The step of partitioning the scene map according to the flow of people at each coordinate includes: For any two coordinates, calculate the spatial distance; Calculate the difference in the flow of people at the two coordinates as the flow distance; Clustering all coordinates according to the spatial distance and the flow distance, each type of coordinates forming a sub-area; Among them, the clustering process is:.

5. The coordinated control method of multiple UAVs according to claim 1, characterized in that: The steps of determining the motion path of the UAV according to the partitioned scene map, and synchronously generating a control instruction based on the motion path and sending it to the UAV include: Read the partitioned scene map and calculate the sum of the flow of people at all coordinates in each sub-area as the feature value of the sub-area; Select UAVs in turn based on the characteristic value of each sub-area, and simultaneously determine the movement path of the UAVs; Count the selected UAVs and their motion paths as a monitoring plan; wherein the data structure of the monitoring plan is a table, including a sequence number column and a motion path column; the sequence number in the sequence number column represents the sequence number of the selected UAV, and the motion path in the motion path column represents the motion path corresponding to each UAV; The best solution is selected from all monitoring solutions, and the control instructions for each drone are determined based on the best solution and sent to the drones.

6. The coordinated control method of multiple UAVs according to claim 5, characterized in that: The step of selecting drones in sequence based on the characteristic values ​​of each sub-area and synchronously determining the motion path of the drones comprises: Randomly select a sub-area in the scene map as the starting area; Selecting the next area in a chain based on the starting area as the passing area; Calculate the total eigenvalues ​​of all passed areas, and stop the chain selection process when the total eigenvalue reaches the preset threshold; A new UAV is selected, and a motion path of the UAV is determined in the selected passing area as the motion path of the UAV; the condition for determining the motion path is that the union of the collection areas of the UAV on the motion path is not less than the passing area, and the motion path includes at least an initial segment, which is a segment of the path from the storage place of the UAV to the starting area; In the scene map, the areas that have been used as passing areas are eliminated, and the above process is recursively performed until all passing areas correspond to at least one motion path.

7. The coordinated control method of multiple UAVs according to claim 6, characterized in that: The step of chain-selecting the next area based on the starting area as the passing area comprises: Calculate the eigenvalue differences of adjacent regions and select the region with the smallest eigenvalue difference as the next region.

8. The coordinated control method of multiple UAVs according to claim 5, characterized in that: The steps of selecting the best solution from all monitoring solutions, determining the control instructions for each drone based on the best solution, and sending the control instructions to the drones include: For any monitoring scheme, the sum of the path lengths of all motion paths is calculated, and the monitoring scheme with the smallest sum is selected as the optimal scheme.

9. A coordinated control system for multiple UAVs, characterized in that: The system comprises: An image receiving module is used to query the image collector registered in the scene and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle; A personnel identification module is used to map the image into a scene map according to the acquisition parameters, identify personnel on the scene map containing the image, obtain personnel information, and calculate the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes personnel position and personnel orientation; Map partitioning module, used to partition the scene map according to the flow of people at each coordinate; A control instruction generation module is used to determine the motion path of the UAV according to the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV; The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

10. The coordinated control system of multiple UAVs according to claim 9, characterized in that: The image receiving module comprises: A map reading unit, used for reading a scene map of a scene; A mapping area determination unit, used to determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a boundary coordinate set containing a time tag; A grid building unit is used to create a map grid according to a preset grid length, and each grid unit is used as the belonging range of the grid node at the upper left end point; The interception and insertion unit is used to determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the belonging range in the latest mapping area, and insert it into the belonging range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold; A map output unit is used to obtain the latest scene map after each grid node has been processed; A crowd flow calculation unit, used to identify people on the latest scene map, obtain people information, and calculate the crowd flow at each coordinate in the scene map according to the people information; When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

Images