A method and system for coordinated control of multiple unmanned aerial vehicles

Abstract

The application relates to the technical field of unmanned plane control, and particularly discloses a coordinated control method and system for multiple unmanned planes, which comprises the following steps: partitioning a scene map according to the passenger flow at each coordinate; determining the movement path of the unmanned plane according to the partitioned scene map, synchronously generating a control instruction based on the movement path, and sending the control instruction to the unmanned plane; the application makes the total passenger flow in the monitoring area of each unmanned plane similar, when the total passenger flow in the monitoring area of the unmanned plane is less, the unmanned plane monitors more areas, when the total passenger flow in the monitoring area of the unmanned plane is more, the unmanned plane monitors less areas, and finally the unmanned plane monitoring less areas returns to the total end more frequently to feed back data; in addition, the determination process of the working range of the unmanned plane is very easy, can be quickly determined by a computer device, can be quickly changed according to the actual situation, is high in dynamicity, and is high in fitting degree with the actual situation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to a coordinated control method and system for multiple unmanned aerial vehicles. BACKGROUND

[0002] Natural scenic area refers to a natural environment area with natural landscape as the main body and without large-scale artificial modification or development. Considering the cost, the existing natural scenic area such as forest area and mountain area can only install cameras at important intersections, and cannot monitor the whole scene, so the information acquisition process is slightly insufficient in comprehensiveness, and therefore a supplementary acquisition process based on unmanned aerial vehicles needs to be introduced.

[0003] However, the existing supplementary acquisition process based on unmanned aerial vehicles is mostly a fixed acquisition scheme, such as dividing the scene into multiple areas, assigning different unmanned aerial vehicles to different areas, and then determining a comprehensive path in advance to collect information in the areas at a fixed time. This way is relatively simple in logic, but it assumes that all areas are the same, while in fact the personnel distribution is different and different areas are not the same, and the fixed acquisition scheme ignores this difference and does not fit the actual situation well. SUMMARY

[0004] The present application aims to provide a coordinated control method and system for multiple unmanned aerial vehicles to solve the problems in the background.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0006] A coordinated control method and system for multiple unmanned aerial vehicles, the method comprising:

[0007] Querying the recorded image collectors in the scene, receiving the images containing acquisition parameters fed back by the image collectors; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide-angle;

[0008] Mapping the images to the scene map according to the acquisition parameters, identifying the personnel in the scene map containing the images, obtaining personnel information, and calculating the passenger flow at each coordinate in the scene map according to the personnel information; the personnel information includes personnel position and personnel orientation;

[0009] Dividing the scene map according to the passenger flow at each coordinate;

[0010] Determining the motion path of the unmanned aerial vehicles according to the divided scene map, synchronously generating control instructions based on the motion path, and sending them to the unmanned aerial vehicles;

[0011] The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

[0012] As a further solution of the present invention, the steps of mapping the image into a scene map according to the acquisition parameters, performing personnel identification on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include:

[0013] Read the scene map of the scene;

[0014] Determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates containing a time tag;

[0015] Create a map grid based on the preset grid length, with each grid cell as the membership range of the grid node at the upper left endpoint;

[0016] Determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the membership range in the latest mapping area, and insert it into the membership range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold;

[0017] After each grid node is processed, the latest scene map is obtained;

[0018] Identify people on the latest scene map, obtain person information, and calculate the flow of people at each coordinate in the scene map based on the person information;

[0019] When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

[0020] As a further solution of the present invention, the steps of identifying people on the latest scene map, obtaining people information, and calculating the flow of people at each coordinate in the scene map according to the people information include:

[0021] Traverse the scene map based on the preset personnel feature table, locate the personnel, and obtain the personnel position and orientation;

[0022] Determine the instantaneous flow of people at each coordinate in the scene map based on the positions and orientations of all people;

[0023] Record the instantaneous flow of people at each coordinate corresponding to the scene map at each moment within a preset time period, and calculate the flow of people at each coordinate;

[0024] The calculation process of the instantaneous flow of people is:

[0025] As a further scheme of the present application, the step of partitioning the scene map according to the crowd flow at each coordinate comprises:

[0026] For any two coordinates, calculate the spatial distance;

[0027] Calculate the difference in crowd flow of the two coordinates as the crowd flow distance;

[0028] Cluster all coordinates according to the spatial distance and the crowd flow distance, and each class of coordinates constitutes a sub-region;

[0029] Wherein, the clustering process is:.

[0030] As a further scheme of the present application, the step of determining the movement path of the unmanned aerial vehicle according to the partitioned scene map, synchronously generating the control instruction based on the movement path, and sending to the unmanned aerial vehicle comprises:

[0031] Read the partitioned scene map, calculate the sum of the crowd flow of all coordinates in each sub-region as the characteristic value of the sub-region;

[0032] Select the unmanned aerial vehicle based on the characteristic value of each sub-region in turn, and synchronously determine the movement path of the unmanned aerial vehicle;

[0033] Statistical selected unmanned aerial vehicle and its movement path as a monitoring scheme; wherein, the data structure of the monitoring scheme is a table, including a serial number column and a movement path column; the serial number in the serial number column represents the serial number of the selected unmanned aerial vehicle, and the movement path in the movement path column represents the corresponding movement path of each unmanned aerial vehicle;

[0034] Select the optimal scheme from all monitoring schemes, determine the control instruction of each unmanned aerial vehicle based on the optimal scheme, and send to the unmanned aerial vehicle.

[0035] As a further scheme of the present application, the step of selecting the unmanned aerial vehicle based on the characteristic value of each sub-region in turn, and synchronously determining the movement path of the unmanned aerial vehicle comprises:

[0036] Randomly select a sub-region in the scene map as a starting area;

[0037] Chain select the next area based on the starting area as a passing area;

[0038] Calculate the total value of the characteristic values of all passing areas, and when the total value of the characteristic values reaches a preset threshold, stop the chain selection process;

[0039] A new drone is selected, and its motion path is determined within the selected transit area. This path is then used as the drone's motion path. The conditions for determining the motion path are that the union of the drone's collection areas along the motion path is no smaller than the transit area, and the motion path includes at least an initial segment, which is the path from the drone's storage location to the starting area.

[0040] In the scene map, the areas that have been used as passing areas are eliminated, and the above process is recursively performed until all passing areas correspond to at least one motion path.

[0041] As a further solution of the present invention, the step of chain-selecting the next area based on the starting area as the passing area includes:

[0042] Calculate the eigenvalue differences of adjacent regions and select the region with the smallest eigenvalue difference as the next region.

[0043] As a further solution of the present invention, the steps of selecting the best solution from all monitoring solutions, determining the control instructions for each drone based on the best solution, and sending the control instructions to the drones include:

[0044] For any monitoring scheme, the sum of the path lengths of all motion paths is calculated, and the monitoring scheme with the smallest sum is selected as the optimal scheme.

[0045] The technical solution of the present invention also provides a coordinated control system for multiple UAVs, the system comprising:

[0046] An image receiving module is used to query the image collector registered in the scene and receive images with acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle;

[0047] A personnel identification module is used to map the image into a scene map according to the acquisition parameters, identify personnel on the scene map containing the image, obtain personnel information, and calculate the flow of personnel at each coordinate in the scene map based on the personnel information; the personnel information includes the position and orientation of the personnel;

[0048] Map partitioning module, used to partition the scene map according to the flow of people at each coordinate;

[0049] The control instruction generation module is used to determine the motion path of the UAV based on the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV;

[0050] The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

[0051] As a further solution of the present invention: the image receiving module includes:

[0052] A map reading unit, used for reading a scene map of a scene;

[0053] A mapping area determination unit, configured to determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates including a time tag;

[0054] A grid building unit is used to create a map grid according to a preset grid length, with each grid unit as the belonging range of the grid node at the upper left endpoint;

[0055] The interception and insertion unit is used to determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the membership range in the latest mapping area, and insert it into the membership range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold;

[0056] Map output unit, used to obtain the latest scene map after each grid node is processed;

[0057] A pedestrian flow calculation unit is used to identify people on the latest scene map, obtain personnel information, and calculate the pedestrian flow at each coordinate in the scene map based on the personnel information;

[0058] When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The present invention determines the distribution of people in the entire scene based on the existing monitoring system, and determines the working range of different drones based on the distribution of people, so that the total flow of people in the monitoring area of ​​each drone is similar. When the total flow of people in the monitoring area of ​​the drone is small, it will monitor more areas. When the total flow of people in the monitoring area of ​​the drone is large, it will monitor fewer areas. Ultimately, the drones monitoring fewer areas will return to the main end more frequently to provide data feedback. In addition, the process of determining the working range of the drone is very easy, and can be quickly determined by computer equipment. It can be quickly changed as the actual situation changes. It is highly dynamic and highly consistent with the actual situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention.

[0062] Figure 1A total flow chart of a coordinated control method of multiple unmanned aerial vehicles is shown.

[0063] Figure 2 A structural diagram of a coordinated control system of multiple unmanned aerial vehicles is shown. DETAILED DESCRIPTION

[0064] In order to make the technical problems to be solved by the present application, technical solutions and beneficial effects clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0065] Figure 1 For a total flow chart of a coordinated control method and system of multiple unmanned aerial vehicles, a coordinated control method of multiple unmanned aerial vehicles in the embodiment of the present application, the method comprises:

[0066] Step S100: querying an image collector recorded in a scene, receiving an image containing collection parameters fed back by the image collector; the collection parameters include collection time, collection coordinates, collection direction and collection wide-angle;

[0067] The image collector in the present application is slightly different from the traditional image collector. The traditional image collector is a camera installed in a scene. These cameras are generally arranged at key intersections and are used to obtain videos in the scene. When the area of the scene is large, the monitoring area of the camera generally does not completely cover the entire scenic area. The image collector in the present application includes a smart device of a person in the scene, such as a mobile phone, etc. The smart device can also take pictures. If a connection channel is established between the person in the scene and the main body of the method, the person in the scene can completely upload some images through the smart device, which has a similar function to the camera.

[0068] It is worth mentioning that the camera of the present application also has some differences compared with the traditional camera, that is, the video obtained by the camera does not need to be uploaded in real time, but only needs to send some image frames at regular intervals.

[0069] Step S200: mapping the image to the scene map according to the collection parameters, performing personnel identification on the scene map containing the image, obtaining personnel information, and calculating the passenger flow at each coordinate in the scene map according to the personnel information; the personnel information includes personnel position and personnel orientation;

[0070] When the image collector acquires an image, it will synchronously record the acquisition parameters. The acquisition parameters indicate when and where the image is acquired, the acquisition direction, and the wide angle of the image collector. These four parameters are acquisition time, acquisition coordinates, acquisition direction, and acquisition wide angle. The acquired image is a planar image. Combined with the acquisition parameters, the planar image can be mapped to the scene map. The process of converting the planar image to a horizontal plane based on the acquisition parameters belongs to the prior art and will not be discussed in this application. After mapping the image to the scene map, personnel identification is performed on the scene map to obtain the distribution of personnel in the scene, which is called personnel information. The personnel information is analyzed to calculate the flow of people at each coordinate in the scene map. It should be noted that the personnel information in this application includes the position and orientation of each person, which also means that the calculation process of the flow of people is not only related to the position of the person, but also to the orientation of the person.

[0071] Step S300: partitioning the scene map according to the flow of people at each coordinate;

[0072] After the flow of people at each coordinate in the scene map is calculated, the scene map can be partitioned based on the flow of people. The partitioning rule is to classify two coordinates with similar flow of people into one area. Similar flow of people means that the difference in flow of people is less than the preset difference threshold.

[0073] Step S400: determining the motion path of the UAV based on the partitioned scene map, and synchronously generating a control instruction based on the motion path and sending it to the UAV;

[0074] After the scene map is partitioned, a drone is selected based on the partitioned scene map, and the drone's motion path is determined. Based on the motion path, control instructions are generated and sent to the selected drone.

[0075] Specifically, regarding the drone selection process, the drone selection process of the present application is a sequential selection process. For each sub-area in the scene map (the small area obtained by partitioning is called a sub-area), a string of sub-areas is selected as the monitoring objects of a drone. After the selection is completed, the string of sub-areas is removed from the original scene map, and a string of sub-areas is selected from the remaining sub-areas as the monitoring objects of the next drone. The above process is continuously repeated. Each time a string of sub-areas is selected, a drone is equipped for detection until all sub-areas are monitored by at least one drone.

[0076] Specifically, the process of selecting a string of sub-areas is to first randomly select an initial sub-area, then select the next sub-area from the adjacent sub-areas, then further select the next sub-area from the adjacent sub-area of ​​the next sub-area, and so on, to obtain a string of sub-areas; the length of the sub-area is determined by a preset pedestrian flow condition, for example, the sum of all pedestrian flows in the sub-area reaches a preset pedestrian flow threshold.

[0077] In addition, there is an implicit condition in the process of determining the UAV's motion path in a string of sub-areas, that is, the sum of the collection areas of the UAV on the motion path needs to completely cover the corresponding string of sub-areas.

[0078] Furthermore, regarding the significance of the above-mentioned drone control process, it can be understood that the function of the drone is actually to obtain video, and the camera of the present application can also obtain video. If a wireless transmission channel is established, the execution subject of the present method (the master control end) can obtain the situation in the scene in real time, and the real-time performance is higher. For this reason, the selection of drones for video acquisition seems to be superfluous, but in fact it is not the case. First, the scenes targeted by the present application are generally large, such as forests or mountainous areas. Cameras will only be installed at important intersections and will not be installed globally. Its comprehensiveness is slightly insufficient. Using drones, images of all locations in the scene can be obtained, which is more comprehensive. Second, the camera uploads video in real time, which consumes a lot of resources, and the amount of data itself is very large, and the transmission process will be very slow. Using drones to obtain images of all locations, when it returns to the master control end, it is very convenient to directly use a card reader to obtain the collected data, which consumes very little resources and has a faster transmission speed. Therefore, in summary, the monitoring architecture of the present application reduces real-time performance, but improves data comprehensiveness, reduces the difficulty of data acquisition, and reduces costs, including the cost of camera deployment (the number can be reduced) and the resource cost required for real-time data upload.

[0079] It should be noted that regarding the movement paths of all drones, the process of determining the movement paths of drones is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold; when monitoring the corresponding movement paths of all drones, the union of the monitoring ranges completely covers the scene.

[0080] Regarding step S200, the steps of mapping the image into the scene map according to the acquisition parameters, performing personnel identification on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include:

[0081] Read the scene map of the scene;

[0082] Determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates containing a time tag;

[0083] Create a map grid based on the preset grid length, with each grid cell as the membership range of the grid node at the upper left endpoint;

[0084] Determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the membership range in the latest mapping area, and insert the membership range; the latest mapping area is the mapping area closest in time, and the time range is not greater than a preset time threshold;

[0085] After each grid node is processed, the latest scene map is obtained;

[0086] Personnel identification is performed on the latest scene map to obtain personnel information, and the personnel information is used to calculate the passenger flow at each coordinate in the scene map.

[0087] In an example of the technical solution of the application, the identification and analysis process of the scene map is described. The scene map of each scene is known data. The scene map of the scene is read. A mapping area is determined in the scene map according to the collection parameters. The mapping area is mainly a two-dimensional contour, that is, a range in the scene map. A map grid is superimposed on the scene map. The map grid is the grid function in many existing software. Each grid unit is a rectangle. Each grid unit is the membership range of the grid node at the upper left end point. In other words, each grid unit belongs to the range of the upper left end point. For each grid node, the latest mapping area corresponding to it (the mapping area corresponding to the image collected at the latest time) is queried. The area of the latest mapping area is large. The range belonging to the grid node is intercepted and retained. After each grid node is operated once, the latest scene map is obtained. The purpose of this point-based interception and splicing scheme is to coordinate the conflict problem of multiple mapping areas. Since the obtained image may overlap in area, it is a small problem to select which one in the overlapping part. The application directly splits the scene map into multiple small blocks (grid units) and processes each small block separately, thereby solving the overlapping selection problem.

[0088] Finally, personnel identification is performed on the latest scene map to obtain personnel information, and the personnel information is used to calculate the passenger flow at each coordinate in the scene map.

[0089] It should be noted that when there is no mapping area at a certain grid node, the data within the membership range is emptied. The meaning of this process is that each grid unit only queries the mapping area within the preset time range when querying the latest mapping area. For example, the mapping area within one day. If the mapping area at the latest time exceeds one day, the original map is retained, that is, the data within the membership range is emptied.

[0090] Specifically, the step of performing personnel identification on the latest scene map, obtaining personnel information, and calculating the passenger flow at each coordinate in the scene map according to the personnel information includes:

[0091] Traverse the scene map based on the preset personnel feature table, locate the personnel, and obtain the personnel position and orientation;

[0092] Determine the instantaneous flow of people at each coordinate in the scene map based on the positions and orientations of all people;

[0093] Record the instantaneous flow of people at each coordinate corresponding to the scene map at each moment within a preset time period, and calculate the flow of people at each coordinate.

[0094] The above content explains the calculation process of pedestrian flow. Pedestrian flow is a very common parameter in the prior art. Conventional pedestrian flow is the number of people per unit area. On this basis, this application defines the parameter of pedestrian flow. First, based on the preset personnel feature table, the scene map is traversed to locate personnel, and the personnel position and personnel orientation are obtained at the same time. The instantaneous pedestrian flow at each location is determined by combining the two parameters of personnel position and personnel orientation. The same operation is performed on the scene map at each moment. At this time, the instantaneous pedestrian flow at different times can be obtained at each coordinate, and the final pedestrian flow can be calculated based on the instantaneous pedestrian flow at different times; among them, the final pedestrian flow can be the sum of the instantaneous pedestrian flow or the average of the instantaneous pedestrian flow.

[0095] Specifically, the calculation process of the instantaneous flow of people is as follows:

[0096]

[0097]

[0098] In the formula, S(x, y) represents the instantaneous flow of people at the location (x, y), N represents the total number of people located, and T i represents the projection of the i-th person at position (x, y), x i Indicates the horizontal coordinate of the i-th person, y i Indicates the vertical coordinate of the i-th person; a i is the direction vector in coordinate form of the person facing the i-th person.

[0099] The above calculation process is explained as follows: for a certain position, the influence of all people at that position needs to be considered. For any person, the coordinates of the person are obtained, and the vector between the person and the position is calculated based on the coordinates of the person as the relative position vector. Then, the person's orientation is obtained. The person's orientation also uses a vector in coordinate form. The projection of the person's orientation on the relative position vector is calculated. Finally, the sum of the projections corresponding to all people is calculated to obtain the instantaneous flow of people with direction.

[0100] The above process introduces the concept of personnel orientation on the basis of tradition. If a certain location is in the personnel orientation of multiple people, it means that the location is the personnel's focus location. Even if the personnel have not reached the location, the location is also considered to have human traffic. In actual scenarios, there are many places that personnel cannot reach. After the above processing, the locations that personnel cannot reach can also be counted.

[0101] It should be noted that in the process of calculating the projection amount, when a person's direction is opposite to the position to be analyzed, and the projection amount is non-positive, the projection amount is set to zero. This means that the concept of negative influence is not introduced. Because as the person moves, the person's position changes. Even if he was facing a certain position before, as the position changes, he will eventually turn his back to that position. If a negative influence is introduced, the projection amount previously facing that position is actually eliminated. Therefore, this application does not consider negative values.

[0102] In addition, when the projection amount is positive, The term is the coefficient, and the denominator represents the distance. The greater the distance, the smaller the projection amount. The term represents the projection of the person's orientation on the relative position vector, which belongs to the projection calculation formula.

[0103] It is worth mentioning that during the calculation of the projection amount, Introducing α into the term i term, as the numerator, α i The item is a weight coefficient item, which is used to indicate the importance of different personnel. For example, VIP personnel can choose a larger weight coefficient item. This needs to be specifically defined according to actual conditions and will not be elaborated in this application.

[0104] Regarding step S300, the step of partitioning the scene map according to the flow of people at each coordinate includes:

[0105] For any two coordinates, calculate the spatial distance;

[0106] Calculate the difference in the flow of people at the two coordinates as the flow distance;

[0107] All coordinates are clustered according to the spatial distance and the pedestrian flow distance, and each type of coordinates constitutes a sub-area.

[0108] In an example of the technical solution of the present invention, the partitioning process of the scene map is explained. The partitioning process is very simple, which is to classify similar coordinates into one category. The focus of this application is that similar coordinates are not only similar in distance in the traditional sense, but also require similar flow of people. Specifically, for any two coordinates, the spatial distance and the flow of people distance are calculated, and all coordinates are clustered together according to the spatial distance and the flow of people distance.

[0109] Among them, the clustering process is:.

[0110] Regarding step S400, the steps of determining the motion path of the drone based on the partitioned scene map, and simultaneously generating and sending control instructions based on the motion path to the drone include:

[0111] Read the partitioned scene map and calculate the sum of the pedestrian flow of all coordinates in each sub-area as the characteristic value of the sub-area;

[0112] Based on the characteristic values ​​of each sub-area, drones are selected in sequence and the movement paths of the drones are determined simultaneously;

[0113] Counting the selected drones and their motion paths as a monitoring plan; wherein the data structure of the monitoring plan is a table, including a sequence number column and a motion path column; the sequence number in the sequence number column represents the sequence number of the selected drone, and the motion path in the motion path column represents the motion path corresponding to each drone;

[0114] The best solution is selected from all monitoring solutions, and the control instructions for each drone are determined based on the best solution and sent to the drones.

[0115] In an example of the technical solution of the present invention, the partitioned scene map is read, and the sum of the pedestrian flow of all coordinates in each sub-area is calculated as the characteristic value of the sub-area. The larger the characteristic value, the greater the pedestrian flow; based on the characteristic value of each sub-area, drones are selected in turn, and the movement path of the drone is determined synchronously. There are many ways to select drones and determine the movement path of the drone, each of which is called a monitoring plan. The best plan is selected from the multiple monitoring plans determined as the final plan; based on the final plan, the control instructions for each drone are determined and sent to the drone.

[0116] Specifically, the steps of sequentially selecting drones based on the characteristic values ​​of each sub-area and synchronously determining the motion paths of the drones include:

[0117] Randomly select a sub-area in the scene map as the starting area;

[0118] Selecting the next area in a chain based on the starting area as the passing area;

[0119] Calculate the total eigenvalues ​​of all passed areas. When the total eigenvalues ​​reaches the preset threshold, stop the chain selection process.

[0120] A new drone is selected, and its motion path is determined within the selected transit area. This path is then used as the drone's motion path. The conditions for determining the motion path are that the union of the drone's collection areas along the motion path is no smaller than the transit area, and the motion path includes at least an initial segment, which is the path from the drone's storage location to the starting area.

[0121] In the scene map, the areas that have been used as passing areas are eliminated, and the above process is recursively performed until all passing areas correspond to at least one motion path.

[0122] The process of selecting drones in sequence based on the eigenvalues ​​of each sub-area and simultaneously determining the motion path of the drones is as follows: the process of selecting drones is sequential selection, and the motion path of the drone is determined by the eigenvalues ​​of each sub-area. Specifically:

[0123] The process of selecting drones in this application is a sequential selection process. First, one drone is selected and its movement path is determined. Then the next drone is selected and its movement path is determined. This process is repeated continuously until the monitoring areas of all drones on the movement paths completely cover the entire scene.

[0124] A sub-area is randomly selected in the scene map as the starting area, and the next area is chain-selected based on the starting area as the passing area. The meaning of chain selection is that the selected area must be adjacent to the previous area, and the obtained passing area is a "string of areas". When the process of chain selection of the next area stops is determined by the total value of the eigenvalues ​​of all passing areas. When the total value of the eigenvalues ​​reaches the preset threshold, the chain selection process is stopped; at this time, the selected passing area is used as a monitoring area for a drone, and the movement path of the drone is determined in the selected passing area as the movement path of the drone.

[0125] After the above processing, a drone and its motion path are obtained. On this basis, the areas that have been passed through in the scene map are eliminated, and the above process is recursively executed to obtain multiple drones and their motion paths. When all passed through areas correspond to at least one motion path, it means that all areas in the entire scene have at least one drone for image collection. At this time, the selected drones and their motion paths are statistically selected as a monitoring plan.

[0126] Since the initially selected area, i.e., the starting area, is random, and the next area selected in the chain selection process is also random, the movement path of each drone is not fixed. By executing the monitoring plan generation process multiple times, multiple monitoring plans can be obtained.

[0127] It is worth mentioning that regarding the generation process of the motion path of each drone, when the passing area is known, the generation process of the motion path is a plan. Its requirement is that the union of the collection area of ​​the drone on the motion path is not less than the passing area, and the motion path includes at least an initial segment. The initial segment is a section of the path from the drone storage place to the starting area, that is, the drone needs to reach the starting area first and then completely cover all the passing areas; a conventional motion path generation plan is to obtain the single monitoring radius of the drone, divide the passing area into multiple strip areas by the single monitoring radius, and connect the center lines of each strip area to obtain the final motion path.

[0128] As an example of the technical solution of the present invention, the step of chain-selecting the next area based on the starting area as the passing area includes:

[0129] Calculate the eigenvalue differences of adjacent regions and select the region with the smallest eigenvalue difference as the next region.

[0130] The selection process of the starting area in the above content is random, and the next area selected in the chain selection process is also random. The randomness of this method is too high. In order to reduce a certain degree of randomness, this application limits the process of selecting the next area in the chain selection process. For a certain area, when selecting the next area, the difference between the characteristic values ​​of the adjacent area and the current area is calculated, and the next area with the smallest difference is selected as the selection result.

[0131] As a preferred embodiment of the technical solution of the present invention, the steps of selecting the best solution from all monitoring solutions, determining the control instructions for each drone based on the best solution, and sending the control instructions to the drones include:

[0132] For any monitoring scheme, the sum of the path lengths of all motion paths is calculated, and the monitoring scheme with the smallest sum is selected as the optimal scheme.

[0133] In an example of the technical solution of the present invention, the process of selecting the optimal solution is described. For any monitoring solution, the sum of the path lengths of all motion paths is calculated, and the monitoring solution with the smallest sum is selected as the optimal solution.

[0134] It is worth mentioning that there is another selection scheme, which is to use each passing area as a benchmark, query the number of motion paths corresponding to each passing area, and calculate the total number. The total number indicates how many drones will pass through the passing area during the movement process (drones generally pass through other passing areas in the initial stage). The monitoring scheme with the least total number is selected as the optimal scheme.

[0135] Figure 2The structure diagram of the coordinated control system of multiple UAVs is shown. In a preferred embodiment of the technical solution of the present invention, a coordinated control system of multiple UAVs is also provided. The system 10 includes:

[0136] The image receiving module 11 is used to query the image collector registered in the scene and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle;

[0137] A person identification module 12 is configured to map the image into a scene map according to the acquisition parameters, identify people on the scene map containing the image, obtain person information, and calculate the flow of people at each coordinate in the scene map based on the person information; the person information includes person position and person orientation;

[0138] A map partitioning module 13 is used to partition the scene map according to the flow of people at each coordinate;

[0139] A control instruction generation module 14 is used to determine the motion path of the UAV based on the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV;

[0140] The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

[0141] Furthermore, the image receiving module 11 includes:

[0142] A map reading unit, used for reading a scene map of a scene;

[0143] A mapping area determination unit, configured to determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates including a time tag;

[0144] A grid building unit is used to create a map grid according to a preset grid length, with each grid unit as the belonging range of the grid node at the upper left endpoint;

[0145] The interception and insertion unit is used to determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the membership range in the latest mapping area, and insert it into the membership range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold;

[0146] Map output unit, used to obtain the latest scene map after each grid node is processed;

[0147] A pedestrian flow calculation unit is used to identify people on the latest scene map, obtain personnel information, and calculate the pedestrian flow at each coordinate in the scene map based on the personnel information;

[0148] When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

[0149] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A coordinated control method for multiple UAVs, characterized in that: The method comprises: Query the image collector registered in the scene and receive the image containing acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle; Mapping the image into a scene map according to the acquisition parameters, performing personnel identification on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information; the personnel information includes the position and orientation of the people; Partition the scene map according to the flow of people at each coordinate; Determine the UAV's motion path based on the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV; The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

2. The coordinated control method of multiple UAVs according to claim 1, characterized in that: The steps of mapping the image into a scene map according to the acquisition parameters, performing personnel identification on the scene map containing the image, obtaining personnel information, and calculating the flow of people at each coordinate in the scene map according to the personnel information include: Read the scene map of the scene; Determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates containing a time tag; Create a map grid based on the preset grid length, with each grid cell as the membership range of the grid node at the upper left endpoint; Determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the membership range in the latest mapping area, and insert it into the membership range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold; After each grid node is processed, the latest scene map is obtained; Identify people on the latest scene map, obtain person information, and calculate the flow of people at each coordinate in the scene map based on the person information; When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

3. The coordinated control method of multiple UAVs according to claim 2, characterized in that: The steps of identifying people on the latest scene map, obtaining people information, and calculating the flow of people at each coordinate in the scene map according to the people information include: Traverse the scene map based on the preset personnel feature table, locate the personnel, and obtain the personnel position and orientation; Determine the instantaneous flow of people at each coordinate in the scene map based on the positions and orientations of all people; Record the instantaneous flow of people at each coordinate corresponding to the scene map at each moment within a preset time period, and calculate the flow of people at each coordinate; The calculation process of the instantaneous flow of people is as follows: ； ； Where, Indicates location The instantaneous flow of people at Indicates the total number of located people. Indicates the Person at location The projection amount at Indicates the The horizontal coordinate of each person, Indicates the The vertical coordinate of each person; It is The direction vector of the person's direction in coordinate form.

4. The coordinated control method of multiple UAVs according to claim 1, characterized in that: The step of partitioning the scene map according to the flow of people at each coordinate includes: For any two coordinates, calculate the spatial distance; Calculate the difference in the flow of people at the two coordinates as the flow distance; All coordinates are clustered according to the spatial distance and the pedestrian flow distance, and each type of coordinates constitutes a sub-area.

5. The coordinated control method of multiple UAVs according to claim 1, characterized in that: The steps of determining the motion path of the UAV according to the partitioned scene map, and synchronously generating a control instruction based on the motion path and sending it to the UAV include: Read the partitioned scene map and calculate the sum of the pedestrian flow of all coordinates in each sub-area as the characteristic value of the sub-area; Based on the characteristic values ​​of each sub-area, drones are selected in sequence and the motion paths of the drones are determined simultaneously; Counting the selected drones and their motion paths as a monitoring plan; wherein the data structure of the monitoring plan is a table, including a sequence number column and a motion path column; the sequence number in the sequence number column represents the sequence number of the selected drone, and the motion path in the motion path column represents the motion path corresponding to each drone; The best solution is selected from all monitoring solutions, and the control instructions for each drone are determined based on the best solution and sent to the drones.

6. The coordinated control method of multiple UAVs according to claim 5, characterized in that: The step of sequentially selecting drones based on the characteristic values ​​of each sub-area and synchronously determining the motion paths of the drones includes: Randomly select a sub-area in the scene map as the starting area; Selecting the next area in a chain based on the starting area as the passing area; Calculate the total eigenvalues ​​of all passed areas. When the total eigenvalues ​​reaches the preset threshold, stop the chain selection process. A new drone is selected, and its motion path is determined within the selected transit area. This path is then used as the drone's motion path. The conditions for determining the motion path are that the union of the drone's collection areas along the motion path is no smaller than the transit area, and the motion path includes at least an initial segment, which is the path from the drone's storage location to the starting area. In the scene map, the areas that have been used as passing areas are eliminated, and the above process is recursively performed until all passing areas correspond to at least one motion path.

7. The coordinated control method of multiple UAVs according to claim 6, characterized in that: The step of chain-selecting the next area based on the starting area as the passing area includes: Calculate the eigenvalue differences of adjacent regions and select the region with the smallest eigenvalue difference as the next region.

8. The coordinated control method of multiple UAVs according to claim 5, characterized in that: The steps of selecting the best solution from all monitoring solutions, determining the control instructions for each drone based on the best solution, and sending the instructions to the drones include: For any monitoring scheme, the sum of the path lengths of all motion paths is calculated, and the monitoring scheme with the smallest sum is selected as the optimal scheme.

9. A coordinated control system for multiple UAVs, characterized in that: The system comprises: An image receiving module is used to query the image collector registered in the scene and receive images with acquisition parameters fed back by the image collector; the acquisition parameters include acquisition time, acquisition coordinates, acquisition direction and acquisition wide angle; A personnel identification module is used to map the image into a scene map according to the acquisition parameters, identify personnel on the scene map containing the image, obtain personnel information, and calculate the flow of personnel at each coordinate in the scene map based on the personnel information; the personnel information includes the position and orientation of the personnel; Map partitioning module, used to partition the scene map according to the flow of people at each coordinate; The control instruction generation module is used to determine the motion path of the UAV based on the partitioned scene map, and simultaneously generate control instructions based on the motion path and send them to the UAV; The process of determining the movement path of the drone is as follows: cluster each area in the partitioned scene, and assign each type of area to the same drone for data collection. The total flow of people in each type of area is less than the preset threshold.

10. The coordinated control system of multiple UAVs according to claim 9, characterized in that: The image receiving module includes: A map reading unit, used for reading a scene map of a scene; A mapping area determination unit, configured to determine a mapping area in the scene map according to the acquisition parameters; the mapping area is a set of boundary coordinates including a time tag; A grid building unit is used to create a map grid according to a preset grid length, with each grid unit as the belonging range of the grid node at the upper left endpoint; The interception and insertion unit is used to determine the latest mapping area corresponding to each grid node in the scene map, intercept and retain the area within the membership range in the latest mapping area, and insert it into the membership range; the latest mapping area is the mapping area with the most recent time, and the time range is not greater than a preset time threshold; Map output unit, used to obtain the latest scene map after each grid node is processed; A pedestrian flow calculation unit is used to identify people on the latest scene map, obtain personnel information, and calculate the pedestrian flow at each coordinate in the scene map based on the personnel information; When there is no mapping area at a certain grid node, the data within the belonging range will be cleared.

Images