Unmanned aerial vehicle swarm collaborative control system and method based on image recognition in complex environment

Abstract

The present invention relates to the field of drone control technology, and specifically to a drone swarm collaborative control system and method in a complex environment based on image recognition. The system preprocesses collected real-time environment image data of the drone swarm to generate real-time environment image feature data of the drone swarm, builds real-time environment three-dimensional model data of the drone swarm based on the real-time environment image feature data, and matches the data with the collected preset flight route data of the drone swarm. According to the matching result, preset flight route adjustment data of the drone swarm is generated. If the preset flight route adjustment data of the drone swarm needs to be adjusted, the flight route of the drone swarm is planned by using a constructed drone swarm flight route planning model to generate drone swarm flight route planning data. Finally, drone swarm collaborative control data is built, and drone swarm flight route adjustment operations are performed to achieve collaborative control of the drone swarm and scientific adjustment of the flight route.

Description

Technical Field

[0001] The present invention relates to the field of drone control technology, and in particular to a drone group collaborative control system and method in a complex environment based on image recognition. Background Art

[0002] With the continuous development of artificial intelligence technology and electronic information technology, drone control technology has gradually been applied to many fields. Among them, obstacle avoidance and route adjustment are still the top priorities of drone swarm collaborative control.

[0003] The Chinese invention patent application with publication number CN115686068B introduces a collaborative path planning method in a multi-threat environment, in which threat avoidance path planning is performed through the starting and ending positions of each drone in the drone cluster; a basic path is determined for each drone from all planned paths according to the expected path length, and the determined basic path is optimized and adjusted, and n basic paths with lower length errors after optimization are used as an optimized path set; collision detection is performed on the optimized path set of each drone, and the detected collision-free optimized path is used as a collaborative path, so that the drone cluster can generate collaborative paths of almost equal length under the condition of threat avoidance function, but the path planning process is relatively cumbersome, and the timeliness of the acquired path cannot be guaranteed. At the same time, it is easily affected by environmental changes in a complex environment. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In order to solve the deficiencies in the background technology, the present invention provides a cooperative control system and method of a swarm of drones in a complex environment based on image recognition, so as to realize the cooperative control of a swarm of drones and the scientific adjustment of flight routes.

[0006] (II) Technical solution

[0007] A method for cooperative control of a swarm of unmanned aerial vehicles in a complex environment based on image recognition comprises the following steps:

[0008] S1, collect real-time environmental image data of drone swarm;

[0009] S2, performing data preprocessing on the real-time environment image data of the drone swarm to generate real-time environment image feature data of the drone swarm;

[0010] S3, constructing the real-time environment three-dimensional model data of the drone swarm based on the real-time environment image feature data of the drone swarm;

[0011] S4, collecting the preset flight route data of the drone group;

[0012] S5, performing a matching process on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data, and generating drone swarm preset flight route adjustment data according to the matching result;

[0013] S6. Constructing a drone swarm flight route planning model. If the preset flight route adjustment data of the drone swarm needs to be adjusted, the drone swarm flight route planning model is used to plan the flight route of the drone swarm, and the drone swarm flight route planning data is generated.

[0014] S7. Constructing drone swarm collaborative control data based on the drone swarm real-time environment three-dimensional model data and the drone swarm flight route planning data, and executing the drone swarm flight route adjustment operation.

[0015] The present invention generates real-time environment image feature data of a drone swarm by performing data preprocessing on collected real-time environment image data of a drone swarm, constructs real-time environment three-dimensional model data of a drone swarm based on the real-time environment image feature data of a drone swarm, performs matching processing on a preset flight route of a drone swarm with collected preset flight route data of a drone swarm, generates preset flight route adjustment data of a drone swarm according to the matching result, and if the preset flight route adjustment data of a drone swarm needs to be adjusted, performs flight route planning on the drone swarm through a constructed flight route planning model of a drone swarm, generates flight route planning data of a drone swarm, finally constructs collaborative control data of a drone swarm, and executes flight route adjustment operations of a drone swarm, thereby realizing collaborative control of a drone swarm and scientific adjustment of a flight route.

[0016] Preferably, the specific steps of collecting real-time environmental image data of the drone group are as follows:

[0017] S11. Use the image capture device installed on each drone in the drone group to collect image data of the surrounding environment in real time, and generate a real-time environment image data set of the drone group. ,in, Indicates the number of drones in the swarm The real-time environmental image data of the drone group collected by the image capture device installed on each drone, It represents the total number of real-time environmental image data of the drone group, and the image shooting device represents any one of a high-definition camera, a multi-spectral camera and a hyperspectral camera.

[0018] Preferably, the specific steps of performing data preprocessing on the drone swarm real-time environment image data to generate the drone swarm real-time environment image feature data are as follows:

[0019] S21, using an adaptive median filtering method to preprocess the real-time environment image data of the drone swarm to generate a real-time environment image feature data set of the drone swarm. ,in, Indicates the number of drones in the swarm The real-time environment image data of the drone swarm collected by the image shooting device installed on each drone is preprocessed to obtain the real-time environment image feature data of the drone swarm; the data preprocessing process is as follows:

[0020] S211, randomly selecting any drone swarm real-time environment image data in the drone swarm real-time environment image data set as sample image data;

[0021] S212, set a A sliding filter window is formed, and the center point of the sliding filter window is aligned with each pixel point in the sample image data in sequence;

[0022] S213, determine in turn whether the pixel value of the center point of the sliding filter window is the maximum value in the neighborhood of the sliding filter window. If the pixel value of the center point of the sliding filter window is not the maximum value in the neighborhood of the sliding filter window, no processing is performed; otherwise, the pixel values ​​of all the pixels in the sliding filter window are arranged from large to small to obtain a pixel value sequence. ,in, Represents the pixel value sequence pixel values, Represents the total number of pixels in the sliding filter window;

[0023] S214, calculating the median of the pixel value sequence, and using the median of the pixel value sequence to replace the pixel value of the center point of the sliding filter window; the calculation formula is as follows:

[0024] ,

[0025] in, represents the pixel value of the center point of the sliding filter window after data preprocessing, represents the function of taking the median;

[0026] S215, repeating the operations in S211 to S214 until all the drone swarm real-time environment image data in the drone swarm real-time environment image data set are traversed to generate a drone swarm real-time environment image feature data set.

[0027] An adaptive median filtering method is used to preprocess the real-time environmental image data of the drone swarm, and the noise data in the real-time environmental image data of the drone swarm is accurately removed, making the acquired data smoother. At the same time, the detailed information in the real-time environmental image data of the drone swarm is retained, providing a good data foundation for subsequent operations.

[0028] Preferably, the specific steps of constructing the real-time environment three-dimensional model data of the drone swarm based on the real-time environment image feature data of the drone swarm are as follows:

[0029] S31, three-dimensionally construct the drone swarm real-time environment image feature data in the drone swarm real-time environment image feature data set through three-dimensional modeling software to generate drone swarm real-time environment three-dimensional model data The 3D modeling software includes any one of Inpho, ContextCapture and PhotoMesh.

[0030] Preferably, the specific steps for collecting the preset flight route data of the drone group are as follows:

[0031] S41. Collect the preset flight route data of each drone in the drone group online through the flight route data management system to generate a preset flight route data set for the drone group ,in, Indicates the number of drones in the group collected online through the flight route data management system. The preset flight route data of each drone.

[0032] Preferably, the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data are matched with the drone swarm preset flight route data, and the specific steps of generating the drone swarm preset flight route adjustment data according to the matching result are as follows:

[0033] S51, using an artificial bee colony algorithm to calculate the real-time three-dimensional model data of the drone swarm environment The drone group preset flight route data in the drone group preset flight route data set are matched with the drone group preset flight route data, and the drone group preset flight route adjustment data is generated according to the matching result. ;

[0034] S511. Construct an artificial bee population and set the size of the artificial bee population to , the maximum number of nectar source searches is , the current number of iterations is The maximum number of iterations is And the search space dimension of the preset flight route data of the drone group is ;

[0035] Randomly select from the artificial bee population An artificial bee individual is used as an employed bee individual, and other artificial bee individuals in the artificial bee population are used as follower bee individuals;

[0036] S512: randomly generate in the data search space by using the honey source initialization formula The preset flight route data of the drone group is used as the nectar source individual, and each nectar source individual is assigned an employed bee individual; the nectar source initialization formula is as follows:

[0037] ,

[0038] in, Indicates The nectar source individuals are The preset flight route data of the drone group searches for the location of the spatial dimension, represents a random number that is uniformly distributed between (0,1). and Respectively represent The search upper limit and search lower limit of the preset flight route data search space of the drone swarm;

[0039] S513, calculate the fitness function value of each nectar source individual, arrange the nectar source individuals from large to small according to the fitness function value, and select the nectar source individual with the highest fitness function value as the current optimal individual; the fitness function formula is as follows:

[0040] ,

[0041] in, Indicates The fitness function value of a nectar source individual, Indicates that according to The preset flight route data of the drone group corresponding to each nectar source individual is obtained by using the proxy value of the real-time three-dimensional environment model of the drone group. Indicates the correction value;

[0042] S514. The hired bee individuals near each nectar source individual will search for a new nectar source individual in the preset flight route data search space of the drone group, and calculate the fitness function value of the new nectar source individual. If the fitness function value of the new nectar source individual is greater than the fitness function value of the current nectar source individual, the new nectar source individual will replace the current nectar source individual; otherwise, the search times of the current nectar source individual will be increased by one, and the current nectar source individual will be retained; the search formula for the new nectar source individual is as follows:

[0043] ,

[0044] in, Indicates The hired bee individuals near the nectar source individuals are in The preset flight route data of the drone group searches for the location of the new nectar source individuals in the spatial dimension. represents a random number that is uniformly distributed between (-1,1). represents the first The nectar source individuals are The preset flight route data of the drone group searches for the location of the spatial dimension;

[0045] S515. Each follower bee individual will select an employed bee individual to follow according to probability in the preset flight route data search space of the drone group, and search for a new nectar source individual near the nectar source individual where the employed bee individual is located. If the fitness function value of the new nectar source individual is greater than the fitness function value of the current nectar source individual, the new nectar source individual will replace the current nectar source individual; otherwise, the search times of the current nectar source individual will be increased by one, and the current nectar source individual will be retained; the probability calculation formula of the employed bee individual being followed is as follows:

[0046] ,

[0047] in, Indicates The probability that an employed bee individual is followed;

[0048] S516: If the number of searches for a nectar source individual is greater than or equal to the maximum number of searches for a nectar source , then the nectar source individual is abandoned, and the employed bee individuals near the nectar source individual randomly generate a new nectar source individual in the preset flight route data search space of the drone group through the nectar source initialization formula, and calculate the fitness function value of the new nectar source individual; otherwise, directly enter S517;

[0049] S517, arranging all nectar source individuals from large to small according to the fitness function value, and selecting the nectar source individual with the highest fitness function value as the new current optimal individual;

[0050] S518, determine the current number of iterations Is it less than the maximum number of iterations? , if the current number of iterations Less than the maximum number of iterations , then the current number of iterations Add 1 and return to S514; otherwise, take the current optimal individual as the global optimal solution;

[0051] S519, setting a fitness function threshold, if the fitness function value of the global optimal solution is greater than the fitness function threshold, then determining the real-time environment three-dimensional model data of the drone swarm The match with the drone group preset flight route data in the drone group preset flight route data set is unsuccessful, indicating that the drone group will collide when moving according to the preset flight route, and output the drone group preset flight route adjustment data If necessary, make adjustments; otherwise, determine the real-time three-dimensional model data of the drone group environment The preset flight route data of the drone group in the preset flight route data set of the drone group is successfully matched, indicating that the drone group will not collide when moving according to the preset flight route, and the preset flight route adjustment data of the drone group is output No adjustment is required.

[0052] The artificial bee colony algorithm is used to match the drone swarm's preset flight route with the drone swarm's real-time environment three-dimensional model data and the drone swarm's preset flight route data, thereby achieving accurate matching of the drone swarm's preset flight route and ensuring the accuracy of the drone swarm's preset flight route adjustment data. At the same time, the artificial bee colony algorithm has good robustness and can ensure the stability and reliability of the matching process.

[0053] Preferably, a drone swarm flight route planning model is constructed. If the preset flight route adjustment data of the drone swarm needs to be adjusted, the drone swarm flight route planning model is used to plan the flight route of the drone swarm. The specific steps of generating the drone swarm flight route planning data are as follows:

[0054] S61. Collect 3D model data of different drone safe flight environments through the Internet of Things platform and generate 3D model sample data sets ,in, Indicates 3D model sample data, Indicates the total number of 3D model sample data;

[0055] S62, constructing a UAV swarm flight route planning model based on the three-dimensional model sample data set;

[0056] S621, setting a training data ratio and a test data ratio, and dividing the 3D model sample data set according to the training data ratio and the test data ratio to generate a 3D model training sample data set and a 3D model test sample data set;

[0057] S622, construct an initial convolutional neural network model, and set the number of input layer nodes of the initial convolutional neural network model to , the number of hidden layer nodes is , the number of output layer nodes is The initial weight is And the initial bias is ;

[0058] S623, setting a training error threshold, inputting the three-dimensional model training sample data set into the initial convolutional neural network model for training, and when the training error is less than the training error threshold, stopping the training to obtain a trained convolutional neural network model; otherwise, using a gradient descent optimization algorithm to adjust the initial weights and initial biases and retraining until the training error is less than the training error threshold;

[0059] S624, setting a test accuracy threshold, inputting the three-dimensional model test sample data set into the trained convolutional neural network model for testing, and calculating the test result accuracy. When the test result accuracy is greater than the test accuracy threshold, the trained convolutional neural network model is used as the UAV swarm flight route planning model; otherwise, returning to S623 for retraining until the test result accuracy is greater than the test accuracy threshold;

[0060] S63: If the preset flight route adjustment data of the drone group needs to be adjusted, the real-time environment three-dimensional model data of the drone group is Input the UAV swarm flight route planning model to perform UAV swarm flight route planning and generate UAV swarm flight route planning data .

[0061] By constructing a drone swarm flight route planning model to plan the flight route of the drone swarm, a reliable tool is provided for adjusting the flight route of the drone swarm, realizing the scientific planning of the flight route of the drone swarm, and at the same time improving the speed and efficiency of the adjustment of the flight route of the drone swarm.

[0062] Preferably, the specific steps of constructing the drone swarm collaborative control data based on the drone swarm real-time environment three-dimensional model and the drone swarm flight route planning data, and performing the drone swarm flight route adjustment operation are as follows:

[0063] S71: Combine the real-time three-dimensional model data of the drone group environment and the flight route planning data of the drone group to construct the drone group collaborative control data. ;

[0064] S72. Push the drone swarm collaborative control data to the drone swarm collaborative control platform through the Internet of Things communication network, and perform the drone swarm flight route adjustment operation.

[0065] The present invention also discloses a drone swarm collaborative control system in a complex environment based on image recognition, including a drone swarm real-time environment image data acquisition module, a drone swarm real-time environment image data preprocessing module, a drone swarm real-time environment three-dimensional model construction module, a drone swarm preset flight route data acquisition module, a drone swarm preset flight route data matching module, a drone swarm flight route planning module and a drone swarm collaborative control data construction module;

[0066] The drone swarm real-time environment image data acquisition module collects image data of the surrounding environment in real time through the image shooting device installed on each drone in the drone swarm, and generates real-time environment image data of the drone swarm;

[0067] The drone swarm real-time environment image data preprocessing module uses an adaptive median filtering method to perform data preprocessing on the drone swarm real-time environment image data to generate drone swarm real-time environment image feature data;

[0068] The drone swarm real-time environment three-dimensional model construction module performs three-dimensional construction on the drone swarm real-time environment image feature data in the drone swarm real-time environment image feature data set through three-dimensional modeling software to generate drone swarm real-time environment three-dimensional model data;

[0069] The drone group preset flight route data collection module collects the preset flight route data of each drone in the drone group online through the flight route data management system to generate the drone group preset flight route data;

[0070] The drone swarm preset flight route data matching module performs drone swarm preset flight route matching processing on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data through an artificial bee colony algorithm, and generates drone swarm preset flight route adjustment data according to the matching result;

[0071] The drone swarm flight route planning module constructs a drone swarm flight route planning model through the collected three-dimensional model sample data set, and when the drone swarm preset flight route adjustment data needs to be adjusted, the drone swarm real-time environment three-dimensional model data is input into the drone swarm flight route planning model to perform drone swarm flight route planning and generate drone swarm flight route planning data;

[0072] The drone swarm collaborative control data construction module constructs the drone swarm collaborative control data by combining the real-time environment three-dimensional model data of the drone swarm and the drone swarm flight route planning data, and then pushes it to the drone swarm collaborative control platform through the Internet of Things communication network, and executes the drone swarm flight route adjustment operation.

[0073] (III) Beneficial effects

[0074] 1. The present invention generates real-time environment image feature data of a drone swarm by performing data preprocessing on the collected real-time environment image data of a drone swarm, constructs real-time environment three-dimensional model data of a drone swarm based on the real-time environment image feature data of a drone swarm, and performs matching processing of a preset flight route of a drone swarm with the collected preset flight route data of a drone swarm, generates preset flight route adjustment data of a drone swarm according to the matching result, and if the preset flight route adjustment data of a drone swarm needs to be adjusted, performs flight route planning on the drone swarm through the constructed drone swarm flight route planning model, generates flight route planning data of a drone swarm, finally constructs cooperative control data of a drone swarm, and performs flight route adjustment operation of a drone swarm, so as to realize cooperative control of a drone swarm and scientific adjustment of a flight route;

[0075] 2. Adopting the adaptive median filtering method to preprocess the real-time environment image data of the drone swarm, accurately remove the noise data in the real-time environment image data of the drone swarm, make the acquired data smoother, and retain the detailed information in the real-time environment image data of the drone swarm, providing a good data basis for subsequent operations;

[0076] 3. The artificial bee colony algorithm is used to match the real-time three-dimensional environment model of the drone swarm and the preset flight route data of the drone swarm to the preset flight route of the drone swarm, so as to achieve accurate matching of the preset flight route of the drone swarm and ensure the accuracy of the preset flight route adjustment data of the drone swarm. At the same time, the artificial bee colony algorithm has good robustness and can ensure the stability and reliability of the matching process;

[0077] 4. By constructing a UAV swarm flight route planning model, the flight route of the UAV swarm is planned, which provides a reliable tool for adjusting the flight route of the UAV swarm, realizes the scientific planning of the flight route of the UAV swarm, and improves the speed and efficiency of the adjustment of the flight route of the UAV swarm. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.

[0079] Figure 1 A flowchart of a method for cooperative control of a swarm of unmanned aerial vehicles in a complex environment based on image recognition provided by the present invention;

[0080] Figure 2A module schematic diagram of a cooperative control system of a swarm of unmanned aerial vehicles in a complex environment based on image recognition provided by the present invention. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0083] Embodiment 1 is as follows:

[0084] See also Figure 1 , a cooperative control method of a drone group in a complex environment based on image recognition, comprising the following steps:

[0085] S1, collect real-time environmental image data of drone swarm;

[0086] S11. Use the image capture device installed on each drone in the drone group to collect image data of the surrounding environment in real time, and generate a real-time environment image data set of the drone group. ,in, Indicates the number of drones in the swarm The real-time environmental image data of the drone group collected by the image capture device installed on each drone, It represents the total number of real-time environmental image data of the drone group, and the image shooting device represents any one of a high-definition camera, a multi-spectral camera and a hyperspectral camera.

[0087] S2, performing data preprocessing on the real-time environment image data of the drone swarm to generate real-time environment image feature data of the drone swarm;

[0088] S21, using an adaptive median filtering method to preprocess the real-time environment image data of the drone swarm to generate a real-time environment image feature data set of the drone swarm. ,in, Indicates the number of drones in the swarm The real-time environment image data of the drone swarm collected by the image shooting device installed on each drone is preprocessed to obtain the real-time environment image feature data of the drone swarm; the data preprocessing process is as follows:

[0089] S211, randomly selecting any drone swarm real-time environment image data in the drone swarm real-time environment image data set as sample image data;

[0090] S212, set a A sliding filter window is formed, and the center point of the sliding filter window is aligned with each pixel point in the sample image data in sequence;

[0091] S213, determine in turn whether the pixel value of the center point of the sliding filter window is the maximum value in the neighborhood of the sliding filter window. If the pixel value of the center point of the sliding filter window is not the maximum value in the neighborhood of the sliding filter window, no processing is performed; otherwise, the pixel values ​​of all the pixels in the sliding filter window are arranged from large to small to obtain a pixel value sequence. ,in, Represents the pixel value sequence pixel values, Represents the total number of pixels in the sliding filter window;

[0092] S214, calculating the median of the pixel value sequence, and using the median of the pixel value sequence to replace the pixel value of the center point of the sliding filter window; the calculation formula is as follows:

[0093] ,

[0094] in, represents the pixel value of the center point of the sliding filter window after data preprocessing, represents the function of taking the median;

[0095] S215, repeating the operations in S211 to S214 until all the drone swarm real-time environment image data in the drone swarm real-time environment image data set are traversed to generate a drone swarm real-time environment image feature data set.

[0096] S3, constructing the real-time environment three-dimensional model data of the drone swarm based on the real-time environment image feature data of the drone swarm;

[0097] S31, three-dimensionally construct the drone swarm real-time environment image feature data in the drone swarm real-time environment image feature data set through three-dimensional modeling software to generate drone swarm real-time environment three-dimensional model data The 3D modeling software includes any one of Inpho, ContextCapture and PhotoMesh.

[0098] S4, collecting the preset flight route data of the drone group;

[0099] S41. Collect the preset flight route data of each drone in the drone group online through the flight route data management system to generate a preset flight route data set for the drone group ,in, Indicates the number of drones in the group collected online through the flight route data management system. The preset flight route data of each drone.

[0100] S5, performing a matching process on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data, and generating drone swarm preset flight route adjustment data according to the matching result;

[0101] S51, using an artificial bee colony algorithm to calculate the real-time three-dimensional model data of the drone swarm environment The drone group preset flight route data in the drone group preset flight route data set are matched with the drone group preset flight route data, and the drone group preset flight route adjustment data is generated according to the matching result. ;

[0102] S511. Construct an artificial bee population and set the size of the artificial bee population to , the maximum number of nectar source searches is , the current number of iterations is The maximum number of iterations is And the search space dimension of the preset flight route data of the drone group is ;

[0103] Randomly select from the artificial bee population An artificial bee individual is used as an employed bee individual, and other artificial bee individuals in the artificial bee population are used as follower bee individuals;

[0104] S512: randomly generate in the data search space by using the honey source initialization formula The preset flight route data of the drone group is used as the nectar source individual, and each nectar source individual is assigned an employed bee individual; the nectar source initialization formula is as follows:

[0105] ,

[0106] in, Indicates The nectar source individuals are The preset flight route data of the drone group searches for the location of the spatial dimension, represents a random number that is uniformly distributed between (0,1). and Respectively represent The search upper limit and search lower limit of the preset flight route data search space of the drone swarm;

[0107] S513, calculate the fitness function value of each nectar source individual, arrange the nectar source individuals from large to small according to the fitness function value, and select the nectar source individual with the highest fitness function value as the current optimal individual; the fitness function formula is as follows:

[0108] ,

[0109] in, Indicates The fitness function value of a nectar source individual, Indicates that according to The preset flight route data of the drone group corresponding to each nectar source individual is obtained by using the proxy value of the real-time three-dimensional environment model of the drone group. Indicates the correction value;

[0110] S514. The hired bee individuals near each nectar source individual will search for a new nectar source individual in the preset flight route data search space of the drone group, and calculate the fitness function value of the new nectar source individual. If the fitness function value of the new nectar source individual is greater than the fitness function value of the current nectar source individual, the new nectar source individual will replace the current nectar source individual; otherwise, the search times of the current nectar source individual will be increased by one, and the current nectar source individual will be retained; the search formula for the new nectar source individual is as follows:

[0111] ,

[0112] in, Indicates The hired bee individuals near the nectar source individuals are in The preset flight route data of the drone group searches for the location of the new nectar source individuals in the spatial dimension. represents a random number that is uniformly distributed between (-1,1). represents the first The nectar source individuals are The preset flight route data of the drone group searches for the location of the spatial dimension;

[0113] S515. Each follower bee individual will select an employed bee individual to follow according to probability in the preset flight route data search space of the drone group, and search for a new nectar source individual near the nectar source individual where the employed bee individual is located. If the fitness function value of the new nectar source individual is greater than the fitness function value of the current nectar source individual, the new nectar source individual will replace the current nectar source individual; otherwise, the search times of the current nectar source individual will be increased by one, and the current nectar source individual will be retained; the probability calculation formula of the employed bee individual being followed is as follows:

[0114] ,

[0115] in, Indicates The probability that an employed bee individual is followed;

[0116] S516: If the number of searches for a nectar source individual is greater than or equal to the maximum number of searches for a nectar source , then the nectar source individual is abandoned, and the employed bee individuals near the nectar source individual randomly generate a new nectar source individual in the preset flight route data search space of the drone group through the nectar source initialization formula, and calculate the fitness function value of the new nectar source individual; otherwise, directly enter S517;

[0117] S517, arranging all nectar source individuals from large to small according to the fitness function value, and selecting the nectar source individual with the highest fitness function value as the new current optimal individual;

[0118] S518, determine the current number of iterations Is it less than the maximum number of iterations? , if the current number of iterations Less than the maximum number of iterations , then the current number of iterations Add 1 and return to S514; otherwise, take the current optimal individual as the global optimal solution;

[0119] S519, setting a fitness function threshold, if the fitness function value of the global optimal solution is greater than the fitness function threshold, then determining the real-time environment three-dimensional model data of the drone swarm The match with the drone group preset flight route data in the drone group preset flight route data set is unsuccessful, indicating that the drone group will collide when moving according to the preset flight route, and output the drone group preset flight route adjustment data If necessary, make adjustments; otherwise, determine the real-time three-dimensional model data of the drone group environment The preset flight route data of the drone group in the preset flight route data set of the drone group is successfully matched, indicating that the drone group will not collide when moving according to the preset flight route, and the preset flight route adjustment data of the drone group is output No adjustment is required.

[0120] S6. Constructing a drone swarm flight route planning model. If the preset flight route adjustment data of the drone swarm needs to be adjusted, the drone swarm flight route planning model is used to plan the flight route of the drone swarm, and the drone swarm flight route planning data is generated.

[0121] S61. Collect 3D model data of different drone safe flight environments through the Internet of Things platform and generate 3D model sample data sets ,in, Indicates 3D model sample data, Indicates the total number of 3D model sample data;

[0122] S62, constructing a UAV swarm flight route planning model based on the three-dimensional model sample data set;

[0123] S621, setting a training data ratio and a test data ratio, and dividing the 3D model sample data set according to the training data ratio and the test data ratio to generate a 3D model training sample data set and a 3D model test sample data set;

[0124] S622, construct an initial convolutional neural network model, and set the number of input layer nodes of the initial convolutional neural network model to , the number of hidden layer nodes is , the number of output layer nodes is The initial weight is And the initial bias is ;

[0125] S623, setting a training error threshold, inputting the three-dimensional model training sample data set into the initial convolutional neural network model for training, and when the training error is less than the training error threshold, stopping the training to obtain a trained convolutional neural network model; otherwise, using a gradient descent optimization algorithm to adjust the initial weights and initial biases and retraining until the training error is less than the training error threshold;

[0126] S624, setting a test accuracy threshold, inputting the three-dimensional model test sample data set into the trained convolutional neural network model for testing, and calculating the test result accuracy. When the test result accuracy is greater than the test accuracy threshold, the trained convolutional neural network model is used as the UAV swarm flight route planning model; otherwise, returning to S623 for retraining until the test result accuracy is greater than the test accuracy threshold;

[0127] S63: If the preset flight route adjustment data of the drone group needs to be adjusted, the real-time environment three-dimensional model data of the drone group is Input the UAV swarm flight route planning model to perform UAV swarm flight route planning and generate UAV swarm flight route planning data .

[0128] S7, constructing drone swarm collaborative control data based on the drone swarm real-time environment three-dimensional model data and the drone swarm flight route planning data, and performing drone swarm flight route adjustment operations;

[0129] S71: Combine the real-time three-dimensional model data of the drone group environment and the flight route planning data of the drone group to construct the drone group collaborative control data. ;

[0130] S72. Push the drone swarm collaborative control data to the drone swarm collaborative control platform through the Internet of Things communication network, and perform the drone swarm flight route adjustment operation.

[0131] Embodiment 2 is as follows:

[0132] See also Figure 2 , a drone swarm collaborative control system in a complex environment based on image recognition, including a drone swarm real-time environment image data acquisition module, a drone swarm real-time environment image data preprocessing module, a drone swarm real-time environment three-dimensional model construction module, a drone swarm preset flight route data acquisition module, a drone swarm preset flight route data matching module, a drone swarm flight route planning module and a drone swarm collaborative control data construction module;

[0133] The drone swarm real-time environment image data acquisition module collects image data of the surrounding environment in real time through the image shooting device installed on each drone in the drone swarm, and generates real-time environment image data of the drone swarm;

[0134] The drone swarm real-time environment image data preprocessing module uses an adaptive median filtering method to perform data preprocessing on the drone swarm real-time environment image data to generate drone swarm real-time environment image feature data;

[0135] The drone swarm real-time environment three-dimensional model construction module performs three-dimensional construction on the drone swarm real-time environment image feature data in the drone swarm real-time environment image feature data set through three-dimensional modeling software to generate drone swarm real-time environment three-dimensional model data;

[0136] The drone group preset flight route data collection module collects the preset flight route data of each drone in the drone group online through the flight route data management system to generate the drone group preset flight route data;

[0137] The drone swarm preset flight route data matching module performs drone swarm preset flight route matching processing on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data through an artificial bee colony algorithm, and generates drone swarm preset flight route adjustment data according to the matching result;

[0138] The drone swarm flight route planning module constructs a drone swarm flight route planning model through the collected three-dimensional model sample data set, and when the drone swarm preset flight route adjustment data needs to be adjusted, the drone swarm real-time environment three-dimensional model data is input into the drone swarm flight route planning model to perform drone swarm flight route planning and generate drone swarm flight route planning data;

[0139] The drone swarm collaborative control data construction module constructs the drone swarm collaborative control data by combining the real-time environment three-dimensional model data of the drone swarm and the drone swarm flight route planning data, and then pushes it to the drone swarm collaborative control platform through the Internet of Things communication network, and executes the drone swarm flight route adjustment operation.

[0140] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0141] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. A method for cooperative control of drone groups in complex environments based on image recognition, characterized in that: The steps include: S1, collect real-time environmental image data of drone swarm; S2, performing data preprocessing on the real-time environment image data of the drone swarm to generate real-time environment image feature data of the drone swarm; S21, using an adaptive median filtering method to preprocess the real-time environment image data of the drone swarm, generating a real-time environment image feature data set A' = {a'1, a'2, ..., a' i ,…,a' k }, where a' i represents the real-time environment image feature data of the drone swarm obtained after data preprocessing of the real-time environment image data of the drone swarm collected by the image shooting device installed on the i-th drone in the drone swarm; S3, constructing the real-time environment three-dimensional model data of the drone swarm based on the real-time environment image feature data of the drone swarm; S4, collecting the preset flight route data of the drone group; S41, collect the preset flight route data of each drone in the drone group online through the flight route data management system, and generate the drone group preset flight route data set D = {d1, d2, ..., d i ,…,d k }, where d i represents the preset flight route data of the i-th drone in the drone group collected online through the flight route data management system; S5, performing a matching process on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data, and generating drone swarm preset flight route adjustment data according to the matching result; S51, performing drone swarm preset flight route matching processing on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data in the drone swarm preset flight route data set by using an artificial bee colony algorithm, and generating drone swarm preset flight route adjustment data E according to the matching result; S6. Constructing a drone swarm flight route planning model. If the preset flight route adjustment data of the drone swarm needs to be adjusted, the drone swarm flight route planning model is used to plan the flight route of the drone swarm, and the drone swarm flight route planning data is generated. S61. Collect 3D model data of different UAV safe flight environments through the Internet of Things platform to generate a 3D model sample data set F = {f1, f2, ..., f i ,…,f p }, where f i represents the i-th 3D model sample data, and p represents the total number of 3D model sample data; S62, constructing a UAV swarm flight route planning model based on the three-dimensional model sample data set; S63, if the preset flight route adjustment data of the drone group needs to be adjusted, input the drone group real-time environment three-dimensional model data into the drone group flight route planning model to perform drone group flight route planning, and generate drone group flight route planning data Q; S7. Constructing drone swarm collaborative control data based on the drone swarm real-time environment three-dimensional model data and the drone swarm flight route planning data, and executing the drone swarm flight route adjustment operation.

2. According to the method of cooperative control of drone groups in complex environments based on image recognition in claim 1, it is characterized in that: The S1 comprises the following steps: S11, collect image data of the surrounding environment in real time through the image shooting device installed on each drone in the drone group, and generate a drone group real-time environment image data set A = {a1, a2, ..., a i ,…,a k }, where a i represents the real-time environmental image data of the drone swarm collected by the image capture device installed on the i-th drone in the drone swarm, k represents the total number of real-time environmental image data of the drone swarm, and the image capture device represents any one of a high-definition camera, a multispectral camera and a hyperspectral camera.

3. According to the method of cooperative control of drone groups in complex environments based on image recognition in claim 2, it is characterized in that: The S21 comprises the following steps: S211, randomly selecting any drone swarm real-time environment image data in the drone swarm real-time environment image data set as sample image data; S212, setting an m×m sliding filter window, and aligning the center point of the sliding filter window with each pixel point in the sample image data in sequence; S213, determining in sequence whether the pixel value of the center point of the sliding filter window is the maximum value in the neighborhood of the sliding filter window, if the pixel value of the center point of the sliding filter window is not the maximum value in the neighborhood of the sliding filter window, no processing is performed; otherwise, the pixel values ​​of all the pixels in the sliding filter window are arranged from large to small to obtain a pixel value sequence; S214, calculating the median of the pixel value sequence, and replacing the pixel value of the center point of the sliding filter window with the median of the pixel value sequence; S215, repeating the operations in S211 to S214 until all the drone swarm real-time environment image data in the drone swarm real-time environment image data set are traversed to generate a drone swarm real-time environment image feature data set.

4. The method for cooperative control of drone groups in complex environments based on image recognition according to claim 3 is characterized in that: The S3 comprises the following steps: S31. Perform three-dimensional construction on the drone swarm real-time environment image feature data in the drone swarm real-time environment image feature data set by using three-dimensional modeling software to generate three-dimensional model data C of the drone swarm real-time environment.

5. The method for cooperative control of drone groups in complex environments based on image recognition according to claim 4 is characterized in that: The S51 comprises the following steps: S511, build an artificial bee population, set the size of the artificial bee population to N, the maximum number of nectar source searches to lim it, the current number of iterations to t, and the maximum number of iterations to t max And the dimension of the search space for the preset flight route data of the drone group is P; Randomly select N / 2 worker bee individuals from the artificial bee population as employed bee individuals, and select other worker bee individuals in the artificial bee population as follower bee individuals; S512, randomly generating N / 2 preset flight route data of the drone group in the data search space as nectar source individuals through the nectar source initialization formula, and assigning one employed bee individual to each nectar source individual; S513, calculating the fitness function value of each nectar source individual, arranging the nectar source individuals from large to small according to the fitness function value, and selecting the nectar source individual with the highest fitness function value as the current optimal individual; S514, the hired bee individuals near each nectar source individual will search for a new nectar source individual in the preset flight route data search space of the drone group, and calculate the fitness function value of the new nectar source individual. If the fitness function value of the new nectar source individual is greater than the fitness function value of the current nectar source individual, the new nectar source individual will replace the current nectar source individual; otherwise, the search times of the current nectar source individual will be increased by one, and the current nectar source individual will be retained; S515, each follower bee individual will select an employed bee individual to follow according to probability in the preset flight route data search space of the drone group, and search for a new nectar source individual near the nectar source individual where the employed bee individual is located. If the fitness function value of the new nectar source individual is greater than the fitness function value of the current nectar source individual, the new nectar source individual will replace the current nectar source individual; otherwise, the search times of the current nectar source individual will be increased by one, and the current nectar source individual will be retained; S516. If the search times of the nectar source individual is greater than or equal to the maximum nectar source search times lim it, the nectar source individual is abandoned, and the hired bee individuals near the nectar source individual randomly generate a new nectar source individual in the preset flight route data search space of the drone group through the nectar source initialization formula, and calculate the fitness function value of the new nectar source individual; otherwise, directly enter S517; S517, arranging all nectar source individuals from large to small according to the fitness function value, and selecting the nectar source individual with the highest fitness function value as the new current optimal individual; S518: Determine whether t is less than t max , if t is less than t max , then t is increased by 1, and the process returns to S514; otherwise, the current optimal individual is taken as the global optimal solution; S519. Set a fitness function threshold. If the fitness function value of the global optimal solution is greater than the fitness function threshold, it is determined that the drone swarm real-time environment three-dimensional model data C and the drone swarm preset flight route data set in the drone swarm preset flight route data set are not matched successfully, indicating that the drone swarm will collide when moving according to the preset flight route, and the drone swarm preset flight route adjustment data E is output as needing adjustment; otherwise, it is determined that the drone swarm real-time environment three-dimensional model data C and the drone swarm preset flight route data set are matched successfully, indicating that the drone swarm will not collide when moving according to the preset flight route, and the drone swarm preset flight route adjustment data E is output as needing no adjustment.

6. The method for cooperative control of drone groups in complex environments based on image recognition according to claim 5 is characterized in that: The S7 comprises the following steps: S71, combining the real-time three-dimensional model data of the drone group environment and the flight route planning data of the drone group to construct drone group collaborative control data W={C,Q}; S72. Push the drone swarm collaborative control data to the drone swarm collaborative control platform through the Internet of Things communication network, and perform the drone swarm flight route adjustment operation.

7. A system for implementing the method for cooperative control of a swarm of unmanned aerial vehicles in a complex environment based on image recognition according to any one of claims 1 to 6, comprising a swarm of unmanned aerial vehicles real-time environment image data acquisition module, a swarm of unmanned aerial vehicles real-time environment image data preprocessing module, a swarm of unmanned aerial vehicles real-time environment three-dimensional model construction module, a swarm of unmanned aerial vehicles preset flight route data acquisition module, a swarm of unmanned aerial vehicles preset flight route data matching module, a swarm of unmanned aerial vehicles flight route planning module and a swarm of unmanned aerial vehicles cooperative control data construction module, characterized in that: The drone swarm real-time environment image data acquisition module collects image data of the surrounding environment in real time through the image shooting device installed on each drone in the drone swarm, and generates real-time environment image data of the drone swarm; The drone swarm real-time environment image data preprocessing module uses an adaptive median filtering method to perform data preprocessing on the drone swarm real-time environment image data to generate drone swarm real-time environment image feature data; The drone swarm real-time environment three-dimensional model construction module performs three-dimensional construction on the drone swarm real-time environment image feature data in the drone swarm real-time environment image feature data set through three-dimensional modeling software to generate drone swarm real-time environment three-dimensional model data; The drone group preset flight route data collection module collects the preset flight route data of each drone in the drone group online through the flight route data management system to generate the drone group preset flight route data; The drone swarm preset flight route data matching module performs drone swarm preset flight route matching processing on the drone swarm real-time environment three-dimensional model data and the drone swarm preset flight route data through an artificial bee colony algorithm, and generates drone swarm preset flight route adjustment data according to the matching result; The drone swarm flight route planning module constructs a drone swarm flight route planning model through the collected three-dimensional model sample data set, and when the drone swarm preset flight route adjustment data needs to be adjusted, the drone swarm real-time environment three-dimensional model data is input into the drone swarm flight route planning model to perform drone swarm flight route planning and generate drone swarm flight route planning data; The drone swarm collaborative control data construction module constructs the drone swarm collaborative control data by combining the drone swarm real-time environment three-dimensional model data and the drone swarm flight route planning data, and then pushes it to the drone swarm collaborative control platform through the Internet of Things communication network, and executes the drone swarm flight route adjustment operation.

Images