Unmanned aerial vehicle cluster information fusion method

Abstract

The invention discloses an unmanned aerial vehicle cluster information fusion method. The method comprises the steps that a first unmanned aerial vehicle node acquires a photoelectric investigation image, first category information and first position information of a target object; cutting the photoelectric investigation image according to the first position information to obtain a target image; inputting the target image into a pre-trained convolutional neural network to obtain a first feature code; sending the category information, the position information and the feature code to a second unmanned aerial vehicle node; the second unmanned aerial vehicle node divides a plurality of corresponding target objects into a plurality of target clusters according to the received first position information and the first feature code sent by each first unmanned aerial vehicle node, wherein the target objects in each target cluster are the same; and under the condition that the target cluster comprises at least two target objects, fusing the first category information and the first position information corresponding to the at least two target objects to obtain target category information and target position information. The transmission efficiency and the multi-machine information fusion efficiency can be improved.

Description

Technical Field

[0001] The present application belongs to the field of UAV technology, and in particular, relates to a method for fusion of UAV cluster information. Background Art

[0002] In recent years, with the rapid development of science and technology, all walks of life are undergoing digital and intelligent transformation. The continuous advancement of sensor technology, network communication technology and data processing capabilities has significantly improved the performance of drones in information acquisition, transmission and processing. As an innovative application form, drone swarms have received widespread attention in the fields of agricultural monitoring, environmental monitoring, traffic management, disaster relief, etc. Through the collaborative work of multiple drones, it can achieve more efficient monitoring, data collection and multi-angle data analysis to meet the needs of modern industries.

[0003] The information fusion of drone swarms is to integrate data from different platforms and sensors, and provide more valuable reference information for decision makers through comprehensive and in-depth analysis. At present, the principle of multi-machine information fusion is achieved by transmitting target pictures, but when the number of targets is large, this method of target matching by transmitting pictures will occupy a large amount of network communication bandwidth, resulting in low data transmission efficiency and multi-machine information fusion efficiency. Summary of the invention

[0004] An embodiment of the present application provides a method for fusion of drone cluster information, which is used to at least solve the problems of low data transmission efficiency and low efficiency of multi-machine information fusion in related technologies.

[0005] The embodiment of the present application provides a method for fusion of drone cluster information, wherein the drone cluster includes a plurality of first drone nodes and a second drone node;

[0006] The method comprises:

[0007] The first UAV node obtains an optoelectronic reconnaissance image, first category information, and first position information of the target object;

[0008] The first UAV node crops the photoelectric reconnaissance image according to the first position information to obtain a target image;

[0009] The first drone node inputs the target image into a pre-trained convolutional neural network to obtain a first feature code corresponding to the target image;

[0010] The first drone node sends the first category information, the first location information and the first feature code to the second drone node;

[0011] The second UAV node divides the corresponding target objects into a plurality of target clusters according to the first position information and the first feature codes respectively sent by the first UAV nodes, wherein the number of target clusters is less than or equal to the number of target objects, and the target objects in each of the target clusters are the same;

[0012] For each target cluster, when the target cluster includes one target object, the first category information and the first position information of the target object are respectively determined as the target category information and the target position information; when the target cluster includes at least two target objects, the first category information and the first position information corresponding to the at least two target objects are respectively fused to obtain the target category information and the target position information.

[0013] The drone cluster information fusion method of the embodiment of the present application uses a neural network to extract features of the target image in each node drone, and then only transmits the feature vector of the target to the fusion drone node for target matching to achieve information fusion. This can greatly reduce the amount of information transmission, reduce the bandwidth occupied by network communication transmission, improve transmission efficiency, and thus improve the efficiency of multi-machine information fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 It is a flowchart of a method for fusion of drone cluster information provided by an embodiment of the present application;

[0016] Figure 2 is a schematic diagram of an exemplary cropping to a target image provided in an embodiment of the present application;

[0017] Figure 3 is a schematic diagram of training a convolutional neural network provided in an embodiment of the present application;

[0018] Figure 4 is a schematic diagram of obtaining a first feature encoding of a target image using a pre-trained convolutional neural network, as provided in an exemplary embodiment of the present application;

[0019] Figure 5 It is a flow chart of another method for fusion of drone cluster information provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0021] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0022] In recent years, with the rapid development of science and technology, all walks of life are undergoing digital and intelligent transformation. The continuous advancement of sensor technology, network communication technology and data processing capabilities has significantly improved the performance of drones in information acquisition, transmission and processing. As an innovative application form, drone swarms have received widespread attention in the fields of agricultural monitoring, environmental monitoring, traffic management, disaster relief, etc. Through the collaborative work of multiple drones, it can achieve more efficient monitoring, data collection and multi-angle data analysis to meet the needs of modern industries.

[0023] The information fusion of drone swarms is to integrate data from different platforms and sensors, and provide more valuable reference information for decision makers through comprehensive and in-depth analysis. At present, the principle of multi-machine information fusion is achieved by transmitting target pictures, but when the number of targets is large, this method of target matching by transmitting pictures will occupy a large amount of network communication bandwidth, resulting in low data transmission efficiency and multi-machine information fusion efficiency.

[0024] In order to solve the problems of related technologies, an embodiment of the present application provides a method for fusing drone cluster information.

[0025] The following is a detailed description of the drone cluster information fusion method provided in the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0026] It should be noted that the drone cluster in the embodiment of the present application includes a plurality of first drone nodes and a second drone node. It can be understood that the second drone node is a fusion drone node.

[0027] refer to Figure 1 , is a flow chart of a method for fusion of drone cluster information according to an embodiment of the present application. Figure 1 As shown, the UAV cluster information fusion method may specifically include the following steps:

[0028] S101, a first UAV node obtains an optoelectronic reconnaissance image, first category information, and first location information of a target object;

[0029] S102, the first UAV node crops the photoelectric reconnaissance image according to the first position information to obtain a target image;

[0030] S103, the first drone node inputs the target image into a pre-trained convolutional neural network to obtain a first feature code corresponding to the target image;

[0031] S104, the first drone node sends the first category information, the first location information and the first feature code to the second drone node;

[0032] S105, the second UAV node divides the corresponding target objects into a plurality of target clusters according to the first position information and the first feature codes respectively sent by the first UAV nodes, the number of target clusters is less than or equal to the number of target objects, and the target objects in each of the target clusters are the same;

[0033] S106. For each target cluster, when the target cluster includes one target object, the first category information and the first position information of the target object are respectively determined as the target category information and the target position information; when the target cluster includes at least two target objects, the first category information and the first position information corresponding to the at least two target objects are respectively fused to obtain the target category information and the target position information.

[0034] Therefore, each node drone uses a neural network to extract the features of the target image, and then only transmits the target's feature vector to the fusion drone node for target matching to achieve information fusion. This can greatly reduce the amount of information transmission, reduce the bandwidth occupied by network communication transmission, and improve transmission efficiency, thereby improving the efficiency of multi-machine information fusion.

[0035] The specific implementation methods of the above steps are introduced below.

[0036] In some embodiments, in S101, each first drone node in the drone cluster obtains the category and location information of the reconnaissance target object, i.e., first category information and first location information, through the target recognition and target positioning algorithm on the airborne end based on the optoelectronic payload reconnaissance data.

[0037] During specific implementation, the drone position information is obtained; the position of the drone position information is solved to obtain the first position information of the target object.

[0038] In some embodiments, in S102, based on the first position information, a recognition algorithm is used to identify the target object in the photoelectric detection image to obtain a first area including the target object; a preset multiple of the first side length of the first area is determined as the target side length, and the center of the first area is determined as the target center; a preset shape area corresponding to the target side length and the target center is determined as the target area; and the area other than the target area in the photoelectric detection image is cropped to obtain a target image.

[0039] Figure 2 FIG. 4 shows an exemplary schematic diagram of cropping a target image. Figure 2 As shown in the figure, the red box is the recognition box of the recognition algorithm, which corresponds to the first area; the green box is the cropping box, which corresponds to the target area. If the length of the long side of the red box (i.e. the first side length of the first area) is defined as L, the target side length can be set to 3 / 2L. Optionally, the preset shape can be a square, then Figure 2 The green frame corresponding to the target area is a square that coincides with the center of the red frame and has a side length of 3 / 2L. In this way, each first UAV node crops the optoelectronic reconnaissance image according to the detected target position and size information to obtain the target image.

[0040] As an optional embodiment, a convolutional neural network (CNN) is obtained by training a preset model. Specifically, the preset model is a Triplet network, and the convolutional neural network is obtained by training the Triplet network based on Triplet Loss.

[0041] refer to Figure 3 , is a schematic diagram of training a convolutional neural network. Figure 3As shown, the Triplet network includes three identical CNNs and weights are shared between CNNs. The specific implementation method of training a convolutional neural network may include: obtaining an image sample set, the image sample set includes a number of triplets, each of the triplets includes a reference sample, a positive sample, and a negative sample; inputting the triplets into a preset model to obtain sample feature codes corresponding to the reference sample, the positive sample, and the negative sample, respectively, the preset model is a Triplet network, the Triplet network includes three identical CNNs and weights are shared between CNNs; according to the sample feature codes corresponding to the reference sample, the positive sample, and the negative sample, the loss value of the triplets is calculated; according to the loss value, the network weight is updated, and the updated preset model is trained to obtain a trained convolutional neural network.

[0042] Optionally, according to the sample feature codes corresponding to the reference sample, the positive sample and the negative sample, the loss value of the triplet is calculated by the following formula (1):

[0043] Loss=max(d(a,p)-d(a,n)+margin,0); (1)

[0044] Wherein, Loss represents the loss value of the triple; d(a,p) represents the cosine distance between the sample feature code of the reference sample and the sample feature code of the positive sample; d(a,n) represents the cosine distance between the sample feature code of the reference sample and the sample feature code of the negative sample; margin represents a pre-set interval value, which is used to control the distance between the positive sample and the negative sample and can be set by the cross-validation result. It should be noted that in this embodiment, the cosine distance is used to calculate the distance between two feature codes.

[0045] In this way, by using Triplet Loss, the model can learn feature representations that make similar samples closer and dissimilar samples farther apart, thereby improving the performance of the target feature representation.

[0046] That is to say, during training, a triplet (including a reference sample anchor, a positive sample and a negative sample) is randomly selected from the training sample set and input into the network to obtain the feature encoding of the three samples respectively, and finally the network weight is updated by calculating the Triplet Loss of the triplet. In this way, by training the Triplet network to extract the features of the target image, the expressiveness of the features can be enhanced, so that the distance between the features of the same type of target samples is very small, while the distance between the features of different target samples is very large, which is very helpful for improving the accuracy of target matching in subsequent steps.

[0047] Thus, a trained convolutional neural network is obtained. Specifically, the convolutional neural network includes a base network ResNet18 and two fully connected layers connected in sequence. Thus, after the convolutional neural network training is completed, S103 is executed.

[0048] In the specific implementation, the target image is sampled to a preset size and then input into the basic network ResNet18 of the convolutional neural network. The output of the average pooling layer is extracted to obtain a feature vector of the first preset dimension, and then the first feature code of the second preset dimension is obtained through the output of two fully connected layers.

[0049] Figure 4 FIG. 4 shows an exemplary schematic diagram of obtaining the first feature encoding of a target image using a pre-trained convolutional neural network. Figure 4 As shown in the figure, the target image is first uniformly sampled to a size of 64×64, and then input into the basic network ResNet18 model. The output of the average pooling layer is extracted to obtain a 512-dimensional feature vector, and then a 128-dimensional feature vector is obtained through two fully connected layers to represent the first feature code of the input target image.

[0050] Further, in some optional embodiments, S105 may be implemented in the following manner:

[0051] Arrange a number of target objects in descending order according to recognition confidence to obtain a target list; create a first target cluster, determine the first target object in the target list as the first member of the first target cluster, and determine the mean of the first position information and the mean of the first feature code of all members in the target cluster as the second position information and the second feature code of the target cluster respectively;

[0052] Iterate the following steps A to B until all target objects in the target list are traversed to obtain several target clusters:

[0053] Step A: for the next target object of the current target object in the target list, the appearance distance between the target object and all existing target clusters is calculated according to the first feature code of the target object and the second feature code of the target cluster, and the spatial distance between the target object and all existing target clusters is calculated according to the first position information of the target object and the second position information of the target cluster;

[0054] Step B: When the appearance distance and the spatial distance meet the preset conditions, the similarity between the target object and the target cluster meeting the preset conditions is calculated; the target object is divided into the target cluster corresponding to the highest similarity, and the mean of the first position information and the mean of the first feature code of all members in the updated target cluster are respectively determined as the second position information and the second feature code of the updated target cluster;

[0055] When the appearance distance and the spatial distance do not satisfy the preset conditions, a new target cluster is created; the target object is divided into the newly created target cluster; and the first position information and the first feature code of the target object are respectively determined as the second position information and the second feature code of the newly created target cluster.

[0056] In specific implementation, the appearance distance between the target object and the target cluster can be calculated according to the following formula (2):

[0057]

[0058] Among them, D 外观 represents the appearance distance; f1 represents the first feature code of the target object; f2 represents the second feature code of the target cluster.

[0059] In a specific implementation, the first location information includes first longitude information and first latitude information, and the second location information includes second longitude information and second latitude information. Then, the spatial distance between the target object and the target cluster can be calculated according to the following formula (3):

[0060]

[0061] Among them, D 空间 represents the spatial distance; x1 represents the first longitude information of the target object; x2 represents the second longitude information of the target cluster; y1 represents the first latitude information of the target object; y2 represents the second latitude information of the target cluster.

[0062] In a specific implementation, the preset condition is that the appearance distance is less than a first preset threshold and the spatial distance is less than a second preset threshold. Then, the similarity between the target object and the target cluster that meets the preset condition can be calculated according to the following formula (4):

[0063]

[0064] Among them, S represents similarity; T 外观 represents the first preset threshold; T 空间 Indicates the second preset threshold.

[0065] That is to say, target matching is performed in the fusion drone node. Assuming that the fusion drone node collects a total of n targets, the output of the target matching algorithm is m target clusters, and the target in each cluster is the same target. The detailed matching steps are as follows:

[0066] ① Sort all targets from high to low according to target recognition confidence, and get the target list, recorded as Obj = [Obj1, Obj2, Obj3, ... Obj n ], where n is the total number of targets;

[0067] ② Denote the first target cluster after target matching as Cluster1, and take the first target Obj1 in the target list Obj as the first member of Cluster1. Then Cluster1 = [Obj1], and use the mean of the feature encodings and the mean of the positions of all members in the cluster as the feature encoding and position information of the cluster.

[0068] ③ Traverse the next target in Obj, and calculate the appearance distance D 外观 and the spatial distance D 空间 ;

[0069] ④ If there exists a target cluster that satisfies D 外观 <T 外观 and D 空间 <T 空间 (where T 外观 is the set threshold for the appearance distance, and T 空间 is the set threshold for the spatial distance), then calculate the similarity S between the current target and the current target cluster; after the calculation, sort all the target clusters that meet the conditions according to the S value, and the current target belongs to the target cluster corresponding to the maximum S value. Add the current target to this target cluster, and recalculate the mean of the feature encodings and the mean of the positions of all members in the target cluster to update the feature encoding and position information of the target cluster;

[0070] ⑤ If there is no cluster that satisfies D 外观 <T_appearance and D 空间 <T 空间 , then create a new target cluster, add the current target to the newly created target cluster, and the feature encoding and position information of the newly created target cluster are the feature encoding and position information of the newly added target;

[0071] ⑥ Repeat steps ③ - ⑤ until all targets in Obj are traversed, and finally output [Cluster1, Cluster2, Cluster3..., Cluster m , where m is the total number of target clusters.

[0072] In this way, the target matching method based on the combination of spatial and appearance information realizes the matching of the same target objects among several target objects, can improve the accuracy of target matching, and is very helpful for improving the information fusion accuracy in the subsequent steps.

[0073] Furthermore, in some embodiments, in S106, based on the Dempster combination rule, data fusion is performed on the first category information corresponding to at least two target objects respectively, and the fusion probability corresponding to each category after fusion is calculated: the category corresponding to the maximum fusion probability is determined as the target category information.

[0074] Specifically, the target categories are fused using DS evidence theory. The detailed steps are as follows:

[0075] ① Determine the recognition framework Θ = {class1, class2, class3..., class n}, where n is the type of target recognition;

[0076] ②The power of the identification frame Θ

[0077] 2 Θ =

[0078] {φ,class1,class2,class3...,class n ,{class1,class2}...{class n-1 ,class n}...,Θ}, is determined as the set of all subsets of the recognition frame Θ, which contains 2 n elements;

[0079] ③Determine the basic probability assignment BPA (Basic Probability Assignment), BPA is a 2 Θ →[0,1], and satisfies m(φ)=0 and Assuming that the target cluster contains a targets, it contains a m functions, represented as m1, m2, m3...m i ...m a . It is the probability distribution of target recognition, satisfying:

[0080] m i (class1)+m i (class2)+....+m i (class n )=1,power 2 Θ The probability of the remaining elements in is 0;

[0081] ④ Perform data fusion based on Dempster synthesis rule to obtain the fusion probability corresponding to each category. The calculation formula is as follows:

[0082]

[0083] ⑤After obtaining the fusion probability of all categories, the target category is the category corresponding to the maximum fusion probability.

[0084] In another embodiment, the specific implementation of location information fusion may include:

[0085] Iterate the following steps C to E until the number of target objects in the target cluster remains unchanged, and determine the mean of the position information of the first position information corresponding to each target object in the final target cluster as the target position information:

[0086] Step C: Calculate the mean of the first position information corresponding to all target objects in the target cluster, that is, Where a is the number of targets in the target cluster;

[0087] Step D: According to the formula Calculate the distance between each target object in the target cluster and the mean value (mean point) of the position information respectively;

[0088] Step E: When the distance between the target object and the mean value of the position information is greater than a third preset threshold, the target object is determined to be an abnormal point, and the corresponding target object is removed from the target cluster.

[0089] That is to say, after removing the target object corresponding to the outlier from the target cluster each time, the mean position information of the remaining targets in the target cluster is recalculated, that is, Where b is the number of remaining targets in the target cluster; then, the distance between all remaining targets in the target cluster and the mean point is calculated. If the distance is greater than the set threshold (the third preset threshold), the target is judged as an outlier and removed from the target cluster; the above steps are repeated until the number of targets in the target cluster remains unchanged, and the mean of the position information of the remaining targets in the final target cluster is used as the fused position mean.

[0090] Thus, by executing S106, after obtaining the target cluster, information fusion is performed in each target cluster in turn to obtain the fusion information of the target. It should be understood that if there is only one target in the target cluster, fusion is not required, and the information of the target is directly used as the fusion information. Among them, information fusion includes target category fusion and position information fusion.

[0091] also, Figure 5 FIG. 2 is a flow chart showing another method for fusion of information of a drone cluster according to an embodiment of the present application. Figure 5As shown, another method for fusion of information of drone clusters may specifically include the following steps: all drone nodes in the drone cluster obtain the category and position information of the target through photoelectric payload reconnaissance; each drone node crops the target image according to the target position information, and inputs the cropped target image into the pre-trained convolutional neural network for feature extraction to obtain the feature expression of the target; each drone node transmits the category, position and feature expression of the target to the fusion drone node through the communication network; the fusion drone node associates the target based on the target information (position and feature expression) of all drone nodes through the matching algorithm; the associated target is fused based on the target category information of all drone nodes through the DS evidence theory algorithm to obtain the fused target category information; the associated target is fused based on the target position information of all drone nodes through the average algorithm after removing outliers to obtain the fused target position information. Thus, the final fused target information is obtained.

[0092] It can be seen that the low information transmission volume UAV cluster information fusion method of the embodiment of the present application is based on the target information transmission method of feature coding, which has a small target information transmission volume and high efficiency. That is to say, the information fusion method for target matching by transmitting target feature coding can not only improve the data transmission efficiency and occupy a smaller communication bandwidth, but also improve the efficiency of multi-machine information fusion; at the same time, the target matching method combining appearance and spatial features has stronger target feature expression and can improve the target matching accuracy, providing a guarantee for improving the accuracy of information fusion. It solves the problems of large data transmission volume and insufficient target matching accuracy in the current UAV cluster fusion technology. It realizes the use of multi-machine information fusion technology to improve the efficiency and accuracy of UAV clusters in the information acquisition process, which is of great significance for improving the performance of the entire system, meeting the growing industry needs and coping with complex and changing application scenarios.

[0093] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] It should also be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0095] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0096] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for fusion of drone cluster information, characterized in that: The drone cluster includes a plurality of first drone nodes and a second drone node; The method comprises: The first UAV node obtains an optoelectronic reconnaissance image, first category information, and first position information of the target object; The first UAV node crops the photoelectric reconnaissance image according to the first position information to obtain a target image; The first drone node inputs the target image into a pre-trained convolutional neural network to obtain a first feature code corresponding to the target image; The first drone node sends the first category information, the first location information and the first feature code to the second drone node; The second UAV node divides the corresponding target objects into a plurality of target clusters according to the first position information and the first feature codes respectively sent by the first UAV nodes, wherein the number of target clusters is less than or equal to the number of target objects, and the target objects in each of the target clusters are the same; For each target cluster, when the target cluster includes one target object, the first category information and the first position information of the target object are respectively determined as the target category information and the target position information; when the target cluster includes at least two target objects, the first category information and the first position information corresponding to the at least two target objects are respectively fused to obtain the target category information and the target position information.

2. The method according to claim 1, characterized in that The pre-trained convolutional neural network includes a base network ResNet18 and two fully connected layers connected in sequence; The first drone node inputs the target image into a pre-trained convolutional neural network to obtain a first feature code corresponding to the target image, including: The target image is sampled into a preset size and input into the basic network ResNet18, the output of the average pooling layer is extracted to obtain a feature vector of a first preset dimension, and then the first feature code of a second preset dimension is obtained through the output of two fully connected layers.

3. The method according to claim 1, characterized in that: Before the first drone node inputs the target image into a pre-trained convolutional neural network to obtain a first feature code corresponding to the target image, the method further includes: Acquire an image sample set, wherein the image sample set includes a plurality of triplets, each of the triplets includes a reference sample, a positive sample, and a negative sample; Input the triplet into a preset model to obtain sample feature codes corresponding to the reference sample, the positive sample and the negative sample, respectively, wherein the preset model is a Triplet network, and the Triplet network includes three identical CNNs and weights are shared between the CNNs; Calculating the loss value of the triplet according to the sample feature codes corresponding to the benchmark sample, the positive sample and the negative sample; The network weights are updated according to the loss value, and the updated preset model is trained to obtain a trained convolutional neural network.

4. The method according to claim 3, characterized in that The calculating the loss value of the triplet according to the sample feature codes corresponding to the reference sample, the positive sample and the negative sample respectively includes: According to the sample feature codes corresponding to the reference sample, the positive sample and the negative sample, the loss value of the triplet is calculated by the following formula: Loss=max(d(a,p)-d(a,n)+margin,0); Among them, Loss represents the loss value of the triplet; d(a,p) represents the cosine distance between the sample feature encoding of the benchmark sample and the sample feature encoding of the positive sample; d(a,n) represents the cosine distance between the sample feature encoding of the benchmark sample and the sample feature encoding of the negative sample; margin represents a pre-set interval value used to control the distance between the positive sample and the negative sample.

5. The method according to claim 1, characterized in that The second UAV node divides the corresponding target objects into a plurality of target clusters according to the first position information and the first feature codes respectively sent by the first UAV nodes, including: Arrange several target objects in descending order according to recognition confidence to obtain a target list; Create a first target cluster, determine the first target object in the target list as the first member of the first target cluster, and determine the mean of the first position information and the mean of the first feature code of all members in the target cluster as the second position information and the second feature code of the target cluster respectively; Iterate the following steps A to B until all target objects in the target list are traversed to obtain several target clusters: Step A: for the next target object of the current target object in the target list, the appearance distance between the target object and all existing target clusters is calculated according to the first feature code of the target object and the second feature code of the target cluster, and the spatial distance between the target object and all existing target clusters is calculated according to the first position information of the target object and the second position information of the target cluster; Step B: When the appearance distance and the spatial distance meet the preset conditions, the similarity between the target object and the target cluster meeting the preset conditions is calculated; the target object is divided into the target cluster corresponding to the highest similarity, and the mean of the first position information and the mean of the first feature code of all members in the updated target cluster are respectively determined as the second position information and the second feature code of the updated target cluster; When the appearance distance and the spatial distance do not satisfy the preset conditions, a new target cluster is created; the target object is divided into the newly created target cluster; and the first position information and the first feature code of the target object are respectively determined as the second position information and the second feature code of the newly created target cluster.

6. The method according to claim 5, characterized in that The preset condition is that the appearance distance is less than a first preset threshold and the spatial distance is less than a second preset threshold, the first location information includes first longitude information and first latitude information, and the second location information includes second longitude information and second latitude information; The step of calculating the appearance distance between the target object and all existing target clusters according to the first feature code of the target object and the second feature code of the target cluster comprises: The appearance distance between the target object and the target cluster is calculated according to the following formula: Among them, D 外观 represents the appearance distance; f1 represents the first feature code of the target object; f2 represents the second feature code of the target cluster; The step of calculating the spatial distance between the target object and all existing target clusters according to the first position information of the target object and the second position information of the target cluster includes: The spatial distance between the target object and the target cluster is calculated according to the following formula: Among them, D 空间 represents the spatial distance; x1 represents the first longitude information of the target object; x2 represents the second longitude information of the target cluster; y1 represents the first latitude information of the target object; y2 represents the second latitude information of the target cluster; When the appearance distance and the spatial distance meet the preset conditions, calculating the similarity between the target object and the target cluster meeting the preset conditions includes: The similarity between the target object and the target cluster is calculated according to the following formula: Among them, S represents similarity; T 外观 represents the first preset threshold; T 空间 Indicates the second preset threshold.

7. The method according to claim 1, characterized in that In the case where the target cluster includes at least two target objects, respectively fusing the first category information and the first location information corresponding to the at least two target objects to obtain the target category information and the target location information includes: Based on the Dempster synthesis rule, data fusion is performed on the first category information corresponding to at least two target objects, and the fusion probability corresponding to each category after fusion is calculated: The category corresponding to the maximum fusion probability is determined as the target category information.

8. The method according to claim 1, characterized in that In the case where the target cluster includes at least two target objects, respectively fusing the first category information and the first location information corresponding to the at least two target objects to obtain the target category information and the target location information includes: Iterate the following steps C to E until the number of target objects in the target cluster remains unchanged, and determine the mean of the position information of the first position information corresponding to each target object in the final target cluster as the target position information: Step C: calculating the position information mean of the first position information corresponding to all target objects in the target cluster; Step D: Calculate the distance between each target object in the target cluster and the mean value of the position information respectively; Step E: when the distance between the target object and the mean value of the position information is greater than a third preset threshold, remove the corresponding target object from the target cluster.

9. The method according to claim 1, characterized in that: The first UAV node crops the photoelectric reconnaissance image according to the first position information to obtain a target image, including: According to the first position information, a target object in the photoelectric detection image is identified by using a recognition algorithm to obtain a first area including the target object; Determine a preset multiple of a first side length of the first area as a target side length, and determine a center of the first area as a target center; Determine a preset shape area corresponding to the target side length and the target center as the target area; The regions other than the target region in the photoelectric detection image are cut out to obtain a target image.

10. The method according to claim 1, characterized in that The first drone node obtains first location information of the target object, including: Get the drone’s position information; The position information of the drone is calculated to obtain the first position information of the target object.

Images