An air-ground cooperative situation fusion method and system based on a drone and an unmanned vehicle

Abstract

The application discloses an air-ground cooperative situation fusion method and system based on unmanned aerial vehicles and unmanned vehicles, and belongs to the fields of image processing and computer vision. The method comprises the following steps: uniformly describing the feature structures of N targets detected by the unmanned aerial vehicle cluster and the unmanned vehicle cluster, and generating N target feature vectors; searching for similar feature vectors which are similar to a target feature vector and have been detected and stored in a target feature vector database, so that the target feature vector and the similar feature vectors form a target feature vector set; clustering the target feature vectors based on multi-feature frequency weight and multi-target cluster set quality clustering criteria, and clustering the target feature vector set, so as to realize air-ground cooperative situation fusion. The application solves many problems of the air-ground cooperative situation fusion system based on unmanned aerial vehicles and unmanned vehicles.

Description

Technical Field

[0001] This invention belongs to the fields of image processing and computer vision, and specifically relates to an air-ground cooperative situational awareness fusion algorithm based on UAVs and unmanned vehicles. Background Technology

[0002] In the past decade or so, with the development of artificial intelligence, robots have demonstrated excellent characteristics in high-risk, repetitive, and complex tasks, leading to the rapid development of robotics technology. In particular, drones and unmanned vehicles have been widely used in tactical reconnaissance, dangerous area operations, disaster relief, target tracking, personnel search and rescue, guidance and navigation, and other fields.

[0003] Unmanned aerial vehicles (UAVs) possess strong reconnaissance capabilities. Their swarm reconnaissance methods can capture images of the reconnaissance site from multiple angles, exhibiting strong stealth, adaptability, and mobility. They can quickly achieve situational awareness over wide areas. However, limitations in size, weight, payload power, and communication capabilities restrict the endurance of UAV swarms for extended missions. Unmanned vehicles (UAVs) have strong carrying capacity and long endurance. Their swarm reconnaissance methods can approach multiple individual targets at close range and obtain detailed information about them. However, they cannot acquire overall situational awareness, and their mobility is significantly reduced when ground roads are blocked, making them susceptible to ground environmental constraints.

[0004] The method of using UAV swarms and unmanned vehicles for air-ground collaborative situational awareness integrates the advantages of ground unmanned vehicles, such as long endurance, high level of ground detail reconnaissance, and strong carrying capacity, as well as the advantages of UAV swarms, such as high mobility, strong concealment, strong survivability, and relatively low price. At the same time, it makes up for the disadvantages of ground unmanned vehicles, such as limited reconnaissance field of view and low travel speed, and UAV swarms, such as relatively weak endurance. It meets the needs of future missions with poor visibility and complex electromagnetic environment, and can quickly achieve all-round and multi-dimensional situational awareness.

[0005] The air-ground collaborative situational awareness fusion system based on UAVs and unmanned vehicles primarily processes video images. It uses a formation of unmanned vehicle swarms carrying UAV swarms. After reaching the core area, the ground-based unmanned vehicle swarm releases the UAV swarm, utilizing the wide high-altitude field of view of the UAV swarm to obtain high-altitude reconnaissance images. These images are then combined with the ground-based unmanned vehicles' detailed images of ground targets. This deep fusion of target information from different dimensions, angles, and scales acquired by the UAV and unmanned vehicle swarms allows for rapid acquisition of battlefield situational awareness. Currently, air-ground collaborative situational awareness fusion technology based on UAVs and unmanned vehicles mainly faces the following challenges:

[0006] 1) The shooting perspectives, angles, and levels of detail differ between unmanned vehicles and drones, leading to cross-domain and cross-scale issues in air-to-ground information fusion. Cross-domain refers to the fact that aerial drone swarms and ground unmanned vehicle swarms are conducting information detection in different domains, while cross-scale refers to the different scales of information detection conducted by aerial drone swarms and ground unmanned vehicle swarms.

[0007] 2) The reconnaissance methods and focuses of drone swarms and unmanned vehicle swarms are different, and the target feature dimensions they acquire are different. This results in inconsistent target feature representation and makes it impossible to complete information fusion.

[0008] 3) The target information of drone swarms and unmanned vehicle swarms is large. When the same target is detected by multiple drones and unmanned vehicles multiple times, there is a lack of effective means to identify duplicate targets, resulting in the same target being counted multiple times. Summary of the Invention

[0009] Based on the shortcomings of UAV and unmanned vehicle air-ground cooperative situation fusion, and taking into account the algorithm performance and adaptability, this invention proposes an air-ground cooperative situation fusion method and system based on UAV and unmanned vehicle, which solves many problems of air-ground cooperative situation fusion system based on UAV and unmanned vehicle.

[0010] According to a first aspect of the technical solution of the present invention, an air-ground cooperative situation fusion method based on unmanned aerial vehicles and unmanned vehicles is provided, the method comprising the following steps:

[0011] Step 1: Use target feature description technology to uniformly describe the feature structure of the target information detected by the unmanned vehicle cluster and the drone cluster, and generate multiple target feature vectors;

[0012] Step 2: For a target feature vector, quickly retrieve similar feature vectors that have been detected and stored in the target feature vector database and are similar to the target feature vector through feature retrieval. The target feature vector and its similar feature vectors together form the target feature vector set.

[0013] Step 3: Cluster the target features based on the multi-feature frequency weight and multi-target cluster quality clustering criteria, and cluster the target feature vector set to achieve air-ground coordinated situational awareness fusion.

[0014] Furthermore, step 1 specifically includes: constructing a unified target feature description structure, and performing unified target feature description on each of the N targets detected by the UAV swarm and the unmanned vehicle swarm in a coordinated manner, thereby generating N target feature vectors.

[0015] Furthermore, in step 1, the target feature vector uses the target detection category, target detection confidence, target identification name code, target identification confidence, target longitude, target latitude, target altitude, target color code, target height, target width, and target depth as feature dimensions.

[0016] Furthermore, in step 2, the fast feature retrieval technology is a feature vector retrieval technology based on a sparse-table indexing algorithm.

[0017] Further, step 2 specifically includes: using the i-th target feature vector as the retrieval basis, i∈[1,N], and using the feature vector retrieval based on the Sparse-Table indexing algorithm, searching in the target feature vector database to retrieve M similar feature vectors that have been detected and stored in the database and are similar to the target feature. The i-th target feature vector and its M similar feature vectors together form the target feature vector set U, where M is a positive integer.

[0018] Furthermore, step 3 specifically includes:

[0019] Step 31: For the M+1 target feature vectors in the target feature vector set U, calculate the feature weight values ​​based on rough set theory and using the multi-feature frequency criterion;

[0020] Step 32: Detect and filter out the feature vectors of noisy targets using the method of minimizing interval dispersion;

[0021] Step 33: Perform hierarchical agglomerative clustering based on multi-objective clustering criteria;

[0022] Step 34: Determine the relevant feature subspace for each cluster;

[0023] Step 35: When the target feature vector of the i-th target belongs to the relevant feature subspace of a certain cluster, the target is determined to be a target that has been detected and stored in the database, and its feature vector is stored in the database again; when the target feature vector of the i-th target does not belong to the relevant feature subspace of any cluster, the target is determined to be a new detection target, and its feature vector is stored in the database.

[0024] Step 36: Perform clustering of the (i+1)th target until the target of the UAV swarm and unmanned vehicle swarm air-ground collaborative reconnaissance is integrated in the current situation.

[0025] Furthermore, step 3 also includes: removing duplicate targets in empty areas to increase target credibility.

[0026] Furthermore, step 31 specifically includes:

[0027] Suppose that the (M+1) feature vectors in the target feature vector set U are divided into several class cluster sets C. s And the noise set NOS, the feature vector x k It consists of D classification features, formally represented as x k ={x k1 ,x k2 ,x k3 ,···,x kD}, x kd For the feature vector x k The d-th classification feature a d Eigenvalues ​​on [1, M+1], d ∈ [1, D]; in the eigensubspace SA s Below, cluster C s From the feature vector set SD s Composition, C s Represented as a pair (SD) s SA s ),

[0028] From classification feature a d To measure the eigenvalue x from the perspective of kd If the single feature weights are given, then x kd Single feature weight Defined as

[0029]

[0030] in, For containing feature vector x k a d Equivalence classes The feature vector within the classification feature a d The value of x k same, Reflects the eigenvalue x kd In a d The number of times it appears above;

[0031] From relevant feature a e To measure x from the angle kd The multi-feature weights, then the multi-feature weights Defined as

[0032]

[0033] in, This indicates that the feature vector x is included. k a e Equivalence classes The eigenvalue x represents the number of elements in the intersection of two equivalence classes. kd With x ke The number of times they appear together, The definition indicates that the eigenvalue x kd With x ke Co-occurrence frequency accounts for The larger the proportion, the better from the classification feature a e The angle reflected by x kd The greater the clustering effect,

[0034] Using W(x) kd ) represents the overall weight, and is normalized to make 0 <W(x kd If ) < 1, the calculation formula is defined as

[0035]

[0036] Therefore, x is obtained. kd The feature weight values.

[0037] Furthermore, step 32 specifically includes:

[0038] Step 321: Calculate the eigenvector x k Degree of polymerization O S (x k ):

[0039]

[0040] Step 322: According to the degree of polymerization O S (x k Sort all feature vectors in ascending order;

[0041] Step 323: Utilize interval dispersion Find the optimal split point x h The value of h satisfies That is, the partition point makes the vector interval [x1, x...] h ] and vector [x h+1 ,x M+1 The sum of the discretenesses of ] is the smallest among all partition points, where:

[0042] Interval Dispersion Variance is used to describe the vector interval [x] f ,x g The degree of separation of feature vectors in [ ]:

[0043]

[0044] Represents the vector interval [x] f ,x g The average degree of polymerization; |gf| represents the absolute value of the difference between g and f; The smaller the value, the closer the feature vectors are within the interval;

[0045] Step 324: With the goal of minimizing interval dispersion, a clustering approach similar to k-means is used to divide the feature vectors into two classes, where the low-aggregation vector interval [x1, x2, x3, x4, x5, x6, x7, x8, x9, x1, x1, x2, x3, x1, x2, x3, x4, x5, x6, x7, x8, x9, x1, x1, x2, x1, x h The feature vectors in [ ] are the feature vectors of noisy targets with poor clustering ability.

[0046] Furthermore, step 33 specifically includes:

[0047] Step 331: In the initial clustering stage, a feature vector is randomly selected as the first sub-cluster. Using the sub-cluster quality function, according to the principle of maximizing the sub-cluster quality, the cluster quality of other feature vectors to be clustered and the first sub-cluster is calculated in turn. Thus, each feature vector is sequentially selected and assigned to an existing sub-cluster or a newly generated sub-cluster.

[0048] Step 332: In the cluster merging phase, using the overall cluster quality function, iteratively merge each sub-cluster based on maximizing the cluster quality function, until no more sub-cluster merging operations are generated during the entire iteration process.

[0049] Furthermore, in step 331, the sub-cluster set quality function is:

[0050] Let x kd For the feature vector x k The d-th classification feature a d Eigenvalues ​​and eigenvectors x k Let C1 be the first subset, and Q(C1) be used to describe the quality of C1. The quality values ​​are measured by intra-cluster compactness and inter-cluster separation, respectively. Com(x) kd ) is used to represent the characteristic value x kd The measured intra-cluster compactness, Sep(x) kd ) is the eigenvalue x kd Inter-cluster separation as a metric:

[0051]

[0052] Intra-cluster compactness Com(x) kd Evaluate from two aspects: eigenvalue x kd The distribution within the first subset C1 is determined by the eigenvalue x. kd In classification feature a d The probability P(a) on d =x kd ) and the probability P(a) of that value within the first sub-cluster C1. d =x kd The product of |C1) is used as the metric, and this value reflects the eigenvalue x. kdConcentration on the first sub-cluster C1; eigenvalue x kd The importance of the first subset C1 is determined by the weight W(x) of this value. kd )express;

[0053] Inter-cluster separation degree Sep(x) kd It depends on the eigenvalue x. kd The degree to which it belongs exclusively to the first subset C1 is denoted by x. kd Classification feature a appearing in the first sub-cluster C1 d The probability P(a) on d =x kd ∧(x k ∈C1)) and x on the entire dataset kd The probability of occurrence P(a) d =x kd The proportion of 'a' indicates that the larger the value, the stronger the classification feature 'a'. d eigenvalues ​​x on kd The more concentrated the occurrences, the more likely they are to appear in the first sub-cluster C1; count(x kd ,a d C1) represents the projection onto the classification feature a within the first subset C1. d The value on is x kd The number of feature vectors; n represents the total amount of data in the dataset; W(x kd ) is the eigenvalue x kd The weights; count(x) kd ,a d ) refers to the classification feature a d Above, x kd Total number of occurrences.

[0054] Furthermore, in step 332, the overall cluster quality function is:

[0055] Let the cluster set C = {C1, C2, ..., C} K If we use Q(C) to describe the overall quality of the cluster, then:

[0056]

[0057] Q(C s Then it represents cluster C. S The quality, P(C) S ) represents C S The proportion of feature vectors in the dataset.

[0058] Furthermore, step 34 specifically includes;

[0059] Step 341: Calculate the classification feature a d For cluster C SDegree of dependence R(a) d C s The formula is as follows:

[0060]

[0061] Step 342: According to the degree of dependence R(a) d C s Sort all categories in ascending order;

[0062] Step 343: Utilize interval dispersion Find the optimal split point a o The value of o satisfies That is, the dividing point makes the vector interval [a1, a...] o ] and vectors [a o+1 ,a D The sum of the discretenesses of ] is the smallest among all partition points, where:

[0063] Interval dispersion Variance is used to describe the vector interval [a f ,a g The degree of separation of feature vectors in [ ]:

[0064]

[0065] Represents the vector interval [a f ,a g The average degree of polymerization; |gf| represents the absolute value of the difference between g and f; The smaller the value, the closer the feature vectors are within the interval;

[0066] Step 344: With the goal of minimizing interval dispersion, a clustering approach similar to k-means is used to divide the classification vectors into two classes, where the high-dependency interval [[a o+1 ,a D That is, cluster C. S The relevant feature subspace.

[0067] According to a second aspect of the technical solution of the present invention, an air-ground cooperative situational awareness fusion system based on unmanned aerial vehicles (UAVs) and unmanned vehicles (UAVs) is provided, the system comprising:

[0068] The target feature unified description component is used to uniformly describe the feature structure of target information detected by the unmanned vehicle cluster and the drone cluster respectively through target feature description technology, and generate multiple target feature vectors.

[0069] The feature fast retrieval component is used to retrieve similar feature vectors that have been detected and stored in the target feature vector database for a certain target feature vector. The target feature vector and its similar feature vectors together form the target feature vector set.

[0070] The target feature fusion and clustering component is used to cluster target features based on multi-feature frequency weights and multi-target cluster quality clustering criteria, and to cluster the target feature vector set to achieve air-ground coordinated situational fusion.

[0071] Compared with the prior art, the present invention has the following advantages:

[0072] 1) This invention introduces a unified target feature description model, which not only effectively solves the cross-domain and cross-scale problems of air-to-ground information fusion based on UAV swarms and unmanned vehicle swarms under different shooting perspectives, shooting angles and shooting details, but also effectively solves the problem of inconsistent target feature representations obtained by UAV swarms and unmanned vehicle swarms.

[0073] 2) This invention introduces target feature fusion clustering technology, which solves the problem of duplicate counting when the same target is detected by multiple drones and unmanned vehicles. Attached Figure Description

[0074] Figure 1 Overall flowchart of the air-ground collaborative situational awareness fusion algorithm based on UAVs and unmanned vehicles. Detailed Implementation

[0075] First, a unified target feature description structure is constructed using target feature description technology, unifying the feature structure descriptions of targets detected by both unmanned vehicle (UAV) and drone (UAV) swarms. Then, a rapid feature retrieval technique is used to search a target feature vector database, retrieving feature vectors that have already been detected and are similar to the target's features to form a target feature vector set. Finally, a target feature clustering technique based on multi-feature frequency weights and multi-target cluster quality clustering criteria is employed to cluster the retrieved target feature vectors, removing duplicate targets between air and ground, increasing target credibility, and achieving air-ground coordinated situational awareness fusion. The overall process is as follows: Figure 1 As shown.

[0076] The algorithm can be summarized into the following steps:

[0077] 1) A unified target feature description technique is adopted to uniformly describe the target information of both UAV swarm reconnaissance and UAV swarm reconnaissance, generating target feature vectors. The target feature vectors use target detection category, target detection confidence, target identification name / code, target identification confidence, target longitude, target latitude, target altitude, target color code, target height, target width, and target depth as feature dimensions. Assuming there are N reconnaissance targets, N one-dimensional target feature vectors are generated.

[0078] for i = 1; I <= N:

[0079] 2) Based on the feature vector value of the i-th target, the feature vector retrieval technology based on the Sparse-Table indexing algorithm is used to search the target feature vector database. A total of M feature vectors that have been detected and stored in the database and are similar to the features of the target are retrieved to form the target feature vector set.

[0080] 3) For the target feature vector set composed of these M+1 target feature vectors, a target clustering technique based on multi-feature frequency weights and multi-target cluster quality clustering criteria is used to cluster these M+1 target feature vectors. The specific process is as follows:

[0081] ① Calculate feature weights. Considering the influence of the correlation between multiple feature dimensions on the feature weights, the intersection of equivalence classes of feature vectors under different features is used to count the co-occurrence frequency of feature values ​​on multiple features. Based on rough set theory, the weights of each feature value are calculated using the multi-feature frequency criterion.

[0082] ② Detect noise vectors. Utilize the method of minimizing interval dispersion to filter out low-weight vectors, thereby improving subsequent clustering results;

[0083] ③ Hierarchical agglomerative clustering based on multi-objective clustering criteria. Utilizing the clustering criteria of intra-cluster compactness and inter-cluster separation, a cluster quality function is given. A bottom-up agglomerative clustering strategy is adopted in the clustering process, which is divided into two stages: initial clustering and merging clustering. In the initial clustering stage, the most similar feature vectors are used to generate sub-clusters using the multi-objective cluster quality function. In the merging clustering stage, the sub-clusters are iteratively merged to improve the overall cluster quality, forming the final clusters.

[0084] ④ Identify the feature-related subspace. Based on the dependence of each feature within a cluster on the cluster, determine the relevant feature subspace for each cluster.

[0085] 4) When the feature vector of the i-th target belongs to the relevant feature subspace of a cluster, the target is determined to be a target that has already been detected and stored in the database, and its feature vector is stored in the database again; when the feature vector of the i-th target belongs to the relevant feature subspace of a cluster, the target is determined to be a new detection target, and its feature vector is stored in the database. Then proceed to step 2) to perform clustering of the (i+1)-th target, thereby realizing the fusion of targets in the current situation for UAV swarm and unmanned vehicle swarm air-ground collaborative detection.

[0086] 1. Unified description of target features

[0087] Drone swarms and unmanned vehicle swarms differ in their target reconnaissance methods and focuses, resulting in different dimensions of target features. This leads to inconsistent target feature representations and hinders information fusion. Therefore, it is necessary to establish a unified feature table for the targets detected by both drone and unmanned vehicle swarms to obtain a unified feature structure representing the detected targets.

[0088] This invention combines the reconnaissance characteristics of UAVs and unmanned vehicles, employing target detection category, target detection confidence score, target identification name code, target identification confidence score, target longitude, target latitude, target altitude, target color code, target height, target width, and target depth features as the feature dimensions of a unified target feature description structure. Specifically, this invention introduces a CNN network model, where the target depth feature is composed of the last-level convolutional features extracted from the reconnaissance target image based on the CNN network model. The unified target feature description structure is shown in Table 1.

[0089] Table 1 Unified Description Structure of Target Features

[0090]

[0091] In actual air-to-ground information data collection, a multi-method collaborative detection approach can significantly improve detection accuracy. When unmanned aerial vehicles (UAVs) reconnoiter a reconnaissance area from multiple angles and directions, the shooting angles differ, but the shooting time is approximately the same. Therefore, the detection processes are independent of each other. Assuming there are n UAVs shooting at the same area, and there are K types of targets in the scene, the detection process of the i-th UAV is denoted by f. i (·) indicates that the confidence level of the test result is represented by P. i (·) represents the target. Therefore, for target O, if the i-th unmanned device detects target O as belonging to the k-th class, then the result f of the i-th unmanned device detecting target O is... i (O) = k, and the confidence level of the detection result is P. i (O).

[0092] For target O, the overall confidence level of detecting target O as class k during the same shooting process is denoted by C. k In other words, then

[0093]

[0094] In practical use, the category with the highest overall confidence level for target O is taken as the final category K for detection. O .

[0095] At this point, the category of the detected target O is defined as K. O Let M be a set of unmanned devices. Then the probability of a detection error in M ​​is:

[0096]

[0097] Since each detection process is independent, the probability of a detection error when all individuals in set M detect that target O belongs to category k can be expressed as:

[0098]

[0099] Therefore, the probability of a correct detection is:

[0100]

[0101] In practice, by incorporating comprehensive confidence scores, it is possible to correct individual classification errors caused by shooting angles or other conditions, thereby improving classification accuracy. Simultaneously, the combined air-to-ground detection results enhance the final image detection accuracy. Based on experience with single unmanned target detection, the confidence score of a single detected target is typically around 0.6. Therefore, if two targets are detected simultaneously belonging to the same class, the probability of correct detection can be calculated using equation (4): P = 1 - 0.4 * 0.4 = 0.86. If three devices simultaneously detect a target belonging to the same class, the accuracy can be increased to P = 1 - 0.4 * 0.4 * 0.4 = 0.936. In practice, the confidence score of a single detection is usually determined by a combination of factors such as shooting angle and image clarity.

[0102] 2. Feature-based rapid retrieval technology

[0103] In actual target retrieval, the obtained target vector may not be completely consistent with the vector data in the database. Therefore, this invention adopts the approximate nearest neighbor query method, which selects the more similar targets by comparing the similarity between the vector data in the database and the retrieved target.

[0104] In the process of approximate nearest neighbor query, for any query vector q, calculate q and any vector x in the database. i When determining the distance between q and x, an asymmetric distance computation method is used for distance calculation. The starting point of the hash method is to avoid directly calculating q and x.i The Euclidean distance between them is D(q,x′) i If the distance between q and every vector in the database is calculated, the time cost of the query is too high. In the approximate nearest neighbor query process, q and x are used... i The asymmetric distance AD(q,x) between them i The original distance D(q,x′) is approximately represented by the following. i ), where AD(q,x) i )=D(q,x′ i ), x′ i It is x i The cluster center to which it belongs. D(q,x′) i The distances can be calculated and stored in a lookup table beforehand. During subsequent comparisons, the asymmetric distances in the lookup table are used to approximate the original distances. In the specific algorithm flow, we first calculate the distance between the corresponding sub-vector of q in the subspace and the cluster center in the subspace, and store the calculated distances in a lookup table. Now we calculate the distance between the query vector q and each data point in the database. In each subspace, since the distances between the sub-vector and the cluster center are already stored in the lookup table, we can find the approximate distance between each data point and vector q. Finally, we sum the distances of the same vector in different subspaces. This gives us the distance between the query vector q and each vector in the database. By linearly scanning the distance array once, we can quickly obtain the top k nearest neighbor vectors.

[0105] Approximate nearest neighbor search methods can generally be divided into two main categories: binary tree-based indexes and hash-based indexes. Since the target data retrieved in this invention is a one-dimensional vector, this invention uses a Sparse-Table index based on a binary tree structure to implement the retrieval.

[0106] The Sparse Table algorithm, or ST algorithm for short, can be used to solve the Range Query (RMQ) problem. The RMQ problem typically takes the form of: given a large array, the task is to quickly find the maximum or minimum value within a given range.

[0107] A naive approach is to scan all numbers from the starting point to the ending point and maintain the maximum and minimum values ​​among them. This approach has a time complexity of O(n^2). 2 The previous method was too slow. The ST algorithm uses a dynamic programming approach similar to binary search, with a complexity of O(nlogn), making it very fast. The execution process of the ST algorithm (taking finding the maximum value as an example):

[0108] 1) Initialization:

[0109] Let the original array be x[N]. Allocate an array dp[N]. Here, dp[i][j] represents the array starting from the element at index i and ending at index (i+2). j The maximum value among these elements is defined up to the element with respect to -1). For integers, the value will not exceed 2^32, so a second dimension size of 33 is sufficient. Therefore, dp[i][0] represents the element itself, and can be initialized as dp[i][0] = x[i]. For the other dp[i][j], dynamic programming can be used to find the maximum value among them. The recurrence relation is dp[i][j] = max(dp[i][j-1], dp[i+2)). (j-1) The algorithm [j-1] essentially divides an interval into two equal-sized intervals. The maximum value of the current interval is the larger of the maximum values ​​of the two sub-intervals. The initialization complexity is O(nlogn).

[0110] 2) Solution:

[0111] Given a starting point `beg` and an ending point `end`, the size of the interval is `range = end - beg + 1`. Therefore, we can find an integer `k = (int)(log(range) / log2)`. This allows the interval to be divided into sub-interval 1, i.e., [beg, beg + (2...]. k )-1], subinterval 2, i.e. [end-(2 k These two may overlap, but the overlap will not affect the solution of the maximum value. Therefore, for beg and end, the solution can be obtained as res = max(dp[beg][k], dp[end-(2)). k The time complexity of solving this problem is O(1).

[0112] For a certain RMQ problem, the total complexity is O(nlogn) + O(1) = O(nlogn), so the maximum or minimum value of the interval can be obtained in a sufficiently fast time.

[0113] This invention employs the Sparse Table algorithm to perform approximate nearest neighbor lookups on stored data. Compared to the original linear scanning method that calculates similarity one-to-one with a complexity of O(n^2), the Sparse Table algorithm has a complexity of only O(nlogn). As the amount of data increases, the retrieval speed will also be greatly improved.

[0114] 3. Target Feature Fusion Clustering Technique

[0115] This invention proposes to employ multi-feature frequency weighting and a multi-objective cluster quality clustering criterion. The algorithm utilizes equivalence classes from rough set theory to define a multi-feature weighting calculation method, effectively improving the clustering discrimination ability of features. Based on the multi-objective cluster quality function, a hierarchical agglomeration strategy is adopted to iteratively merge sub-clusters, effectively measuring clusters at various scales. Interval dispersion is utilized to solve the parameter problems caused by using thresholds to remove noise points. The degree of feature dependence on clusters is used to determine the feature-related subspace of clusters, improving the interpretability of clusters.

[0116] For the target feature vector set consisting of the i-th target feature vector and its M retrieved similar feature vectors, the spatial clustering objective of this invention is to divide the (M+1) feature vectors in the target feature vector set U into several clusters C. s And the noise set NOS, the feature vector x k It consists of d classification features, which can be formally represented as x k ={x k1 ,x k2 ,x k3 ,…,x kd}, x kd For the feature vector x k The value of the d-th categorical feature. In the feature subspace SA s Below, cluster C s From the feature vector set SD s Composition, C s It can be represented as a pair (SD) s SA s ).

[0117] As shown in Table 2, assuming U has 10 feature vectors, each feature vector consists of 8 classification feature values, the feature vector set can be divided into three clusters and a noise set, denoted as C1, C2, C3 and NOS, respectively, x1, x2, x3, ..., x 10 Let ai be a set of 10 feature vectors, where a1, a2, a3, ..., a8 are 8 classification features within a single feature vector. Using multi-feature frequency to calculate feature weights effectively reflects the distribution characteristics of feature values ​​in the feature space, solving problems such as decreased clustering discriminative power caused by single-feature weight calculation. For example, analyzing the feature space distribution, since the feature value x... 43 With x 42 (RR), x 43 With x 44 (RV) appears simultaneously in the feature vector set at a frequency of 30%, while x 35 (K) has a co-occurrence frequency of only 10% with other features. Therefore, using categorical features a2 and a4, the feature value x is calculated based on the multi-feature frequency. 43With x 33 The weights can effectively distinguish the weight differences between two feature values. Based on the goal of minimizing intra-cluster distance, combining the goal of maximizing inter-cluster separation as a clustering criterion helps to correctly partition marginal data. The inability of a single cluster center to reflect the characteristics of the major class is also one of the reasons for the homogenization of imbalanced data. Therefore, the hierarchical agglomerative clustering idea is adopted, using sub-clusters to represent multiple cluster centers of the major class. By iteratively merging sub-clusters, the sub-clusters of the major class are grouped together, thereby solving the clustering problem of imbalanced data.

[0118] Table 2 Examples of Clustering

[0119]

[0120] Therefore, the steps for solving the data clustering problem in this invention are as follows:

[0121] 1) Calculation of feature weights. Based on rough set theory, the weights of each feature value are calculated using the frequency of multiple features. This method not only considers the influence of the correlation between multiple feature dimensions on the feature weights, but also, based on rough set theory, uses the intersection of equivalence classes under different features of the feature vector to count the co-occurrence frequency of feature values ​​on multiple features, without having to traverse the entire data space, thus improving time efficiency.

[0122] 2) Noise vector detection. Before clustering, the method of minimizing interval dispersion is used to filter out low-weight feature vectors, i.e., noise points, to improve the clustering effect;

[0123] 3) Hierarchical agglomerative clustering based on multi-objective clustering criteria. Utilizing the clustering objectives of intra-cluster compactness and inter-cluster separation, a cluster quality function is given. A bottom-up agglomerative clustering strategy is adopted in the clustering process, which is divided into two stages: initial clustering and merging clustering. In the initial clustering stage, the most similar feature vectors are used to generate sub-clusters using the multi-objective cluster quality function. In the merging clustering stage, the sub-clusters are iteratively merged to improve the overall cluster quality, forming the final clusters.

[0124] 4) Feature-related subspace identification. Based on the dependence of each feature within a cluster on the cluster, the relevant feature subspace of each cluster is determined.

[0125] The detailed solution process for each step is as follows.

[0126] ● Calculation of feature weight values

[0127] In rough set theory, R denotes an equivalence relation on the universe of discourse, [x] RLet represent an equivalence class R containing x, where all objects within the equivalence class are equivalent in relation R. The problem of calculating the co-occurrence frequency of statistical feature values ​​can be transformed into solving the problem of finding the intersection of feature equivalence classes. This invention employs an equivalence class calculation algorithm, utilizing the idea of ​​quicksort, to pre-sort the dataset based on the feature values. For any classification feature a... i Let x ki For the feature vector in the classification feature a i The feature values ​​on the categorical feature a i To measure the eigenvalue x from the perspective of ki If the single feature weights are given, then x ki Single feature weight It can be defined as

[0128]

[0129] in, For containing feature vector x k a i Equivalence classes The feature vector within the classification feature a i The value of x k same, Reflects the eigenvalue x ki In a i The number of times it appears.

[0130] From relevant feature a j To measure x from the angle ki The multi-feature weights, then the multi-feature weights It can be defined as

[0131]

[0132] in, This indicates that the feature vector x is included. k a j Equivalence classes The eigenvalue x represents the number of elements in the intersection of two equivalence classes. ki With x kj The number of times they appear together, The definition indicates that the eigenvalue x ki With x kj Co-occurrence frequency accounts for The larger the proportion, the better from the classification feature a j The angle reflected by x ki The greater the clustering effect.

[0133] Using W(x) ki ) represents the overall weight, and is normalized to make 0 <W(x ki If ) < 1, the calculation formula is defined as

[0134]

[0135] ● Noise Vector Detection

[0136] Noise vectors are special points that significantly differ from normal feature vectors. For feature vectors with relatively low weights for each feature, it can be assumed that no similar vectors can be found when projected onto any dimension. To identify noise vectors, the concept of aggregation degree O, as defined below, can be introduced. S (x k ).

[0137]

[0138] O S (x k This reflects the eigenvector x k Clustering ability across all feature dimensions. S (x k The smaller the value, the stronger x is. k It is more likely to be a noise vector.

[0139] Specific identification steps: According to the degree of aggregation O S (x k First, sort all feature vectors in ascending order; then utilize interval dispersion. Find the optimal split point x m The value of m satisfies That is, the partition point makes the vector interval [x1, x...] m ] and vector [x m+1 ,x n The sum of the dispersion of the vectors in the interval [x1, x2, ..., x3] is minimized among all the partition points. To minimize the interval dispersion, a clustering approach similar to k-means is used to divide the feature vectors into two classes, where the low-aggregation vector interval [x1, x2, ..., x3] is the smallest. m The feature vectors in [ ] are noise vectors with poor clustering ability.

[0140] The interval dispersion mentioned in the above steps is used It is indicated that variance is used to describe the vector interval [x]. i ,x j The degree of separation of feature vectors in [ ]:

[0141]

[0142] Represents the vector interval [x] i ,x j The average degree of aggregation on [i]. |ji| represents the absolute value of the difference between j and i. The smaller the value, the closer the feature vectors are within the interval.

[0143] ● Hierarchical agglomerative clustering based on multi-objective clustering criteria

[0144] Clustering criteria are the main basis for judging the partitioning of feature vectors during the clustering process, and are usually represented by a cluster quality function. Cluster quality refers to the quality of the clustering results. This value represents the reasonableness of the data partitioning. Cluster quality can be achieved using single-objective and multi-objective methods. Multi-objective cluster quality is more conducive to mining the internal structure of the dataset.

[0145] Cluster quality is evaluated by considering both intra-cluster compactness and inter-cluster separation. 1) Cluster quality depends on the compactness of intra-cluster data. Intra-cluster compactness is related to the eigenvalues ​​projected onto important features. The larger the weight of the eigenvalue and the higher its frequency, the better the cluster quality. 2) Cluster quality is also related to the degree of inter-cluster data separation. To separate different clusters as much as possible, the eigenvalues ​​projected onto important features should be as concentrated as possible, and the values ​​of different clusters on high-weight feature dimensions should be as different as possible. To calculate the overall cluster quality, the quality of each cluster needs to be calculated separately. Based on the eigenvalues, Q(C) is defined as follows: s To describe cluster C s The quality of the cluster can be measured by intra-cluster compactness and inter-cluster separation, respectively. ki ) is used to represent the characteristic value x ki The measured intra-cluster compactness, Sep(x) ki ) is the eigenvalue x ki The measure of inter-cluster separation.

[0146]

[0147] Intra-cluster compactness Com(x) ki It can be evaluated from two aspects: eigenvalues ​​x ki In cluster C s The distribution within can be determined by the eigenvalue x. ki In classification feature a i The probability P(a) on i =x ki ) and the value in cluster C s The probability P(a) within the range i =x ki |C s The product of () is used to measure this value, which mainly reflects the eigenvalue x. ki In C s The degree of concentration on, eigenvalue x ki The importance of a cluster is determined by the weight W(x) of this value. ki The inter-cluster separation is represented by Sep(x). ki Then it depends on the eigenvalue x. ki Exclusively for cluster C sThe degree can be expressed as x ki Appears in cluster C s Feature a i The probability P(a) on i =x ki ∧(x k ∈C s )) with the entire dataset x ki The probability of occurrence P(a) i =x ki The proportion represents the value of a; the larger the value, the greater the a. i eigenvalues ​​x on ki The more concentrated the occurrence of C s In the middle. Through derivation, Q(C s The expression can be represented by the following symbols: count(x) ki ,a i C s ) indicates that in cluster C s Inside, the projection is on a i The value on is x ki The number of feature vectors; n represents the total amount of data in the dataset; W(x ki ) is the eigenvalue x ki The weights; count(x) ki ,a i ) refers to feature a i Above, x ki Total number of occurrences.

[0148] Suppose a cluster set C = {C1, C2, ..., C} K The overall quality of a cluster is described by Q(C), and the quality of a cluster can be characterized by the following formula.

[0149]

[0150] Q(C) reflects the overall quality of the data distribution under the partitioning scheme of cluster C. The larger this value, the more reasonable the partitioning scheme. s Then it represents cluster C. S The quality, P(C) S ) represents C S The proportion of feature vectors in the entire dataset, P(C) S The main function of clustering is to coordinate the interactions between clusters to achieve the best overall clustering effect.

[0151] Based on multi-objective clustering criteria, hierarchical agglomerative clustering can be divided into two stages: initial clustering and merging clustering. In the initial clustering stage, a feature vector is randomly selected as the first sub-cluster. The cluster quality of other feature vectors to be clustered and existing sub-clusters is calculated sequentially using a formula. This stage is characterized by high purity and small size, and can represent multiple cluster centers of a large class. The formation of sub-clusters not only effectively avoids the homogenization of imbalanced data but also improves the time efficiency of the iterative merging stage. In the initial clustering stage, using a cluster quality function and following the principle of maximizing cluster quality, each feature vector is sequentially assigned to an existing sub-cluster or a newly generated sub-cluster. The task of merging clustering is to find all sub-clusters belonging to the same class and iteratively merge these sub-clusters to improve the clustering quality of the dataset until no further merging operations occur during the entire iteration process. This process adopts the hierarchical agglomerative idea, using the maximization of the cluster quality function as measured by a formula as the basis, and merges the two closest sub-clusters. This stage fully utilizes high-purity sub-clusters to iteratively merge the most similar sub-clusters, effectively accelerating the clustering process.

[0152] ● Feature-related subspace identification

[0153] The feature-related subspace is composed of a set of properties that best reflect the characteristics of the clusters. Calculating the dependence of each feature on the cluster is key to determining the property subspace; R(a) is used. i C s Characteristic a i The degree of cluster dependency can be described by the following formula:

[0154]

[0155] The method for identifying feature subspaces is similar to that for identifying noise points, based on the feature dependency R(a i C s With the goal of minimizing interval dispersion, the descending characteristic interval [a1, a2] is divided into... d The interval is divided into [a1, a...]. m ] and [a m+1 ,a d The high-dependency interval is the feature subspace.

[0156] Therefore, through the above solution, the technical solution of this application solves the following problems:

[0157] 1) Cross-domain and cross-scale problems in air-to-ground information fusion based on UAV swarms and unmanned vehicle swarms under different shooting perspectives, shooting angles and shooting details;

[0158] 2) The problem of inconsistent target feature representation between drone swarms and unmanned vehicle swarms;

[0159] 3) The problem of duplicate counting when the same target is detected by multiple drones and unmanned vehicles.

[0160] The embodiments of the present invention have been described above, but the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and these forms are all within the protection scope of the present invention.

Claims

1. A method for air-ground cooperative situation fusion based on a UAV and an unmanned vehicle, characterized in that, The method comprises the following steps: Step 1: uniformly describing N targets detected by the UAV cluster and the unmanned vehicle cluster in cooperation through a target feature description structure, generating N target feature vectors, N being a positive integer; Step 2: for a target feature vector, searching for similar feature vectors that have been detected and stored in a target feature vector database through feature fast retrieval, the target feature vector and the similar feature vectors together forming a target feature vector set; Step 3: clustering the target features based on multi-feature frequency weight and multi-target cluster set quality clustering criteria, clustering the target feature vector set, and realizing air-ground cooperative situation fusion; In the step 2, the i-th target feature vector is taken as the retrieval basis, i∈[1, N], the feature vector retrieval based on the sparse table index algorithm is used to search in the target feature vector database, M similar feature vectors that have been detected and stored in the target feature vector database are searched out, the i-th target feature vector and the M similar feature vectors together form a target feature vector set U, and M is a positive integer; In the step 3, the step 3 specifically comprises: Step 31: based on the rough set theory, the feature weight value is calculated for the M+1 target feature vectors in the target feature vector set U by using a multi-feature frequency criterion, wherein the multi-feature frequency criterion specifically comprises: the co-occurrence times of the feature values in the multi-features are counted by using the intersection between the equivalent classes in different features of the feature vector; Step 32: the noise target feature vectors are detected and filtered out by using the minimum interval dispersion method; Step 33: hierarchical agglomerative clustering is performed based on a multi-target clustering criterion, wherein the multi-target clustering criterion specifically comprises: a cluster set quality function is given by using the clustering target of the cluster compactness and the cluster separation; Step 34: the relevant feature subspace of each cluster set is determined; Step 35: when the target feature vector of the i-th target belongs to the relevant feature subspace of a certain cluster set, the target is determined to be a target that has been detected and stored, and the feature vector is stored again; when the target feature vector of the i-th target does not belong to the relevant feature subspace of any cluster set, the target is determined to be a new target to be detected, and the feature vector is stored; Step 36: the clustering of the i+1-th target is performed until the fusion of the targets detected by the UAV cluster and the unmanned vehicle cluster in the current situation is realized.

2. The method of claim 1, wherein, The step 31 specifically comprises: Suppose that the (M+1) feature vectors in the target feature vector set U are divided into several class cluster sets C. s And the noise set NOS, the feature vector x k It consists of D classification features, formally represented as x k ={x k1 ,x k2 ,x k3 ,···,x kD }, x kd For the feature vector x k The d-th classification feature a d Eigenvalues ​​on [1, M+1], d ∈ [1, D]; in the eigensubspace SA s Below, cluster C s From the feature vector set SD s Composition, C s Represented as a pair (SD) s SA s ), From the perspective of the classification feature a d measuring the single feature weight of the feature value x kd , x kd single feature weight is defined as wherein, is a feature vector x k of a d equivalence class, the feature vector within the equivalence class has the same value on the classification feature a d as x k , reflects the number of times the feature value x kd occurs on a d ; n represents the total amount of data of the data set; From the perspective of relevant features a e measuring the multi-feature weight of x kd , the multi-feature weight is defined as in, This indicates that the feature vector x is included. k a e Equivalence classes The eigenvalue x represents the number of elements in the intersection of two equivalence classes. kd With x ke The number of times they appear together, The definition indicates that the eigenvalue x kd With x ke Co-occurrence frequency accounts for The larger the proportion, the better from the relevant feature a e The angle reflected by x kd The greater the clustering effect, W(x kd ) represents the comprehensive weight, and is normalized so that 0 < W(x kd ) < 1, and the calculation formula is defined as Thus, the feature weight value of x is obtained kd is obtained.

3. The method of claim 1, wherein, The step 32 specifically comprises: Step 321 : Calculate the feature vector x k of the degree of polymerization O S (x k ): Step 322: Sort all feature vectors in ascending order according to the degree of polymerization O S (x k ) Sort all feature vectors in ascending order; Step 323: Utilize interval dispersion Find the optimal split point x h The value of h satisfies That is, the partition point makes the vector interval [x1, x...] h ] and vector interval [x h+1 ,x M+1 The sum of the discretenesses of ] is the smallest among all partition points, where: Interval dispersion The interval dispersion is described by the variance, which reflects the degree of separation of the eigenvectors in the vector interval [x f , x g ]. represents the average value of the aggregation degree on the vector interval [x f ,x g ]; |g-f| represents the absolute value of the difference between g and f; The smaller the value, the closer the characteristic vectors in the interval. Step 324: using the clustering idea of k-means, the feature vectors are divided into two categories with the goal of minimizing the interval dispersion, wherein the feature vectors in the low-aggregation-degree vector interval [x1, x h ] are the noise target feature vectors with poor clustering ability.

4. The method of claim 1, wherein, The step 33 specifically comprises: Step 331: in the initial clustering stage, a feature vector is randomly selected as a first sub-cluster set, and the cluster set quality function is used to calculate the cluster set quality of other clustering feature vectors and the first sub-cluster in sequence according to the maximum principle of the sub-cluster set quality, so that each feature vector is selected and distributed to an existing sub-cluster or a new sub-cluster generated in sequence; Step 332: in the merging clustering stage, the cluster set overall quality function is used to iteratively merge the sub-clusters based on the maximum principle of the cluster set quality function until no merging sub-cluster operation is generated in the whole iteration process.

5. The method of claim 4, wherein, In the step 331, the sub-cluster set quality function is: Let x kd be the feature vector k The feature value on the dth classification feature a d is denoted by x k The first sub-cluster set is denoted by C1, and the quality of the first sub-cluster set C1 is described by Q(C1). The quality value of Q(C1) is measured from the intra-cluster compactness and inter-cluster separation, respectively, Com(x kd ) is used to represent the intra-cluster compactness measured by the feature value x kd , and Sep(x kd ) is the inter-cluster separation measured by the feature value x kd . Compactness within cluster Com(x kd ) is evaluated from two aspects: the eigenvalue x kd The distribution within the first sub-cluster set Cl, by the eigenvalue x kd The probability P(a d = x d ) on the classification feature a kd and the probability P(a d = x kd | Cl) of the value within the first sub-cluster set Cl, which reflects the concentration of the eigenvalue x kd on the first sub-cluster set Cl; Feature value x kd The importance for the first sub-cluster set C1 is indicated by the weight W(x kd ) of the value. inter-cluster separation Sep(x kd ) depends on the eigenvalue x kd The degree of exclusivity of the first sub-cluster set C1 is represented by the ratio of the probability P(a kd = x d ∧(x d ∈ C1)) that a feature value x kd occurs on the classification feature a k in the first sub-cluster set C1 to the probability P(a kd = x d ) that x kd occurs on the entire data set. The larger this value, the more concentrated the feature values x d on the classification feature a kd are in the first sub-cluster set C1. count(x kd , a d , C1) represents the number of feature vectors in the first sub-cluster set C1 whose projection on the classification feature a d is x kd ; n represents the total number of data in the data set; W(x kd ) is the weight of the feature value x kd ; and count(x kd , a d ) is the total number of occurrences of x d on the classification feature a kd .

6. The method of claim 4, wherein, The step 332 includes: Let C = {C1, C2, …, C K} be a cluster set, and let Q(C) be the overall quality of the cluster set, then: Q(C s ) then represents the quality of the cluster C S , P(C S ) represents the proportion of the feature vectors in C S to the entire data set.

7. The method of claim 5, wherein, The step 34 specifically includes; Step 341 : Calculate the classification feature a d For the cluster set C S the degree of dependence R(a d , C s ) is calculated as follows: Step 342: Sort all categories in ascending order according to the dependency degree R(a d ,C s ). Step 343: using interval dispersion Finding the best split point a o , o is chosen such that That is, the split point is such that the sum of the dispersions of the vector intervals [ai, ai+1] and [ai+1, ai+2] is the smallest among all split points, where: o ai+1= a + o o+1 ai+2= a + o + 1 D and o is an integer. The interval dispersion is used to Variance is used to describe the degree of separation of the eigenvectors in the vector interval [a f ,a g ] represents the average value of the aggregation degree on the vector interval [a f ,a g ]; |g-f| represents the absolute value of the difference between g and f; The smaller the value, the closer the characteristic vectors in the interval. Step 344: using the clustering idea of k-means, the classification vector is divided into two categories, wherein, the high-attachment degree interval [[a o+1 ,a D ] is the relevant feature subspace of cluster C S .

8. An unmanned aerial vehicle and unmanned vehicle based air-ground cooperative situation fusion system, characterized in that, The system operates based on the method according to any one of claims 1 to 7, and the system comprises: The target feature uniform description component is used for uniformly describing the target information respectively detected by the unmanned vehicle cluster and the unmanned aerial vehicle cluster through a target feature description technology, and generating a plurality of target feature vectors; The feature fast retrieval component is used for retrieving, for a certain target feature vector, similar feature vectors which have been detected and stored and are similar to the target feature vector in a target feature vector database through feature fast retrieval, and the target feature vector and the similar feature vectors together form a target feature vector set; The target feature fusion clustering component is used for clustering the target feature vectors based on a multi-feature frequency weight and a multi-target cluster quality clustering criterion, and realizing air-ground cooperative situation fusion.

Images