Global unattended intelligent monitoring method based on multi-source data fusion

Abstract

The application relates to the technical field of data processing, in particular to a global unattended intelligent monitoring method based on multi-source data fusion, which comprises the following steps: acquiring a real-time feature vector of a to-be-identified target to obtain a global discrimination score; evaluating the category uncertainty of the to-be-identified target; adaptively generating a dynamic weight for model fusion according to the category uncertainty; obtaining a local discrimination score by using a pre-trained local detail model, and performing weighted fusion of the local discrimination score and the global discrimination score by using the dynamic weight to obtain a final discrimination score, and determining the category of the to-be-identified target based on the final discrimination score, so as to realize adaptive identification of unmanned aerial vehicles and birds. Through the technical scheme, a nonlinear decision boundary of unmanned aerial vehicles and birds can be effectively constructed, and the identification accuracy of fuzzy targets in a complex scene can be significantly improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method for full-domain unattended intelligent monitoring based on multi-source data fusion. Background Technology

[0002] In complex environments such as densely populated urban areas and critical infrastructure zones, monitoring low-altitude targets using multi-source sensors, such as radar, photoelectric sensors, and radio sensors, is a core requirement for ensuring safety. Among these, accurately distinguishing between drones and birds is a crucial link in the entire monitoring process, directly affecting the accuracy of early warnings and the effectiveness of subsequent response measures.

[0003] Currently, the mainstream method is to use the classic Linear Discriminant Analysis (LDA) algorithm for differentiation. The core of the LDA algorithm is to find an optimal linear projection direction to project high-dimensional feature data onto a one-dimensional space, so that the inter-class distance between the two categories of drones and birds is maximized and the intra-class distance is minimized in the one-dimensional space, thereby hoping to complete the classification through a threshold. This method is widely studied and applied because of its high computational efficiency and clear principle.

[0004] However, the standard LDA algorithm is a linear classification model, and its success hinges on the fact that data from different categories are linearly separable, or at least approximately linearly separable, in the feature space. But in real-world, densely populated urban environments, due to obstructions such as buildings, complex electromagnetic interference, and the diversity of target flight behaviors, the boundary between the characteristics of drones and birds is often blurred, presenting a complex and non-linear overlapping region. For example, a multi-rotor drone may glide briefly when it loses power, its trajectory temporarily resembling that of a bird; similarly, a large bird circling between buildings may have its radar signal and trajectory misidentified as a drone hovering or slow-moving. In such cases, the feature distribution boundary between the two target classes is no longer a simple straight line, but a complex non-linear curve. The LDA algorithm cannot accurately fit this non-linear boundary, leading to frequent misjudgments of targets in the feature ambiguity region, thus severely reducing the reliability of the entire monitoring system. Summary of the Invention

[0005] To address the technical problem that existing LDA algorithms cannot adapt to the nonlinear separability of drone and bird features in complex scenarios, this application provides a full-domain unattended intelligent monitoring method based on multi-source data fusion.

[0006] This application provides a method for full-domain unattended intelligent monitoring based on multi-source data fusion, comprising: acquiring real-time feature vectors of a target to be identified, and projecting the real-time feature vectors using a pre-trained global discriminant model to obtain a global discriminant score; evaluating the category uncertainty of the target to be identified based on the distribution of neighboring samples of the real-time feature vectors in historical sample data; adaptively generating dynamic weights for model fusion based on the category uncertainty; projecting the real-time feature vectors using a pre-trained local detail model to obtain a local discriminant score, and using the dynamic weights to perform weighted fusion of the local discriminant score and the global discriminant score to obtain a final discriminant score; and determining the category of the target to be identified based on the final discriminant score, thereby achieving adaptive identification of drones and birds.

[0007] This application constructs two linear discrimination models, a global model and a local model, and dynamically fuses their discrimination results based on the uncertainty of the target's own category. This creates a piecewise adaptive nonlinear decision boundary, which can effectively handle the nonlinear overlap problem in the feature space. For targets located in the blurred region between UAV and bird features, the recognition accuracy is significantly improved.

[0008] In one embodiment, obtaining the real-time feature vector of the target to be identified includes: simultaneously acquiring radar data and photoelectric image data of the target to be identified; extracting radar Doppler features from the radar data, extracting motion trajectory features from the target location information, and extracting image morphology features from the photoelectric image data, and fusing all extracted features into a real-time feature vector.

[0009] In one embodiment, assessing the category uncertainty of the target to be identified includes: searching for the K nearest historical samples to the real-time feature vector in the feature space of historical sample data; calculating the average distance from the real-time feature vector to these K historical samples; normalizing the average distance to obtain a feature space sparsity index; counting the number of samples belonging to the drone category and the bird category in these K historical samples respectively to calculate a nearest neighbor category entropy index; and combining the feature space sparsity index with the nearest neighbor category entropy index to obtain the category uncertainty.

[0010] By comprehensively assessing the suspiciousness of a target from two dimensions, the difficulty of the identification task can be accurately evaluated, providing a reliable basis for subsequent adaptive fusion and making the decision-making process more intelligent.

[0011] In one embodiment, the feature space sparsity index satisfies the following relationship:

[0012] ;in, It is a sparsity index for the feature space. For real-time feature vectors, The average distance from the real-time feature vector to the K nearest historical samples. and These are the maximum and minimum values ​​of the average distance of all historical samples collected during the training phase, respectively.

[0013] In one embodiment, the nearest neighbor category entropy index satisfies the following relationship: ;in, The nearest neighbor category entropy index The proportion of drone category samples in the K historical samples. The percentage of bird-type samples in the K historical samples.

[0014] In one embodiment, the dynamic weights satisfy the following relation: ;in, For dynamic weights, It is a sparsity index for the feature space. The nearest neighbor category entropy index and These are the preset hyperparameters.

[0015] By dynamically determining the discrimination strategy based on the relationship between the target and massive historical data, the model becomes less sensitive to minor changes in data distribution, has stronger generalization ability, and can better adapt to complex and ever-changing real-world monitoring environments such as densely populated urban areas.

[0016] In one embodiment, the global discrimination model and the local detail model are pre-trained as follows: the global discrimination model is trained using all historical sample data; fuzzy samples located at the boundary of the feature distribution of UAVs and birds are selected from all historical sample data using a density-based clustering algorithm; and the local detail model is trained using the fuzzy samples.

[0017] By using a global model to quickly process clear targets and a specially trained local model to finely correct fuzzy targets, a reasonable allocation of computing resources is achieved while ensuring high accuracy.

[0018] In one embodiment, the final discriminant satisfies the following relation: ;in, For the final judgment score, For dynamic weights, For global discrimination, For local discrimination.

[0019] The technical solution of this application has the following beneficial technical effects:

[0020] This application uses a global model to handle targets with significant feature differences, while a local detail model specifically learns and processes targets whose features are located in fuzzy, overlapping regions. During real-time recognition, the classification uncertainty of the current target is assessed, and weights are dynamically allocated based on this uncertainty. The judgment results of the two models are adaptively fused to generate the final classification result.

[0021] Furthermore, a nonlinear decision boundary was constructed, which can significantly improve the recognition accuracy in complex scenarios. The model dynamically decides which discrimination strategy to adopt based on the relationship between the target and massive historical data. This adaptability makes the model insensitive to small changes in data distribution, has stronger generalization ability, and can better adapt to complex and ever-changing real-world monitoring environments such as dense urban areas. Attached Figure Description

[0022] Figure 1 This is a flowchart of a fully unattended intelligent monitoring method based on multi-source data fusion according to an embodiment of this application.

[0023] Figure 2 This is a schematic diagram comparing the embodiments of this application with the prior art. Detailed Implementation

[0024] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0025] Figure 1 This is a flowchart illustrating a fully unattended intelligent monitoring method based on multi-source data fusion, according to an embodiment of this application. Figure 1 As shown, the full-domain unattended intelligent monitoring method based on multi-source data fusion includes steps S101 to S104, which are described in detail below.

[0026] S101: Obtain the real-time feature vector of the target to be identified, and project the real-time feature vector using a pre-trained global discrimination model to obtain the global discrimination score.

[0027] In one embodiment, when the monitoring system locks onto a low-altitude target to be identified, it simultaneously invokes multiple sensors, including radar, electro-optical, and radio spectrum sensors, to collect data such as radar data and electro-optical image data. Subsequently, the system extracts a series of predefined numerical features from this raw data that reflect the essential differences between drones and birds.

[0028] Specifically, radar Doppler features are extracted from radar data: short-time Fourier transform is performed on the radar echo to generate a micro-motion spectrum, from which features such as spectral bandwidth, main peak frequency, number and energy ratio of harmonics, and spectral line periodicity are extracted to distinguish the different modes of UAV propeller rotation and bird wing flapping.

[0029] Extracting motion trajectory features from target location information: By integrating target location information provided by multi-source data, the mean and variance of velocity, the mean and variance of acceleration, the rate of change of heading angle, the rate of vertical climb, and the trajectory curvature are calculated to assess the dynamic differences between the mechanical motion of UAVs and the organic flight of birds.

[0030] Extracting image morphological features from photoelectric image data: When lighting conditions permit, extract the target's contour compactness, morphological factor, optical flow divergence, texture complexity, etc. from the photoelectric video stream to help determine the target's shape and motion pattern.

[0031] Furthermore, all extracted features can be fused into a real-time feature vector. To form a unified feature vector, one or more representative values ​​can be selected from each of the three feature classes. In this embodiment, one representative value is extracted from each feature class, and they are fused into a single 3-dimensional real-time feature vector, denoted as […]. .

[0032] In this optional embodiment, a pre-trained offline global discriminant model can be used to perform a linear projection on the real-time feature vector to obtain a global discriminant score. It is worth noting that this global discriminant score is a signed numerical value, and its sign and magnitude have clear physical meaning. In this embodiment, a positive value indicates that the target is more likely to be classified as a drone, while a negative value indicates that it is more likely to be classified as a bird. The larger the absolute value of the score, the further the target's features are from the fuzzy boundary between the two categories, and the higher the model's confidence in this classification.

[0033] For example, suppose that at a certain moment the system extracts the real-time feature vector of a certain target as follows: The real-time feature vector is then input into the offline-trained global discriminant model for a linear projection. ,in For global discrimination, Let be the projection vector of the global discriminant model obtained through offline training. ,but

[0034] Therefore, the global discrimination model classifies the target as a drone.

[0035] Thus, by extracting and fusing the various physical features of the target and using a global model for rapid discrimination, a preliminary and efficient judgment result can be obtained on the target category.

[0036] S102 assesses the class uncertainty of the target to be identified based on the distribution of neighboring samples in the historical sample data of the real-time feature vector.

[0037] In one embodiment, the K-Nearest Neighbors (KNN) search algorithm is used to quickly find the feature vector of the target being tested from a massive offline training sample feature library. Find the K nearest historical samples, and then calculate... The average Euclidean distance to these K samples This distance reflects The local sample density in the feature space. Finally, the maximum value of the average distance of all historical samples pre-statistically collected during the training phase. and minimum value Normalizing this average Euclidean distance yields a dimensionless sparsity index for the feature space, which satisfies the following relationship:

[0038]

[0039] in, It is a sparsity index for the feature space. The closer the value is to 1, the rarer the target feature is and the lower the classification confidence; the closer the value is to 0, the more closely the target feature matches a certain historically dense region and the higher the classification confidence is.

[0040] For example, let's set K to 5, and use KNN search to find the nearest neighbors. The most recent 5 historical samples were used to calculate The average distance to these 5 samples is 0.25. Assuming that during the offline training phase, the maximum average neighbor distance of all historical samples is 0.5 and the minimum is 0.05, the sparsity index of the feature space can be calculated to be 0.44.

[0041] Furthermore, the number of samples belonging to the drone category and the number of samples belonging to the bird category are counted separately in these K historical samples to calculate the nearest neighbor category entropy index. The nearest neighbor category entropy index satisfies the following relationship:

[0042]

[0043] in, The nearest neighbor category entropy index The proportion of drone category samples in the K historical samples. The percentage of bird-type samples in the K historical samples. The closer the value is to 1, the more mixed the identities of the target's neighbors are, and the target is in the core ambiguity zone of the decision boundary; the closer the value is to 0, the more consistent the identities of its neighbors are, and the clear category is.

[0044] For example, among the 5 nearest neighbor samples found, 3 were labeled as drones and 2 were labeled as birds. It is 0.6. The nearest neighbor category entropy index is 0.4, and the calculated value is 0.971.

[0045] Finally, the sparsity index of the feature space is combined with the entropy index of the nearest neighbor categories to obtain the category uncertainty. In this embodiment, the combination method is direct summation, which is used to reflect the difficulty of identification of the target due to the rarity of its feature location and the mixed nature of its neighborhood categories.

[0046] Thus, by comprehensively analyzing the spatial density and neighborhood category composition of target features in historical data, and combining them through summation, a single and comprehensive category uncertainty index can be obtained.

[0047] S103, based on category uncertainty, adaptively generates dynamic weights for model fusion.

[0048] In this optional embodiment, the feature space sparsity index is added to the nearest neighbor class entropy index to obtain the class uncertainty of the target to be identified. The higher the sum, the more suspicious and difficult to distinguish the target. Then, the sum is input into a preset hyperbolic tangent activation function to generate dynamic weights for model fusion. The dynamic weights satisfy the following relationship:

[0049]

[0050] in, For dynamic weights, It is a sparsity index for the feature space. The nearest neighbor category entropy index and These are preset hyperparameters, for example. and The values ​​are 2 and 1 respectively.

[0051] Specifically, when category uncertainty is low, the value of the dynamic weight is close to 0; when the overall uncertainty measure increases and exceeds the parameter... At the inflection point of control, the value of the dynamic weight will increase smoothly and rapidly and approach 1; parameters These parameters control the severity of the transition; both are hyperparameters determined during the offline phase based on validation set performance tuning.

[0052] For example, the feature space sparsity index is 0.44, and the neighbor class entropy index is 0.971. and With values ​​of 2 and 1 respectively, the dynamic weight can be calculated to be 0.974, which is very close to 1. This indicates that the system judges the current target to be highly suspicious based on the comprehensive uncertainty measure, and should mainly rely on the local detail model for fine discrimination.

[0053] Thus, through a nonlinear mapping function, the uncertainty measure can be transformed into a smooth weight between 0 and 1, providing a direct control signal for subsequent dynamic fusion decision-making.

[0054] S104 uses a pre-trained local detail model to project the real-time feature vector to obtain a local discrimination score. Then, it uses dynamic weights to weight and fuse the local discrimination score with the global discrimination score to obtain the final discrimination score. Based on the final discrimination score, it determines the category of the target to be identified, thereby achieving adaptive recognition of drones and birds.

[0055] In one embodiment, the feature vector of the target to be tested The input is fed into a pre-trained offline local detail model, and a second linear projection is performed to obtain a local discriminant score. This score is specifically optimized for the feature differences in blurred regions and can be regarded as a correction to the global discriminant score.

[0056] For example, Let the projection vector of the local detail model obtained through offline training be... The corresponding local discriminant is divided into Therefore, the target is identified as a bird in the local judgment, which is contrary to the initial judgment of the global model, thus forming a correction.

[0057] Furthermore, dynamic weights can be used to weight and fuse the local and global discriminant scores to obtain the final discriminant score, which satisfies the following relation:

[0058]

[0059] in, For the final judgment score, For dynamic weights, For global discrimination, This is for local discrimination. The calculation process is adaptive; for clear targets, The final discrimination score is close to 0, therefore the final discrimination score is close to the global discrimination score. For fuzzy targets, Since the value is close to 1, the final discriminant score is close to the local discriminant score.

[0060] For example, the global discrimination score is 0.64, the local discrimination score is -0.5, and the dynamic weight is 0.974. Therefore, the final discrimination score can be calculated as -0.47, meaning that the system ultimately determines that the target is a bird.

[0061] In this way, by dynamically weighting and fusing the global preliminary judgment results with the local detailed judgment results, different decision-making strategies can be adopted for targets of different difficulties, ultimately achieving high-precision adaptive recognition of drones and birds.

[0062] like Figure 2 The diagram shows a comparison between the embodiments of this application and the prior art. The left diagram shows the classification result after applying the existing standard LDA, while the right diagram shows the classification result of the embodiment of this invention. It can be seen that the decision boundary of the prior art is a straight line, and no matter how it is adjusted, it cannot correctly classify two categories at the same time, and a large number of misclassifications will occur in the overlapping area. In contrast, the decision boundary in the embodiment is a smooth curve, which can effectively adaptively segment the two categories and greatly reduce misclassification.

[0063] It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this patent application shall be determined by the appended claims.

Claims

1. A global unattended intelligent monitoring method based on multi-source data fusion, characterized in that, The method comprises the following steps: obtaining a real-time feature vector of a target to be identified, and projecting the real-time feature vector using a pre-trained global discrimination model to obtain a global discrimination score; based on the distribution of neighboring samples of the real-time feature vector in historical sample data, evaluating the class uncertainty of the target to be identified; the evaluation of the class uncertainty of the target to be identified comprises: searching for K historical samples closest to the real-time feature vector in the feature space of the historical sample data, calculating the average distance from the real-time feature vector to the K historical samples, and normalizing the average distance to obtain a feature space sparsity index; respectively counting the number of historical samples belonging to the unmanned aerial vehicle class and the bird class among the K historical samples to calculate a nearest neighbor class entropy index; combining the feature space sparsity index and the nearest neighbor class entropy index to obtain a class uncertainty; the feature space sparsity index satisfies the relationship: wherein, is a feature space sparsity index, is a real-time feature vector, is an average distance of the real-time feature vector to K nearest historical samples, and are respectively a maximum and a minimum of the average distance of the whole historical samples in the training phase statistics; the near neighbor class entropy index satisfies a relationship: wherein, is a near neighbor class entropy indicator, is a proportion of drone class samples in the K historical samples, is a proportion of bird class samples in the K historical samples; adaptively generating a dynamic weight for model fusion according to the class uncertainty; the dynamic weight satisfies the relationship: wherein, is a dynamic weight, is a feature space sparsity index, is a near neighbor class entropy index, and is a pre-set hyper-parameter; projecting the real-time feature vector using a pre-trained local detail model to obtain a local discrimination score, and using the dynamic weight to perform weighted fusion of the local discrimination score and the global discrimination score to obtain a final discrimination score, and determining the class of the target to be identified based on the final discrimination score, thereby realizing adaptive identification of unmanned aerial vehicles and birds.

2. The global unattended intelligent monitoring method based on multi-source data fusion according to claim 1, characterized in that, The method comprises the following steps: synchronously collecting radar data and photoelectric image data of the target to be identified; extracting radar Doppler features from the radar data, extracting motion trajectory features from target position information, and extracting image shape features from the photoelectric image data, and fusing all the extracted features into a real-time feature vector. 3.The global unattended intelligent monitoring method based on multi-source data fusion of claim 1, wherein, The global discrimination model and the local detail model are pre-trained in the following way: using all historical sample data to train the global discrimination model; using a density-based clustering algorithm to filter out fuzzy samples located at the boundary of the feature distribution of the two classes of samples, i.e. unmanned aerial vehicles and birds, from all historical sample data; using the fuzzy samples to train the local detail model.

4. The global unattended intelligent monitoring method based on multi-source data fusion according to claim 1, characterized in that, The final discrimination score satisfies the relationship: wherein, is the final discriminant score, is the dynamic weight, is the global discriminant score, is the local discriminant score.

Images