A method and system for identifying and warning of unknown clusters of unmanned aerial vehicles in complex scenarios

Abstract

This invention discloses a method and system for identifying and warning of unknown drones in complex scenarios. The method includes: acquiring raw radio frequency signals in a complex electromagnetic environment and extracting multimodal features from the raw radio frequency signals; dynamically fusing the multimodal features to generate high-dimensional robust fused features; processing the high-dimensional robust fused features through a shared feature extraction network to generate general features; generating preset drone model labels for the general features through decoupled representation learning; identifying drone models based on the preset drone model labels; dynamically fusing the identified drone model-specific features, comparing the residuals of the fused drone model-specific features with the overall deep embedding of the raw radio frequency signals, and determining whether an unknown drone model exists based on the comparison results and triggering a warning mechanism.

Description

Technical Field

[0001] This invention relates to the field of multimodal intelligent question answering technology, and more specifically, to a method and system for identifying and providing early warning of unknowns in complex scenarios using swarm drones. Background Technology

[0002] With the advent of the low-altitude economy era, the application of civilian drones is becoming increasingly widespread. However, this has also led to a surge in drone-related illegal activities, such as illegally entering no-fly zones near airports, nuclear facilities, and government offices, and infringing on personal privacy. How to effectively identify and regulate drones has become a critical issue that urgently needs to be addressed. The "low-altitude, slow-speed, and small" characteristics of drones make them difficult to identify in a timely and accurate manner. These characteristics result in existing identification systems generally exhibiting the following core limitations when facing drone targets: The dynamic response speed is slow. Some drone detection and identification methods rely on offline deep analysis of communication signals, requiring the extraction of multi-dimensional electronic fingerprints such as channel features, modulation and coding schemes, and sequence structures to build a feature database. During online detection, the system must perform the same complex feature extraction on real-time signals and perform global matching in the feature database. The process has a high computational load and low matching efficiency, resulting in significant delays when facing sudden or high-density targets, making it difficult to meet the real-time response requirements of security scenarios.

[0003] Significant misjudgments occur in swarm scenarios: In complex real-world scenarios such as between urban buildings, near-ground flight, and complex electromagnetic environments, multiple drones may appear simultaneously and form dense swarms. The superposition of signals in the same frequency band and the confusion of modulation features lead to a sharp decline in the performance of existing recognition models.

[0004] However, existing technologies are essentially closed-set classifiers, lacking the ability to perceive unknown categories. With the continuous emergence of new and heterogeneous drones, the training databases of existing identification systems cannot fully cover them, resulting in a large number of new drone targets that cannot be effectively identified by existing systems, creating significant blind spots. This problem often leaves systems in a passive and unaware state when facing new drone threats, greatly weakening the effectiveness of air defense systems.

[0005] Addressing the aforementioned issues and improving existing drone signal processing and identification systems is a comprehensive task involving national security, social stability, and the safety of people's lives and property. It has profound strategic significance for building a comprehensive and multi-layered security protection system, safeguarding national security, and combating illegal and criminal activities. Summary of the Invention

[0006] The present invention provides a method and system for identifying and providing early warning of unknowns in swarm drones in complex scenarios, in order to solve the problem of how to identify and provide early warning of unknowns in swarm drones in complex scenarios.

[0007] To address the aforementioned problems, this invention provides a method for identifying and providing early warning of unknowns in complex scenarios using swarm drones, the method comprising: Raw radio frequency signals are acquired in a complex electromagnetic environment, and multimodal features are extracted from the raw radio frequency signals; the multimodal features are dynamically fused to generate high-dimensional robust fused features; The high-dimensional robust fused features are processed by a shared feature extraction network to generate general features; a preset aircraft model label is generated for the general features through decoupled representation learning; and the aircraft model of the UAV is identified based on the preset aircraft model label. The identified drone-specific features are dynamically fused, and the fused drone-specific features are compared with the overall deep embedding of the original radio frequency signal. Based on the comparison results, it is determined whether there is an unknown drone model and an early warning mechanism is triggered.

[0008] Preferably, the step of acquiring raw radio frequency signals in a complex electromagnetic environment, extracting multimodal features from the raw radio frequency signals, and dynamically fusing the multimodal features to generate high-dimensional robust fused features includes: The original radio frequency signal is converted to a two-dimensional time-frequency domain through short-time Fourier transform to generate a time-frequency spectrum; a multi-scale convolutional neural network is used to extract multi-scale deep features from the time-frequency spectrum to obtain time-frequency domain features; The original radio frequency signal is converted to the frequency domain through discrete Fourier transform, and the spectrum of the original radio frequency signal is analyzed and modeled through a frequency domain feature learner to extract frequency domain features. The original radio frequency signal is input into a time-series convolutional network to obtain a time-domain waveform. Based on the time-domain waveform, time-domain features characterizing the long-range time dependence and transient dynamic characteristics of the signal are extracted. The time-frequency domain features, the frequency domain features, and the time domain features are input into the cross-attention fusion module. The cross-attention fusion module calculates the cross-domain feature correlation, dynamically assigns weights to the time-frequency domain features, the frequency domain features, and the time domain features, and performs adaptive weighted fusion to generate high-dimensional robust fusion features.

[0009] Preferably, the cross-attention fusion module assigns differentiated weights to the time-domain features, frequency-domain features, and time-frequency-domain features by calculating the correlation coefficients of the time-domain features, frequency-domain features, and time-frequency-domain features.

[0010] Preferably, the step of processing the high-dimensional robust fused features through a shared feature extraction network to generate general features; generating preset aircraft model labels for the general features through decoupled representation learning; and identifying the aircraft model of the UAV based on the preset aircraft model labels includes: Based on the decoupled representation learning mechanism, a corresponding feature representation space is constructed for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores the discriminative features corresponding to the known UAV model label, generating a high-dimensional feature vector corresponding to the preset model label; By constructing a decoupled neural network model, the original joint multi-label classification task is decoupled at the feature level and decomposed into multiple independent and parallel single-label classification subtasks. Based on an independent feature encoder, each single-label subtask is transformed into a parameter-independent lightweight sub-classifier. By minimizing the classification loss of the input channel of the lightweight sub-classifier, each channel can independently identify a single UAV model and determine the label of the UAV model. Using known drone model labels as nodes, edges between nodes are defined based on label co-occurrence relationships or prior knowledge to construct a label relationship graph. The decoupled features corresponding to each label are used as the initial feature vectors of the graph nodes in the label relationship graph. The message passing mechanism of the graph neural network is used to aggregate the neighbor node information of each node and update the enhanced features that incorporate label relevance to improve the robustness of multi-drone model recognition in complex scenarios.

[0011] Preferably, the decoupled representation learning mechanism constructs a corresponding feature representation space for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores discriminative features corresponding to the known UAV model label, including: The high-dimensional robust fusion features are processed by a shared feature extraction network to obtain general features; an independent feature processing slot is constructed for each model label, and each feature processing slot extracts only the discriminative features corresponding to the known UAV model labels; the general features are input into the feature processing slot, and a high-dimensional feature vector corresponding to the preset model labels is generated based on the decoupled representation learning mechanism.

[0012] Preferably, the tag co-occurrence relationship is the probability distribution relationship of different drone models appearing simultaneously in the same monitoring scene.

[0013] Preferably, the step of dynamically fusing the identified drone features, comparing the fused drone features with the overall deep embedding of the original radio frequency signal, and determining whether there is an unknown drone type based on the comparison result and triggering an early warning mechanism includes:

[0014] The model-specific features of all known models are obtained, and a dynamic fusion embedding representation is generated through a fusion mechanism of weighted summation and projection activation. Calculate the similarity between the dynamically fused embedded representation and the overall deep embedding from the original radio frequency, and generate a scalarized residual metric. The residual metric is calculated based on all samples in the training set, and the 95th percentile of the residual metric is used as the preset threshold for detecting unknown models. The residual metric of the sample to be tested is compared with the preset threshold. If it is greater than the threshold, an early warning mechanism is triggered.

[0015] Preferably, the weights of the weighted summation are adaptively assigned weight values ​​based on the discriminativeness and feature contribution of each known model-specific feature.

[0016] Preferably, it further includes: The overall deep embedding is obtained by performing high-dimensional feature extraction on the original radio frequency signal; The complex electromagnetic environment includes: the coexistence of multiple UAV models, or electromagnetic interference intensity exceeding the interference threshold.

[0017] Based on another aspect of the present invention, the present invention provides a system for identifying and warning of unknowns in complex scenarios using swarm drones, the system comprising: An initial unit is used to acquire raw radio frequency signals in a complex electromagnetic environment, extract multimodal features from the raw radio frequency signals, and dynamically fuse the multimodal features to generate high-dimensional robust fused features. The identification unit is used to process the high-dimensional robust fused features through a shared feature extraction network to generate general features; generate a preset model label for the general features through decoupled representation learning; and identify the model of the UAV based on the preset model label. The early warning unit is used to dynamically fuse the identified model-specific features, compare the fused model-specific features with the overall deep embedding of the original radio frequency signal, and determine whether there is an unknown drone model based on the comparison result and trigger the early warning mechanism.

[0018] This invention provides a method and system for identifying and warning of unknown drone swarms in complex scenarios. The method includes: acquiring raw radio frequency (RF) signals from a complex electromagnetic environment and extracting multimodal features from the raw RF signals; dynamically fusing the multimodal features to generate high-dimensional robust fused features; processing the high-dimensional robust fused features through a shared feature extraction network to generate general features; generating preset drone model labels for the general features through decoupled representation learning; identifying drone models based on the preset drone model labels; dynamically fusing the identified drone model-specific features; comparing the residuals of the fused drone model-specific features with the overall deep embedding of the raw RF signals; and determining the presence of unknown drone models based on the comparison results and triggering a warning mechanism. This invention provides a method that combines deep neural networks and unsupervised learning to perform real-time identification of multiple known targets in drone swarms in complex real-world scenarios and to achieve unsupervised automatic detection and warning of unknown or novel drone targets. Attached Figure Description

[0019] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures: Figure 1 This is a flowchart of a method for identifying and providing early warning of unknowns in complex scenarios according to a preferred embodiment of the present invention. Figure 2 This is a flowchart of a method for identifying and providing early warning of unknowns in complex scenarios according to a preferred embodiment of the present invention. Figure 3 This is a flowchart of a dynamic multimodal feature extraction and fusion method according to a preferred embodiment of the present invention; Figure 4 This is a flowchart of a decoupled multi-known UAV identification method according to a preferred embodiment of the present invention; Figure 5 This is a diagram of an unsupervised early warning module for unknown aircraft models according to a preferred embodiment of the present invention; Figure 6 This is a structural diagram of a clustered UAV identification and early warning system in a complex scenario according to a preferred embodiment of the present invention. Detailed Implementation

[0020] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0021] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0022] Figure 1 This is a flowchart of a method for identifying and providing early warning of unknowns in complex scenarios according to a preferred embodiment of the present invention.

[0023] like Figure 1 As shown, this invention provides a method for identifying and providing early warning of unknowns in complex scenarios using swarm drones. The method includes: Step 101: Acquire raw radio frequency signals in a complex electromagnetic environment and extract multimodal features from the raw radio frequency signals; dynamically fuse the multimodal features to generate high-dimensional robust fused features; Preferably, raw radio frequency (RF) signals are acquired in a complex electromagnetic environment, and multimodal features are extracted from the raw RF signals; the multimodal features are dynamically fused to generate high-dimensional robust fused features, including: The original radio frequency signal is transformed into a two-dimensional time-frequency domain through short-time Fourier transform to generate a time-frequency spectrum; a multi-scale convolutional neural network is used to extract multi-scale deep features from the time-frequency spectrum to obtain time-frequency domain features; The original radio frequency signal is transformed to the frequency domain through discrete Fourier transform, and the spectrum of the original radio frequency signal is analyzed and modeled through a frequency domain feature learner to extract frequency domain features. The original radio frequency signal is input into a time-series convolutional network to obtain a time-domain waveform. Based on the time-domain waveform, time-domain features characterizing the long-range time dependence and transient dynamic characteristics of the signal are extracted. Time-frequency domain features, frequency domain features, and time-domain features are input into the cross-attention fusion module. The cross-attention fusion module calculates the correlation between cross-domain features, dynamically assigns weights to the time-frequency domain features, frequency domain features, and time-domain features, and performs adaptive weighted fusion to generate high-dimensional robust fused features.

[0024] Preferably, the cross-attention fusion module assigns differentiated weights to the time-domain features, frequency-domain features, and time-frequency-domain features by calculating the correlation coefficients of the time-domain features, frequency-domain features, and time-frequency-domain features.

[0025] This invention performs dynamic multimodal feature extraction and fusion: it collects raw radio frequency signals in a complex electromagnetic environment, extracts multimodal features of the signals in the time and frequency domains respectively, and achieves adaptive dynamic fusion of multimodal information through a feature fusion module based on an attention mechanism.

[0026] The specific steps of dynamic multimodal feature extraction and fusion in step 101 of this invention are as follows: 101-1 Time-Frequency Domain Feature Extraction: The original radio frequency signal is converted to a two-dimensional time-frequency domain through short-time Fourier transform to generate a time-frequency spectrum; a multi-scale convolutional neural network is used to perform multi-scale deep feature extraction on the time-frequency spectrum to obtain time-frequency domain features; The specific process of extracting time-frequency domain features from 101-1 is as follows: Unmanned aerial vehicle (UAV) communication signals exhibit significant time-varying, non-stationary, and complex modulation characteristics, making it difficult for traditional analysis methods based on a single time or frequency dimension to fully characterize their dynamic evolution. To fully explore the essential properties of these signals, we first introduce the Short-Time Fourier Transform (SFT) for joint time-frequency analysis, thus mapping the original one-dimensional sampling sequence into a two-dimensional time-frequency graph. The core idea of ​​the SFT is the windowed Fourier Transform, which continuously slides a finite-length window function along the time axis, performing a Fourier transform on each local signal segment to obtain the signal's energy distribution in the time-frequency domain. in The original signal, It is a sliding window function (such as Hamming window or Gaussian window), where t is time and f is frequency.

[0027] Building upon this foundation, a multi-scale convolutional neural network was constructed. By setting convolutional kernels of different sizes, large-scale trends and small-scale details were captured in parallel within the same network layer, extracting deep abstract features from the time-frequency plot layer by layer. This process significantly improved the completeness and robustness of the UAV communication signal representation, laying a solid foundation for subsequent tasks.

[0028] 101-2 Frequency Domain Feature Extraction: The original radio frequency signal is converted to the frequency domain through discrete Fourier transform, and the signal spectrum is analyzed and modeled through a frequency domain feature learner to extract global frequency domain features; The specific process of frequency domain feature extraction for 101-2 is as follows: The core logic of frequency domain features is to first map the time series to the complex frequency domain through discrete Fourier transform, and then use a multilayer perceptron designed specifically for complex numbers to mine global dependencies, thereby overcoming the information bottleneck and lack of long-range correlation faced by traditional multilayer perceptrons that only operate in the time domain.

[0029] Discrete Fourier Transform transforms finite-length discrete signals Transformed into complex-valued spectrum : Where N is the total number of sampling points of the signal, and j is the imaginary unit.

[0030] The transformed frequency components are processed separately for their real and imaginary parts by an improved frequency-domain multilayer perceptron, utilizing the weights and biases of complex values ​​to learn the time and channel dependencies in the frequency domain. Since the input data is presented in complex form, the model processes the real and imaginary parts of the input data separately.

[0031] in and These are the real and imaginary parts of the output from the previous layer, respectively. and These are the real and imaginary weight matrices of the current layer, respectively. and These are the real and imaginary part biases for the current layer, respectively. After each layer's computation, the outputs of the real and imaginary parts are stacked back into a complex number form, thus preserving the complexity of the frequency domain data. Finally, the model learns the frequency domain patterns and features of the signal through the computation of the multilayer perceptron.

[0032] 101-3 Time-domain feature extraction: The original radio frequency signal is input into a time-series convolutional network, its time-domain waveform is analyzed, and time-domain features characterizing the long-range time dependence and transient dynamic characteristics of the signal are extracted; The specific process of time-domain feature extraction for 101-3 is as follows: The low-sampling-rate sequence obtained after downsampling is then input into a temporal convolutional network for local feature extraction. Because the amount of data has been significantly reduced, the temporal convolutional network can achieve deep receptive field coverage with limited computational resources, thereby efficiently capturing key temporal features and ensuring overall recognition or prediction accuracy.

[0033] 101-4 Cross-domain dynamic fusion: The time-frequency domain features, the frequency domain features, and the time domain features are input into the cross-attention fusion module. The cross-attention fusion module calculates the correlation between cross-domain features, dynamically assigns weights to the time-frequency domain features, the frequency domain features, and the time domain features, and performs adaptive weighted fusion to generate high-dimensional robust fused features.

[0034] The specific process of cross-domain dynamic fusion in 101-4 is as follows: A cross-attention mechanism is applied between three feature maps: those from the time domain, those from the frequency domain, and those from the time-frequency domain. This mechanism operates between each pair of image modalities, extracting the correlation between them.

[0035] in , , Let represent the linear projection transformations of the query, key, and value of the ith attention head in the k-th module, respectively. , and These represent the time-domain features, frequency-domain features, and joint time-frequency features extracted from the input signal, respectively. This represents the dimension of the key vector and is used as a scaling factor to adjust the numerical range of the dot product to stabilize the gradient.

[0036] Following the cross-attention operation, features from multiple domains are concatenated. The concatenated feature maps include: in , , These represent the original features in the time domain, frequency domain, and time-frequency domain, respectively; while This corresponds to the deep association representation between different domains captured by the aforementioned cross-attention mechanism.

[0037] The concatenated feature maps undergo a 1x1 convolution operation to compress their dimensions and adjust their shape to fit the input requirements of the subsequent transformer encoder. The final fused feature map is represented as follows: Obtaining stacked representations of multidimensional information Then, the model uses convolution operations. The stacked features are subjected to nonlinear mapping and dimensionality compression to eliminate redundant information and achieve deep semantic fusion.

[0038] The above steps fuse features from the time domain, frequency domain, and time-frequency domain. Cross-attention mechanisms help the model effectively capture correlations between different modalities, and by concatenating and compressing feature maps, we fuse this information into a unified feature space. For example... Figure 3 As shown.

[0039] This invention addresses the problems of insufficient single-modal feature representation and inadequate information utilization in complex electromagnetic environments. It simultaneously acquires raw radio frequency signals and extracts their multimodal features in both the time and frequency domains, then uses an attention-based fusion module for adaptive weighted fusion. This method achieves complementarity and enhancement of multimodal information, effectively improving the robustness and information richness of the features relied upon by subsequent recognition models.

[0040] Step 102: Process the high-dimensional robust fusion features through a shared feature extraction network to generate general features; generate preset aircraft type labels for the general features through decoupled representation learning; identify the aircraft type of the UAV based on the preset aircraft type labels; Preferably, the high-dimensional robust fused features are processed by a shared feature extraction network to generate general features; a preset aircraft model label is generated for the general features through decoupled representation learning; and the aircraft model of the UAV is identified based on the preset aircraft model label, including: Based on the decoupled representation learning mechanism, a corresponding feature representation space is constructed for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores the discriminative features corresponding to the known UAV model label, generating a high-dimensional feature vector corresponding to the preset model label; By constructing a decoupled neural network model, the original joint multi-label classification task is decoupled at the feature level and decomposed into multiple independent and parallel single-label classification subtasks. Based on an independent feature encoder, each single-label subtask is transformed into a parameter-independent lightweight sub-classifier. By minimizing the classification loss of the input channels of the lightweight sub-classifier, the independent identification of a single UAV model by each channel is achieved, and the label of the UAV model is determined. Using known drone model labels as nodes, edges between nodes are defined based on label co-occurrence relationships or prior knowledge to construct a label relationship graph. The decoupled features corresponding to each label are used as the initial feature vectors of the graph nodes in the label relationship graph. The message passing mechanism of the graph neural network is used to aggregate the neighbor node information of each node and update the enhanced features that incorporate label relevance to improve the robustness of multi-drone model recognition in complex scenarios.

[0041] Preferably, based on a decoupled representation learning mechanism, a corresponding feature representation space is constructed for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores discriminative features corresponding to the known UAV model label, including: High-dimensional robust fusion features are processed by a shared feature extraction network to obtain general features; an independent feature processing slot is constructed for each aircraft type label, and each feature processing slot extracts only the discriminative features corresponding to the known UAV aircraft type labels; the general features are input into the feature processing slots, and high-dimensional feature vectors corresponding to the preset aircraft type labels are generated based on the decoupled representation learning mechanism.

[0042] Preferably, the tag co-occurrence relationship is the probability distribution relationship of different drone models appearing simultaneously in the same monitoring scenario.

[0043] This invention performs decoupled multi-known UAV identification: it adopts a decoupled representation learning mechanism to assign independent feature channels to each potential UAV target in the signal and outputs the model of each identified known UAV target; The specific steps of decoupled multi-known UAV identification in step 102 are as follows: 102-1. Decoupled Representation Learning and Feature Slot Allocation: A decoupled representation learning mechanism is introduced in the feature extraction stage to construct a dedicated feature representation space for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores the discriminative features of a specific label, realizing the structured separation of features at the semantic level. The specific steps of decoupling representation learning and feature slot allocation in 102-1 are as follows: Extract basic visual or signal features related to the task from the raw input data to provide a unified feature representation for subsequent fine decoupling; Establish an independent, parameter-isolated feature processing channel for each known drone model label, thereby forcing the model to learn separate feature subspaces for different labels from an architectural perspective. The general features are input into the feature processing slot, supervised and driven decoupled representation learning, and a high-dimensional feature vector corresponding to the preset model label is generated. An independent feature processing slot is built for each model tag, and each channel includes a dedicated feature encoder and decoder, forming multiple parallel processing channels.

[0044] By using labeled supervised data and a carefully designed loss function, each feature slot is driven to truly learn to extract its corresponding model-specific, highly discriminative features, thereby achieving semantic decoupling of representations.

[0045] 102-2. Decoupled Neural Network: A decoupled neural network model is constructed to decouple the original joint multi-label classification task at the feature level, decomposing it into multiple independent and parallel single-label classification subtasks. The model can process and optimize multiple subtasks in parallel at the same time. By establishing a targeted lightweight quantum classifier for each UAV model label, the learning difficulty of the multi-target joint recognition task is significantly reduced, while the complexity and number of parameters of the overall model structure are reduced, improving training efficiency and recognition real-time performance. The specific steps of the decoupled neural network in 102-2 are as follows: The multi-label classification task for drone models is structurally decoupled, decomposing it from a traditional joint multi-output prediction problem into multiple independent, parallel single-label binary classification subtasks. Each subtask focuses solely on determining whether its corresponding specific drone model exists. Figure 4 As shown.

[0046] Based on the decoupled task structure, a decoupled neural network architecture is constructed. Each single-label subtask is equipped with a parameter-independent lightweight sub-classifier. These sub-classifiers each have completely independent feature encoding and classification decision layers that do not interfere with each other.

[0047] During the model training phase, a parallel loss calculation mechanism is designed. The classification loss is calculated independently for each sub-task channel. The parameters of each sub-classifier are independently optimized using the backpropagation algorithm. This process drives each independent feature channel to adaptively focus on learning and extracting unique signal patterns and discriminative features highly correlated with its associated device model, thereby independently achieving an accurate determination of the device model's existence.

[0048] Where: K is the total number of drone models. is an adjustable weight coefficient for the i-th category, which can handle class imbalance.

[0049] Where K represents the total number of drone models to be identified, and N is the sample size. and Let represent the true label and predicted probability of the nth sample in the i-th class, respectively. Specifically, an adjustable weighting coefficient is introduced. The aim is to effectively alleviate the problem of class imbalance in training data by assigning different loss weights to different categories.

[0050] 102-3. Label Relevance Modeling Based on Graph Neural Networks: To utilize the potential semantic and functional associations between known UAV model labels, graph neural network technology is introduced; using each model label as a node and defining edges based on label co-occurrence relationships or prior knowledge, a label relationship graph is constructed to explicitly model and fuse the structured correlations between labels, thereby enhancing the robustness of recognition and contextual reasoning ability when multiple models coexist in complex scenarios.

[0051] The specific steps for label relevance modeling in the neural network in 102-3 are as follows: Construct a label relationship graph using model labels as nodes and relationships between labels as edges.

[0052] in, This represents the element in the i-th row and j-th column of the adjacency matrix of the constructed label relationship graph. For adjustable temperature coefficient hyperparameters, and These are the conditional co-occurrence probability distribution vectors for the i-th and j-th model tags, respectively.

[0053] Use the decoupled features corresponding to each label as the initial feature vector of its graph node; By leveraging the message passing mechanism of graph neural networks, information about neighboring nodes of each node is aggregated, and the interactive propagation of label features is achieved through graph convolution operations to update and obtain enhanced features incorporating label relevance.

[0054] in, It is the first graph neural network. Layer feature matrix, yes The trainable weight matrix of the layer, It is a non-linear activation function.

[0055] The label relevance modeling module uses graph neural network technology to explicitly capture the semantic relationships between labels. Finally, a multi-layer graph convolutional network is used as a label relationship extractor, which combines the features of the signal itself and the label information through dot product operations.

[0056] This invention processes the input through a shared feature extraction network to obtain general features; An independent feature processing slot is built for each model label, and only exclusive feature information is extracted for each channel; The general features are input into the feature processing slot, supervised and driven decoupled representation learning, and a high-dimensional feature vector corresponding to the preset model label is generated. The specific steps of the decoupled neural network in this invention are as follows: The multi-label classification task for drone models is decoupled and decomposed into multiple independent single-label classification sub-tasks. Based on an independent feature encoder, a parameter-independent lightweight sub-classifier is instantiated for each single-label subtask; By minimizing the classification loss of each channel, each channel learns independent information about a single type of drone and completes the category determination.

[0057] The specific steps of label relevance modeling based on graph neural networks in this invention are as follows: Construct a label relationship graph using model labels as nodes and relationships between labels as edges; Use the decoupled features corresponding to each label as the initial feature vector of its graph node; By leveraging the message passing mechanism of graph neural networks, information about neighboring nodes of each node is aggregated to update and obtain enhanced features that incorporate label relevance.

[0058] This invention addresses the problem of overlapping features and difficulty in accurately distinguishing signals from multiple known UAV models when they coexist. By introducing a decoupled representation learning mechanism, it assigns and learns independent feature channels for each potential target in the signal, thereby significantly improving the recognition accuracy in multi-target scenarios.

[0059] Step 103: Dynamically fuse the identified model-specific features, compare the residuals of the fused model-specific features with the overall deep embedding of the original radio frequency signal, and determine whether there is an unknown drone model based on the comparison results and trigger the early warning mechanism.

[0060] Preferably, the identified drone features are dynamically fused, and the fused drone features are compared with the overall deep embedding of the original radio frequency signal for residual comparison. Based on the comparison result, it is determined whether there is an unknown drone type and an early warning mechanism is triggered, including:

[0061] The model-specific features of all known models are obtained, and a dynamic fusion embedding representation is generated through a fusion mechanism of weighted summation and projection activation. Calculate the similarity between the dynamically fused embedded representation and the overall deep embedding from the original radio frequency, and generate a scalarized residual metric. The residual metric is calculated based on all samples in the training set, and the 95th percentile of the residual metric is used as the preset threshold for detecting unknown models. The residual metric of the sample to be tested is compared with a preset threshold. If it is greater than the threshold, an early warning mechanism is triggered.

[0062] Preferably, the weights for the weighted summation are adaptively assigned weight values ​​based on the discriminativeness and feature contribution of each known model-specific feature.

[0063] Preferably, the method further includes: The overall deep embedding is obtained by performing high-dimensional feature extraction on the original radio frequency signal; Complex electromagnetic environments include: the coexistence of multiple drone models, or electromagnetic interference intensity exceeding the interference threshold.

[0064] Unsupervised early warning module for unknown aircraft types: This module constructs a fusion metric module to adaptively fuse high-dimensional decoupled feature embeddings, effectively integrating information from different targets; by comparing feature differences before and after dynamic recombination, it determines whether an unknown type of drone exists. For example... Figure 2 As shown.

[0065] The specific steps of the unsupervised early warning module for unknown aircraft models in step 103 of this invention are as follows: 103-1. Dynamic Adaptive Fusion of Known Features: Dynamically fuse the specific features of each known aircraft model obtained in stage 102 to generate a dynamic fused embedding representation that can comprehensively represent the joint features of all known aircraft models in the current observation; The specific steps for dynamic adaptive fusion of known features in 103-1 are as follows: After obtaining the unique feature representations of all identified models, various dynamic fusion mechanisms can be used to generate fused embedded representations.

[0066] The first type is a dynamic fusion mechanism based on weighted summation: in, This represents the total number of identified device models. The dynamic weight coefficient corresponding to the i-th model feature is used to measure the contribution of different model features to the final fused representation.

[0067] The second state fusion mechanism is based on projection activation: in, and These represent the weight matrices for the first and second layer linear transformations, respectively. and For the corresponding bias term, and the formula utilizes Activation functions introduce nonlinear representation capabilities.

[0068] 103-2 Residual Construction and Unknown Model Detection: The dynamically fused embedding representation is compared with the overall embedding extracted from the original input, and the difference between the two is calculated to quantify the components in the current sample that cannot be explained by known model features. The residual measurement results are analyzed using a preset threshold or a statistically based unsupervised anomaly detection algorithm. If the residual exceeds a set range, it is determined that an unknown model or abnormal target exists, thereby triggering an early warning signal, achieving unsupervised detection and early warning of newly emerging or unregistered drone types. Figure 5 As shown.

[0069] The specific steps of the known feature dynamic adaptive fusion of the present invention are as follows: Obtain the unique feature representations of all identified and known device models; Dynamic fused embedding representations are generated through fusion mechanisms such as weighted summation and projection activation.

[0070] The specific steps of residual construction and unknown model detection in this invention are as follows: The similarity between the dynamic fusion embedding representation and the overall deep embedding extracted from the original input is calculated to generate a scalarized residual metric. The residual metric is calculated based on all samples in the training set, and its 95th percentile is taken as the preset threshold for detecting unknown models.

[0071] The residual metric is compared with a preset threshold. If it is greater than the threshold, an unknown model warning signal is triggered.

[0072] This invention addresses the problem that unknown novel drones not present in training data are difficult to detect using traditional supervised models. It constructs a fusion metric module to adaptively integrate decoupled high-dimensional features and innovatively constructs detection criteria by comparing the differences in features before and after dynamic recombination. This achieves unsupervised perception and real-time early warning of the presence of unknown drones, enhancing the system's scalability and security in responding to new threats.

[0073] The specific steps for unsupervised early warning of unknown aircraft models in this invention are as follows.

[0074] Dynamic fusion embedding representation Compared with the overall depth embedding extracted from the original input Calculate the similarity and generate scalarized residual metrics: in It is a dynamic fusion and embedding. It is a deep embedding of the original input, and the cosine similarity is calculated. Generate standardized residual metrics.

[0075] After the training phase, the scalarized residual distribution is calculated using all training samples. The 95th percentile of the residual distribution is taken as the preset detection threshold. The specific steps are as follows: in This represents the training dataset containing all samples used to calculate the threshold. This represents the total number of samples in the training set. Let represent the similarity score of the t-th sample. This indicates the number of samples in the set that meet the conditions.

[0076] During the inference phase, residual metrics are calculated for the inference samples. Compare it with a preset threshold By comparing the data, it can be determined whether the aircraft is an unknown model. To provide more refined early warning information, the continuous confidence level of unknown target anomalies can be calculated as follows: in To generate standardized residual metrics, The desired preset detection threshold is... As a scaling factor, after The mapping process ultimately produces a continuous confidence level for unknown target anomalies.

[0077] This invention discloses a method for identifying and warning of unknown drones in complex scenarios, comprising the following steps: First, multimodal features of the collected raw radio frequency signals are extracted in the time domain, frequency domain, and time-frequency domain, and robust fusion features are generated through a cross-domain dynamic fusion module based on an attention mechanism; Second, a decoupled representation learning mechanism is adopted to assign independent feature channels to each potential drone target, and label correlation is modeled by combining decoupled neural networks and graph neural networks to achieve parallel identification of multiple known drone types; Finally, the known drone type features are dynamically fused and compared with the overall deep embedding residuals, and the existence of unknown drone types is determined based on unsupervised anomaly detection, triggering an early warning. Compared with existing technologies, this invention has the following outstanding advantages: 1) By using parallel multimodal feature extraction and adaptive fusion mechanism, the bottleneck of traditional serial processing is broken through, significantly improving processing efficiency and model generalization; 2) The decoupled identification architecture combined with label correlation modeling significantly improves the identification accuracy and anti-interference ability in multi-target coexistence scenarios; 3) An innovative dynamic residual detection mechanism is introduced, which can achieve unsupervised real-time perception and early warning of unknown threats without relying on known samples. This invention enables efficient and accurate identification of swarm drones in complex electromagnetic environments, while also possessing the ability to proactively detect unknown threats, thus forming a complete closed loop of monitoring technology.

[0078] This invention provides a method for identifying and warning of unknown drones in complex scenarios, with the following advantages: 1) It fully mines and fuses multimodal features. Breaking through the traditional serial processing mode of layer-by-layer unpacking, it significantly improves processing speed and enhances the robustness and discriminative power of features in complex electromagnetic environments through parallel extraction and adaptive fusion mechanisms. 2) It significantly improves the recognition accuracy and robustness in scenarios where multiple targets coexist. Through a decoupled architecture and feature slot mechanism, this invention achieves structured separation and dedicated representation of signals from multiple drone types at the feature level. Combined with label correlation modeling, it significantly improves the recognition accuracy and anti-interference capability in scenarios where multiple targets coexist. 3) It achieves unsupervised real-time perception and warning of unknown new drones, breaking through the capability boundary of closed set recognition. Traditional methods can only identify predefined drone types and are completely ineffective against new threats that have not been seen before. This invention innovatively introduces dynamic fusion and residual detection mechanisms, enabling real-time warning of new drones without training with unknown samples, breaking through the limitations of traditional closed set recognition.

[0079] Figure 6 This is a structural diagram of a clustered UAV identification and early warning system in a complex scenario according to a preferred embodiment of the present invention.

[0080] like Figure 6 As shown, this invention provides a system for identifying and warning of unknowns in complex scenarios using swarm drones. The system includes: The initial unit 601 is used to acquire raw radio frequency signals in a complex electromagnetic environment, extract multimodal features from the raw radio frequency signals, and dynamically fuse the multimodal features to generate high-dimensional robust fused features. Preferably, the initial unit 601 is used to acquire raw radio frequency signals in a complex electromagnetic environment, extract multimodal features from the raw radio frequency signals, and dynamically fuse the multimodal features to generate high-dimensional robust fused features, including: The original radio frequency signal is transformed into a two-dimensional time-frequency domain through short-time Fourier transform to generate a time-frequency spectrum; a multi-scale convolutional neural network is used to extract multi-scale deep features from the time-frequency spectrum to obtain time-frequency domain features; The original radio frequency signal is transformed to the frequency domain through discrete Fourier transform, and the spectrum of the original radio frequency signal is analyzed and modeled through a frequency domain feature learner to extract frequency domain features. The original radio frequency signal is input into a time-series convolutional network to obtain a time-domain waveform. Based on the time-domain waveform, time-domain features characterizing the long-range time dependence and transient dynamic characteristics of the signal are extracted. Time-frequency domain features, frequency domain features, and time-domain features are input into the cross-attention fusion module. The cross-attention fusion module calculates the correlation between cross-domain features, dynamically assigns weights to the time-frequency domain features, frequency domain features, and time-domain features, and performs adaptive weighted fusion to generate high-dimensional robust fused features.

[0081] Preferably, the cross-attention fusion module assigns differentiated weights to the time-domain features, frequency-domain features, and time-frequency-domain features by calculating the correlation coefficients of the time-domain features, frequency-domain features, and time-frequency-domain features.

[0082] The identification unit 602 is used to process high-dimensional robust fused features through a shared feature extraction network to generate general features; generate preset aircraft type labels for the general features through decoupled representation learning; and identify the aircraft type of the UAV based on the preset aircraft type labels. Preferably, the identification unit 602 is used to process the high-dimensional robust fusion features through a shared feature extraction network to generate general features; generate preset aircraft model labels for the general features through decoupled representation learning; and identify the aircraft model of the UAV based on the preset aircraft model labels, including: Based on the decoupled representation learning mechanism, a corresponding feature representation space is constructed for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores the discriminative features corresponding to the known UAV model label, generating a high-dimensional feature vector corresponding to the preset model label; By constructing a decoupled neural network model, the original joint multi-label classification task is decoupled at the feature level and decomposed into multiple independent and parallel single-label classification subtasks. Based on an independent feature encoder, each single-label subtask is transformed into a parameter-independent lightweight sub-classifier. By minimizing the classification loss of the input channels of the lightweight sub-classifier, the independent identification of a single UAV model by each channel is achieved, and the label of the UAV model is determined. Using known drone model labels as nodes, edges between nodes are defined based on label co-occurrence relationships or prior knowledge to construct a label relationship graph. The decoupled features corresponding to each label are used as the initial feature vectors of the graph nodes in the label relationship graph. The message passing mechanism of the graph neural network is used to aggregate the neighbor node information of each node and update the enhanced features that incorporate label relevance to improve the robustness of multi-drone model recognition in complex scenarios.

[0083] Preferably, based on a decoupled representation learning mechanism, a corresponding feature representation space is constructed for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores discriminative features corresponding to the known UAV model label, including: High-dimensional robust fusion features are processed by a shared feature extraction network to obtain general features; an independent feature processing slot is constructed for each aircraft type label, and each feature processing slot extracts only the discriminative features corresponding to the known UAV aircraft type labels; the general features are input into the feature processing slots, and high-dimensional feature vectors corresponding to the preset aircraft type labels are generated based on the decoupled representation learning mechanism.

[0084] Preferably, the tag co-occurrence relationship is the probability distribution relationship of different drone models appearing simultaneously in the same monitoring scenario.

[0085] The early warning unit 603 is used to dynamically fuse the identified model-specific features, compare the fused model-specific features with the overall deep embedding of the original radio frequency signal, and determine whether there is an unknown drone model based on the comparison result and trigger the early warning mechanism.

[0086] Preferably, the early warning unit 603 is used to dynamically fuse the identified aircraft features, compare the fused aircraft features with the overall deep embedding of the original radio frequency signal for residual comparison, and determine whether there is an unknown drone type based on the comparison result and trigger an early warning mechanism, including:

[0087] The model-specific features of all known models are obtained, and a dynamic fusion embedding representation is generated through a fusion mechanism of weighted summation and projection activation. Calculate the similarity between the dynamically fused embedded representation and the overall deep embedding from the original radio frequency, and generate a scalarized residual metric. The residual metric is calculated based on all samples in the training set, and the 95th percentile of the residual metric is used as the preset threshold for detecting unknown models. The residual metric of the sample to be tested is compared with a preset threshold. If it is greater than the threshold, an early warning mechanism is triggered.

[0088] Preferably, the weights for the weighted summation are adaptively assigned weight values ​​based on the discriminativeness and feature contribution of each known model-specific feature.

[0089] Preferably, it further includes: The overall deep embedding is obtained by performing high-dimensional feature extraction on the original radio frequency signal; Complex electromagnetic environments include: the coexistence of multiple drone models, or electromagnetic interference intensity exceeding the interference threshold.

[0090] The present invention has been described with reference to a few embodiments. However, it will be apparent to those skilled in the art that other embodiments besides those disclosed above fall equivalently within the scope of the present invention.

[0091] Generally, all terms used in this invention are interpreted according to their ordinary meaning in the art, unless otherwise expressly defined herein. All references to “a / the / the [device, component, etc.]” ​​are openly interpreted as at least one instance of said device, component, etc., unless otherwise expressly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed unless explicitly stated otherwise.

[0092] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for identifying and providing early warning of unknowns in swarm drones in complex scenarios, the method comprising: Acquire raw radio frequency signals in a complex electromagnetic environment and extract multimodal features from the raw radio frequency signals; The multimodal features are dynamically fused to generate high-dimensional robust fused features; The high-dimensional robust fused features are processed by a shared feature extraction network to generate general features; Preset aircraft model labels are generated for the general features through decoupled representation learning; the aircraft model is identified based on the preset aircraft model labels; The identified drone-specific features are dynamically fused, and the fused drone-specific features are compared with the overall deep embedding of the original radio frequency signal. Based on the comparison results, it is determined whether there is an unknown drone model and an early warning mechanism is triggered.

2. The method according to claim 1, wherein the original radio frequency signal in a complex electromagnetic environment is acquired, and the multimodal features in the original radio frequency signal are extracted; Dynamically fusing the multimodal features to generate high-dimensional robust fused features includes: The original radio frequency signal is converted to a two-dimensional time-frequency domain through short-time Fourier transform to generate a time-frequency spectrum; a multi-scale convolutional neural network is used to extract multi-scale deep features from the time-frequency spectrum to obtain time-frequency domain features; The original radio frequency signal is converted to the frequency domain through discrete Fourier transform, and the spectrum of the original radio frequency signal is analyzed and modeled through a frequency domain feature learner to extract frequency domain features. The original radio frequency signal is input into a time-series convolutional network to obtain a time-domain waveform. Based on the time-domain waveform, time-domain features characterizing the long-range time dependence and transient dynamic characteristics of the signal are extracted. The time-frequency domain features, the frequency domain features, and the time domain features are input into the cross-attention fusion module. The cross-attention fusion module calculates the cross-domain feature correlation, dynamically assigns weights to the time-frequency domain features, the frequency domain features, and the time domain features, and performs adaptive weighted fusion to generate high-dimensional robust fusion features.

3. According to the method of claim 2, the cross-attention fusion module assigns differentiated weights to the time-domain features, frequency-domain features, and time-frequency-domain features by calculating the correlation coefficients of the time-domain features, frequency-domain features, and time-frequency-domain features.

4. The method according to claim 1, wherein the high-dimensional robust fused features are processed by a shared feature extraction network to generate general features; Preset model labels are generated for the general features by decoupling representation learning; Identifying the drone model based on the preset model tag includes: Based on the decoupled representation learning mechanism, a corresponding feature representation space is constructed for each known UAV model label; through a learnable feature slot mechanism, each slot adaptively captures and stores the discriminative features corresponding to the known UAV model label, generating a high-dimensional feature vector corresponding to the preset model label; By constructing a decoupled neural network model, the original joint multi-label classification task is decoupled at the feature level and decomposed into multiple independent and parallel single-label classification subtasks. Based on an independent feature encoder, each single-label subtask is transformed into a parameter-independent lightweight sub-classifier. By minimizing the classification loss of the input channel of the lightweight sub-classifier, each channel can independently identify a single UAV model and determine the label of the UAV model. Using known drone model labels as nodes, edges between nodes are defined based on label co-occurrence relationships or prior knowledge to construct a label relationship graph. The decoupled features corresponding to each label are used as the initial feature vectors of the graph nodes in the label relationship graph. The message passing mechanism of the graph neural network is used to aggregate the neighbor node information of each node and update the enhanced features that incorporate label relevance to improve the robustness of multi-drone model recognition in complex scenarios.

5. The method according to claim 4, wherein a corresponding feature representation space is constructed for each known UAV model label based on the decoupled representation learning mechanism; Through a learnable feature slot mechanism, each slot adaptively captures and stores discriminative features corresponding to known drone model labels, including: The high-dimensional robust fused features are processed by a shared feature extraction network to obtain general features; An independent feature processing slot is constructed for each model label, and each feature processing slot extracts only the discriminative features corresponding to the known drone model labels; the general features are input into the feature processing slot, and a high-dimensional feature vector corresponding to the preset model label is generated based on the decoupled representation learning mechanism.

6. The method according to claim 4, wherein the tag co-occurrence relationship is the probability distribution relationship of different UAV models appearing simultaneously in the same monitoring scenario.

7. The method according to claim 1, wherein the identified aircraft type features are dynamically fused, and the fused aircraft type features are compared with the overall deep embedding of the original radio frequency signal using residual comparison, and the existence of an unknown drone type is determined based on the comparison result and an early warning mechanism is triggered. include:. The model-specific features of all known models are obtained, and a dynamic fusion embedding representation is generated through a fusion mechanism of weighted summation and projection activation. Calculate the similarity between the dynamically fused embedded representation and the overall deep embedding from the original radio frequency, and generate a scalarized residual metric. The residual metric is calculated based on all samples in the training set, and the 95th percentile of the residual metric is used as the preset threshold for detecting unknown models. The residual metric of the sample to be tested is compared with the preset threshold. If it is greater than the threshold, an early warning mechanism is triggered.

8. The method according to claim 7, wherein the weights of the weighted summation are weight values ​​adaptively assigned based on the discriminability and feature contribution of each known model-specific feature.

9. The method according to claim 1, further comprising: The overall deep embedding is obtained by performing high-dimensional feature extraction on the original radio frequency signal; The complex electromagnetic environment includes: the coexistence of multiple UAV models, or electromagnetic interference intensity exceeding the interference threshold.

10. A system for identifying and warning of unknowns in complex scenarios using swarm drones, the system comprising: The initial unit is used to acquire raw radio frequency signals in a complex electromagnetic environment and extract multimodal features from the raw radio frequency signals; The multimodal features are dynamically fused to generate high-dimensional robust fused features; The identification unit is used to process the high-dimensional robust fused features through a shared feature extraction network to generate general features; Preset model labels are generated for the general features by decoupling representation learning; The drone model is identified based on the preset model label; The early warning unit is used to dynamically fuse the identified model-specific features, compare the fused model-specific features with the overall deep embedding of the original radio frequency signal, and determine whether there is an unknown drone model based on the comparison result and trigger the early warning mechanism.

Images