A radar emitter individual identification method based on multimodal information fusion

Through a neural network with multimodal information fusion and self-attention mechanism, combined with medium frequency and full pulse data, the problem of insufficient feature extraction in the radiation source individual recognition in the prior art is solved, and more efficient radiation source individual recognition is achieved.

CN118690320BActive Publication Date: 2024-12-06BEIJING HUAYUE MAIPU TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410775959.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2024-12-06
Estimated Expiration
2044-06-17

AI Technical Summary

Technical Problem

The existing radar radiation source individual recognition method based on deep learning is limited in the receptive field range, and the relationship between features and globality cannot be fully extracted, and the inter-vein characteristics of radar description characters are not fully utilized, resulting in poor individual recognition effect.

Method used

Using multimodal information fusion method, combining medium frequency and full pulse data, a neural network designed by self-attention mechanism is used to build medium frequency branches and full pulse branches, and features are extracted using Transformer and full connection layer, and feature styling and fusion are performed to achieve the Effective identification of individual radiation source.

Benefits of technology

The accuracy of individual recognition of radiation sources is improved, and the recognition effect is improved by mining the correlation between particle characteristics and time-frequency diagrams from multiple angles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118690320B_ABST
    Figure CN118690320B_ABST
Patent Text Reader

Abstract

The present invention discloses a radar radiation source individual identification method based on multimodal information fusion, the method comprising: step 1, data acquisition; step 2, using short-time Fourier transform, using intermediate frequency data Q for time-frequency conversion; step 3, normalizing the full pulse data; step 4, constructing a two-branch radar radiation source feature fusion network, which is composed of an intermediate frequency branch and a full pulse branch; step 5, constructing an MLP neural network classifier to obtain the identification result of the radiation source individual; step 6, using the data processed by steps 2 and 3, training the fusion network constructed in step 4 and the MLP neural network classifier constructed in step 5, and applying the trained network to the radar radiation source individual identification. The present application integrates intermediate frequency and full pulse data, uses a targeted designed neural network, and through a self-attention mechanism, mines the particle characteristics of the radiation source individual from multiple angles to achieve effective identification of the radiation source individual.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of radar radiation source individual identification, and in particular relates to a radar radiation source individual identification method based on multi-modal information fusion. Background Art

[0002] Specific Emitter Identification (SEI) mainly refers to the process of obtaining fingerprint information reflecting the identity of the target due to differences in components during production and manufacturing and loss during use of radars of the same model. This is a key link in the field of electronic countermeasures. Research on radar radiator individual identification will help improve battlefield situation awareness and provide information support for the assessment of strategic intentions. However, with the improvement of manufacturing technology and the increasing complexity of the electromagnetic environment in recent years, traditional radiator identification methods have more and more limitations, resulting in problems such as poor individual identification effect and instability across environments.

[0003] As an important branch of artificial intelligence in recent years, deep learning is based on neural networks. By building a multi-layer neural network, the model fits the data distribution, thereby completing the algorithm for data representation learning. In the field of individual identification of radar emitters in a non-cooperative environment today, the self-learning ability of deep learning will effectively realize the mining of differences between individual emitters. Qu Lingyun proposed a method that combines a double-layer attention mechanism and a residual neural network to realize individual identification of emitters. Wang X proposed a method based on pulse graph images and CNN to realize the conversion of time domain waveforms into two-dimensional images for radar individual identification. The patent application with application number 202211283087.9 proposed a method for individual identification of emitters based on transfer learning. Liu Gaohui used a deep belief network to identify individual emitters. The patent application with application number 202211118749.7 proposed a method for individual identification of emitters based on one-dimensional neural networks.

[0004] Most of the existing studies on individual radiation sources based on deep learning are realized by mining intermediate frequency data through one-dimensional neural networks or convolutional neural networks. However, although these two algorithms can achieve deep extraction of data features, they are limited by the range of the receptive field. The network structure cannot fully extract the relationship between features and the global situation, and the inter-pulse feature of radar descriptors is not fully utilized. Therefore, how to fully mine the fingerprint characteristics of individual radiation sources without increasing the complexity of the algorithm has become an urgent problem to be solved in the identification of individual radar radiation sources. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a radar emitter individual identification method based on multimodal information fusion. The method integrates intermediate frequency and full pulse data, utilizes a targeted designed neural network, and through a self-attention mechanism, mines the particle characteristics of the individual emitter from multiple angles to achieve effective identification of the individual emitter based on multimodal information fusion.

[0006] The object of the present invention is achieved through the following technical solution: a radar emitter individual identification method based on multimodal information fusion, the method comprising:

[0007] Step 1: Data acquisition: record the full pulse sequence intercepted by the detection device as P1, P2, P3, ... P i ,…,P n , where P represents the full pulse data, n is the number of full pulse data intercepted; P i Represents the intercepted i-th full pulse data, expressed as P i = {RF i ,PA i ,PW i ,PRI i}, where RF i Represents the pulse-to-pulse carrier frequency component, PA i Represents the pulse-to-pulse amplitude component, PW i Represents the pulse width component between pulses, PRI i represents the pulse repetition interval component; the intercepted intermediate frequency data is recorded as

[0008] Q1,Q2,Q3,…Q i ,…,Q n , Q represents the intermediate frequency data, Q i represents the i-th intermediate frequency data intercepted; the intermediate frequency data Q corresponds to the full pulse data P one by one;

[0009] Step 2: Use short-time Fourier transform to perform time-frequency conversion using intermediate frequency data Q to obtain a time-frequency diagram;

[0010] Step 3: normalize the full pulse data P to obtain dimensionless data;

[0011] Step 4: Construct a two-branch radar radiation source feature fusion network, which consists of an intermediate frequency branch and a full pulse branch. The intermediate frequency branch uses Transformer as the basic component unit, and realizes feature extraction of the time-frequency graph through multi-layer superposition. The full pulse branch extracts full pulse information features based on the fully connected layer. Then, the features of the two branches are spliced, and the spliced ​​features are fused using the fully connected layer. The specific construction process is as follows:

[0012] Step 4.1: Construct an intermediate frequency neural network branch to extract the features of the time-frequency graph. The specific steps are as follows:

[0013] Step 4.1.1, construct the Embedding layer to convert the input time-frequency graph into a data format that can be used by the Transformer Encoder layer;

[0014] Step 4.1.2: Construct a Transformer Encoder layer. The Transformer Encoder layer consists of multiple sequential Transformer modules and multiple sequential MLP fully connected layers. The output of the last Transformer module is used as the input of the first MLP fully connected layer.

[0015] The Transformer module first obtains the query vector matrix Q, the key-value vector matrix K, and the value vector matrix V through linear transformation:

[0016] Q=F C W Q

[0017] K=F C W K

[0018] V=F C W

[0019] Where W Q , W K , W V All are linear parameter matrices with training capabilities;

[0020] Secondly, the query vector matrix Q, the key vector matrix K, and the value vector matrix V are used to calculate the feature layer with autocorrelation attention weights using the softmax operation:

[0021]

[0022] where d k is the length of vector K, K T is the data after K is transposed, Attention(Q,K,V) is the feature layer with attention weight;

[0023] Finally, we construct the MLP fully connected layer to convert Attention(Q,K,V) into F C Matrix vectors of the same dimensions;

[0024] Step 4.2, construct a full pulse neural network branch composed of a multi-layer MLP neural network to extract full pulse features;

[0025] Step 4.3: Construct a splicing fusion network to splice and fuse the intermediate frequency neural network branch with the full pulse neural network branch. The specific steps are as follows:

[0026] Step 4.3.1: Output T of the intermediate frequency neural network branch L Perform linear transformation to obtain a one-dimensional sequence

[0027] Step 4.3.2: Dimensionality reduction is performed and then combined with the output of the full pulse neural network branch Use the splicing operation to get the spliced ​​features

[0028] Step 4.3.3: The spliced ​​features By building a multi-layer MLP neural network for data fusion, the fused features are obtained.

[0029] Step 5: Construct an MLP neural network classifier to classify the fused features As input, the identification result Pred of the radiation source individual is obtained through the MLP neural network classifier;

[0030] Step 6: Use the data processed in steps 2 and 3 to train the fusion network built in step 4 and the MLP neural network classifier built in step 5, and apply the trained network to individual identification of radar emitters.

[0031] In step S3, the process of normalization is as follows:

[0032] Step 3.1: Calculate the mean value of component C in the radar description word

[0033]

[0034] Among them C i represents the i-th sample in the C component, where the C component is composed of n inter-pulse carrier frequency components, inter-pulse amplitude components, inter-pulse pulse width components or inter-pulse repetition interval components;

[0035] Step 3.2: Calculate the minimum value C of component C in the radar description word min :

[0036] C min =min(C);

[0037] Step 3.3 Calculate the maximum value C of component C in the radar description word max :

[0038] C max =max(C);

[0039] Step 3.4: Calculate and obtain the normalized result of the i-th sample of the radar description word C component:

[0040]

[0041] in Represents the i-th sample after normalization of the C component.

[0042] The beneficial effects of the present invention are:

[0043] 1. A multimodal information fusion method is proposed, which uses the received intermediate frequency data and full pulse data in the task of radiation source individual identification. From the perspectives of intra-pulse and inter-pulse, multimodal information fusion is used to provide more abundant target information for radiation source individual identification, which plays a certain role in improving the accuracy of radiation source individual identification.

[0044] 2. In the process of extracting features from the time-frequency graph, Transformer is used for feature extraction. Due to its self-attention mechanism, the Transformer algorithm can more carefully explore the particle features of the time-frequency graph and the correlation between high-dimensional features, thereby improving the accuracy of individual radiation source identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the radar radiation source individual identification method based on multi-modal information fusion of the present invention;

[0046] Figure 2 It is a schematic diagram of the intermediate frequency branch of the present invention;

[0047] Figure 3 It is a schematic diagram of the full pulse branch of the present invention;

[0048] Figure 4 It is a schematic diagram of the multimodal information fusion recognition structure of the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0050] like Figure 1 As shown, a radar emitter individual identification method based on multimodal information fusion of the present invention comprises:

[0051] Step 1: Data acquisition: The full pulse sequence intercepted by the detection and receiving equipment is recorded as P1, P a ,P3,…P i ,…,P n , where P represents the full pulse data, n is the number of full pulse data intercepted; P iRepresents the intercepted i-th full pulse data, expressed as P i = {RF i ,PA i ,PW i ,PRI i}, where RF i Represents the pulse-to-pulse carrier frequency component, PA i Represents the pulse-to-pulse amplitude component, PW i Represents the pulse width component between pulses, PRI i represents the pulse repetition interval component; the intercepted intermediate frequency data is recorded as

[0052] Q1,Q2,Q3,…Q i ,…,Q n , Q represents the intermediate frequency data, Q i represents the i-th intermediate frequency data intercepted; the intermediate frequency data Q corresponds to the full pulse data P one by one; the data used in this example are the intermediate frequency and full pulse data of multiple individuals of a certain type of radar, the length of the intermediate frequency data is not less than 2048 points, and the full pulse data contains information such as carrier frequency, pulse width, pulse amplitude, and repetition interval. It should be noted that the length of the intermediate frequency data and the full pulse data can be adjusted accordingly according to actual needs to obtain the optimal sample, or the sample set construction method in the prior art can be used for construction.

[0053] Step 2: Use short-time Fourier transform to perform time-frequency conversion using intermediate frequency data Q to obtain a time-frequency diagram; in this example, the Hanning frame is used in the Fourier transform calculation, the Fourier transform window length is 256, the overlap rate is 25%, and after the time-frequency diagram is generated, the image size is scaled to 224. It should be noted that the Fourier transform window length, overlap rate, window form and image size can be adjusted accordingly according to actual needs.

[0054] Step 3: normalize the full pulse data P to obtain dimensionless data; the normalization process is as follows:

[0055] Step 3.1: Calculate the mean value of component C in the radar description word

[0056]

[0057] Among them C i represents the i-th sample in the C component, where the C component is composed of n inter-pulse carrier frequency components, inter-pulse amplitude components, inter-pulse pulse width components or inter-pulse repetition interval components;

[0058] Step 3.2: Calculate the minimum value C of component C in the radar description word min :

[0059] Cmin =min(C);

[0060] Step 3.3 Calculate the maximum value C of component C in the radar description word max :

[0061] C max =max(C);

[0062] Step 3.4: Calculate and obtain the normalized result of the i-th sample of the radar description word C component:

[0063]

[0064] in Represents the i-th sample after normalization of the C component.

[0065] The pulse amplitude components {PA1, PA2, PA3, ..., PA n}, pulse width components {PW1, PW2, PW3, ..., PW n}, {PRI1,PRI2,PRI3,…,PRI n}repetition interval component, and carrier frequency components {RF1, RF2, RF3, …, RF n}, the four components are normalized in turn to form a normalized pulse sequence

[0066] Step 4: Construct a two-branch radar radiation source feature fusion network, which consists of an intermediate frequency branch and a full pulse branch. The intermediate frequency branch uses Transformer as the basic component unit, and realizes feature extraction of the time-frequency graph through multi-layer superposition. The full pulse branch extracts full pulse information features based on the fully connected layer. Then, the features of the two branches are spliced, and the spliced ​​features are fused using the fully connected layer. The specific construction process is as follows:

[0067] Step 4.1: Construct the intermediate frequency neural network branch to extract the features of the time-frequency graph. Its structure is as follows: Figure 2 As shown; the specific steps are:

[0068] Step 4.1.1: Construct the Embedding layer to convert the input frequency spectrum into a data format that can be used by the Transformer Encoder layer. First, split the frequency spectrum of size [H, W, C] into N 2 Sizes [H / N,W / N,C*

[0069] N 2 ]Patch block, where H is the width of the image, W is the height of the image, and C is the dimension of the image; secondly, through linear mapping, the patch is converted from a three-dimensional vector to a size The vector matrix F C ;

[0070] Specifically, in this example, a convolutional neural network is used for time-frequency graph segmentation, a 16*16 convolution kernel is used, a step size of 16, and the number of output convolution kernels is 1024. Through convolution, the time-frequency graph of size [224,224,3] is converted into a feature graph of [14,14,768]. Finally, the width and height dimensions of the feature graph are flattened to obtain a two-dimensional matrix of [196,1024]. It should be noted that the size, step size, and number of output convolution kernels of the convolution kernel can be adjusted according to actual needs, or the existing technology can be used to improve the time-frequency graph segmentation method.

[0071] Step 4.1.2, build the Transformer Encoder layer. The main function of the Transformer Encoder layer is to achieve feature extraction from low dimensions to high dimensions through the self-attention mechanism. The Transformer Encoder layer consists of multiple sequential Transformer modules and multiple sequential MLP fully connected layers. The output of the last Transformer module is used as the input of the first MLP fully connected layer. L layers of Transformer modules and N MLP fully connected layers can be stacked according to the data situation;

[0072] The Transformer module first obtains the query vector matrix Q, the key-value vector matrix K, and the value vector matrix V through linear transformation:

[0073] Q=F C W Q

[0074] K=F C W K

[0075] V=F C W

[0076] Where W Q , W K , W V All are linear parameter matrices with training capabilities;

[0077] Secondly, the query vector matrix Q, the key vector matrix K, and the value vector matrix V are used to calculate the feature layer with autocorrelation attention weights using the softmax operation:

[0078]

[0079] where d k is the length of vector K, K Tis the data after K is transposed, Attention(Q,K,V) is the feature layer with attention weight;

[0080] Finally, we construct the MLP fully connected layer to convert Attention(Q,K,V) into F C Matrix vectors of the same dimension. This implementation example consists of four Transformer modules. In the Transformer module, first, the feature layer with attention weights is obtained through the query vector matrix Q, the key-value vector matrix K, and the value vector matrix V. The dimension of the feature layer is still [196, 1024]. Then, through an MLP fully connected layer, the dimension of the feature layer is increased to [196, 2048]. Finally, another MLP fully connected layer is connected to restore the dimension of the feature layer to the same dimension as the Transformer module [196, 1024].

[0081] Step 4.2: Construct a full-pulse neural network branch composed of a multi-layer MLP neural network to extract full-pulse features; the input of this branch is the normalized full-pulse data. By building a multi-layer neural network, the full-pulse network branch is built. In this example, normalized full-pulse data is used, and the data scale is [1,4]. First, an MLP fully connected layer is used to increase the dimension of the feature layer to [1,8]. Then, another MLP fully connected layer is connected to transform the dimension of the feature layer to [1,2]. The network structure is as follows: Figure 3 shown.

[0082] Step 4.3: Construct a splicing fusion network, and splice and fuse the intermediate frequency neural network branch with the full pulse neural network branch, such as Figure 4 As shown; the specific steps are:

[0083] Step 4.3.1: Output T of the intermediate frequency neural network branch L Perform linear transformation to obtain a one-dimensional sequence

[0084] Step 4.3.2: Dimensionality reduction is performed and then combined with the output of the full pulse neural network branch Use the Concat operation to get the concatenated features

[0085] Step 4.3.3: The spliced ​​features By building a multi-layer MLP neural network for data fusion, the fused features are obtained. In this example, the output of the intermediate frequency neural network branch is first linearly transformed from [196, 2048] to [1, 401408]; then the data dimension is converted to [1, 2046] through a multi-layer MLP fully connected neural network, and the MLP fully connected layers are connected using the Relu activation function; then the Concat operation is used to concatenate the intermediate frequency data features of dimension [1, 2046] and the full pulse data features of dimension [1, 2] to obtain a multimodal information fusion feature of dimension [1, 2048].

[0086] Step 5: Construct an MLP neural network classifier to classify the fused features As input, the MLP neural network classifier is used to obtain the identification result Pred of the radiation source individual; in this example, a K-layer MLP neural network fully connected layer is constructed, the feature layer with a dimension of [1,2048] is reduced in dimension, and finally the Softmax layer function is used to obtain the confidence of each individual.

[0087] Step 6: Use the data processed in steps 2 and 3 to train the fusion network built in step 4 and the MLP neural network classifier built in step 5, and apply the trained network to individual identification of radar emitters.

[0088] The present application constructs a radar emitter individual identification method based on multimodal information fusion. The method integrates intermediate frequency and full pulse data, uses a targeted designed neural network, and through a self-attention mechanism, mines the particle characteristics of the individual emitter from multiple angles to achieve effective identification of the individual emitter.

[0089] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A radar emitter individual identification method based on multimodal information fusion, characterized in that: The method comprises: Step 1: Data acquisition: record the full pulse sequence intercepted by the detection device as P1, P2, P3, ... P i ,…,P n , where P represents the full pulse data, n is the number of full pulse data intercepted; P i Represents the intercepted i-th full pulse data, expressed as P i = {RF i , P.A. i , PW i , PRI i }, where RF i Represents the pulse-to-pulse carrier frequency component, PA i Represents the pulse-to-pulse amplitude component, PW i Represents the pulse width component between pulses, PRI i represents the inter-pulse repetition interval component; the intercepted intermediate frequency data is recorded as Q1, Q2, Q3, ... Q i , …, Q n , Q represents the intermediate frequency data, Q i represents the i-th intermediate frequency data intercepted; the intermediate frequency data Q corresponds to the full pulse data P one by one; Step 2: Use short-time Fourier transform to perform time-frequency conversion using intermediate frequency data Q to obtain a time-frequency diagram; Step 3: normalize the full pulse data P to obtain dimensionless data; Step 4: Construct a two-branch radar radiation source feature fusion network, which consists of an intermediate frequency branch and a full pulse branch. The intermediate frequency branch uses Transformer as the basic component unit, and realizes feature extraction of the time-frequency graph through multi-layer superposition. The full pulse branch extracts full pulse information features based on the fully connected layer. Then, the features of the two branches are spliced, and the spliced ​​features are fused using the fully connected layer. The specific construction process is as follows: Step 4.1: Construct an intermediate frequency neural network branch to extract the features of the time-frequency graph. The specific steps are as follows: Step 4.1.1, construct the Embedding layer to convert the input time-frequency graph into a data format that can be used by the Transformer Encoder layer; Step 4.1.2: Construct a Transformer Encoder layer. The Transformer Encoder layer consists of multiple sequential Transformer modules and multiple sequential MLP fully connected layers. The output of the last Transformer module is used as the input of the first MLP fully connected layer. The Transformer module first obtains the query vector matrix Q, the key-value vector matrix K, and the value vector matrix V through linear transformation: Q=F C W Q K=F C W K V=F C W V Where W Q , W K , W v All are linear parameter matrices with training capabilities; Secondly, the query vector matrix Q, the key vector matrix K, and the value vector matrix V are used to calculate the feature layer with autocorrelation attention weights using the softmax operation: where d k is the length of vector K, K T is the data after K is transposed, Attention(Q, K, V) is the feature layer with attention weight; Finally, we construct the MLP fully connected layer to convert Attention(Q, K, V) into F C Matrix vectors of the same dimensions; Step 4.2, construct a full pulse neural network branch composed of a multi-layer MLP neural network to extract full pulse features; Step 4.3: Construct a splicing fusion network to splice and fuse the intermediate frequency neural network branch with the full pulse neural network branch. The specific steps are as follows: Step 4.3.1: Output T of the intermediate frequency neural network branch L Perform linear transformation to obtain a one-dimensional sequence Step 4.3.2: Dimensionality reduction is performed and then combined with the output of the full pulse neural network branch Use the splicing operation to get the spliced ​​features Step 4.3.3: The spliced ​​features By building a multi-layer MLP neural network for data fusion, the fused features are obtained. Step 5: Construct an MLP neural network classifier to classify the fused features As input, the identification result Pred of the radiation source individual is obtained through the MLP neural network classifier; Step 6: Use the data processed in steps 2 and 3 to train the fusion network built in step 4 and the MLP neural network classifier built in step 5, and apply the trained network to individual identification of radar emitters.

2. According to claim 1, a radar emitter individual identification method based on multimodal information fusion is characterized in that: In step 3, the process of normalization is as follows: Step 3.1: Calculate the mean value of component C in the radar description word Among them C i represents the i-th sample in the C component, where the C component is composed of n inter-pulse carrier frequency components, inter-pulse amplitude components, inter-pulse pulse width components or inter-pulse repetition interval components; Step 3.2: Calculate the minimum value C of component C in the radar description word min : C min =min(C); Step 3.3 Calculate the maximum value C of component C in the radar description word max : C max =max(C); Step 3.4: Calculate and obtain the normalized result of the i-th sample of the radar description word C component: in Represents the i-th sample after normalization of the C component.