Radar active jamming visual language joint recognition method, system, medium and equipment

Abstract

The present invention discloses a radar active jamming visual language joint recognition method, system, medium and device, including: obtaining a radar active jamming signal data set with jamming type labels; dividing the radar active jamming signal data set into a training set, a verification set and a test set; using the training set, the verification set and the test set to train, verify and test a pre-built visual language joint recognition model to obtain a target visual language joint recognition model; using the target visual language joint recognition model to test the radar active jamming signal to be tested to obtain a classification prediction probability. The present invention can effectively improve the recognition accuracy of radar jamming recognition tasks under small samples.

Description

Technical Field

[0001] The present invention relates to the field of radar jamming signal processing technology, and in particular to a radar active jamming visual language joint recognition method, system, medium and equipment. Background Art

[0002] With the continuous advancement of electronic jamming technology, especially the widespread application of digital radio frequency storage technology, radar systems are facing increasingly complex active deception jamming threats. In order to effectively deal with the impact of such interference on the normal operation of radar systems, corresponding anti-interference measures must be taken. In this process, the radar system accurately identifies and classifies interference signals and takes targeted measures to weaken or eliminate interference, which is a key link to ensure the normal operation of the radar system. Therefore, correctly identifying and processing various types of radar active deception jamming signals is of great significance for the anti-interference of radar systems in complex electromagnetic environments.

[0003] At present, many deep learning algorithms have been successfully applied in radar jamming signal recognition and have achieved high recognition accuracy. However, this method usually requires a large amount of jamming sample data to support training. However, a large amount of jamming sample data is difficult to obtain in a real complex electromagnetic confrontation environment. At the same time, most of the previous radar jamming recognition networks are limited to the use of two-dimensional time-frequency images, ignoring the different dimensions of the jamming signal such as time domain, frequency domain and statistical domain, as well as the extraction and use of multi-domain features contained in each dimension. Summary of the invention

[0004] In order to solve the technical problems existing in the background technology, the present invention proposes a radar active jammer visual language joint recognition method, system, medium and device.

[0005] The present invention proposes a radar active jammer visual language joint recognition method based on multi-domain feature guidance, comprising:

[0006] Obtain radar active jamming signal dataset with jamming type labels;

[0007] The radar active jamming signal dataset is divided into a training set, a validation set, and a test set;

[0008] The pre-built visual-language joint recognition model is trained, verified and tested using the training set, validation set and test set to obtain the target visual-language joint recognition model;

[0009] The target visual language joint recognition model is used to test the radar active jamming signal to obtain the classification prediction probability;

[0010] The visual-language joint recognition model includes a language branch, a visual branch, a visual-language alignment module and a classification head. The output of the language branch and the output of the visual branch are respectively connected to the input of the visual-language alignment module, and the output of the visual branch and the output of the visual-language alignment module are respectively connected to the input of the classification head.

[0011] The language branch includes a multi-domain feature optimization unit based on an attention mechanism and a text encoder. The multi-domain feature optimization unit based on the attention mechanism is used to obtain the effective multi-domain features corresponding to each type of interference according to the input radar active interference signal data, and the text encoder is used to obtain a text feature vector according to the effective multi-domain features corresponding to each type of interference.

[0012] The visual branch includes a short-time Fourier transform unit and an image encoder, wherein the short-time Fourier transform unit is used to obtain a time-frequency image corresponding to each type of interference according to the input radar active interference signal data, and the image encoder is used to obtain an image feature vector according to the time-frequency image corresponding to each type of interference;

[0013] The vision-language alignment module is used to map the text feature vector and the image feature vector to the same feature shared space and align them to obtain the alignment vector; the classification head is used to classify according to the image feature vector and the alignment vector to obtain the classification prediction probability.

[0014] Preferably, the multi-domain feature optimization unit based on the attention mechanism includes a feature extraction module and an attention module, wherein the feature extraction module is used to obtain multi-domain features of the radar active jamming signal in the time domain, frequency domain, time-frequency domain and statistical domain according to the input radar active jamming signal data;

[0015] The attention module is used to obtain the effective multi-domain features corresponding to each type of interference based on the multi-domain features of the radar active interference signal in time domain, frequency domain, time-frequency domain and statistical domain.

[0016] Preferably, the attention module structure includes two fully connected layers, an activation function layer and a Softmax layer.

[0017] Preferably, the text encoder adopts the Transformer structure as the backbone structure.

[0018] Preferably, the Transformer structure includes 8 attention heads, 3 residual blocks and 3 fully connected layers.

[0019] Preferably, the image encoder adopts the EfficientNetV2 structure as the backbone structure.

[0020] Preferably, the EfficientNetV2 structure includes 3 fused inverted residual convolution layers, 3 inverted residual convolution layers, a pooling layer and a fully connected layer.

[0021] Preferably, the pre-built visual-language joint recognition model is trained using the training set, specifically including:

[0022] The radar active jamming signals in the training set are input into the pre-built visual language joint recognition model in batches;

[0023] In the language branch, the multi-domain feature optimization unit based on the attention mechanism obtains the effective multi-domain features corresponding to each type of interference according to the radar active interference signal, and the text encoder obtains the text feature vector according to the effective multi-domain features corresponding to each type of interference;

[0024] In the visual branch, the short-time Fourier transform unit obtains the time-frequency image corresponding to each type of interference according to the radar active interference signal, and the image encoder obtains the image feature vector according to the time-frequency image corresponding to each type of interference;

[0025] The visual-language alignment module maps the text feature vector and the image feature vector to the same feature sharing space and aligns them to obtain an alignment vector.

[0026] The classification head performs classification based on the image feature vector and alignment vector to obtain the classification prediction probability;

[0027] According to the radar active interference signal, the text feature vector, the image feature vector and the classification prediction probability, the classification prediction loss function of the classification prediction probability and the corresponding interference type label and the alignment loss function between the image feature vector and the text feature vector are calculated respectively;

[0028] According to the classification prediction loss function and the alignment loss function, the total loss function is calculated;

[0029] The classification prediction probability is iterated through the gradient descent algorithm until the total loss function converges.

[0030] Preferably, the classification prediction loss function is

[0031]

[0032] In the formula, is the classification prediction loss function, is the cross entropy loss function, y i For x i The one-hot encoding of the label information, c is the index of the interference category, p i is the predicted probability of the classification head of the visual branch.

[0033] Preferably, the cross entropy loss function is:

[0034]

[0035] In the formula, is the cross entropy loss function, y i is the one-hot encoding of the label information, c is the index of the interference category, and p i is the predicted probability of the classification head of the visual branch.

[0036] Preferably, the alignment loss function of the image feature vector and the text feature vector is:

[0037]

[0038] In the formula, is the alignment loss function of the image feature vector and the text feature vector, λ is the hyperparameter of the balanced alignment loss function, P(i) and A(i) are the positive and negative sample sets, respectively, |P(i)| is the number of positive samples, and are one of the positive sample and negative sample respectively, v i is a single image feature vector and l i is a single text feature vector, P v (i) and A v (i) are the positive sample set and negative sample set of image feature vectors, P l (i) and A l (i) are the positive sample set and negative sample set of text feature vectors, and |P v (i)|=|P l (i)|=|P(i)|, τ is a hyperparameter, which is the temperature coefficient for adjusting the Softmax classification probability.

[0039] Preferably, the total loss function is

[0040]

[0041] In the formula, is the total loss function, is the classification prediction loss function, is the alignment loss function of the image feature vector and the text feature vector, and λ is the hyperparameter of the balanced alignment loss function.

[0042] Preferably, the radar active jamming signal data set is divided into a training set, a validation set and a test set, specifically including:

[0043] The radar active jamming signals in the radar active jamming signal data set are divided into corresponding data subsets according to corresponding jamming type labels;

[0044] Samples are randomly selected from each data subset according to a preset ratio to form the training set, validation set, and test set.

[0045] The present invention also proposes a radar active jammer visual language joint recognition system based on multi-domain feature guidance, comprising:

[0046] An acquisition module, used for acquiring a radar active interference signal data set with an interference type label;

[0047] A partitioning module, used for partitioning the radar active jamming signal data set into a training set, a validation set and a test set;

[0048] A processing module is used to train, verify and test the pre-built visual-language joint recognition model using a training set, a verification set and a test set to obtain a target visual-language joint recognition model; the visual-language joint recognition model includes a language branch, a visual branch, a visual-language alignment module and a classification head, the output end of the language branch and the output end of the visual branch are respectively connected to the input end of the visual-language alignment module, and the output end of the visual branch and the output end of the visual-language alignment module are respectively connected to the input end of the classification head;

[0049] The language branch includes a multi-domain feature optimization unit based on an attention mechanism and a text encoder. The multi-domain feature optimization unit based on the attention mechanism is used to obtain the effective multi-domain features corresponding to each type of interference according to the input radar active interference signal data, and the text encoder is used to obtain a text feature vector according to the effective multi-domain features corresponding to each type of interference.

[0050] The visual branch includes a short-time Fourier transform unit and an image encoder, wherein the short-time Fourier transform unit is used to obtain a time-frequency image corresponding to each type of interference according to the input radar active interference signal data, and the image encoder is used to obtain an image feature vector according to the time-frequency image corresponding to each type of interference;

[0051] The visual-language alignment module is used to map the text feature vector and the image feature vector to the same feature sharing space and align them to obtain the alignment vector; the classification head is used to classify according to the image feature vector and the alignment vector to obtain the classification prediction probability;

[0052] The recognition module uses the target visual language joint recognition model to test the radar active interference signal to obtain the classification prediction probability.

[0053] The present invention also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described above when executing the computer program.

[0054] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described above are implemented.

[0055] In the present invention, the proposed radar active interference visual language joint recognition method, system, medium and equipment adopt a language branch in the visual language joint recognition model to obtain effective multi-domain features corresponding to each type of interference through a multi-domain feature optimization unit based on an attention mechanism, and a text encoder obtains a text feature vector according to the effective multi-domain features corresponding to each type of interference, which is conducive to the subsequent full use of multiple modal features of a small number of interference signal samples for interference signal recognition and classification, saving the manpower and time cost of obtaining a large amount of training sample data required for the training model; moreover, the dual-stream architecture of the language branch and the visual branch respectively extracts image feature vectors and text feature items, and maps the text feature vector and the image feature vector to the same feature sharing space and in the feature sharing space, narrows the gap between text features and image features by category to achieve multi-modal alignment, and completes visual language alignment through supervised contrast learning in the feature sharing space, thereby effectively improving the recognition accuracy of radar interference recognition tasks under small samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The present invention is a flowchart of a method for joint recognition of radar active jammers based on multi-domain feature guidance and visual language in one embodiment of the present invention.

[0057] Figure 2 A schematic diagram of the structure of a visual language joint recognition model in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0059] Reference Figure 1 and Figure 2 The present invention proposes a radar active jammer visual language joint recognition method based on multi-domain feature guidance, comprising:

[0060] Obtain radar active jamming signal dataset with jamming type labels;

[0061] The radar active jamming signal dataset is divided into a training set, a validation set, and a test set;

[0062] The pre-built visual-language joint recognition model is trained, verified and tested using the training set, validation set and test set to obtain the target visual-language joint recognition model.

[0063] The target visual language joint recognition model is used to test the radar active jamming signal to obtain the classification prediction probability;

[0064] The visual-language joint recognition model includes a language branch, a visual branch, a visual-language alignment module and a classification head. The output of the language branch and the output of the visual branch are respectively connected to the input of the visual-language alignment module, and the output of the visual branch and the output of the visual-language alignment module are respectively connected to the input of the classification head.

[0065] The language branch includes a multi-domain feature optimization unit based on an attention mechanism and a text encoder. The multi-domain feature optimization unit based on the attention mechanism is used to obtain the effective multi-domain features corresponding to each type of interference according to the input radar active interference signal data, and the text encoder is used to obtain a text feature vector according to the effective multi-domain features corresponding to each type of interference.

[0066] The visual branch includes a short-time Fourier transform unit and an image encoder, wherein the short-time Fourier transform unit is used to obtain a time-frequency image corresponding to each type of interference according to the input radar active interference signal data, and the image encoder is used to obtain an image feature vector according to the time-frequency image corresponding to each type of interference;

[0067] The vision-language alignment module is used to map the text feature vector and the image feature vector to the same feature shared space and align them to obtain the alignment vector; the classification head is used to classify according to the image feature vector and the alignment vector to obtain the classification prediction probability.

[0068] The language branch in the visual-language joint recognition model adopted in the present invention obtains the effective multi-domain features corresponding to each type of interference through a multi-domain feature optimization unit based on an attention mechanism, and the text encoder obtains the text feature vector according to the effective multi-domain features corresponding to each type of interference, which is conducive to the subsequent full use of multiple modal features of a small number of interference signal samples for interference signal recognition and classification, saving the manpower and time costs spent on obtaining a large amount of training sample data required for the training model; moreover, the dual-stream architecture of the language branch and the visual branch extracts the image feature vector and the text feature vector respectively, and maps the text feature vector and the image feature vector to the same feature sharing space and in the feature sharing space, narrows the gap between the text feature vector and the image feature vector by category to achieve multi-modal alignment, and completes the visual language alignment through supervised contrast learning in the feature sharing space, thereby effectively improving the recognition accuracy of the radar interference recognition task under small samples.

[0069] In order to use multiple modal features of a small number of interference signal samples to identify and classify interference signals, in this embodiment, the multi-domain feature optimization unit based on the attention mechanism includes a feature extraction module and an attention module. The feature extraction module is used to obtain multi-domain features of the radar active interference signal in the time domain, frequency domain, time-frequency domain and statistical domain according to the input radar active interference signal data;

[0070] The attention module is used to obtain the effective multi-domain features corresponding to each type of interference based on the multi-domain features of the radar active interference signal in time domain, frequency domain, time-frequency domain and statistical domain.

[0071] Specifically, the attention module structure includes two fully connected layers, an activation function layer and a Softmax layer.

[0072] Specifically, the feature extraction module obtains the multi-domain feature value corresponding to each type of interference by writing the corresponding multi-domain feature calculation formula code, and forms a data table with i rows and j columns, where each row of the table represents different multi-domain feature values ​​of a certain type of interference, and each column of the table represents the feature value of different interference signals under a certain multi-domain feature. Among them, the i row indicates that there are i types of radar active interference signals in the current embodiment, and the j column indicates that there are j multi-domain features in the current embodiment.

[0073] It should be noted that the definition of an effective multi-domain feature is that when a multi-domain feature of a type of interference signal is the maximum value of all multi-domain features and this maximum value is also the maximum value among all multi-domain features of the interference, it is recognized as an effective multi-domain feature of the current interference signal.

[0074] Specifically, define The row dimension representation of the data table of the input attention module for the i-th row and j-th column record, and then the row dimension attention score is obtained by paying attention to the records in the same row

[0075] Similarly, if is the Levi representation of the record in the i-th row and j-th column, and then the Levi attention score is obtained by paying attention to the records in the same column

[0076] Then multiply the row-dimensional attention with the column-dimensional attention to obtain the attention score of all input interference types on each multi-domain feature. i,j =score row ×score col ;

[0077] Finally, the multi-domain features are screened by the size of the attention score, and the effective multi-domain features corresponding to a certain type of interference are automatically selected.

[0078] In this embodiment, the text encoder adopts the Transformer structure as the backbone structure.

[0079] In one specific embodiment, the Transformer structure includes 8 attention heads, 3 residual blocks and 3 fully connected layers.

[0080] In this embodiment, the image encoder adopts the EfficientNetV2 structure as the backbone structure.

[0081] In one specific embodiment, the EfficientNetV2 structure includes 3 fused inverted residual convolution layers, 3 inverted residual convolution layers, a pooling layer and a fully connected layer.

[0082] In order to obtain the target visual-language joint recognition model, in this embodiment, the pre-built visual-language joint recognition model is trained using the training set, specifically including:

[0083] The radar active jamming signals in the training set are input into the pre-built visual language joint recognition model in batches;

[0084] In the language branch, the multi-domain feature optimization unit based on the attention mechanism obtains the effective multi-domain features corresponding to each type of interference according to the radar active interference signal, and the text encoder obtains the text feature vector according to the effective multi-domain features corresponding to each type of interference;

[0085] In the visual branch, the short-time Fourier transform unit obtains the time-frequency image corresponding to each type of interference according to the radar active interference signal, and the image encoder obtains the image feature vector according to the time-frequency image corresponding to each type of interference;

[0086] The visual-language alignment module maps the text feature vector and the image feature vector to the same feature sharing space and aligns them to obtain an alignment vector.

[0087] The classification head performs classification based on the image feature vector and alignment vector to obtain the classification prediction probability;

[0088] According to the radar active interference signal, the text feature vector, the image feature vector and the classification prediction probability, the classification prediction loss function of the classification prediction probability and the corresponding interference type label and the alignment loss function between the image feature vector and the text feature vector are calculated respectively;

[0089] According to the classification prediction loss function and the alignment loss function, the total loss function is calculated;

[0090] The classification prediction probability is iterated through the gradient descent algorithm until the total loss function converges.

[0091] Among them, the classification prediction loss function is

[0092]

[0093] In the formula, is the classification prediction loss function, is the cross entropy loss function, y i For x i The one-hot encoding of the label information, c is the index of the interference category, pi is the predicted probability of the classification head of the visual branch.

[0094] Among them, the cross entropy loss function is:

[0095]

[0096] In the formula, is the cross entropy loss function, y i is the one-hot encoding of the label information, c is the index of the interference category, and p i is the predicted probability of the classification head of the visual branch.

[0097] Among them, the alignment loss function of the image feature vector and the text feature vector is:

[0098]

[0099] In the formula, is the alignment loss function of the image feature vector and the text feature vector, λ is the hyperparameter of the balanced alignment loss function, P(i) and A(i) are the positive and negative sample sets, respectively, |P(i)| is the number of positive samples, and are one of the positive sample and negative sample respectively, v i is a single image feature vector and l i is a single text feature vector, P v (i) and A v (i) are the positive sample set and negative sample set of image feature vectors, P l (i) and A l (i) are the positive sample set and negative sample set of text feature vectors, and |P v (i)|=|P l (i)|=|P(i)|, τ is a hyperparameter, which is the temperature coefficient for adjusting the Softmax classification probability.

[0100] Among them, the total loss function is

[0101]

[0102] In the formula, is the total loss function, is the classification prediction loss function, is the alignment loss function of the image feature vector and the text feature vector, and λ is the hyperparameter of the balanced alignment loss function.

[0103] In this embodiment, the target visual language joint recognition model is used to test the radar active interference signal to obtain the classification prediction probability, which specifically includes:

[0104] Input the radar active jamming signal to be tested into the target vision and language joint recognition model;

[0105] The short-time Fourier transform unit obtains the time-frequency image corresponding to each type of interference according to the active interference signal of the radar to be tested;

[0106] The image encoder obtains an image feature vector according to the time-frequency image corresponding to each type of interference;

[0107] The classification head outputs the classification prediction probability based on the image feature vector.

[0108] In this embodiment, the radar active jamming signal data set is divided into a training set, a validation set, and a test set, specifically including:

[0109] The radar active jamming signals in the radar active jamming signal data set are divided into corresponding data subsets according to corresponding jamming type labels;

[0110] Samples are randomly selected from each data subset according to a preset ratio to form the training set, validation set, and test set.

[0111] Specifically, the radar active jamming signal data set includes 10 categories of radar active jamming signals, including 7 single-type deception jamming signals and 3 compound-type jamming signals, with 200 items in each category, totaling 1,000 items of radar active jamming signal data.

[0112] Specifically, the ratio of training set, validation set and test set is 0.25:0.75:9.

[0113] In this embodiment, the interference type labels specifically include: distance deception interference label DDJ, coherent suppression interference label CSJ, spectrum dispersion interference label SMSP, intermittent sampling direct forwarding interference label ISDJ, intermittent sampling repeated forwarding interference label ISRJ, intermittent sampling cyclic forwarding interference label ISLJ, distance deception interference and intermittent sampling repeated forwarding interference mixed label DDJ+ISRJ, intermittent sampling cyclic forwarding interference and coherent suppression interference mixed label ISRJ+CSJ, and intermittent sampling cyclic forwarding interference and spectrum dispersion interference mixed label ISRJ+SMSP.

[0114] The present invention also proposes a radar active jammer visual language joint recognition system based on multi-domain feature guidance, comprising:

[0115] An acquisition module, used for acquiring a radar active interference signal data set with an interference type label;

[0116] A partitioning module, used for partitioning the radar active jamming signal data set into a training set, a validation set and a test set;

[0117] A processing module, used for training, verifying and testing the pre-built visual-language joint recognition model using a training set, a verification set and a test set to obtain a target visual-language joint recognition model;

[0118] The recognition module uses the target visual language joint recognition model to test the radar active interference signal to obtain the classification prediction probability;

[0119] The visual-language joint recognition model includes a language branch, a visual branch, a visual-language alignment module and a classification head. The output of the language branch and the output of the visual branch are respectively connected to the input of the visual-language alignment module, and the output of the visual branch and the output of the visual-language alignment module are respectively connected to the input of the classification head.

[0120] The language branch includes a multi-domain feature optimization unit based on an attention mechanism and a text encoder. The multi-domain feature optimization unit based on the attention mechanism is used to obtain the effective multi-domain features corresponding to each type of interference according to the input radar active interference signal data, and the text encoder is used to obtain a text feature vector according to the effective multi-domain features corresponding to each type of interference.

[0121] The visual branch includes a short-time Fourier transform unit and an image encoder, wherein the short-time Fourier transform unit is used to obtain a time-frequency image corresponding to each type of interference according to the input radar active interference signal data, and the image encoder is used to obtain an image feature vector according to the time-frequency image corresponding to each type of interference;

[0122] The vision-language alignment module is used to map the text feature vector and the image feature vector to the same feature shared space and align them to obtain the alignment vector; the classification head is used to classify according to the image feature vector and the alignment vector to obtain the classification prediction probability.

[0123] In this embodiment, the multi-domain feature optimization unit based on the attention mechanism includes a feature extraction module and an attention module, and the feature extraction module is used to obtain multi-domain features of the radar active interference signal in the time domain, frequency domain, time-frequency domain and statistical domain according to the input radar active interference signal data;

[0124] The attention module is used to obtain the effective multi-domain features corresponding to each type of interference based on the multi-domain features of the radar active interference signal in time domain, frequency domain, time-frequency domain and statistical domain.

[0125] In this embodiment, the attention module structure includes two fully connected layers, an activation function layer and a Softmax layer.

[0126] In this embodiment, the text encoder adopts the Transformer structure as the backbone structure.

[0127] In this embodiment, the Transformer structure includes 8 attention heads, 3 residual blocks and 3 fully connected layers.

[0128] In this embodiment, the image encoder adopts the EfficientNetV2 structure as the backbone structure.

[0129] In this embodiment, the EfficientNetV2 structure includes 3 fused inverted residual convolution layers, 3 inverted residual convolution layers, a pooling layer and a fully connected layer.

[0130] The present invention also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described above when executing the computer program.

[0131] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described above are implemented.

[0132] The effects of the present invention can be further illustrated by the following comparative examples and embodiments.

[0133] Example 1

[0134] Under the conditions that the training sample size is 2, 3, 4, and 5, the radar active jammer visual language joint recognition method based on multi-domain feature guidance proposed in the present invention is used to identify the samples to be tested, and the results are shown in Table 2.

[0135] Comparative Example 1

[0136] When the number of training samples is 2, 3, 4, and 5, the model trained, verified, and tested using only the visual branch in the present invention is used to identify the samples to be tested. The results are shown in Table 2.

[0137] Comparative Example 2

[0138] Under the condition that the number of training samples for each type of interference signal is 4, the existing support vector machine (SVM), two-dimensional convolutional neural network (2D-CNN), deep fusion model (DFM), small sample interference recognition network based on time-frequency self-attention and global knowledge distillation (JR-TFSAD) and model-independent meta-learning method (MAML) are used to identify the test samples, and the results are shown in Table 3.

[0139] It should be noted that this embodiment and all comparative examples use the same data set for training, verification and testing, and the visual encoder used uniformly uses EfficientNetV2 as the backbone structure. The model is uniformly trained using a stochastic gradient descent optimizer for 350 rounds with a batch size of 32. All experiments were conducted on a server with an Nvidia GeForceRTX3060 graphics card and CUDA 11.1 installed.

[0140] This embodiment selects accuracy (Accuracy), precision (Precision), F1 score (F1), recall (Recall) and Kappa value (Kappa) to measure the performance of various methods in active interference identification.

[0141] Specifically, F1 is the harmonic mean of precision and recall, representing a measure of the overall performance of the classification head. Its value is between 0 and 1. The closer it is to 1, the better the performance of the classification head. Kappa is a measure of classification accuracy, usually between 0 and 1. If Kappa is between 0.61 and 0.8, the classification result is considered to be highly consistent with the actual class. If Kappa is between 0.81 and 1, the classification result is considered to be basically consistent with the actual class. The K-class multi-classification problem can be regarded as K binary classification problems, and the confusion matrix is ​​shown in Table 1.

[0142] Table 1 Binary classification confusion matrix representation

[0143]

[0144] Therefore, the evaluation index can be expressed as

[0145]

[0146]

[0147]

[0148]

[0149]

[0150] Among them, p o is the overall classification accuracy, Define the sample size of the kth class as a k (k=1,2,...,K), the predicted value of the sample size of the kth class is b k (k=1,2,...,K).

[0151] Table 2 Comparison of evaluation indicators of classification results using the visual language model and using only the visual branch

[0152]

[0153] As can be seen from Table 2, the radar active jamming visual language joint recognition method based on multi-domain feature guidance proposed in the present invention can make full use of the multimodal information of small sample jamming training data to classify radar active jamming signals, and the best results of each data are highlighted in bold. Compared with classification using only visual features, due to the addition of radar active jamming text feature vectors, the present invention has greatly improved the five evaluation indicators of accuracy, precision, F1 score, recall rate and Kappa value, which proves that the introduction of text feature vectors greatly improves the recognition and classification capabilities of radar active jamming signals under small samples compared to the use of single image feature vectors, and proves the necessity and effectiveness of the introduction of text feature vectors.

[0154] Table 3 Evaluation indicators of various recognition methods when the number of training samples for each type of interference signal is 4

[0155] Identification method SVM 2D-CNN DFM JR-TFSAD MAML PP-RVLM(ours) Accuracy 78.31％ 89.69％ 86.12％ 93.42％ 94.92％ 99.94％ Precision 78.31％ 89.69％ 86.12％ 93.42％ 94.92％ 99.94％ Recall 78.31％ 89.69％ 86.12％ 93.42％ 94.92％ 99.94％ F1 78.31％ 89.69％ 86.12％ 93.42％ 94.92％ 99.94％ Kappa 75.90％ 88.55％ 84.58％ 92.69％ 93.98％ 99.93％

[0156] It can be seen from Table 3 that the method proposed in the present application achieves the highest scores in all evaluation indicators.

[0157] In summary, under the same experimental conditions, compared with support vector machine (SVM), two-dimensional convolutional neural network (2D-CNN), deep fusion model (DFM), small sample interference recognition network based on time-frequency self-attention and global knowledge distillation (JR-TFSAD) and model-independent meta-learning method (MAML), the radar active interference visual language joint recognition method based on multi-domain feature guidance proposed in the present invention can significantly improve the accuracy of radar active interference signal recognition and classification under small samples. It can be seen that the method proposed in the present invention has better recognition effect than previous radar interference recognition methods when there are very few samples.

[0158] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A radar active jammer visual language joint recognition method based on multi-domain feature guidance, characterized in that: include: Obtain radar active jamming signal dataset with jamming type labels; The radar active jamming signal dataset is divided into a training set, a validation set, and a test set; The pre-built visual-language joint recognition model is trained, verified and tested using the training set, validation set and test set to obtain the target visual-language joint recognition model; The target visual language joint recognition model is used to test the radar active jamming signal to obtain the classification prediction probability; The visual-language joint recognition model includes a language branch, a visual branch, a visual-language alignment module and a classification head. The output of the language branch and the output of the visual branch are respectively connected to the input of the visual-language alignment module, and the output of the visual branch and the output of the visual-language alignment module are respectively connected to the input of the classification head. The language branch includes a multi-domain feature optimization unit based on an attention mechanism and a text encoder. The multi-domain feature optimization unit based on the attention mechanism is used to obtain the effective multi-domain features corresponding to each type of interference according to the input radar active interference signal data, and the text encoder is used to obtain a text feature vector according to the effective multi-domain features corresponding to each type of interference. The visual branch includes a short-time Fourier transform unit and an image encoder, wherein the short-time Fourier transform unit is used to obtain a time-frequency image corresponding to each type of interference according to the input radar active interference signal data, and the image encoder is used to obtain an image feature vector according to the time-frequency image corresponding to each type of interference; The visual-language alignment module is used to map the text feature vector and the image feature vector to the same feature sharing space and align them to obtain the alignment vector; the classification head is used to classify according to the image feature vector and the alignment vector to obtain the classification prediction probability; The training set is used to train the pre-built visual language joint recognition model, including: The radar active jamming signals in the training set are input into the pre-built visual language joint recognition model in batches; In the language branch, the multi-domain feature optimization unit based on the attention mechanism obtains the effective multi-domain features corresponding to each type of interference according to the radar active interference signal, and the text encoder obtains the text feature vector according to the effective multi-domain features corresponding to each type of interference; In the visual branch, the short-time Fourier transform unit obtains the time-frequency image corresponding to each type of interference according to the radar active interference signal, and the image encoder obtains the image feature vector according to the time-frequency image corresponding to each type of interference; The visual-language alignment module maps the text feature vector and the image feature vector to the same feature sharing space and aligns them to obtain an alignment vector. The classification head performs classification based on the image feature vector and alignment vector to obtain the classification prediction probability; According to the radar active interference signal, the text feature vector, the image feature vector and the classification prediction probability, the classification prediction loss function of the classification prediction probability and the corresponding interference type label and the alignment loss function between the image feature vector and the text feature vector are calculated respectively; According to the classification prediction loss function and the alignment loss function, the total loss function is calculated; The classification prediction probability is iterated through the gradient descent algorithm until the total loss function converges.

2. According to claim 1, the radar active jammer visual language joint recognition method based on multi-domain feature guidance is characterized in that: The multi-domain feature optimization unit based on the attention mechanism includes a feature extraction module and an attention module, wherein the feature extraction module is used to obtain multi-domain features of the radar active jamming signal in the time domain, frequency domain, time-frequency domain and statistical domain according to the input radar active jamming signal data; The attention module is used to obtain the effective multi-domain features corresponding to each type of interference based on the multi-domain features of the radar active interference signal in time domain, frequency domain, time-frequency domain and statistical domain.

3. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 2 is characterized in that: The attention module structure consists of two fully connected layers, an activation function layer, and a Softmax layer.

4. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 1 is characterized in that: The text encoder uses the Transformer structure as the backbone structure.

5. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 4 is characterized in that: The Transformer structure consists of 8 attention heads, 3 residual blocks, and 3 fully connected layers.

6. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 1 is characterized in that: The image encoder uses the EfficientNetV2 structure as the backbone structure.

7. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 6 is characterized in that: The EfficientNetV2 structure consists of 3 fused inverted residual convolutional layers, 3 inverted residual convolutional layers, a pooling layer and a fully connected layer.

8. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 1 is characterized in that: The total loss function is In the formula, is the total loss function, is the classification prediction loss function, is the alignment loss function of the image feature vector and the text feature vector, and λ is the hyperparameter of the balanced alignment loss function.

9. The radar active jammer visual language joint recognition method based on multi-domain feature guidance according to claim 1 is characterized in that: The radar active jamming signal dataset is divided into training set, validation set and test set, including: The radar active jamming signals in the radar active jamming signal data set are divided into corresponding data subsets according to corresponding jamming type labels; Samples are randomly selected from each data subset according to a preset ratio to form the training set, validation set, and test set.

10. A radar active jammer visual language joint recognition system based on multi-domain feature guidance, characterized in that: include: An acquisition module, used for acquiring a radar active interference signal data set with an interference type label; A partitioning module, used for partitioning the radar active jamming signal data set into a training set, a validation set and a test set; A processing module is used to train, verify and test the pre-built visual-language joint recognition model using a training set, a verification set and a test set to obtain a target visual-language joint recognition model; the visual-language joint recognition model includes a language branch, a visual branch, a visual-language alignment module and a classification head, the output end of the language branch and the output end of the visual branch are respectively connected to the input end of the visual-language alignment module, and the output end of the visual branch and the output end of the visual-language alignment module are respectively connected to the input end of the classification head; The language branch includes a multi-domain feature optimization unit based on an attention mechanism and a text encoder. The multi-domain feature optimization unit based on the attention mechanism is used to obtain the effective multi-domain features corresponding to each type of interference according to the input radar active interference signal data, and the text encoder is used to obtain a text feature vector according to the effective multi-domain features corresponding to each type of interference. The visual branch includes a short-time Fourier transform unit and an image encoder, wherein the short-time Fourier transform unit is used to obtain a time-frequency image corresponding to each type of interference according to the input radar active interference signal data, and the image encoder is used to obtain an image feature vector according to the time-frequency image corresponding to each type of interference; The visual-language alignment module is used to map the text feature vector and the image feature vector to the same feature sharing space and align them to obtain the alignment vector; the classification head is used to classify according to the image feature vector and the alignment vector to obtain the classification prediction probability; The recognition module uses the target visual language joint recognition model to test the radar active interference signal to obtain the classification prediction probability; The training process of the processing module using the training set to train the pre-built visual language joint recognition model specifically includes: The radar active jamming signals in the training set are input into the pre-built visual language joint recognition model in batches; In the language branch, the multi-domain feature optimization unit based on the attention mechanism obtains the effective multi-domain features corresponding to each type of interference according to the radar active interference signal, and the text encoder obtains the text feature vector according to the effective multi-domain features corresponding to each type of interference; In the visual branch, the short-time Fourier transform unit obtains the time-frequency image corresponding to each type of interference according to the radar active interference signal, and the image encoder obtains the image feature vector according to the time-frequency image corresponding to each type of interference; The visual-language alignment module maps the text feature vector and the image feature vector to the same feature sharing space and aligns them to obtain an alignment vector. The classification head performs classification based on the image feature vector and alignment vector to obtain the classification prediction probability; According to the radar active interference signal, the text feature vector, the image feature vector and the classification prediction probability, the classification prediction loss function of the classification prediction probability and the corresponding interference type label and the alignment loss function between the image feature vector and the text feature vector are calculated respectively; According to the classification prediction loss function and the alignment loss function, the total loss function is calculated; The classification prediction probability is iterated through the gradient descent algorithm until the total loss function converges.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images