Intelligent recognition method for ultra-short wave signal system based on masked autoencoder

Abstract

The application discloses an ultra-short wave signal system intelligent recognition method based on a Masked autoencoder, which comprises the following steps: pre-processing ultra-short wave broadband data according to signal output time, center frequency and bandwidth, performing short-time Fourier transform parameter extraction on the ultra-short wave narrowband time-frequency characteristics, and constructing an unsupervised learning data set; a random mask mechanism is redesigned for the ultra-short wave signal system recognition task, and a Masked autoencoder is constructed to pre-train in a self-supervised learning mode on a large-scale ultra-short wave narrowband time-frequency diagram data set. A multi-task learning module is designed around the basic structure of the pre-training model, and reconstruction learning and classification learning are performed on the labeled specific system ultra-short wave signal time-frequency diagram, so that the problem of poor model generalization caused by insufficient ultra-short wave signal samples and uneven distribution is solved. The method of large-scale self-supervised learning pre-training and small-scale supervised learning fine-tuning is adopted to realize the specific system ultra-short wave signal recognition problem in a complex electromagnetic environment.

Description

Technical Field

[0001] This invention belongs to the field of space target recognition technology, specifically relating to an intelligent recognition method for ultra-shortwave signal systems based on a Masked autoencoder. Background Technology

[0002] Ultra-shortwave (UHF) communication, operating in the 30MHz-300MHz frequency band, is widely used in tactical communications due to its wide bandwidth, broad coverage, and strong penetration, making it one of the most commonly used communication methods in the current military field. The current battlefield electromagnetic environment is complex, filled with a large amount of electronic interference and noise, and includes conventional signals, frequency hopping, spread spectrum, and radio signals within the UHF band. Therefore, accurately identifying the UHF signal system from intercepted signals is crucial and a vital means of intelligence gathering. Precisely identifying the opponent's UHF communication signal system can effectively prevent information leakage and provide important tactical guidance for counterattacks.

[0003] With the increasing variety of VHF / UHF reconnaissance equipment, the traditional manual analysis and comparison methods for operating and maintaining reconnaissance stations are costly and inefficient. Therefore, it is necessary to leverage the development of intelligent technologies and research AI-based VHF / UHF reconnaissance methods to effectively improve the accuracy and timeliness of communication reconnaissance. Patent application number 201811159958.X provides a method for identifying specific VHF / UHF signals based on convolutional neural networks. This method combines the time-frequency spectrum of VHF / UHF signals with a convolutional neural network, using the obtained signal time-frequency spectrum to train an optimized convolutional neural network model, ultimately achieving the identification of specific VHF / UHF signals. In their paper "A Classification Method for VHF / UHF Time-Frequency Graphs Based on Improved VGG16," Ma Boang et al. from the 54th Research Institute of China Electronics Technology Group Corporation (CETC) proposed converting actual VHF / UHF blind signals collected in the electromagnetic battlefield into time-frequency spectra. Then, through transfer learning, they combined these spectra with an optimized VGG16 convolutional neural network and introduced dilated convolution into the network to complete the classification of VHF / UHF blind signals. The methods described above all use supervised learning to identify specific VHF signals. However, in real-world scenarios, the number of military VHF signals intercepted is small and interference is severe, making it difficult to construct a sufficiently large and standardized VHF signal system identification dataset, resulting in insufficient generalization of VHF system identification algorithms. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an intelligent identification method for UHF signal regimes based on a Masked autoencoder. By constructing a Masked Encoder model, unsupervised learning is performed using a large-scale unlabeled UHF communication signal time-frequency graph dataset. Then, fine-tuning is performed on the time-frequency graph of a specific UHF narrowband signal with labeled information. Reconstruction learning is used to assist in UHF regime identification, thereby improving the model's generalization ability and achieving intelligent identification of specific UHF signals under low signal-to-noise ratio conditions.

[0005] The objective of this invention is achieved through the following technical solution: an intelligent identification method for UHF signal systems based on a Masked autoencoder, comprising the following steps:

[0006] S1. Preprocess the signal according to the transmission time, center frequency and bandwidth of the intercepted VHF signal. The preprocessed VHF narrowband signal must contain the contents of the signal's synchronization frame and part of the data frame.

[0007] Then, a short-time Fourier transform is performed on the preprocessed ultra-shortwave narrowband signal to construct an ultra-shortwave narrowband signal time-frequency graph dataset for unsupervised learning.

[0008] S2. Construct a Masked AutoEncoder intelligent recognition model for ultra-shortwave systems and perform unsupervised learning pre-training using an ultra-shortwave narrowband signal time-frequency graph dataset; the Masked AutoEncoder intelligent recognition model for ultra-shortwave systems includes a random mask layer, an encoding layer, a decoding layer, and a classification output layer.

[0009] S3. Based on the known VHF signals, perform preprocessing and time-frequency transformation according to the process in step S1 to construct a VHF narrowband signal time-frequency map dataset for supervised learning.

[0010] S4. Fine-tune the pre-trained intelligent recognition model of the ultra-shortwave system and train it using the dataset constructed in step S3.

[0011] S5. Use the trained Masked AutoEncoder UHF mode intelligent recognition model to perform mode recognition on specific UHF communication signals.

[0012] The beneficial effects of this invention are as follows: First, this invention uses time-frequency analysis to transform the problem of IQ signal system identification in communication into an image recognition problem based on deep learning. It solves the problem of insufficient effective samples for specific UHF signal systems under actual conditions through unsupervised learning, thereby improving the utilization rate of unlabeled signal data in real battlefield environments. It achieves intelligent identification of specific UHF signal systems by using a method of large-scale unsupervised pre-training combined with small-scale supervised fine-tuning. This effectively solves the common problem of poor generalization of intelligent algorithms in the field of communication reconnaissance and has strong engineering and practical value. Attached Figure Description

[0013] Figure 1 This is a flowchart of an intelligent identification method for UHF signal modes based on a Masked autoencoder, according to the present invention.

[0014] Figure 2 This is a narrowband time-frequency diagram of a navigation signal in the VHF band in this embodiment;

[0015] Figure 3 This is a schematic diagram of the Masked AutoEncoder model architecture used for unsupervised learning in this embodiment;

[0016] Figure 4 This is a schematic diagram of the narrowband time-frequency diagram of a navigation signal in the VHF band after random masking processing in this embodiment;

[0017] Figure 5 This is a schematic diagram of the bidirectional Transformer module structure in this embodiment;

[0018] Figure 6 This is a schematic diagram of the Masked AutoEncoder model architecture used for supervised learning in this embodiment. Detailed Implementation

[0019] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0020] like Figure 1 As shown, the present invention provides an intelligent identification method for UHF signal systems based on a Masked autoencoder, comprising the following steps:

[0021] S1. Obtain the transmission time, center frequency, and bandwidth of the intercepted VHF signal. Preprocess the signal based on the transmission time, center frequency, and bandwidth of the intercepted VHF signal. The preprocessed VHF narrowband signal must contain the content of the synchronization frame and part of the data frame. Preprocessing includes time-domain truncation, mixing, and down-conversion. First, truncate the original signal in the time domain according to the start time of the synchronization frame and the end time of the data frame. Then, mix the signal according to the center frequency of the target signal to ensure that the signal is at zero frequency. Finally, down-convert the signal according to the downsampling factor.

[0022] In this example, a communication signal receiving device is used to acquire IQ data for broadband communication in the ultra-shortwave band.

[0023] S = {S1,S2,S3,…S} i ,…,S n}, where n is the total number of sampling points within a certain sampling time range, the sampling rate is fs, and the center frequency is F. c After digital channelization and signal detection, the output times of the complete target signal are t1 and t2, the center frequency is fc, and the bandwidth is bw.

[0024] First, the IQ data of the ultra-shortwave signal is truncated in the time domain based on the output time of the target signal. The starting and ending sampling points of the truncated data are respectively... and Then, the frequency difference |Fc-fc| between the center frequency of the broadband signal and the center frequency of the target signal is calculated. The frequency of the target signal is shifted to zero frequency through mixing technology. Finally, the optimal downsampling factor is calculated to construct a low-pass filter to achieve downconversion and obtain a standard ultra-shortwave narrowband IQ signal.

[0025] Then, a short-time Fourier transform is performed on the preprocessed VHF narrowband signal to construct a VHF narrowband signal time-frequency graph dataset for unsupervised learning. Specifically, for the preprocessed VHF narrowband IQ signal, an appropriate window function, window length, and step size are selected for short-time Fourier transform to extract the signal's time-frequency graph features, such as... Figure 2 The image shows the time-frequency diagram of a navigation signal in the VHF band. The Short-Time Fourier Transform (STFT) is a key analytical tool in signal processing, prized for its localization properties. Traditional Fourier Transforms, when processing the entire signal, cannot distinguish the frequency characteristics of the signal at different time points. However, STFT, by setting a short-time window, can intuitively display the trend and characteristics of signal parameter changes. It can characterize different modulation schemes, synchronization frame patterns, time slots, bandwidths, and other parameters of different VHF signals through image texture, edge shape, color gamut distribution, etc., making it suitable for the analysis of signals with non-stationary characteristics or multiple frequency components.

[0026] S2. Construct a Masked AutoEncoder intelligent recognition model for ultra-shortwave systems and perform unsupervised learning pre-training using an ultra-shortwave narrowband signal time-frequency graph dataset;

[0027] An autoencoder (AE) is a special type of neural network architecture used for non-linear compression, reconstruction, and learning of latent representations of data. The core idea is to use a neural network model to attempt to reconstruct its input and train itself using only this information. An autoencoder typically consists of two parts: an encoder and a decoder. The encoder compresses high-dimensional data into a lower-dimensional hidden representation space through a series of non-linear transformations. During this process, the encoder captures key features of the input data and expresses them in a concise form; therefore, the final output layer of the encoder is called a "latent vector," which is much smaller than the original input size. The decoder then restores the latent vector to the original dimensional space, reconstructing the original input data as accurately as possible. The decoding process is a mapping from low to high dimensions, which to some extent simulates the behavior of generative models. The training objective of an autoencoder is to minimize the reconstruction error, i.e., the distance or similarity measure (such as mean squared error, cross-entropy, etc.) between the input data and the decoder output data. During training, the network attempts to find a way to preserve key information during compression into a latent vector, so that when it is passed to the decoder, the decoder can reconstruct the original input as accurately as possible. In summary, autoencoders, as a tool for unsupervised learning, primarily aim to extract effective feature representations from raw inputs without relying on labels, and then verify the applicability of these features through the decoding process. Through training, we can discover latent structures, reduce data complexity, and improve the performance of subsequent supervised tasks.

[0028] Therefore, this invention proposes to construct a Masked AutoEncoder model for identifying specific UHF / UHF regime signals. It utilizes the idea of ​​reconstruction learning in unsupervised learning, training a reliable pre-trained model using a large-scale dataset of unlabeled UHF / UHF regime signal time-frequency maps. Then, the encoding layer of the pre-trained Masked AutoEncoder model is used as a feature extractor, connected to a fully connected layer as the output classification layer to construct a specific UHF / UHF regime recognition model. This model is trained using a labeled dataset of specific UHF / UHF regime signal time-frequency maps, overcoming the overfitting problem caused by insufficient sample size of specific UHF / UHF regime signals in real electromagnetic environments, and achieving recognition of specific UHF / UHF regime signals under low signal-to-noise ratio conditions. The Masked AutoEncoder intelligent UHF / UHF regime recognition model of this invention sequentially includes a random masking layer, an encoding layer, a decoding layer, and a classification output layer, as shown in the figure. Figure 3 As shown; this invention optimizes the random mask layer and the coding layer, as detailed below:

[0029] (1) The random masking layer in the Masked AutoEncoder model of this invention borrows from and optimizes the original masking mechanism. Since the discriminative information of different UHF systems is reflected in the shape of the synchronization frame and the texture of the data frame in the time-frequency diagram, random masking with a fixed probability in a single pass is likely to obscure key image regions of the synchronization frame and data frame, causing the AutoEncoder model to be unable to effectively reconstruct and learn the signal, severely affecting the model's convergence speed and reconstruction accuracy. Therefore, this invention first uniformly divides the UHF narrowband time-frequency diagram into multiple pixel blocks in the horizontal and vertical directions, then randomly masks the divided pixel blocks according to three probabilities: 0.25, 0.5, and 0.75. Finally, it sequentially extracts the unmasked portions from the pixel blocks processed according to the three random masking probabilities in the original UHF narrowband time-frequency diagram, randomly shuffles them, resizes them in the image height dimension, splices them in the channel dimension, and sends them to the encoding layer for feature extraction and compression, such as... Figure 3 The random masking mechanism is shown in the figure. In this embodiment, the original image size input to the random masking layer is 224*224, the number of mask blocks in the random masking layer is 8*8, and the size of a single mask block is 28*28. The random masking layer divides the 224*224 time-frequency map into 8*8 pixel blocks in the horizontal and vertical directions, and the size of each block is 28*28 (224 / 8=28). The schematic diagram of the random masking process of a narrowband time-frequency map of a navigation signal in the VHF band in this embodiment is shown in the figure. Figure 4 As shown. This partitioning method ensures that the masking operation covers key areas in the time-frequency map (such as the textures of synchronization frames and data frames) while reducing computational redundancy. Using a random masking layer reduces redundant information in the UHF narrowband time-frequency map by learning only non-masked pixel blocks, reducing computation by up to 75%. It also addresses the problem that traditional convolutional neural networks can only learn local features of data, forcing the model to learn global features of the UHF narrowband time-frequency map and improving the model's reconstruction learning ability for UHF signal images.

[0030] (2) The coding layer includes multiple cascaded bidirectional Transformer modules, each of which includes an input layer, positional encoding, multi-head self-attention mechanism and feedforward neural network in sequence.

[0031] 1) The input layer includes two processes: word embedding and character embedding. Word embedding maps words to a high-dimensional vector space to capture their semantic information; character embedding encodes each character within a word. In VHF signal processing, pixel blocks in the time-frequency map are considered "words," and individual pixels or local features are considered "characters." Word embedding maps each pixel block to a high-dimensional vector, while character embedding captures local details within the pixel block (such as texture and edges).

[0032] 2) Positional encoding, used to add positional codes to the input word embeddings; since the Transformer does not contain a loop structure, positional encoding needs to be added to the input word embeddings to introduce sequence information into the model. Common methods include absolute positional encoding and relative positional encoding, which represent the absolute position of a word relative to the entire sequence or its distance relative to the surrounding words, respectively.

[0033] 3) Multi-Head Self-Attention Mechanism: The multi-head self-attention mechanism processes time-frequency map pixel blocks in two directions: a forward sequence (from left to right) and a reverse sequence (from right to left), obtaining forward and reverse feature vectors respectively. Forward processing captures the signal evolution features in the time dimension, while reverse processing captures the symmetry features in the frequency dimension. The feature vectors from the two directions are fused through the multi-head self-attention mechanism. By concatenating the outputs from both directions through a bidirectional mechanism, the model's ability to express global features of the time-frequency map can be enhanced. In the multi-head self-attention mechanism, the output of each head is calculated through a scaling point attention mechanism, and finally concatenated into a complete feature through a linear transformation (Output weights). To improve the model's learning ability for global features, this invention uses a two-directional self-attention mechanism to capture the dependencies between the feature vectors of the pixel blocks in the UHF signal time-frequency map. The structure of the self-attention mechanism is as follows: Figure 5 As shown. In this embodiment, the encoder is composed of six identical bidirectional Transformer modules stacked together.

[0034] Multi-head self-attention mechanisms divide the input feature vector into multiple "heads" (or channels), each processing features from both the forward and reverse directions simultaneously. An attention score is calculated for each direction, and these scores are then weighted and summed through a scaling-point attention layer to obtain a new vector representation. This design allows for simultaneous attention to information at different scales. The outputs of all heads are then concatenated at the concatenation layer to form a feature representation that integrates bidirectional information. Finally, this feature representation is mapped to the target dimension through a linear transformation matrix of the self-attention output.

[0035] Each head is transformed from three matrices: Query, Key, and Value. Similarity is obtained by calculating the dot product of Query and Key and dividing by sqrt(d). d is the dimension of the feature vector (e.g., the dimension of the vector after embedding each pixel block). When calculating the attention score, the dot product result is scaled by dividing by sqrt(d) to prevent gradient vanishing or exploding. Multiplying the similarity by the Value matrix yields the output of each head.

[0036] 4) Position-wise Feed-Forward Networks: These consist of two sequentially connected fully connected layers, which typically use ReLU or GELU activation functions.

[0037] 5) Residual Connections and Layer Normalization: A normalization layer is added before and after the feedforward neural network; the input and output of the multi-head self-attention mechanism are summed by residuals and then fed into the first normalization layer, with the output of the first normalization layer serving as the input to the feedforward neural network; the input and output of the feedforward neural network are summed by residuals and then output to the second normalization layer.

[0038] The decoding layer adopts a multi-head attention mechanism structure based on Vision Transformer, and the output of the decoding layer is the reconstructed image.

[0039] The size of the classification output layer is determined by the number of specific UHF / UHF communication signal modes.

[0040] S3. Based on the known VHF signals, perform preprocessing and time-frequency transformation according to the process in step S1 to construct a VHF narrowband signal time-frequency map dataset for supervised learning.

[0041] S4. Fine-tune the pre-trained VHF / UHF regime intelligent recognition model and train it using the dataset constructed in step S3. The model is fine-tuned and tested using a labeled VHF / UHF signal dataset with specific regimes. This method employs a multi-task learning approach, jointly training the VHF / UHF narrowband time-frequency map mask reconstruction task and the VHF / UHF regime-specific recognition task to achieve better performance than training the VHF / UHF regime-specific recognition task alone. In the fine-tuning stage, the model simultaneously performs reconstruction and classification tasks. The reconstruction task recovers the masked time-frequency map through the decoding layer, while the classification task predicts the signal regime category through a newly added fully connected layer. The loss functions for both tasks are dynamically adjusted using weights α and β, with the optimization objective being to minimize the total loss L_Total. Figure 6 As shown;

[0042] The specific method for fine-tuning is as follows: after the output of the coding layer, a Dropout layer and a fully connected layer are connected as a branch network for the classification task, wherein the number of neurons in the fully connected layer is the number of neurons in a specific ultra-short wave mode;

[0043] During the fine-tuning phase, the trainable parameters of the first three bidirectional Transformer modules in the encoding layer are frozen. Reconstruction learning and classification learning branches are then constructed and adaptively jointly trained. The reconstruction learning loss function uses the MSE loss function, while the classification loss function uses the label-smoothed cross-entropy loss function. Label smoothing is a regularization technique used in deep learning classification tasks, primarily to alleviate the model's overconfidence in a single class. It creates a soft target by smoothing the actual class labels against a uniform distribution. Replacing the original hard target hot : Where K represents the number of specific ultra-shortwave system types; δ is a hyperparameter, which is set to 0.1 in this invention. Label smoothing can improve the generalization performance of the model when the dataset is small. The generalization performance of the algorithm is improved by combining self-supervised learning and supervised learning; the loss function formula of the model in the fine-tuning stage is shown below:

[0044] L Total =αL MSE +βL LS

[0045] Where L Total Let L be the overall loss function, and α and β be the weights of the reconstruction loss and classification loss functions, respectively. The weights for the two tasks are iteratively optimized through model training to achieve the globally optimal result. MSE To calculate the root mean square error of the reconstructed VHF time-frequency plot and the original time-frequency plot, L LS This is the classification loss function after label smoothing.

[0046] S5. Use the trained Masked AutoEncoder UHF mode intelligent recognition model to perform mode recognition on specific UHF communication signals.

[0047] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for intelligent identification of UHF signal modes based on a Masked autoencoder, characterized in that, Includes the following steps: S1. Preprocess the signal according to the transmission time, center frequency and bandwidth of the intercepted VHF signal. The preprocessed VHF narrowband signal must contain the contents of the signal's synchronization frame and part of the data frame. Then, a short-time Fourier transform is performed on the preprocessed ultra-shortwave narrowband signal to construct an ultra-shortwave narrowband signal time-frequency graph dataset for unsupervised learning. S2. Construct a Masked AutoEncoder intelligent recognition model for ultra-shortwave systems and perform unsupervised learning pre-training using an ultra-shortwave narrowband signal time-frequency image dataset. The Masked AutoEncoder intelligent recognition model for ultra-shortwave systems includes a random masking layer, an encoding layer, a decoding layer, and a classification output layer. The random masking layer first divides the ultra-shortwave narrowband time-frequency image into multiple pixel blocks in the horizontal and vertical directions. Then, the divided pixel blocks are randomly masked according to three probabilities: 0.25, 0.5, and 0.

75. Finally, the unmasked parts of the pixel blocks processed by the three random masking probabilities in the original ultra-shortwave narrowband time-frequency image are randomly shuffled, resized in the height dimension, and then stitched together in the channel dimension before being fed into the encoding layer for feature extraction and compression. S3. Based on the known VHF signals, perform preprocessing and time-frequency transformation according to the process in step S1 to construct a VHF narrowband signal time-frequency map dataset for supervised learning. S4. Fine-tune the pre-trained intelligent recognition model of the ultra-shortwave system and train it using the dataset constructed in step S3. S5. Use the trained Masked AutoEncoder UHF mode intelligent recognition model to perform mode recognition on specific UHF communication signals.

2. The intelligent identification method for UHF signal modes based on a Masked autoencoder according to claim 1, characterized in that, The preprocessing in step S1 includes time-domain truncation, mixing, and down-conversion. First, the original signal is truncated in the time domain according to the start time of the synchronization frame and the end time of the data frame. Then, mixing is performed according to the center frequency of the target signal to ensure that the signal is at zero frequency. Finally, down-conversion is performed according to the downsampling factor.

3. The intelligent identification method for UHF signal modes based on a Masked autoencoder according to claim 1, characterized in that, The coding layer includes multiple cascaded bidirectional Transformer modules, each of which includes an input layer, position encoding, a multi-head self-attention mechanism, and a feedforward neural network in sequence. 1) The input layer includes two processes: word embedding and character embedding. Word embedding maps words to a high-dimensional vector space to capture the semantic information of words. Character embedding encodes each character in a word. In ultra-shortwave signal processing, pixel blocks in the time-frequency map are regarded as "words", and single pixels or local features are regarded as "characters". 2) Positional encoding is used to add positional codes to the input word embeddings; 3) Multi-head self-attention mechanism: The time-frequency image pixel blocks are processed in two directions, forward and reverse, to obtain forward and reverse feature vectors respectively. The multi-head self-attention mechanism divides the input feature vector into multiple "heads", each of which processes features in both the forward and reverse directions simultaneously. An attention score is calculated for each of the two directions, and these attention scores are then weighted and summed through a scaling point attention mechanism layer to obtain a new vector representation. Finally, the outputs of all heads are concatenated in the concatenation layer to form a feature representation that integrates bidirectional information. 4) Feedforward Neural Network: It contains two sequentially connected fully connected layers; a normalization layer is added before and after the feedforward neural network; the input and output of the multi-head self-attention mechanism are added together by residuals and then fed into the first normalization layer, and the output of the first normalization layer is used as the input of the feedforward neural network; the input and output of the feedforward neural network are added together by residuals and then the output is the second normalization layer.

4. The intelligent identification method for UHF signal modes based on a Masked autoencoder according to claim 1, characterized in that, The specific method for fine-tuning in step S4 is as follows: after the output of the coding layer, connect the Dropout layer and the fully connected layer as a branch network for the classification task, wherein the number of neurons in the fully connected layer is the number of neurons in a specific ultra-short wave mode. During the fine-tuning phase, the trainable parameters of the first three bidirectional Transformer modules in the encoding layer are frozen. An adaptive joint training of the reconstruction learning branch and the classification learning branch is then performed. The reconstruction learning loss function uses the MSE loss function, while the classification loss function uses the label-smoothed cross-entropy loss function. The loss function formula for the model during the fine-tuning phase is shown below: ； in For the total loss function, and These are the weights for the reconstruction loss and classification loss functions, respectively. The weights for the two tasks are iteratively optimized through model training to achieve the global optimum. The root mean square error of the reconstructed VHF time-frequency plot and the original time-frequency plot, This is the classification loss function after label smoothing.

Images