A signal modulation recognition method based on mixed modal contrast and fusion

Abstract

The present application relates to a kind of signal modulation identification methods based on mixed mode contrast and fusion, by using the image modalities represented with time-frequency spectrum and the sequence modalities represented with A / P sequence to mine the feature representation of modulation signal, and using the unlabelled double modal contrast self-supervised pre-training to obtain the feature extractor including image encoder and sequence encoder, then by modal level feature fusion, the trained signal modulation identification model is obtained after supervised fine-tuning;Modulation classification result is obtained by using the model to receive the signal modulation identification.This application can extract the hidden features of the signal, and also can obtain higher model performance in complex multipath scene and small sample condition, get accurate recognition result.

Description

Technical Field

[0001] This invention relates, and in particular, to a signal modulation identification method based on mixed-mode comparison and fusion. Background Technology

[0002] With the explosive growth of modern wireless communication technology, an increasing number of IoT terminals are being deployed to provide communication services. The ever-increasing user demand and explosive data volume pose a severe challenge to spectrum resources. An effective way to alleviate this situation is to use Automatic Modulation Recognition (AMR) technology in cognitive radio. As an intermediate step between signal detection and demodulation, AMR can identify the modulation type of a signal and analyze the information contained within it, distinguishing between interference signals and legitimate user signals, thus enabling the monitoring of spectrum resources. Therefore, AMR technology is a prerequisite for signal demodulation at the receiver and a key component of wireless communication, finding widespread application in military and civilian fields, including electronic warfare, surveillance analysis, and radio fault detection.

[0003] Traditional AMR methods generally suffer from low recognition efficiency, high cost, and low accuracy, and can be mainly divided into two categories: likelihood function-based recognition methods and feature-based recognition methods. Likelihood function-based methods use hypothesis testing to determine the likelihood function of a signal and find the optimal Bayesian estimate to obtain the signal classification result. However, they require a large amount of prior information and have high computational complexity, making them difficult to implement in non-cooperative communication and greatly limiting their applicability. Feature-based methods consist of feature extraction and classification, and can be divided into traditional manual methods and deep learning-based methods. Existing deep learning-based AMR methods can be divided into two categories: those using sequence input and those using image input. Sequence representation methods represent the received modulated signal as an in-phase and quadrature (I / Q) sequence or an amplitude and phase (A / P) sequence. Image representation methods map the signal to an image through preprocessing, including constellation diagrams, eye diagrams, cyclic spectra, and time-spectral diagrams.

[0004] However, current deep learning-based AMR methods are mostly based on single-modal inputs, i.e., sequences or images. Furthermore, AMR is a challenging task in complex electromagnetic environments. Firstly, acquiring signal sample data in real-world scenarios is difficult, posing a challenge to improving model performance with small sample sizes. Secondly, signals in real-world environments are inevitably affected by different propagation channels. When the distribution of actual signal samples differs significantly from the training data, the performance of the trained deep neural network drops sharply. Improving model performance under different propagation channel conditions is a pressing issue that needs to be addressed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a signal modulation recognition method based on hybrid mode comparison and fusion, which can extract the hidden features of the signal and obtain high model performance and accurate recognition results even in complex multipath scenarios and small sample conditions.

[0006] The technical solution adopted by this invention to solve its technical problem is: to provide a signal modulation recognition method based on mixed mode comparison and fusion, comprising the following steps:

[0007] A signal modulation recognition model is constructed to automatically identify the modulation of received wireless signals and obtain signal classification results; the signal modulation recognition model includes:

[0008] The time-frequency preprocessing module is used to perform time-frequency analysis on the input signal to obtain its image modal representation;

[0009] The amplitude-phase preprocessing module is used to perform amplitude-phase conversion on the input signal to obtain its sequential mode representation;

[0010] An image encoder is used to extract features from the image modal representation of an input signal to obtain image modal features;

[0011] A sequence encoder is used to extract features from the sequence mode representation of an input signal to obtain sequence mode features;

[0012] The classification head module is used to generate modulation classification results based on image modal features and sequence modal features;

[0013] The signal modulation recognition model is trained using the following method:

[0014] Unlabeled bimodal contrastive self-supervised pre-training was performed on the signal modulation recognition model to optimize the parameters of the image encoder and sequence encoder;

[0015] Supervised training is performed on the pre-trained signal modulation recognition model to obtain a well-trained signal modulation recognition model.

[0016] Furthermore, the unlabeled bimodal contrastive self-supervised pre-training of the signal modulation recognition model includes:

[0017] Image reconstruction based on image modal features;

[0018] Image modal features and sequence modal features are projected onto the same dimensional space to obtain image representation vectors and sequence representation vectors;

[0019] Modal-level contrast loss is calculated using the cross-correlation between image representation vectors and sequence representation vectors;

[0020] The parameters of the image encoder and sequence encoder are optimized based on reconstruction loss and modal-level contrast loss.

[0021] Furthermore, the supervised training of the pre-trained signal modulation recognition model is based on classification loss, which is expressed as:

[0022]

[0023] Among them, L f For modal-level contrast loss, b is the sample index, B is the batch size, k is the modulation type index, K is the total number of modulation types, ∈ is a constant, and y b,k and These are the true label and the predicted probability, respectively.

[0024] Furthermore, during pre-training, the time-frequency preprocessing module is also used to uniformly mask the obtained image modal representation before outputting it to the image encoder.

[0025] Furthermore, the reconstruction loss is expressed as

[0026]

[0027] Among them, L r The reconstruction loss is defined as b, where b is the sample index, B is the batch size, n is the image patch index, and N is the image patch index. m x is the number of image patches to be masked. I and These are the original and reconstructed image modal representations, respectively.

[0028] Furthermore, the modal-level contrast loss is expressed as:

[0029]

[0030] Among them, L c For modal-level contrast loss, C mn Let C be the element of the cross-correlation matrix C calculated along the batch dimension between two distinct vectors. mmThe diagonal elements of the cross-correlation matrix C are η, where η is a constant. and These are the normalized image representation vector and sequence representation vector, respectively.

[0031] Furthermore, the image encoder module includes:

[0032] The pooling dimensionality reduction transformer is used to reshape the image modal representation of the input signal into a two-dimensional space, and then perform multi-scale image feature extraction on the reshaped vector by introducing an attention mechanism.

[0033] The first global average pooling layer is used to convert the obtained multi-scale image features into a set feature dimension to obtain the image modal feature vector.

[0034] Furthermore, the sequence encoder module includes:

[0035] Convolutional networks are used to extract multi-scale temporal features from the sequential modal representation of input signals by introducing an attention mechanism.

[0036] The second global average pooling layer is used to convert the obtained multi-scale temporal features into a set feature dimension to obtain the sequence modality feature vector.

[0037] Furthermore, the classification head module includes:

[0038] The feature concatenation module is used to concatenate image modal features and sequence modal features to obtain fused features;

[0039] A classifier is used to generate modulation classification results based on fused features.

[0040] Furthermore, the step of performing time-frequency analysis on the input signal to obtain its image modal representation includes:

[0041] Perform a short-time Fourier transform on the input signal to extract its amplitude spectrum and phase spectrum;

[0042] After processing the amplitude spectrum and phase spectrum, they are stitched together to form a dual-channel image as the image mode representation of the input signal.

[0043] Furthermore, the step of performing amplitude-phase conversion on the input signal to obtain its sequence mode representation includes:

[0044] Convert the input signal into an amplitude / phase sequence;

[0045] The amplitude / phase sequence is processed using a preprocessing convolutional network to obtain the sequence mode representation of the input signal.

[0046] Beneficial effects

[0047] By adopting the above-mentioned technical solution, this invention has the following advantages and positive effects compared with the prior art: This invention mines the feature representation of modulated signals by using sequence modes represented by A / P sequences and image modes represented by time-spectrum diagrams, and further enhances the feature representation of modulated signals by combining generative learning and contrastive learning, thereby realizing the mining of hidden features of unlabeled signals, and using different signal modes to construct different auxiliary tasks to enhance the feature representation of modulated signals, thereby enhancing the recognition ability in cases with a small number of samples and unlabeled samples, so that the model can achieve high performance and obtain accurate recognition results even in complex multipath scenarios and small sample cases. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of modal preprocessing according to an embodiment of the present invention;

[0050] Figure 3 This is a structural diagram of the image encoder according to an embodiment of the present invention;

[0051] Figure 4 This is a structural diagram of a sequence encoder according to an embodiment of the present invention;

[0052] Figure 5 This is a comparison chart of the classification accuracy of different benchmark methods in different SNRs according to embodiments of the present invention;

[0053] Figure 6 This is a confusion matrix diagram of different modulation classifications in embodiments of the present invention;

[0054] Figure 7 This is a comparison chart of classification accuracy in small sample scenarios according to embodiments of the present invention;

[0055] Figure 8 This is a comparison chart of classification accuracy in complex multipath scenarios according to embodiments of the present invention. Detailed Implementation

[0056] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0057] This invention relates to a Hybrid Modulation Contrast and Fusion-based Signal Modulation Identification (HMCF-AMR) method, considering a single-input single-output wireless communication system. A signal model is given here, incorporating most of the channel effects that radio frequency systems may encounter. The received discrete signal r(n) can be represented as...

[0058]

[0059] Where h represents the channel impulse response, and p represents the square root cosine filter used for impulse shaping. k It is a modulation symbol, modulated by class K {s1, s2, ..., s...} k One of the generation, T s The sampling rate is represented by ΔT. ΔT, Δθ, and ΔT' represent frequency offset, phase offset, and sampling rate offset, respectively. w is additive white Gaussian noise.

[0060] Generally, the in-phase and quadrature (I / Q) components of the received baseband signal r(n) are used, and its expression is as follows:

[0061] r(n) = i(n) + jq(n)

[0062] Here, i(n) and q(n) represent the in-phase and quadrature components of the received baseband signal, respectively. To simplify implementation and computation, the real part i(n) and the imaginary part q(n) can be represented as matrices of dimension 2×N, i.e.

[0063]

[0064] Where N is the signal length.

[0065] AMR can be represented as a time series classification problem with K categories: maximizing the probability using the received baseband signal R. in This represents the i-th modulation type.

[0066] The overall framework of the HMCF-AMR method is as follows: Figure 1As shown, a feature extractor is trained using two common modal representations of unlabeled signal data in a cooperative-supervised manner: the A / P sequence and the time-spectrum. In the pre-training phase, it consists of five parts: modality preprocessing, image encoder, sequence encoder, reconstruction head, and modality contrast head. Specifically, feature extraction is performed using the corresponding encoders, and the projection head maps the feature vectors obtained from different modal representations of the same sample to the same hyperplane representation space. The cooperation and competition among different modalities of the same sample enable the HMCF-AMR encoder to learn the latent information of the signal sample, achieving a more robust feature representation of the modulated signal in the unlabeled case. Then, in the fine-tuning phase, the first three parts are retained, and a classification head is added. After obtaining the pre-trained model, the modality preprocessing, sequence encoder, and image encoder can be applied to classification tasks.

[0067] The internal structure and operating principles of each part will be further explained below.

[0068] (I) Modal preprocessing

[0069] This invention considers using hybrid mode representation to further improve the performance of modulation recognition. Mode-level feature representation uses both sequence and image representations of the same signal samples for training and prediction. This part will be discussed from two aspects: 1) time-frequency preprocessing; 2) amplitude-phase preprocessing. The detailed structure of mode preprocessing is as follows... Figure 2 As shown.

[0070] 1) Time-Frequency Preprocessing: Since the time-frequency transformation of the modulated signal can reflect its frequency change over time, thus enriching the signal's characteristic representation, this invention uses a spectrum based on Short-Time Fourier Transform (STFT) to represent the image modality. Let w(·) and K represent the window function of length J and the window shift size, respectively, m be the time-domain index, and k be the frequency-domain index. The received baseband signal r(n) is processed using STFT as follows:

[0071]

[0072] This contains the amplitude and phase information of the original signal, and after simple calculations, the amplitude spectrum and phase spectrum are obtained. Then, by resizing, normalizing, and stitching, a two-channel image of size 2×128×128 is obtained, and the image mode of the signal is represented as x. I .

[0073] Masked Autoencoder (MAE) methods can obtain high-quality image representations and improve pre-training efficiency; therefore, this invention adopts MAE for the pre-training of the image encoder. To further enable the model to extract multi-scale features and reduce computational complexity, a pyramid-structured encoder is used. To accommodate this structure, a uniform mask is applied to the image. Figure 2 As shown, uniform masking samples one block for every 2×2 image blocks, leaving 75% of the image blocks masked. This means that the masked image blocks will not participate in the image encoding process during the pre-training stage.

[0074] 2) Amplitude and Phase Preprocessing: The amplitude and phase information of the signal are used to represent the sequence mode, i.e., the A / P sequence, which consists of two real-valued vectors: amplitude A(n) and phase P(n), i.e.

[0075]

[0076] The transformation between amplitude and phase sequences can be expressed as follows:

[0077]

[0078] Here, i(n) and q(n) represent the in-phase and quadrature parts of the signal, respectively. To enable the sequence encoder to extract more effective features from the sequence modes, a preprocessing convolutional network (Pre-CNN) is used. This network consists of two convolutional layers, each followed by a batch normalization (BN) layer and a PReLU activation function. The first convolutional kernel is 2×7 in size, and the last convolutional kernel is 1×5 in size, with 16 output channels. After Pre-CNN, the sequence mode of the signal is represented as x. S Its size is 16×3×N.

[0079] (II) Image Encoder

[0080] like Figure 3 As shown in the middle left figure, the image encoder f μ Input image modal representation x IFeature extraction is performed using a designed image encoder, which consists of a Pooling Reduction Transformer (PRFormer) module. The input image size is 2×128×128, and the image patch size is 4×4. The encoder first splits the image into 32×32 patches, then flattens them into vectors of length 32 (4×4×2). The flattened image patches are then input into the image patch encoding module, which consists of a linear projection layer and a normalization layer, followed by fixed-position encoding, and finally input into the RPFormer module. The entire encoder consists of four stages. Except for the first stage, the image patch encoding module combines and stitches the 2×2 image patches and performs linear projection on the feature dimension. The image output sizes for the four stages are 16×32×32, 32×16×16, 64×8×8, and 128×4×4, respectively.

[0081] The design of Pooling Reduction Attention (PRA) is as follows: Figure 3 As shown in the middle right figure. Let Z represent the input feature vector, and reshape it into a two-dimensional space. Average pooling (AvgPool) and depthwise convolution (DWConv) are applied to reduce the spatial dimensionality:

[0082] P=LayerNorm(DWConv(AvgPool(Z)))

[0083] Where P represents the dimensionality-reduced feature map, and DWConv(·) represents a depthwise convolution with a kernel size of 1×1. P is a shorter sequence than the input z, which can replace the input Z during self-attention. Assuming the query, key, and value tensors are Q, K, and V respectively, the calculation process is as follows:

[0084] (Q, K, V) = (ZW) q PW k PW v )

[0085] Among them, W q W k and W v These are the weight matrices for linear projection, used to generate the query, key, and value tensors. Then, Q, K, and V are fed into the attention module to compute the attention feature A, as follows:

[0086]

[0087] Where, d KK is the channel dimension, and the Softmax function is applied along the rows of the matrix. Since the lengths of K and V are smaller than z, the proposed pooling-based dimensionality reduction attention module has lower complexity than traditional attention modules. The Feed-Forward Network (FFN) follows the design of the traditional Vision Transformer (ViT). The workflow of the PRFormer module is as follows... Figure 3 As shown in the middle left figure, the calculation process is as follows:

[0088] Z att =LayerNorm(Z+PRA(Z))

[0089] Z out =LayerNorm(Z) att +FFN(Z att ))

[0090] Among them, Z and Z att and Z out These are the input, the output of the pooling dimensionality reduction attention, and the output of the pooling dimensionality reduction transformer, respectively.

[0091] The features extracted by PRFormer are finally fed into a global average pooling layer to obtain an image representation vector X of size 128×1×1. I .

[0092] (III) Sequence Encoder

[0093] Sequence encoder f v Input sequence mode x s Feature extraction is performed using a sequence encoder, which consists of a ConvNeXt (MKA-ConvNeXt) module based on hybrid convolutional kernel attention, and its design is as follows: Figure 4 As shown, the downsampling layer uses 1×2 convolutions with a stride of 2 for spatial downsampling and channel expansion, extracts multi-scale features using group convolutions based on hybrid convolutional kernel attention (MKA-GConv), followed by layer normalization (LN). The first pointwise convolution (PWConv) expands the number of feature map channels through 1×1 convolutions, then applies the GELU activation function to increase non-linearity. Global response normalization (GRN) aims to increase channel contrast and selectivity. The final PWConv reduces the number of channels using 1×1 convolutions. Finally, residual connections are applied to the entire module.

[0094] MKA-GConv first divides the channels into four groups and applies convolutions of different sizes to each group to learn multi-scale spatial information. The convolution kernel sizes used are 1×3, 1×5, 1×7, and 1×9, respectively, and F... i This represents the multi-scale spatial information learned from each convolutional layer. To effectively measure the importance of features from the multi-scale feature maps, the SEWeight module is used to obtain attention weights.

[0095] E i =SEWeight(F i )=Sigmoid(FC(ReLU(FC(AvgPool(F i ))))

[0096] Among them, E i It is from F i The attention weights are then recalibrated using a softmax function to incorporate all spatial location information. These recalibrated multi-scale channel attention weights are then compared with the feature maps F at the corresponding scales. i Multiplication, the formula is as follows

[0097] H i =F i ⊙Softmax(E i )

[0098] Where ⊙ represents multiplication by channel, H i This involves obtaining feature maps with multi-scale channel attention weights. The final output process can be written as follows:

[0099]

[0100] in, This indicates concatenation by channel. Channel shuffling is used to achieve information fusion between channels to compensate for the defect that grouped convolution can hinder information flow.

[0101] The entire sequence encoder uses three MKA-ConvNeXt modules, each with output feature dimensions of 16×3×64, 32×3×32, and 64×3×16, respectively. The extracted features are then fed into a global average pooling layer to obtain a sequence mode representation vector X of size 64×1×1. S .

[0102] (iv) Reconstructing the Head

[0103] The reconstruction head includes a pixel decoder and a reconstruction loss. The pixel decoder aims to reconstruct the input image using feature maps obtained in the fourth stage of the image encoder; since it is only used in the pre-training stage, a standard ViT approach is sufficient. The reconstruction loss is the mean squared error, and it only calculates the masked image patches between the pixel decoder's prediction and the original image.

[0104]

[0105] Where, N m Indicates the number of image blocks that are masked.

[0106] (V) Modal Comparison Head

[0107] To enable modal-level comparison, the two different modal representations extracted by the encoder need to be projected into a representation space of the same dimension. The projection head consists of two fully connected layers with dimensions of 128 and 64, respectively. For a specific signal sample with two modal representations, the encoder and projection are performed to obtain...

[0108]

[0109] Then, L2 normalization is applied, and the normalized representation vectors are denoted as follows: and

[0110] The cross-correlation matrix of a batch-normalized representation vector is calculated as follows:

[0111]

[0112] Among them, C mn These are the elements of the cross-correlation matrix calculated along the batch dimension between two different vectors, where B is the batch size. The formula for calculating the modal-level contrastive loss is:

[0113]

[0114] Where C mm For the diagonal elements of the cross-correlation matrix C, η is a constant used to weigh the importance of similarity and difference. The cross-correlation matrix C forces different representations of the same sample to be closer together and different samples to be farther apart.

[0115] (vi) Classification Head

[0116] To classify the modulation type of the received signal, the image representation vector X extracted by the image encoder and sequence encoder is used. I And sequence representation vector X S pieced together:

[0117]

[0118] Then, two fully connected layers and a Softmax function are used to identify the modulation format, as follows:

[0119]

[0120] Dropout was used before the last layer to prevent overfitting, and the number of hidden units in the linear layers was 192 and 11, respectively. Classification cross-entropy was used as the classification loss.

[0121]

[0122] Here, label smoothing is used to enhance generalization ability, ∈ is a small constant, and if it belongs to the k-th modulation type, then y b，k =1.

[0123] This invention uses the RadioML.2016.10A public dataset and the RML24.ETU300 dataset with complex multipath channels generated by GNU Radio. During the HMCF-AMR pre-training stage, both the image reconstruction and modal-level feature contrast parts are optimized simultaneously, with a loss function of L. p =L r +L c The dataset used for pre-training was the entire RadioML.2016.10A without any label information. AdamW was used as the optimizer with an initial learning rate of 5e-3 and a batch size of 512. The model was trained for 50 iterations using a cosine learning rate decay strategy. During the HMCF-AMR fine-tuning phase, only L was optimized for parameter updates in the classification task. The dataset used for fine-tuning was split into training (60%), validation (20%), and test (20%) sets for training and evaluation. The batch size was set to 256. The model used the AdamW optimizer with a hierarchical learning rate, i.e., a smaller learning rate of 5e-4 was used except for the classification head to preserve the knowledge of the pre-trained model. An early stopping strategy was employed to reduce overfitting; the learning rate was halved when the validation loss did not improve within 5 iterations. Training was stopped when the validation accuracy did not improve within 10 iterations, and the model weights at the point of maximum validation accuracy were saved. All experiments were conducted using PyTorch 2.1.2 based on Python 3.10.13 on a PC equipped with an Intel Core i5-13490F CPU @ 2.50GHz and an NVIDIA GeForce RTX 4060Ti GPU (16GB of RAM).

[0124] To verify the effectiveness of the HMCF-AMR model designed in this invention, comparative experiments were conducted on the RadioML.2016.10A dataset with sufficient samples, comparing it with AMC-Net, AWN, CNN-LSTM, MCformer, MCLDNN, MCNet, and PET-CGDNN. Under the same experimental conditions, the classification accuracy of the proposed method and other models at different signal-to-noise ratios (SNR) is as follows: Figure 5 As shown in the figure. Notably, our model achieves higher classification accuracy than other methods across almost the entire SNR range. The average classification accuracies of AMC-Net, AWN, CNN-LSTM, MCformer, MCLDNN, MCNet, and PET-CGDNN are 61.14%, 61.23%, 55.54%, 58.31%, 61.03%, 55.62%, and 61.04%, respectively, while our model achieves 62.71%. More importantly, the proposed HMCF-AMR significantly outperforms the other selected models, improving upon the AWN model by 1.48%. Furthermore, HMCF-AMR achieves an average classification accuracy of 92.23% at SNRs above 0dB, and a peak accuracy of 93.59% at 12dB.

[0125] To provide a more in-depth analysis, Figure 6 The left-middle figure shows the confusion matrix for 11 modulation classifications across the entire SNR. The x and y axes represent the predicted and true labels, respectively. The diagonal lines represent correctly identified regions, while the other regions are confusion areas. The figure shows that our model exhibits confusion when identifying WBFM because both WBFM and AM-DSB are continuous analog modulations with silent periods, making their distinguishing features indistinct. While this issue persists in HMCF-AMR, it shows some improvement compared to other models. Note the extremely high accuracy of AM-SSB; this is because the AM-SSB signal waveform is irregular, resembling pure noise. This causes all models to classify signals with low SNR as AM-SSB, thus increasing their accuracy. Figure 6 The right-middle figure shows the confusion matrix for 11 modulation categories at 12dB. HMCF-AMR achieves over 90% accuracy in recognizing QAM16, QAM64, 8PSK, and AM-SSB, and nearly 100% accuracy in recognizing PAM4, QPSK, CPFSK, GMSK, and AM-DSB. However, the model's accuracy in recognizing WBFM is insufficient, primarily because WBFM is a continuous analog modulation with a silence period. HMCF-AMR also effectively mitigates confusion between higher-order modulations QAM64 and QAM16.

[0126] In a small-sample environment, training was performed using only 1% of the signal samples from the RadioML.2016.10A dataset, and comparative experiments were conducted with CNN-LSTM, MCformer, MCLDNN, MCNet, and PET-CGDNN. Under the same experimental conditions, the classification accuracy of the proposed method in a small-sample environment compared with other models is as follows: Figure 7 As shown, our model achieves higher classification accuracy than other methods when the SNR is greater than -4dB, fully demonstrating HMCF-AMR's ability to identify samples with small sample sizes and unlabeled samples. This is thanks to the pre-training gain of HMCF-AMR, which can effectively acquire the latent features of unlabeled signal samples.

[0127] In complex multipath environments, the method was trained using the RML24.ETU300 dataset, and comparative experiments were conducted with CNN-LSTM, MCformer, MCLDNN, MCNet, and PET-CGDNN. Under the same experimental conditions, the classification accuracy of the proposed method in complex multipath environments compared with other models is as follows: Figure 8 As shown, our model achieves a classification accuracy that is nearly higher than other methods across the entire SNR range, showing a significant improvement over CNN-LSTM, MCformer, MCNet, and PET-CGDNN. This fully demonstrates that HMCF-AMR can effectively address the problem of environment transfer and has a certain degree of robustness in complex multipath environments. This is thanks to the signal features acquired during the pre-training stage and the assistance of multimodal representation methods.

Claims

1. A signal modulation recognition method based on mixed-mode comparison and fusion, characterized in that, Includes the following steps: A signal modulation recognition model is constructed to automatically identify the modulation of received wireless signals and obtain signal classification results; The signal modulation recognition model includes: The time-frequency preprocessing module is used to perform time-frequency analysis on the input signal to obtain its image modal representation; The amplitude-phase preprocessing module is used to perform amplitude-phase conversion on the input signal to obtain its sequential mode representation; An image encoder is used to extract features from the image modal representation of an input signal to obtain image modal features; A sequence encoder is used to extract features from the sequence mode representation of an input signal to obtain sequence mode features; The classification head module is used to generate modulation classification results based on image modal features and sequence modal features; The signal modulation recognition model is trained using the following method: Unlabeled bimodal contrastive self-supervised pre-training was performed on the signal modulation recognition model to optimize the parameters of the image encoder and sequence encoder; Supervised training is performed on the pre-trained signal modulation recognition model to obtain a well-trained signal modulation recognition model. The unlabeled bimodal contrastive self-supervised pre-training of the signal modulation recognition model includes: Image reconstruction based on image modal features; Image modal features and sequence modal features are projected onto the same dimensional space to obtain image representation vectors and sequence representation vectors; Modal-level contrast loss is calculated using the cross-correlation between image representation vectors and sequence representation vectors; The parameters of the image encoder and sequence encoder are optimized based on reconstruction loss and modal-level contrast loss. The modal level contrast loss is expressed as: ， ， in, For modal-level contrast loss, The cross-correlation matrix calculated along the batch dimension between two different vectors. elements, For batch size, For this cross-correlation matrix diagonal elements, It is a constant. and These are the normalized image representation vector and sequence representation vector, respectively.

2. The method according to claim 1, characterized in that, The supervised training of the pre-trained signal modulation recognition model is based on classification loss, which is expressed as: ， in, For classifying losses, For sample index, For modulation type index, Total number of modulation types It is a constant. and These are the true label and the predicted probability, respectively.

3. The method according to claim 1, characterized in that, During pre-training, the time-frequency preprocessing module is also used to uniformly mask the obtained image modal representation before outputting it to the image encoder.

4. The method according to claim 1, characterized in that, The reconstruction loss is expressed as ， in, To reconstruct the loss, For sample index, For image patch indexing, The number of image blocks to be masked. and These are the original and reconstructed image modal representations, respectively.

5. The method according to claim 1, characterized in that, The image encoder includes: The pooling dimensionality reduction transformer is used to reshape the image modal representation of the input signal into a two-dimensional space, and then perform multi-scale image feature extraction on the reshaped vector by introducing an attention mechanism. The first global average pooling layer is used to convert the obtained multi-scale image features into a set feature dimension to obtain the image modal feature vector.

6. The method according to claim 1, characterized in that, The sequence encoder includes: Convolutional networks are used to extract multi-scale temporal features from the sequential modal representation of input signals by introducing an attention mechanism. The second global average pooling layer is used to convert the obtained multi-scale temporal features into a set feature dimension to obtain the sequence modality feature vector.

7. The method according to claim 1, characterized in that, The classification header module includes: The feature concatenation module is used to concatenate image modal features and sequence modal features to obtain fused features; A classifier is used to generate modulation classification results based on fused features.

Images