Signal modulation identification method based on mixed mode comparison and fusion

Abstract

The invention relates to a signal modulation identification method based on mixed mode comparison and fusion, and the method comprises the steps: mining the feature representation of a modulation signal through employing an image mode represented by a time-frequency spectrogram and a sequence mode represented by an A / P sequence; a feature extractor comprising an image encoder and a sequence encoder is obtained through label-free bimodal comparison self-supervision pre-training, and then a trained signal modulation recognition model is obtained after supervision fine tuning is conducted through modal-level feature fusion; and carrying out modulation identification on the received signal by adopting the model to obtain a modulation classification result. According to the method, the hidden features of the signals can be extracted, high model performance can be obtained under the conditions of complex multi-path scenes and small samples, and an accurate recognition result is obtained.

Description

Technical Field

[0001] The present invention relates to, in particular to, a signal modulation recognition method based on mixed mode comparison and fusion. Background Art

[0002] With the explosive development of modern wireless communication technology, more and more IoT terminals are deployed to provide communication services. The growing user demand and the explosive amount of data have posed severe challenges to spectrum resources. An effective way to alleviate this situation is to use the Automatic Modulation Recognition (AMR) technology in cognitive radio, which is an intermediate step between signal detection and signal demodulation. It can identify the modulation type of the signal and analyze the information contained in the signal, distinguish interference signals from legitimate user signals, and realize the monitoring of spectrum resources. Therefore, AMR technology is a prerequisite for the demodulation signal at the receiving end, and is also a key link in wireless communication. It has been widely used in military and civilian fields, including electronic countermeasures, surveillance analysis, and radio fault detection.

[0003] Most traditional AMR methods have the disadvantages of low recognition efficiency, high cost and low accuracy. They are mainly divided into two categories: likelihood function-based recognition method and feature-based recognition method. The likelihood function-based method uses the hypothesis test method to determine the likelihood function of the signal and find the best Bayesian estimation to obtain the classification result of the signal. However, it requires a lot of prior information and has high computational complexity. It is difficult to implement in non-cooperative communication, and its applicability is greatly limited. The feature-based method consists of feature extraction and classification. It can be divided into traditional manual-based methods and deep learning-based methods. Existing deep learning-based AMR methods can be divided into two categories: those using sequence input and those using image input. The method using sequence representation is to represent the received modulated signal as an in-phase and quadrature (I / Q) sequence or an amplitude and phase (A / P) sequence. The method using image representation is to map the signal into an image through preprocessing, including constellation diagram, eye diagram, cyclic spectrum diagram and time-frequency spectrum diagram.

[0004] However, the current AMR methods based on deep learning are basically based on single-modal input, i.e., sequence or image. At the same time, in a complex electromagnetic environment, AMR is a challenging task. First, it is difficult to collect signal sample data in actual scenes. How to improve the performance of the model in the case of small samples? Second, in the actual environment, the signal will inevitably be affected by different propagation channels. When the distribution of actual signal samples is very different from that of training data, the performance of the trained deep neural network drops sharply. How to improve the performance of the model in the case of different propagation channels is an urgent problem to be solved. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a signal modulation recognition method based on hybrid mode contrast and fusion, which can extract the hidden features of the signal and obtain higher model performance and accurate recognition results in complex multipath scenarios and small sample conditions.

[0006] The technical solution adopted by the present invention to solve the technical problem is: to provide a signal modulation recognition method based on hybrid mode comparison and fusion, comprising the following steps:

[0007] Construct a signal modulation recognition model to automatically recognize the modulation of the received wireless signal and obtain a signal classification result; the signal modulation recognition model includes:

[0008] A time-frequency preprocessing module is used to perform time-frequency analysis on the input signal to obtain its image modality representation;

[0009] An amplitude-phase preprocessing module is used to perform amplitude-phase conversion on the input signal to obtain its sequence modal representation;

[0010] An image encoder is used to extract features of the image modality representation of the input signal to obtain image modality features;

[0011] A sequence encoder is used to extract features from the sequence modal representation of the input signal to obtain sequence modal features;

[0012] A classification head module, used to generate modulation classification results according to image modality features and sequence modality features;

[0013] The signal modulation recognition model is trained by the following method:

[0014] Unlabeled bimodal contrastive self-supervised pre-training of the signal modulation recognition model to optimize the parameters of the image encoder and sequence encoder;

[0015] The pre-trained signal modulation recognition model is subjected to supervised training to obtain a trained signal modulation recognition model.

[0016] Furthermore, the signal modulation recognition model is subjected to unlabeled bimodal contrast self-supervised pre-training, including:

[0017] Reconstructing images based on image modality features;

[0018] Project the image modality features and the sequence modality features into the same dimensional space to obtain the image representation vector and the sequence representation vector;

[0019] The modality-level contrast loss is calculated using the cross-correlation between the image representation vector and the sequence representation vector.

[0020] The parameters of the image encoder and sequence encoder are optimized according to the reconstruction loss and the modality-level contrastive loss.

[0021] Furthermore, the supervised training of the pre-trained signal modulation recognition model is based on the classification loss, which is expressed as

[0022]

[0023] Among them, L f is the modality-level contrast loss, b is the sample index, B is the batch size, k is the modulation type index, K is the total number of modulation types, ∈ is a constant, and y b,k and are the true label and predicted probability respectively.

[0024] Furthermore, during pre-training, the time-frequency preprocessing module is also used to uniformly mask the obtained image modality representation before outputting it to the image encoder.

[0025] Furthermore, the reconstruction loss is expressed as

[0026]

[0027] Among them, L r is the reconstruction loss, b is the sample index, B is the batch size, n is the image block index, and N m is the number of masked image blocks, x I and are the original and reconstructed image modality representations respectively.

[0028] Furthermore, the modality-level contrast loss is expressed as

[0029]

[0030] Among them, L c is the modality-level contrast loss, C mn is the element of the cross-correlation matrix C calculated along the batch dimension between two different vectors, C mmare the diagonal elements of the cross-correlation matrix C, η is a constant, and They are the normalized image representation vector and sequence representation vector respectively.

[0031] Furthermore, the image encoder module includes:

[0032] The pooling dimension reduction transformer is used to reshape the image modality representation of the input signal into a two-dimensional space, and then perform multi-scale image feature extraction on the reshaped vector by introducing an attention mechanism;

[0033] The first global average pooling layer is used to convert the obtained multi-scale image features into a set feature dimension to obtain an image modality feature vector.

[0034] Furthermore, the sequence encoder module comprises:

[0035] Convolutional network, which is used to extract multi-scale temporal features from the sequence modal representation of the input signal by introducing an attention mechanism;

[0036] The second global average pooling layer is used to convert the obtained multi-scale time series features into a set feature dimension to obtain a sequence modal feature vector.

[0037] Furthermore, the classification header module includes:

[0038] A feature concatenation module is used to concatenate image modality features and sequence modality features to obtain fusion features;

[0039] The classifier is used to generate modulation classification results based on the fused features.

[0040] Furthermore, performing time-frequency analysis on the input signal to obtain its image modality representation includes:

[0041] Perform short-time Fourier transform on the input signal to extract its amplitude spectrum and phase spectrum;

[0042] After the amplitude spectrum diagram and the phase spectrum diagram are processed, they are spliced ​​into a dual-channel image as the image modality representation of the input signal.

[0043] Furthermore, performing amplitude-phase conversion on the input signal to obtain its sequence modal representation includes:

[0044] Convert the input signal into an amplitude / phase sequence;

[0045] The amplitude / phase sequence is processed using a preprocessing convolutional network to obtain a sequence modal representation of the input signal.

[0046] Beneficial Effects

[0047] Due to the adoption of the above-mentioned technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention mines the feature representation of the modulated signal by using the sequence mode represented by the A / P sequence and the image mode represented by the time-frequency spectrum diagram, and further enhances the feature representation of the modulated signal by combining generative learning and contrastive learning, thereby realizing the mining of hidden features of unlabeled signals, and using different signal modes to construct different auxiliary tasks to enhance the feature representation of the modulated signal, thereby enhancing the recognition ability in the case of a small number of samples and unlabeled samples, so that the model can achieve high performance in complex multipath scenarios and small sample conditions, and obtain accurate recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of modal preprocessing according to an embodiment of the present invention;

[0050] Figure 3 is a structural diagram of an image encoder according to an embodiment of the present invention;

[0051] Figure 4 is a structural diagram of a sequence encoder according to an embodiment of the present invention;

[0052] Figure 5 is a comparison chart of classification accuracy of different benchmark methods at different SNRs according to an embodiment of the present invention;

[0053] Figure 6 is a confusion matrix diagram of different modulation classifications according to an embodiment of the present invention;

[0054] Figure 7 is a comparison chart of classification accuracy in a small sample scenario according to an embodiment of the present invention;

[0055] Figure 8 4 is a comparison chart of classification accuracy in a complex multipath scenario according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0057] The embodiments of the present invention relate to a method for signal modulation recognition based on hybrid modal contrast and fusion (HMCF-AMR), considering a single-input single-output wireless communication system. A signal model is given here, which includes most of the channel effects that wireless radio may encounter. The received discrete signal r(n) can be expressed as

[0058]

[0059] Where h is the channel impulse response and p is the square root cosine filter used for pulse shaping. k is a modulation symbol, composed of K types of modulation {s1, s2, …, s k} one of them is generated, T s represents the sampling rate. ΔT, Δθ and ΔT represent the frequency offset, phase offset and sampling rate offset respectively. w is additive white Gaussian noise.

[0060] Generally, the in-phase and quadrature (I / Q) parts of the received baseband signal r(n) are used, and the expression is as follows

[0061] r(n)=i(n)+jq(n)

[0062] Where i(n) and q(n) represent the in-phase and quadrature parts of the received baseband signal, respectively. To simplify implementation and calculation, the real part i(n) and the imaginary part q(n) can be expressed as a matrix of dimension 2×N, that is,

[0063]

[0064] Where N is the signal length.

[0065] AMR can be formulated as a time series classification problem with K categories: using the received baseband signal R to maximize the probability in Indicates the i-th modulation type.

[0066] The overall framework of the HMCF-AMR method is as follows: Figure 1As shown in the figure, a feature extractor is obtained by training in a cooperatively supervised manner using two common modal representations of unlabeled signal data, namely A / P sequence and time-spectrogram. In the pre-training stage, it consists of five parts: modal preprocessing, image encoder, sequence encoder, reconstruction head and modal contrast head. Specifically, the corresponding encoder is used for feature extraction, and the feature vector obtained by different modal representations of the same sample is mapped to the same hyperplane representation space using the projection head. Different modalities of the same sample cooperate and compete, so that the encoder of HMCF-AMR can learn the potential information of the signal sample and realize a more robust feature representation of the modulated signal in the unlabeled case. Then, the first three parts are retained in the fine-tuning stage, and a classification head is added. After obtaining the pre-trained model, the modal preprocessing, sequence encoder and image encoder can be applied to the classification task.

[0067] The following is a further explanation of the internal structure and operating principle of each part.

[0068] (I) Modal preprocessing

[0069] The present invention considers using mixed modal representation to further improve the performance of performance modulation recognition. The modal level feature representation uses the sequence and image representation of the same signal sample for training and prediction. This part will be discussed from two aspects: 1) time-frequency preprocessing; 2) amplitude-phase preprocessing. The detailed structure of modal preprocessing is as follows Figure 2 shown.

[0070] 1) Time-frequency preprocessing: Since the time-frequency transform of the modulated signal can reflect the change of its frequency over time, thereby enriching the characteristic representation of the signal, the present invention uses a spectrum diagram based on short-time Fourier transform (STFT) to represent the image modality. Let w(·) and K represent the window function of length J and the window shift size, respectively, m is the time domain index, and k is the frequency domain index. The received baseband signal r(n) is applied with STFT as follows:

[0071]

[0072] It contains the amplitude and phase information of the original signal. After simple calculation, the amplitude spectrum and phase spectrum are obtained. Then, by resizing, normalizing and splicing, a dual-channel image of size 2×128×128 is obtained, and the image mode of the signal is represented as x I .

[0073] The Masked Autoencoder (MAE) method can obtain high-quality image representation and improve the efficiency of pre-training, so the present invention adopts the MAE method for pre-training the image encoder. In order to further enable the model to extract multi-scale features and reduce the computational complexity, a pyramid structure encoder is used. In order to adapt to this structure, a uniform mask is used for the image. Figure 2 As shown, the uniform mask samples 1 block for every 2×2 image block, and the remaining 75% of the image blocks are masked, which means that the masked image blocks will not participate in the image encoding process in the pre-training stage.

[0074] 2) Amplitude and phase preprocessing: The amplitude and phase information of the signal are used to represent the sequence mode, namely the A / P sequence, which consists of two real-valued vectors: amplitude A(n) and phase P(n), that is,

[0075]

[0076] The conversion of amplitude and phase sequences can be expressed as follows:

[0077]

[0078] Among them, i(n) and q(n) are the in-phase and orthogonal parts of the signal, respectively. In order to enable the sequence encoder to extract more effective features from the sequence modality, a preprocessing convolutional network (Pre-CNN) is used. The network consists of two convolutional layers, each followed by a batch normalization (BN) layer and a PReLU activation function. The size of the first convolution kernel is 2×7, and the size of the last convolution kernel is 1×5, with 16 output channels. After Pre-CNN, the sequence modality of the signal is represented as x S , the size is 16×3×N.

[0079] (II) Image Encoder

[0080] like Figure 3 As shown in the middle left figure, the image encoder f μ Input image modality representation x IThe designed image encoder is used for feature extraction. It consists of a Pooling Reduction Transformer (PRFormer) module. The input image size is 2×128×128 and the image block size is 4×4. The encoder first splits the image into 32×32 blocks and then flattens them into vectors of length 32 (4×4×2). The flattened image blocks are then input into the image block encoding module, which consists of a linear projection layer and a normalization layer, followed by fixed position encoding, and finally input into the RPFormer module. The entire encoder consists of four stages. Except for the first stage, the image block encoding module will combine and splice 2×2 image blocks and perform linear projection on the feature dimension. The image output sizes of the four stages are 16×32×32, 32×16×16, 64×8×8, and 128×4×4, respectively.

[0081] The design of Pooling Reduction Attention (PRA) is as follows Figure 3 As shown in the middle right figure. Let Z represent the input feature vector and reshape it into two-dimensional space. Apply average pooling (AvgPool) and depthwise convolution (DWConv) to reduce the spatial dimension:

[0082] P=LayerNorm(DWConv(AvgPool(Z)))

[0083] Where P represents the feature map after dimensionality reduction, and DWConv(·) represents a deep convolution with a kernel size of 1×1. P is a sequence shorter than the input z and can replace the input Z during self-attention. Assuming that the query, key, and value tensors are Q, K, and V, respectively, the calculation process is as follows:

[0084] (Q, K, V) = (ZW q , PW k , PW v )

[0085] Among them, W q , W k and W v are the weight matrices of linear projection, used to generate query, key, and value tensors. Then, Q, K, and V will be fed into the attention module to calculate the attention feature A. The calculation process is as follows:

[0086]

[0087] Among them, d Kis the channel dimension of K, and the Softmax function is applied along the rows of the matrix. Since the lengths of K and V are smaller than z, the proposed pooling dimension reduction attention module has lower complexity than the traditional attention module. The Feed-Forward Network (FFN) follows the design of the traditional Vision Transformer (ViT). The workflow of the PRFormer module is as follows Figure 3 As shown in the middle left figure, the calculation process is as follows:

[0088] Z att =LayerNorm(Z+PRA(Z))

[0089] Z out =LayerNorm(Z att +FFN(Z att ))

[0090] Among them, Z, Z att and Z out They are the input, the output of the pooled dimensionality reduction attention, and the output of the pooled dimensionality reduction transformer.

[0091] The features extracted by PRFormer are finally fed into the global average pooling layer to obtain an image representation vector X of size 128×1×1. I .

[0092] (III) Sequence Encoder

[0093] Sequence encoder f v Input sequence modality x s And use the sequence encoder for feature extraction, which consists of a ConvNeXt (MKA-ConvNeXt) module based on hybrid convolution kernel attention, which is designed as follows Figure 4 As shown, the downsampling layer uses a 1×2 convolution with a stride of 2 for spatial downsampling and channel expansion, and a group convolution (MKA-GConv) based on hybrid convolution kernel attention is used to extract multi-scale features, followed by a layer normalization (LN) for normalization. The first pointwise convolution (PWConv) expands the number of feature map channels through a 1×1 convolution, and then applies the activation function GELU to increase nonlinearity. The purpose of global response normalization (GRN) is to increase the contrast and selectivity of the channel. The last PWConv reduces the number of channels by using a 1×1 convolution. Finally, a residual connection is applied to the entire module.

[0094] MKA-GConv first divides the channels into 4 groups on average and applies convolutions of different sizes to each group to learn multi-scale spatial information. The convolution kernel sizes used are 1×3, 1×5, 1×7, and 1×9, respectively. i Represents the multi-scale spatial information learned from each set of convolutions. In order to effectively measure the importance of features from multi-scale feature maps, the SEWeight module is used to obtain attention weights:

[0095] E i =SEWeight(F i )=Sigmoid(FC(ReLU(FC(AvgPool(F i ))))

[0096] Among them, E i It is from F i The attention weights are recalibrated by the Softmax function to include all spatial position information. The recalibrated multi-scale channel attention weights are then compared with the feature map F of the corresponding scale. i Multiply, the calculation formula is

[0097] H i =F i ⊙Softmax(E i )

[0098] Among them, ⊙ represents channel multiplication, H i is the feature map of multi-scale channel attention weights. The final output process can be written as

[0099]

[0100] in, It represents channel-by-channel splicing. Channel shuffling is used to achieve information fusion between channels to make up for the defect that grouped convolution hinders the flow of information.

[0101] The entire sequence encoder uses three MKA-ConvNeXt modules, and the output feature dimensions of each module are 16×3×64, 32×3×32, and 64×3×16. The extracted features are finally fed into the global average pooling layer to obtain the sequence modality representation vector X of size 64×1×1 S .

[0102] (IV) Reconstruction Header

[0103] The reconstruction head includes a pixel decoder and a reconstruction loss. The purpose of the pixel decoder is to reconstruct the input image using the feature map obtained in the fourth stage of the image encoder. Since it is only used in the pre-training stage, only the ordinary ViT can be used. The reconstruction loss is the mean squared error, and only the masked image blocks between the pixel decoder's prediction and the original image are calculated:

[0104]

[0105] Among them, N m Indicates the number of masked image patches.

[0106] (V) Modal contrast head

[0107] In order to be able to perform modality-level comparison, the two different modality representations extracted by the encoder need to be projected into a representation space of the same dimension. The projection head consists of two fully connected layers with dimensions of 128 and 64 respectively. For the two modality representations of a specific signal sample, the encoder and projection are performed to obtain

[0108]

[0109] Then l2 normalization is applied, and the normalized representation vectors are recorded as and

[0110] The cross-correlation matrix of the normalized representation vectors within a batch is calculated as follows:

[0111]

[0112] Among them, C mn is the element of the cross-correlation matrix between two different vectors along the batch dimension, B is the batch size, and the formula for modality-level contrast loss is

[0113]

[0114] Among them C mm are the diagonal elements of the cross-correlation matrix C, and η is a constant used to weigh the importance of similarity and difference. The cross-correlation matrix C forces different representations of the same sample to be closer and different samples to be farther away.

[0115] (VI) Classification Head

[0116] In order to classify the modulation type of the received signal, the image representation vector X extracted by the image encoder and the sequence encoder is I and the sequence represents the vector X S Stitching it together:

[0117]

[0118] Then two fully connected layers and a Softmax function are used to identify the modulation format as follows:

[0119]

[0120] Dropout is used before the last layer to prevent overfitting, and the number of hidden units in the linear layer is 192 and 11 respectively. Categorical cross entropy is used as the classification loss:

[0121]

[0122] Among them, label smoothing is used to enhance generalization ability, ∈ is a small constant, if it belongs to the kth modulation type, then y b，k =1.

[0123] The present invention uses the RadioML.2016.10A public dataset and the RML24.ETU300 complex multipath channel dataset generated based on GNU Radio. In the HMCF-AMR pre-training stage, the image reconstruction part and the modality-level feature comparison part are optimized simultaneously, and the loss function is L p =L r +L c The dataset used for pre-training is the entire RadioML.2016.10A without any label information. AdamW is used as the optimizer with an initial learning rate of 5e-3 and a batch size of 512. The model is trained for 50 iterations using a cosine learning rate decay strategy. In the HMCF-AMR fine-tuning phase, only L needs to be optimized to update the parameters of the classification task. The dataset used for fine-tuning is divided into a training set (60%), a validation set (20%), and a test set (20%) for training and evaluation. The batch size is set to 256. The model uses the AdamW optimizer with a hierarchical learning rate, i.e., a smaller learning rate of 5e-4 is used except for the classification head to maintain the knowledge of the pre-trained model. An early stopping strategy is used to reduce the overfitting problem, where the learning rate is halved when the validation loss does not improve within 5 iterations. When the validation accuracy does not improve within 10 iterations, the training process is stopped and the model weights at the time of obtaining the maximum validation accuracy are saved. All experiments were performed using PyTorch 2.1.2 based on Python 3.10.13 on a personal computer equipped with an Intel Core i5-13490F CPU @ 2.50GHz and an NVIDIA GeForce RTX 4060Ti GPU (16GB memory).

[0124] In order to verify the effectiveness of the HMCF-AMR model designed in this paper, a comparative experiment was conducted with AMC-Net, AWN, CNN-LSTM, MCformer, MCLDNN, MCNet and PET-CGDNN on the RadioML.2016.10A dataset with sufficient samples. Under the same experimental environment, the classification accuracy of the proposed method and other models under different signal-to-noise ratios (SNR) is shown in the figure below. Figure 5 As shown. It is worth noting that our model performance achieves higher classification accuracy than other methods in almost the entire SNR range. The average classification accuracy of AMC-Net, AWN, CNN-LSTM, MCformer, MCLDNN, MCNet and PET-CGDNN are 61.14%, 61.23%, 55.54%, 58.31%, 61.03%, 55.62% and 61.04% respectively, while our model reaches 62.71%. More importantly, the proposed HMCF-AMR significantly outperforms other selected models, with an improvement of 1.48% compared to the AWN model. In addition, HMCF-AMR can achieve an average classification accuracy of 92.23% when the SNR is above 0dB, and a maximum accuracy of 93.59% at 12dB.

[0125] To provide a more in-depth analysis, Figure 6 The middle left figure shows the confusion matrix of 11 modulation classifications under the entire SNR. The horizontal and vertical axes represent the predicted labels and the true labels, respectively. The diagonal lines represent the correctly identified areas, and the other areas are confused areas. In the figure, we can see that our model is confused when identifying WBFM, because both WBFM and AM-DSB are continuous analog modulations with silent periods, and the distinguishing features between them are not obvious. Although this problem still exists in HMCF-AMR, it has been improved compared to other models. Note that AM-SSB has an extremely high accuracy rate, because the AM-SSB signal waveform is uneven, similar to pure noise. This causes all models to classify signals at low signal-to-noise ratios as AM-SSB, which improves their accuracy. Figure 6 The middle right figure shows the confusion matrix of 11 modulation classifications at 12dB. The recognition accuracy of HMCF-AMR for QAM16, QAM64, 8PSK, and AM-SSB exceeds 90%, and the recognition accuracy for PAM4, QPSK, CPFSK, GMSK, and AM-DSB is close to 100%. Nevertheless, the recognition accuracy of the model for WBFM is not enough, mainly because WBFM is a continuous analog modulation with a silent period. HMCF-AMR also effectively alleviates the confusion between high-order modulation QAM64 and QAM16.

[0126] In a small sample environment, only 1% of the signal samples in the RadioML.2016.10A dataset are used for training, and comparative experiments are conducted with CNN-LSTM, MCformer, MCLDNN, MCNet, and PET-CGDNN. In the same experimental environment, the classification accuracy of the proposed method and other models in a small sample environment is as follows: Figure 7 Our model achieves higher classification accuracy than other methods when the SNR is greater than -4dB, which fully demonstrates the recognition ability of HMCF-AMR in the case of a small number of samples and unlabeled samples. This is due to the pre-training gain of HMCF-AMR, which can effectively obtain the potential features of unlabeled signal samples.

[0127] In a complex multipath environment, the RML24.ETU300 dataset was used for training and compared with CNN-LSTM, MCformer, MCLDNN, MCNet and PET-CGDNN. Under the same experimental environment, the classification accuracy of the proposed method and other models in a complex multipath environment is as follows: Figure 8 Our model achieves a higher classification accuracy than other methods in the entire SNR range, which is much higher than CNN-LSTM, MCformer, MCNet and PET-CGDNN. This fully proves that HMCF-AMR can effectively deal with the problem of environmental migration and has a certain robustness in complex multipath environments, thanks to the signal features obtained in the pre-training stage and the assistance of multimodal representation methods.

Claims

1. A signal modulation recognition method based on hybrid mode contrast and fusion, characterized in that: The following steps are involved: Construct a signal modulation recognition model to automatically recognize the modulation of the received wireless signal and obtain the signal classification result; The signal modulation identification model includes: A time-frequency preprocessing module is used to perform time-frequency analysis on the input signal to obtain its image modality representation; An amplitude-phase preprocessing module is used to perform amplitude-phase conversion on the input signal to obtain its sequence modal representation; An image encoder is used to extract features of the image modality representation of the input signal to obtain image modality features; A sequence encoder is used to extract features from the sequence modal representation of the input signal to obtain sequence modal features; A classification head module, used to generate modulation classification results according to image modality features and sequence modality features; The signal modulation recognition model is trained by the following method: Unlabeled bimodal contrastive self-supervised pre-training of the signal modulation recognition model to optimize the parameters of the image encoder and sequence encoder; The pre-trained signal modulation recognition model is subjected to supervised training to obtain a trained signal modulation recognition model.

2. The method according to claim 1, characterized in that The unlabeled bimodal contrastive self-supervised pre-training of the signal modulation recognition model includes: Reconstructing images based on image modality features; Project the image modality features and the sequence modality features into the same dimensional space to obtain the image representation vector and the sequence representation vector; The modality-level contrast loss is calculated using the cross-correlation between the image representation vector and the sequence representation vector. The parameters of the image encoder and sequence encoder are optimized according to the reconstruction loss and the modality-level contrastive loss.

3. The method according to claim 2, characterized in that The supervised training of the pre-trained signal modulation recognition model is based on the classification loss, which is expressed as Among them, L f is the classification loss, b is the sample index, B is the batch size, k is the modulation type index, K is the total number of modulation types, ∈ is a constant, and y b,k and are the true label and predicted probability respectively.

4. The method according to claim 2, characterized in that: During pre-training, the time-frequency pre-processing module is also used to uniformly mask the obtained image modality representation before outputting it to the image encoder.

5. The method according to claim 2, characterized in that: The reconstruction loss is expressed as Among them, L r is the reconstruction loss, b is the sample index, B is the batch size, n is the image block index, and N m is the number of masked image blocks, x I and are the original and reconstructed image modality representations respectively.

6. The method according to claim 2, characterized in that: The modality-level contrast loss is expressed as Among them, L c is the modality-level contrast loss, C mn is the element of the cross-correlation matrix C calculated along the batch dimension between two different vectors, C mm are the diagonal elements of the cross-correlation matrix C, η is a constant, and They are the normalized image representation vector and sequence representation vector respectively.

7. The method according to claim 1, characterized in that The image encoder module comprises: The pooling dimension reduction transformer is used to reshape the image modality representation of the input signal into a two-dimensional space, and then perform multi-scale image feature extraction on the reshaped vector by introducing an attention mechanism; The first global average pooling layer is used to convert the obtained multi-scale image features into a set feature dimension to obtain an image modality feature vector.

8. The method according to claim 1, characterized in that The sequence encoder module comprises: Convolutional network, which is used to extract multi-scale temporal features from the sequence modal representation of the input signal by introducing an attention mechanism; The second global average pooling layer is used to convert the obtained multi-scale time series features into a set feature dimension to obtain a sequence modal feature vector.

9. The method according to claim 1, characterized in that: The classification header module includes: A feature concatenation module is used to concatenate image modality features and sequence modality features to obtain fusion features; The classifier is used to generate modulation classification results based on the fused features.

Images