Identification method of multi-model fusion signal modulation mode based on time-frequency diagram

Abstract

The invention discloses a method for identifying a multi-model fusion signal modulation mode based on a time-frequency diagram, and belongs to the field of signal processing. Preprocessing the original signal to obtain a signal sample; constructing a data set by using the normalized signal samples; constructing a fusion model; training a modulation identification module in the fusion model; performing modulation mode identification on an input unknown signal by using the fusion model to obtain a preliminary identification result; and performing comprehensive judgment on the preliminary recognition result of the fusion model by using a comprehensive judgment device to obtain a final recognition result. According to the invention, by combining a plurality of time-frequency analysis methods, the time-frequency domain feature information of the signal is fully extracted; the reliability and robustness of a final decision are improved by adopting a multi-model fusion framework, and the modulation recognition performance of the system under the condition of a low signal-to-noise ratio is improved.

Description

Technical Field

[0001] The present invention relates to the field of signal processing, and in particular to the field of communication signal modulation mode identification. Background Art

[0002] With the rapid development of wireless communication technology, automatic signal modulation identification plays a vital role in a variety of fields, including communication monitoring, spectrum management, cognitive radio, signal interception, and electromagnetic spectrum monitoring. Modulation identification technology is particularly important in military communications, commercial communications, and wireless communication security. Currently, signal modulation identification methods fall into two main categories: traditional feature extraction-based methods and novel deep learning-based methods.

[0003] Traditional feature-based methods primarily identify signal modulation by extracting instantaneous, statistical, and transform-domain features. Common features include high-order cumulants, high-order statistics, power spectrum features, and wavelet transform coefficients. These methods are characterized by low computational complexity, clear physical meaning, and good recognition performance under specific conditions. However, traditional methods rely heavily on the manual design of features, significantly degrading performance in low signal-to-noise ratio environments, limiting their practicality, and being sensitive to non-ideal factors such as frequency and phase offsets, resulting in insufficient robustness.

[0004] With the advancement of machine learning, signal modulation recognition methods based on deep learning have become a research hotspot. These methods primarily leverage the powerful feature learning capabilities of neural networks to directly learn the features required for recognition from the raw IQ signals or their transformed domain representations. Currently, a variety of deep learning models have been applied to modulation recognition, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep residual networks. These methods offer superior feature extraction capabilities and higher recognition accuracy than traditional methods.

[0005] However, existing modulation recognition methods still have some problems:

[0006] 1. The single signal feature representation adopted by most methods cannot fully utilize the rich feature information of the signal in different domains, which limits the performance of the system;

[0007] 2. Under low signal-to-noise ratio conditions, the recognition performance of existing methods is still not ideal;

[0008] 3. Most current methods rely on a single model for recognition decisions, but the recognition results of a single model are not reliable enough;

[0009] Existing systems have poor robustness to non-ideal factors such as signal frequency offset, phase offset, and time offset. Summary of the Invention

[0010] In response to the above problems, the present invention proposes a method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram, which belongs to the field of signal processing. The original signal is preprocessed to obtain a signal sample; a data set is constructed using the normalized signal sample; a fusion model is constructed; the modulation identification module in the fusion model is trained; the fusion model is used to identify the modulation mode of the input unknown signal to obtain a preliminary identification result; a comprehensive decision is made on the preliminary identification result of the fusion model using a comprehensive decision device to obtain a final identification result. The present invention fully extracts the time-frequency domain feature information of the signal by combining a variety of time-frequency analysis methods; a multi-model fusion framework is used to improve the reliability and robustness of the final decision, and the modulation recognition performance of the system under low signal-to-noise ratio conditions is improved.

[0011] A method for identifying a modulation mode of a multi-model fusion signal based on a time-frequency diagram comprises the following steps:

[0012] Step 1: preprocess the original signal to obtain a signal sample; the original signal is an ideal modulated signal, that is, a baseband signal without noise and frequency deviation; the sample signal includes a signal sequence and a signal sequence modulation mode label;

[0013] Step 2: construct a data set using the signal samples: the data set includes a training set and a test set, with the ratio of the training set to the test set being 7:3;

[0014] Step 3: Construct five heterogeneous models respectively, the five heterogeneous models including a first model, a second model, a third model, a fourth model, and a fifth model; the first model includes a first signal processing module and a first modulation identification module; the second model includes a second signal processing module and a second modulation identification module; the third model includes a third signal processing module and a third modulation identification module; the fourth model includes a fourth signal processing module and a fourth modulation identification module; the fifth model includes a fifth signal processing module and a fifth modulation identification module;

[0015] Step 4: independently train and test the five modulation recognition modules to obtain the trained and tested first model, second model, third model, fourth model, and fifth model;

[0016] Step 5: Input the signal to be identified into the five trained heterogeneous models respectively for modulation recognition to obtain preliminary recognition results;

[0017] Step 6: Use the comprehensive judge to make a comprehensive judgment on the preliminary recognition results of the five heterogeneous models to obtain the final recognition result.

[0018] Furthermore, in step 1, the sample signal includes 8 digital modulation modes, which are BPSK, QPSK, 8PSK, 16QAM, 64QAM, OFDM, FSK and MSK;

[0019] The original signal preprocessing process is as follows:

[0020] First, AWGN (Additive White Gaussian Noise) is superimposed on the original signal s(t);

[0021] Next, random frequency offset is introduced on the basis of superimposing AWGN. The calculation formula of the signal s'(t) after superimposing the frequency offset is:

[0022] s'(t)=s(t)e (j2πΔft) ,

[0023] Where Δf is the random frequency deviation, j is the imaginary unit, and t is the time variable;

[0024] Finally, the frequency-shifted signal s'(t) is normalized to eliminate the differences in signal amplitude and power, and the normalized signal sequence x'(n) is obtained:

[0025] x'(n)=x(n) / max(|x(n)|).

[0026] Furthermore, in step 3: the five signal processing modules are all the same; each signal processing module undergoes digital filtering, time-frequency conversion and time-frequency diagram preprocessing in sequence.

[0027] Furthermore, the digital filtering adopts an FIR filter, and the digital filtering adopts a Hamming window design; the frequency response H(f) of the FIR filter is:

[0028]

[0029] Where h(n) represents the impulse response sequence of the filter, the length of the impulse response sequence is N, and it is designed by the Hamming window; f represents the frequency variable; the unit of the frequency variable is Hz; j represents the imaginary unit; and n represents the discrete time index (n = 0, 1, ..., N-1).

[0030] Furthermore, the time-frequency conversion includes five different time-frequency conversion methods, and each signal processing module adopts a different time-frequency conversion method. The time-frequency conversion method adopted by the first signal processing module is short-time Fourier transform (STFT); the time-frequency conversion method adopted by the second signal processing module is wavelet transform (WT); the time-frequency conversion method adopted by the third signal processing module is Hilbert-Huang transform (HHT); the time-frequency conversion method adopted by the fourth signal processing module is Wigner-Ville distribution (WVD); and the time-frequency conversion method adopted by the fifth signal processing module is Chirplet transform.

[0031] The short-time Fourier transform STFT(t,f):

[0032]

[0033] Where x(τ) represents the input signal; ω(τ-t) represents the Hamming window function; τ is the time delay variable;

[0034] During the transformation, STFT is performed frame by frame according to the sampling point step. The result of multiple-frame STFT is the STFT time-frequency diagram.

[0035] The wavelet transform WT(a,b,f):

[0036]

[0037] Among them, x(τ) represents the input signal; ψ is the wavelet basis function, and Morlet wavelet is selected as the basis function, ψ * is the complex conjugate of ψ; a represents the scale parameter (a>0), which controls the frequency resolution; b represents the displacement parameter; the scale range is adaptively selected according to the frequency characteristics of the signal to ensure that the frequency components of the signal are covered;

[0038] The Hilbert-Huang transform HHT(t, f):

[0039]

[0040] Among them, A k (t) represents the instantaneous amplitude of the kth intrinsic mode function (IMF); φ k (t) represents the instantaneous phase of the kth IMF, φ k (t)=∫ω k (t)dt,ω k (t) represents the instantaneous frequency of the kth IMF; K represents the total number of IMFs; the Hilbert-Huang transform is to obtain the intrinsic mode function (IMF) through empirical mode decomposition (EMD), and then perform Hilbert transform (Hilbert) on each IMF;

[0041] The Wigner-Ville distribution WVD(t,f):

[0042]

[0043] in, represents the analytical signal, obtained by Hilbert transform; Wigner-Ville distribution uses the analytical signal to eliminate cross terms and uses a Gaussian smoothing kernel to reduce cross term interference; for The complex conjugate of

[0044] The Chirplet transform Chirp(t, f):

[0045]

[0046] Where x(τ) represents the input signal; g(τ-t) represents the Gaussian window function; β is the chirp rate parameter; and the Chirplet transform is adaptively adjusted according to the signal characteristics.

[0047] Furthermore, the time-frequency graph preprocessing uses non-local mean filtering (NLM) to remove artifacts and noise points in the time-frequency graph, normalizes the pixel values of the time-frequency graph to [0, 1], and uses histogram equalization to enhance the contrast of the time-frequency graph and improve the separability of signal features;

[0048] The time-frequency graph preprocessing process is as follows:

[0049] First, the pixel values of the time-frequency graph are obtained after NLM filtering.

[0050]

[0051] Where n represents the position of the current pixel to be processed, m represents the positions of other pixels in the neighborhood, x(m) represents the pixel value of the image to be processed; ω(n,m) represents the weight function; Z(n) represents the normalization factor, and Ω represents the set range of the pixel neighborhood. When calculating the filter value of pixel n, all possible pixel positions m need to be traversed, but in actual operation, it is limited to a reasonable neighborhood range.

[0052] Then, the pixel values of the time-frequency graph are normalized to the interval [0,1]. Calculation formula:

[0053]

[0054] Among them, p(n) is the pixel value after NLM filtering p(n) min and p(n) maxare the minimum pixel value and maximum pixel value of the time-frequency graph respectively;

[0055] Finally, histogram equalization is used to enhance the contrast of the time-frequency image and obtain the output grayscale s:

[0056]

[0057] Among them, r is the input gray level; L is the total number of gray levels; P r (ω) is the gray level probability density function of the input image, ω represents the integral variable, that is, the dummy variable, which has no actual physical meaning and only represents the intermediate variable in the integration process; T(r) represents the cumulative distribution function mapping, which maps the input gray level r to the output gray level s.

[0058] Furthermore, in step 3, the five modulation recognition modules respectively adopt five different deep learning network structures:

[0059] The first modulation recognition module uses a MobileNetV2 lightweight network; the MobileNetV2 lightweight network includes 17 layers of depth-wise separable convolutional layers; the number of channels in the backbone of the MobileNetV2 lightweight network is [32, 16, 24, 32, 64, 96, 160, 320], and the inverted residual structure uses an expansion-first-then-compression approach with an expansion ratio of 6; each of the 17 layers of depth-wise separable convolutional layers is followed by batch normalization and a ReLU activation function;

[0060] The second modulation recognition module adopts the Inception multi-scale feature fusion network; the Inception multi-scale feature fusion network includes 5 serial Inception modules; each Inception module includes four parallel branches, which are respectively a 1×1 convolution layer branch, a 1×1 convolution layer connected to a 3×3 convolution layer branch, a 1×1 convolution layer connected to a 5×5 convolution layer branch, and a 3×3 pooling layer connected to a 1×1 convolution layer branch; the channel configurations of the 5 serial Inception modules are [64, (96, 128), (16, 32), 32], [128, (128, 192), (32, 96), 64], [192, (96, 208), (16, 48), 64], [160, (112, 224), (24, 64), 64], and [128, (128, 256), (24, 64), 64];

[0061] The third modulation identification module adopts a ResNet residual connection network; the ResNet residual connection network includes 4 residual stages; the number of channels in the 4 residual stages is [64, 128, 256, 512]; each residual stage contains 2 Bottleneck residual blocks, the compression ratio of the Bottleneck residual block is 4, and the Bottleneck residual block is a three-layer structure, which includes a compression channel layer, a feature extraction layer, and a channel expansion layer in sequence; each layer of the three-layer structure is followed by batch normalization and a ReLU activation function; the compression channel layer is a 1×1 convolution, the feature extraction layer is a 3×3 convolution, and the channel expansion layer is a 1×1 convolution; the Bottleneck residual block implements residual learning through a shortcut connection;

[0062] The fourth modulation recognition module adopts a channel-spatial attention mechanism network (CNN+CBAM network for short); the CNN+CBAM network includes a convolutional layer block and a fully connected layer block; the convolutional layer block has 5 convolutional layers, and the fully connected layer block has 3 fully connected layers; the channel attention module and the spatial attention module are sequentially stacked between the convolutional layer block and the fully connected layer block;

[0063] The channel attention module includes global average pooling and a fully connected layer in sequence. The channel compression ratio of the channel attention module is 16. The channel attention module obtains channel weights through parallel processing of maximum pooling and average pooling and then passes through a multi-layer perceptron (MLP). The spatial attention module uses a 7×7 convolution kernel to generate a spatial weight map.

[0064] The fifth modulation recognition module adopts a time-frequency dual-stream network, which includes a time domain branch and a frequency domain branch; the time domain branch uses three layers of one-dimensional convolution to extract time domain features, and the convolution kernel lengths of the three layers of one-dimensional convolution are [9, 7, 5], and the number of channels are [64, 128, 256]; the frequency domain branch uses three layers of two-dimensional convolution to extract frequency domain features, and the convolution kernel sizes of the three layers of two-dimensional convolution are [5×5, 3×3, 3×3], and the number of channels are [64, 128, 256]; the features of the time domain branch and the frequency domain branch are fused through the attention mechanism.

[0065] Furthermore, in step 4, the modulation recognition module training method is as follows;

[0066] First, the five modulation recognition modules described in step 3 are fed with the signal data from the training set, which are time-frequency graphs and their modulation labels. The output target after training is an 8-dimensional probability vector, which represents the predicted probability distribution of the eight digital modulation modes.

[0067] Then, the parameters of the five modulation recognition modules are optimized respectively using the cross entropy loss function L:

[0068] L=-Σy i log(p i )

[0069] Among them, y i is the true label, p i is the predicted probability;

[0070] Finally, the Adam optimizer is used to update the parameters of each modulation recognition module. To avoid overfitting, the Adam optimizer uses the following three strategies to optimize the parameters of the modulation recognition module:

[0071] Dropout strategy: The dropout rates of the five modulation recognition modules are all set to 0.2-0.5;

[0072] Early stopping strategy: stop training when the validation set loss does not improve for 5 consecutive rounds;

[0073] Data augmentation strategy: The time-frequency graph is rotated by ±10°, translated by ±5 pixels, and scaled by 0.9-1.1.

[0074] Furthermore, in step 5, the modulation mode identification process of the signal to be identified is as follows:

[0075] Step 5.1: The signal to be identified is processed sequentially by the first signal processing module, the second signal processing module, the third signal processing module, the fourth signal processing module, and the fifth signal processing module to generate five time-frequency diagrams, which are STFT time-frequency diagram, WT time-frequency diagram, HHT time-frequency diagram, WVD time-frequency diagram, and Chirplet transform time-frequency diagram.

[0076] The time-frequency graphs are processed in different ways before being fed into the five modulation recognition modules:

[0077] For the first modulation identification module to the third modulation identification module: standard normalization is used for preprocessing;

[0078] For the fourth modulation recognition module, maximum and minimum value normalization is used for preprocessing;

[0079] For the fifth modulation identification module, the time domain and frequency domain features are normalized respectively;

[0080] In step 5.2, the STFT time-frequency map is input into the first modulation recognition module, the WT time-frequency map is input into the second modulation recognition module, the HHT time-frequency map is input into the third modulation recognition module, the WVD time-frequency map is input into the fourth modulation recognition module, and the Chirplet transform time-frequency map is input into the fifth modulation recognition module, thereby obtaining five independent prediction results; each prediction result includes category probability, confidence and intermediate features; the intermediate features are used for subsequent fusion.

[0081] Furthermore, in step 6, the comprehensive decision maker adopts a fusion method of the hybrid mechanism of Transformer and MoE expert;

[0082] The comprehensive decision process of the comprehensive decision device is as follows:

[0083] Step 6.1, preprocessing and feature mapping:

[0084] The class probability, confidence, and intermediate features are concatenated into a set of vectors and uniformly mapped to a 256-dimensional feature space through a fully connected layer to ensure that the output data of the five models are in the same format.

[0085] Step 6.2, Transformer encoding:

[0086] The 256-dimensional feature vectors are combined into a 5×256-dimensional matrix and input into a two-layer multi-head self-attention Transformer for encoding. This is used to capture the complementary information between the five models and enhance the feature expression capability. The output of the Transformer encoding is still a 5×256-dimensional matrix, representing five sets of enhanced features.

[0087] Step 6.3, adaptive temperature and confidence estimation:

[0088] Calculate the temperature coefficient T of the five models using a three-layer perceptron (MLP) k and confidence C k The structure of the three-layer perceptron includes an input layer of 64 neurons, a hidden layer of 32 neurons, and an output layer of 1 neuron; the temperature coefficient T k Activate by softplus and ensure positive value, temperature coefficient T k Dynamic adjustment based on prediction entropy; confidence level C k Activated by sigmoid, the value is between (0,1);

[0089] Step 6.4, joint feature construction:

[0090] The enhanced features and temperature coefficient T output by Transformer k and confidence C k Spliced into a 258-dimensional joint feature vector h k , h k ∈R 258 ; Then 5 h k Spliced into a 1290-dimensional global fusion vector R 258 Represents a 258-dimensional vector space; R 1290 Represents a 1290-dimensional vector space;

[0091] Step 6.5, MoE Expert Mixing:

[0092] The global fusion vector The input is sent to the MoE module, which includes a gating network and eight expert networks. The outputs of the eight expert networks are weighted and fused according to the gating weights to obtain a comprehensive judgment result.

[0093] The gating network consists of a linear layer and a softmax layer, and the weight matrix of the linear layer of the gating network is 1290×4. The gating network generates expert weights. Each of the eight expert networks uses a two-layer MLP to generate candidate logits. The first layer of the two-layer MLP fully connected mapping compresses the 1290-dimensional input to 512 dimensions, and the second layer mapping further compresses it to 8 dimensions, matching the number of modulation categories of 8.

[0094] Step 6.6, output: Use softmax operation on the logits after MoE expert network fusion to output the final modulation category probability distribution; the logits are unnormalized prediction scores, representing the model's original confidence in each category; the softmax operation is a normalized exponential function, which maps the logits to a probability distribution in the interval [0,1].

[0095] The beneficial effects of the present invention are as follows: the present invention fully extracts the time-frequency domain feature information of the signal by combining multiple time-frequency analysis methods; adopts a multi-model fusion framework to improve the reliability and robustness of the final decision, and improves the modulation recognition performance of the system under low signal-to-noise ratio conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is the overall block diagram of the signal modulation identification method of the present invention;

[0097] Figure 2 It is a principle block diagram of the signal processing module of the present invention;

[0098] Figure 3 It is a principle block diagram of the comprehensive decision maker of the present invention. DETAILED DESCRIPTION

[0099] The present invention will be further described below with reference to the accompanying drawings and examples.

[0100] like Figure 1 As shown:

[0101] A method for identifying a modulation mode of a multi-model fusion signal based on a time-frequency diagram comprises the following steps:

[0102] Step 1: preprocess the original signal to obtain a signal sample; the original signal is an ideal modulated signal, that is, a baseband signal without noise and frequency deviation; the sample signal includes a signal sequence and a signal sequence modulation mode label;

[0103] Step 2: construct a data set using the signal samples: the data set includes a training set and a test set, with the ratio of the training set to the test set being 7:3;

[0104] Step 3: Construct five heterogeneous models respectively, the five heterogeneous models including a first model, a second model, a third model, a fourth model, and a fifth model; the first model includes a first signal processing module and a first modulation identification module; the second model includes a second signal processing module and a second modulation identification module; the third model includes a third signal processing module and a third modulation identification module; the fourth model includes a fourth signal processing module and a fourth modulation identification module; the fifth model includes a fifth signal processing module and a fifth modulation identification module;

[0105] Step 4: independently train and test the five modulation recognition modules to obtain the trained and tested first model, second model, third model, fourth model, and fifth model;

[0106] Step 5: Input the signal to be identified into the five trained heterogeneous models respectively for modulation recognition to obtain preliminary recognition results;

[0107] Step 6: Use the comprehensive judge to make a comprehensive judgment on the preliminary recognition results of the five heterogeneous models to obtain the final recognition result.

[0108] Furthermore, in step 1, the sample signal includes 8 digital modulation modes, which are BPSK, QPSK, 8PSK, 16QAM, 64QAM, OFDM, FSK and MSK;

[0109] The original signal preprocessing process is as follows:

[0110] First, AWGN (Additive White Gaussian Noise) is superimposed on the original signal s(t);

[0111] Next, random frequency offset is introduced on the basis of superimposing AWGN. The calculation formula of the signal s'(t) after superimposing the frequency offset is:

[0112] s'(t)=s(t)e (j2πΔft) ,

[0113] Where Δf is the random frequency deviation, j is the imaginary unit, and t is the time variable;

[0114] Finally, the frequency-shifted signal s'(t) is normalized to eliminate the differences in signal amplitude and power, and the normalized signal sequence x'(n) is obtained:

[0115] x'(n)=x(n) / max(|x(n)|).

[0116] like Figure 2 As shown:

[0117] Furthermore, in step 3: the five signal processing modules are all the same; each signal processing module undergoes digital filtering, time-frequency conversion and time-frequency diagram preprocessing in sequence.

[0118] Furthermore, the digital filtering adopts an FIR filter, and the digital filtering adopts a Hamming window design; the frequency response H(f) of the FIR filter is:

[0119]

[0120] Where h(n) represents the impulse response sequence of the filter, the length of the impulse response sequence is N, and it is designed by the Hamming window; f represents the frequency variable; the unit of the frequency variable is Hz; j represents the imaginary unit; and n represents the discrete time index (n = 0, 1, ..., N-1).

[0121] Furthermore, the time-frequency conversion includes five different time-frequency conversion methods, and each signal processing module adopts a different time-frequency conversion method. The time-frequency conversion method adopted by the first signal processing module is short-time Fourier transform (STFT); the time-frequency conversion method adopted by the second signal processing module is wavelet transform (WT); the time-frequency conversion method adopted by the third signal processing module is Hilbert-Huang transform (HHT); the time-frequency conversion method adopted by the fourth signal processing module is Wigner-Ville distribution (WVD); and the time-frequency conversion method adopted by the fifth signal processing module is Chirplet transform.

[0122] The short-time Fourier transform STFT(t, f):

[0123]

[0124] Where x(τ) represents the input signal; ω(τ-t) represents the Hamming window function; τ is the time delay variable;

[0125] During the transformation, STFT is performed frame by frame according to the sampling point step. The result of multiple-frame STFT is the STFT time-frequency diagram.

[0126] The wavelet transform WT(a, b, f):

[0127]

[0128] Among them, x(τ) represents the input signal; ψ is the wavelet basis function, and Morlet wavelet is selected as the basis function, ψ * is the complex conjugate of ψ; a represents the scale parameter (a>0), which controls the frequency resolution; b represents the displacement parameter; the scale range is adaptively selected according to the frequency characteristics of the signal to ensure that the frequency components of the signal are covered;

[0129] The Hilbert-Huang transform HHT(t, f):

[0130]

[0131] Among them, A k (t) represents the instantaneous amplitude of the kth intrinsic mode function (IMF); φ k (t) represents the instantaneous phase of the kth IMF, φ k (t)=∫ω k (t)dt,ω k (t) represents the instantaneous frequency of the kth IMF; K represents the total number of IMFs; the Hilbert-Huang transform is to obtain the intrinsic mode function (IMF) through empirical mode decomposition (EMD), and then perform Hilbert transform (Hilbert) on each IMF;

[0132] The Wigner-Ville distribution WVD(t,f):

[0133]

[0134] in, represents the analytical signal, obtained by Hilbert transform; Wigner-Ville distribution uses the analytical signal to eliminate cross terms and uses a Gaussian smoothing kernel to reduce cross term interference; for The complex conjugate of

[0135] The Chirplet transform Chirp(t, f):

[0136]

[0137] Where x(τ) represents the input signal; g(τ-t) represents the Gaussian window function; β is the chirp rate parameter; and the Chirplet transform is adaptively adjusted according to the signal characteristics.

[0138] Furthermore, the time-frequency graph preprocessing uses non-local mean filtering (NLM) to remove artifacts and noise points in the time-frequency graph, normalizes the pixel values of the time-frequency graph to [0, 1], and uses histogram equalization to enhance the contrast of the time-frequency graph and improve the separability of signal features;

[0139] The time-frequency graph preprocessing process is as follows:

[0140] First, the pixel values of the time-frequency graph are obtained after NLM filtering.

[0141]

[0142] Where n represents the position of the current pixel to be processed, m represents the positions of other pixels in the neighborhood, x(m) represents the pixel value of the image to be processed; ω(n,m) represents the weight function; Z(n) represents the normalization factor, and Ω represents the set range of the pixel neighborhood. When calculating the filter value of pixel n, all possible pixel positions m need to be traversed, but in actual operation, it is limited to a reasonable neighborhood range.

[0143] Then, the pixel values of the time-frequency graph are normalized to the interval [0,1]. Calculation formula:

[0144]

[0145] Among them, p(n) is the pixel value after NLM filtering p(n) min and p(n) max are the minimum pixel value and maximum pixel value of the time-frequency graph respectively;

[0146] Finally, histogram equalization is used to enhance the contrast of the time-frequency image and obtain the output grayscale s:

[0147]

[0148] Among them, r is the input gray level; L is the total number of gray levels; P r (ω) is the gray level probability density function of the input image, ω represents the integral variable, that is, the dummy variable, which has no actual physical meaning and only represents the intermediate variable in the integration process; T(r) represents the cumulative distribution function mapping, which maps the input gray level r to the output gray level s.

[0149] Furthermore, in step 3, the five modulation recognition modules respectively adopt five different deep learning network structures:

[0150] The first modulation recognition module uses a MobileNetV2 lightweight network; the MobileNetV2 lightweight network includes 17 layers of depth-wise separable convolutional layers; the number of channels in the backbone of the MobileNetV2 lightweight network is [32, 16, 24, 32, 64, 96, 160, 320], and the inverted residual structure uses an expansion-first-then-compression approach with an expansion ratio of 6; each of the 17 layers of depth-wise separable convolutional layers is followed by batch normalization and a ReLU activation function;

[0151] The second modulation recognition module adopts the Inception multi-scale feature fusion network; the Inception multi-scale feature fusion network includes 5 serial Inception modules; each Inception module includes four parallel branches, which are respectively a 1×1 convolution layer branch, a 1×1 convolution layer connected to a 3×3 convolution layer branch, a 1×1 convolution layer connected to a 5×5 convolution layer branch, and a 3×3 pooling layer connected to a 1×1 convolution layer branch; the channel configurations of the 5 serial Inception modules are [64, (96, 128), (16, 32), 32], [128, (128, 192), (32, 96), 64], [192, (96, 208), (16, 48), 64], [160, (112, 224), (24, 64), 64], and [128, (128, 256), (24, 64), 64];

[0152] The third modulation identification module adopts a ResNet residual connection network; the ResNet residual connection network includes 4 residual stages; the number of channels in the 4 residual stages is [64, 128, 256, 512]; each residual stage contains 2 Bottleneck residual blocks, the compression ratio of the Bottleneck residual block is 4, and the Bottleneck residual block is a three-layer structure, which includes a compression channel layer, a feature extraction layer, and a channel expansion layer in sequence; each layer of the three-layer structure is followed by batch normalization and a ReLU activation function; the compression channel layer is a 1×1 convolution, the feature extraction layer is a 3×3 convolution, and the channel expansion layer is a 1×1 convolution; the Bottleneck residual block implements residual learning through a shortcut connection;

[0153] The fourth modulation recognition module adopts a channel-spatial attention mechanism network (CNN+CBAM network for short); the CNN+CBAM network includes a convolutional layer block and a fully connected layer block; the convolutional layer block has 5 convolutional layers, and the fully connected layer block has 3 fully connected layers; the channel attention module and the spatial attention module are sequentially stacked between the convolutional layer block and the fully connected layer block;

[0154] The channel attention module includes global average pooling and a fully connected layer in sequence. The channel compression ratio of the channel attention module is 16. The channel attention module obtains channel weights through parallel processing of maximum pooling and average pooling and then passes through a multi-layer perceptron (MLP). The spatial attention module uses a 7×7 convolution kernel to generate a spatial weight map.

[0155] The fifth modulation recognition module adopts a time-frequency dual-stream network, which includes a time domain branch and a frequency domain branch; the time domain branch uses three layers of one-dimensional convolution to extract time domain features, and the convolution kernel lengths of the three layers of one-dimensional convolution are [9, 7, 5], and the number of channels are [64, 128, 256]; the frequency domain branch uses three layers of two-dimensional convolution to extract frequency domain features, and the convolution kernel sizes of the three layers of two-dimensional convolution are [5×5, 3×3, 3×3], and the number of channels are [64, 128, 256]; the features of the time domain branch and the frequency domain branch are fused through the attention mechanism.

[0156] Furthermore, in step 4, the modulation recognition module training method is as follows;

[0157] First, the five modulation recognition modules described in step 3 are fed with the signal data from the training set, which are time-frequency graphs and their modulation labels. The output target after training is an 8-dimensional probability vector, which represents the predicted probability distribution of the eight digital modulation modes.

[0158] Then, the parameters of the five modulation recognition modules are optimized respectively using the cross entropy loss function L:

[0159] L=-Σy i log(p i )

[0160] Among them, y i is the true label, p i is the predicted probability;

[0161] Finally, the Adam optimizer is used to update the parameters of each modulation recognition module. To avoid overfitting, the Adam optimizer uses the following three strategies to optimize the parameters of the modulation recognition module:

[0162] Dropout strategy: The dropout rates of the five modulation recognition modules are all set to 0.2-0.5;

[0163] Early stopping strategy: stop training when the validation set loss does not improve for 5 consecutive rounds;

[0164] Data augmentation strategy: The time-frequency graph is rotated by ±10°, translated by ±5 pixels, and scaled by 0.9-1.1.

[0165] Furthermore, in step 5, the modulation mode identification process of the signal to be identified is as follows:

[0166] Step 5.1: The signal to be identified is processed sequentially by the first signal processing module, the second signal processing module, the third signal processing module, the fourth signal processing module, and the fifth signal processing module to generate five time-frequency diagrams, which are STFT time-frequency diagram, WT time-frequency diagram, HHT time-frequency diagram, WVD time-frequency diagram, and Chirplet transform time-frequency diagram.

[0167] The time-frequency graphs are processed in different ways before being fed into the five modulation recognition modules:

[0168] For the first modulation identification module to the third modulation identification module: standard normalization is used for preprocessing;

[0169] For the fourth modulation recognition module, maximum and minimum value normalization is used for preprocessing;

[0170] For the fifth modulation identification module, the time domain and frequency domain features are normalized respectively;

[0171] In step 5.2, the STFT time-frequency map is input into the first modulation recognition module, the WT time-frequency map is input into the second modulation recognition module, the HHT time-frequency map is input into the third modulation recognition module, the WVD time-frequency map is input into the fourth modulation recognition module, and the Chirplet transform time-frequency map is input into the fifth modulation recognition module, thereby obtaining five independent prediction results; each prediction result includes category probability, confidence and intermediate features; the intermediate features are used for subsequent fusion.

[0172] like Figure 3 As shown:

[0173] Furthermore, in step 6, the comprehensive decision maker adopts a fusion method of the hybrid mechanism of Transformer and MoE expert;

[0174] The comprehensive decision process of the comprehensive decision device is as follows:

[0175] Step 6.1, preprocessing and feature mapping:

[0176] The class probability, confidence, and intermediate features are concatenated into a set of vectors and uniformly mapped to a 256-dimensional feature space through a fully connected layer to ensure that the output data of the five models are in the same format.

[0177] Step 6.2, Transformer encoding:

[0178] The 256-dimensional feature vectors are combined into a 5×256-dimensional matrix and input into a two-layer multi-head self-attention Transformer for encoding. This is used to capture the complementary information between the five models and enhance the feature expression capability. The output of the Transformer encoding is still a 5×256-dimensional matrix, representing five sets of enhanced features.

[0179] Step 6.3, adaptive temperature and confidence estimation:

[0180] Calculate the temperature coefficient T of the five models using a three-layer perceptron (MLP) k and confidence C k The structure of the three-layer perceptron includes an input layer of 64 neurons, a hidden layer of 32 neurons, and an output layer of 1 neuron; the temperature coefficient T k Activate by softplus and ensure positive value, temperature coefficient T k Dynamic adjustment based on prediction entropy; confidence level C k Activated by sigmoid, the value is between (0,1);

[0181] Step 6.4, joint feature construction:

[0182] The enhanced features and temperature coefficient T output by Transformer k and confidence C k Spliced into a 258-dimensional joint feature vector h k , h k ∈R 258 ; Then 5 h k Spliced into a 1290-dimensional global fusion vector R 258 Represents a 258-dimensional vector space; R 1290 Represents a 1290-dimensional vector space;

[0183] Step 6.5, MoE Expert Mixing:

[0184] The global fusion vector The input is sent to the MoE module, which includes a gating network and eight expert networks. The outputs of the eight expert networks are weighted and fused according to the gating weights to obtain a comprehensive judgment result.

[0185] The gating network consists of a linear layer and a softmax layer, and the weight matrix of the linear layer of the gating network is 1290×4. The gating network generates expert weights. Each of the eight expert networks uses a two-layer MLP to generate candidate logits. The first layer of the two-layer MLP fully connected mapping compresses the 1290-dimensional input to 512 dimensions, and the second layer mapping further compresses it to 8 dimensions, matching the number of modulation categories of 8.

[0186] Step 6.6, output: Use softmax operation on the logits after MoE expert network fusion to output the final modulation category probability distribution; the logits are unnormalized prediction scores, representing the model's original confidence in each category; the softmax operation is a normalized exponential function, which maps the logits to a probability distribution in the interval [0,1].

[0187] Example 1:

[0188] 1. In this embodiment, the experimental environment is as follows:

[0189] Hardware environment: Ryzen 5600X CPU, NVIDIA RTX 2060SUPER GPU, 32GB RAM;

[0190] Software environment: Python 3.12, PyTorch 2.3, CUDA 11.8;

[0191] 2. In this embodiment, the signal-to-noise ratio range of the signal samples collected in step 1 is {-20dB, -15dB, -10dB, -5dB, 0dB, 5dB, 10dB, 15dB}, the sampling frequency is 5 times the signal carrier frequency, the sampling length of each signal sample is 4096 points, 7000 training samples for each modulation mode, and 3000 test samples for each modulation mode;

[0192] 3. In this embodiment, the signal processing module is as follows Figure 2 As shown, the FIR filter order used in the digital filtering part of step 2 is 64, and the signal sample data obtained in step 1 is filtered by the FIR filter.

[0193] The time-frequency conversion part uses short-time Fourier transform, wavelet transform, Hilbert-Huang transform, Wigner-Ville distribution and Chirplet transform to obtain the time-frequency graph of each signal sample. The size of the time-frequency graph is (512, 512, 1);

[0194] 4. In this embodiment, the main parameters of the five neural network models in step 3 are as follows:

[0195] 1) Model 1 - MobileNetV2 network:

[0196] Input layer: The input size is specified as (512,512,1)

[0197] Depthwise separable convolutional layers: 17 layers, number of channels [32, 16, 24, 32, 64, 96, 160, 320] expansion ratio: 6

[0198] Dropout rate: 0.2

[0199] Output layer: 8 neurons, Softmax activation function

[0200] 2) Model 2-Inception Network:

[0201] Input layer: The input size is specified as (512,512,1)

[0202] Inception module: 5 in series, channel configuration is as follows:

[0203] Module 1: [64, (96, 128), (16, 32), 32]

[0204] Module 2: [128, (128, 192), (32, 96), 64]

[0205] Module 3: [192, (96, 208), (16, 48), 64]

[0206] Module 4: [160, (112, 224), (24, 64), 64]

[0207] Module 5: [128, (128, 256), (24, 64), 64]

[0208] Output layer: 8 neurons, Softmax activation function

[0209] 3) Model 3-ResNet network:

[0210] Input layer: The input size is specified as (512,512,1)

[0211] Residual block: 4 stages × 2 blocks, number of channels [64, 128, 256, 512]

[0212] Bottleneck Compression Ratio: 4

[0213] Output layer: 8 neurons, Softmax activation function

[0214] 4) Model 4 - CNN+CBAM Network:

[0215] Input layer: The input size is specified as (512,512,1)

[0216] Convolutional layer: 5 layers, number of channels [64, 128, 256, 512, 512]

[0217] Attention module compression rate: 16

[0218] Fully connected layer: 3 layers [1024, 512, 8]

[0219] Output layer: Softmax activation function

[0220] 5) Model 5-Two-Stream Network:

[0221] Input layer: The input size is specified as (512,512,1)

[0222] Temporal branch: 3 layers of 1D convolution [64, 128, 256], kernel length [9, 7, 5]

[0223] Frequency domain branch: 3 layers of 2D convolution [64, 128, 256], kernel size [5×5, 3×3, 3×3]

[0224] Output layer: 8 neurons, Softmax activation function

[0225] 5. In this embodiment, the training parameters of step 4 are as follows:

[0226] Batch size: 32;

[0227] Number of training rounds: 100;

[0228] Optimizer parameters:

[0229] Initial learning rate: 0.001

[0230] Adam optimizer parameters: β1 = 0.9, β2 = 0.999

[0231] Weight decay coefficient: 0.0001

[0232] Learning rate scheduling:

[0233] Scheduling strategy: Reduce by 50% every 30 rounds

[0234] Minimum learning rate: 0.00001

[0235] Dropout configuration:

[0236] Model 1-3: 0.2

[0237] Model 4: 0.3

[0238] Model 5: 0.5

[0239] Data augmentation parameters:

[0240] Rotation angle: ±10°

[0241] Translation range: ±5 pixels

[0242] Zoom range: 0.9-1.1

[0243] 6. In this embodiment, the experimental results are as follows:

[0244] Under high signal-to-noise ratio (≥-5dB) conditions, the system's average recognition accuracy reaches 98.1%. Under low signal-to-noise ratio (<-5dB) conditions, the system's average recognition accuracy is 84.2%. Compared with the single time-frequency transformation method, the system's average recognition accuracy is improved by 11.2%.

Claims

1. A method for identifying signal modulation modes based on multi-model fusion of time-frequency diagrams, characterized in that: The identification method comprises the following steps: Step 1: preprocess the original signal to obtain a signal sample; the original signal is an ideal modulated signal, that is, a baseband signal without noise and frequency deviation; the sample signal includes a signal sequence and a signal sequence modulation mode label; Step 2: construct a data set using the signal samples: the data set includes a training set and a test set, with the ratio of the training set to the test set being 7:3; Step 3: Construct five heterogeneous models respectively, the five heterogeneous models including a first model, a second model, a third model, a fourth model, and a fifth model; the first model includes a first signal processing module and a first modulation identification module; the second model includes a second signal processing module and a second modulation identification module; the third model includes a third signal processing module and a third modulation identification module; the fourth model includes a fourth signal processing module and a fourth modulation identification module; the fifth model includes a fifth signal processing module and a fifth modulation identification module; Step 4: independently train and test the five modulation recognition modules to obtain the trained and tested first model, second model, third model, fourth model, and fifth model; Step 5: Input the signal to be identified into the five trained heterogeneous models respectively for modulation recognition to obtain preliminary recognition results; Step 6: Use the comprehensive judge to make a comprehensive judgment on the preliminary recognition results of the five heterogeneous models to obtain the final recognition result.

2. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 1, characterized in that: In step 1, the sample signal includes 8 digital modulation modes, which are BPSK, QPSK, 8PSK, 16QAM, 64QAM, OFDM, FSK and MSK; The original signal preprocessing process is as follows: First, AWGN (Additive White Gaussian Noise) is superimposed on the original signal s(t); Next, random frequency offset is introduced on the basis of superimposing AWGN. The calculation formula of the signal s'(t) after superimposing the frequency offset is: s'(t)=s(t)e (j2πΔft) , Where Δf is the random frequency deviation, j is the imaginary unit, and t is the time variable; Finally, the frequency-shifted signal s'(t) is normalized to eliminate the differences in signal amplitude and power, and the normalized signal sequence x'(n) is obtained: x'(n)=x(n) / max(|x(n)|).

3. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 1, characterized in that: In step 3: the five signal processing modules are all the same; each signal processing module undergoes digital filtering, time-frequency conversion and time-frequency graph preprocessing in sequence.

4. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 3, characterized in that: The digital filtering adopts FIR filter, and the digital filtering adopts Hamming window design; the frequency response H(f) of the FIR filter is: Where h(n) represents the impulse response sequence of the filter, the length of the impulse response sequence is N, and it is designed by the Hamming window; f represents the frequency variable; the unit of the frequency variable is Hz; j represents the imaginary unit; and n represents the discrete time index (n = 0, 1, ..., N-1).

5. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 3, characterized in that: The time-frequency conversion includes five different time-frequency conversion methods. Each signal processing module adopts a different time-frequency conversion method. The time-frequency conversion method adopted by the first signal processing module is STFT; the time-frequency conversion method adopted by the second signal processing module is WT; the time-frequency conversion method adopted by the third signal processing module is HHT; the time-frequency conversion method adopted by the fourth signal processing module is WVD; and the time-frequency conversion method adopted by the fifth signal processing module is Chirplet transform. The short-time Fourier transform STFT(t, f): Where x(τ) represents the input signal; ω(τ-t) represents the Hamming window function; τ is the time delay variable; During the transformation, STFT is performed frame by frame according to the sampling point step. The result of multiple-frame STFT is the STFT time-frequency diagram. The wavelet transform WT(a, b, f): Among them, x(τ) represents the input signal; ψ is the wavelet basis function, and Morlet wavelet is selected as the basis function, ψ * is the complex conjugate of ψ; a represents the scale parameter (a>0), which controls the frequency resolution; b represents the displacement parameter; the scale range is adaptively selected according to the frequency characteristics of the signal to ensure that the frequency components of the signal are covered; The Hilbert-Huang transform HHT(t, f): Among them, A k (t) represents the instantaneous amplitude of the kth eigenmode function; φ k (t) represents the instantaneous phase of the kth IMF, φ k (t)=∫ω k (t)dt,ω k (t) represents the instantaneous frequency of the kth IMF; K represents the total number of IMFs; the Hilbert-Huang transform is to obtain the intrinsic mode function through empirical mode decomposition, and then perform Hilbert transform on each IMF; The Wigner-Ville distribution WVD(t,f): in, represents the analytical signal, obtained by Hilbert transform; Wigner-Ville distribution uses the analytical signal to eliminate cross terms and uses a Gaussian smoothing kernel to reduce cross term interference; for The complex conjugate of The Chirplet transform Chirp(t, f): Where x(τ) represents the input signal; g(τ-t) represents the Gaussian window function; β is the chirp rate parameter; and the Chirplet transform is adaptively adjusted according to the signal characteristics.

6. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 3, characterized in that: The time-frequency graph preprocessing uses non-local mean filtering to remove artifacts and noise points in the time-frequency graph, normalizes the pixel values of the time-frequency graph to [0, 1], and uses histogram equalization to enhance the contrast of the time-frequency graph and improve the separability of signal features; The time-frequency graph preprocessing process is as follows: First, the pixel values of the time-frequency graph are obtained after NLM filtering. Where n represents the position of the current pixel to be processed, m represents the positions of other pixels in the neighborhood, x(m) represents the pixel value of the image to be processed; ω(n,m) represents the weight function; Z(n) represents the normalization factor, and Ω represents the set range of the pixel neighborhood. When calculating the filter value of pixel n, all possible pixel positions m need to be traversed, but in actual operation, it is limited to a reasonable neighborhood range. Then, the pixel values of the time-frequency graph are normalized to the interval [0,1]. Calculation formula: Among them, p(n) is the pixel value after NLM filtering p(n) min and p(n) max are the minimum pixel value and maximum pixel value of the time-frequency graph respectively; Finally, histogram equalization is used to enhance the contrast of the time-frequency image and obtain the output grayscale s: Among them, r is the input gray level; L is the total number of gray levels; P r (ω) is the gray level probability density function of the input image, ω represents the integral variable, that is, the dummy variable, which has no actual physical meaning and only represents the intermediate variable in the integration process; T(r) represents the cumulative distribution function mapping, which maps the input gray level r to the output gray level s.

7. The method for identifying signal modulation modes based on multi-model fusion of time-frequency diagrams according to claim 1, characterized in that: In step 3, the five modulation recognition modules respectively adopt five different deep learning network structures: The first modulation recognition module uses a MobileNetV2 lightweight network; the MobileNetV2 lightweight network includes 17 layers of depth-wise separable convolutional layers; the number of channels in the backbone of the MobileNetV2 lightweight network is [32, 16, 24, 32, 64, 96, 160, 320], and the inverted residual structure uses an expansion-first-then-compression approach with an expansion ratio of 6; each of the 17 layers of depth-wise separable convolutional layers is followed by batch normalization and a ReLU activation function; The second modulation recognition module adopts the Inception multi-scale feature fusion network; the Inception multi-scale feature fusion network includes 5 serial Inception modules; each Inception module includes four parallel branches, which are respectively a 1×1 convolution layer branch, a 1×1 convolution layer connected to a 3×3 convolution layer branch, a 1×1 convolution layer connected to a 5×5 convolution layer branch, and a 3×3 pooling layer connected to a 1×1 convolution layer branch; the channel configurations of the 5 serial Inception modules are [64, (96, 128), (16, 32), 32], [128, (128, 192), (32, 96), 64], [192, (96, 208), (16, 48), 64], [160, (112, 224), (24, 64), 64], and [128, (128, 256), (24, 64), 64]; The third modulation identification module adopts a ResNet residual connection network; the ResNet residual connection network includes 4 residual stages; the number of channels in the 4 residual stages is [64, 128, 256, 512]; each residual stage contains 2 Bottleneck residual blocks, the compression ratio of the Bottleneck residual block is 4, and the Bottleneck residual block is a three-layer structure, which includes a compression channel layer, a feature extraction layer, and a channel expansion layer in sequence; each layer of the three-layer structure is followed by batch normalization and a ReLU activation function; the compression channel layer is a 1×1 convolution, the feature extraction layer is a 3×3 convolution, and the channel expansion layer is a 1×1 convolution; the Bottleneck residual block implements residual learning through a shortcut connection; The fourth modulation recognition module adopts a CNN+CBAM network; the CNN+CBAM network includes a convolutional layer block and a fully connected layer block; the convolutional layer block has 5 convolutional layers, and the fully connected layer block has 3 fully connected layers; a channel attention module and a spatial attention module are sequentially stacked between the convolutional layer block and the fully connected layer block; The channel attention module includes global average pooling and a fully connected layer in sequence. The channel compression ratio of the channel attention module is 16. The channel attention module obtains channel weights through parallel processing of maximum pooling and average pooling and then passes through a multi-layer perceptron. The spatial attention module uses a 7×7 convolution kernel to generate a spatial weight map. The fifth modulation recognition module adopts a time-frequency dual-stream network, which includes a time domain branch and a frequency domain branch; the time domain branch uses three layers of one-dimensional convolution to extract time domain features, and the convolution kernel lengths of the three layers of one-dimensional convolution are [9, 7, 5], and the number of channels are [64, 128, 256]; the frequency domain branch uses three layers of two-dimensional convolution to extract frequency domain features, and the convolution kernel sizes of the three layers of two-dimensional convolution are [5×5, 3×3, 3×3], and the number of channels are [64, 128, 256]; the features of the time domain branch and the frequency domain branch are fused through the attention mechanism.

8. The method for identifying modulation modes of multi-model fusion signals based on time-frequency diagram according to claim 1, characterized in that: In step 4, the modulation recognition module training method is as follows: First, the five modulation recognition modules described in step 3 are fed with the signal data from the training set, which are time-frequency graphs and their modulation labels. The output target after training is an 8-dimensional probability vector, which represents the predicted probability distribution of the eight digital modulation modes. Then, the parameters of the five modulation recognition modules are optimized respectively using the cross entropy loss function L: L=-Σy i log(p i ) Among them, y i is the true label, p i is the predicted probability; Finally, the Adam optimizer is used to update the parameters of each modulation recognition module. To avoid overfitting, the Adam optimizer uses the following three strategies to optimize the parameters of the modulation recognition module: Dropout strategy: The dropout rates of the five modulation recognition modules are all set to 0.2-0.5; Early stopping strategy: stop training when the validation set loss does not improve for 5 consecutive rounds; Data augmentation strategy: The time-frequency graph is rotated by ±10°, translated by ±5 pixels, and scaled by 0.9-1.

1.

9. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 1, characterized in that: In step 5, the modulation mode identification process of the signal to be identified is as follows: Step 5.1: The signal to be identified is processed sequentially by the first signal processing module, the second signal processing module, the third signal processing module, the fourth signal processing module, and the fifth signal processing module to generate five time-frequency diagrams, which are STFT time-frequency diagram, WT time-frequency diagram, HHT time-frequency diagram, WVD time-frequency diagram, and Chirplet transform time-frequency diagram. The time-frequency graphs are processed in different ways before being fed into the five modulation recognition modules: For the first modulation identification module to the third modulation identification module: standard normalization is used for preprocessing; For the fourth modulation recognition module, maximum and minimum value normalization is used for preprocessing; For the fifth modulation identification module, the time domain and frequency domain features are normalized respectively; Step 5.2: Input the STFT time-frequency map into the first modulation recognition module, the WT time-frequency map into the second modulation recognition module, the HHT time-frequency map into the third modulation recognition module, the WVD time-frequency map into the fourth modulation recognition module, and the Chirplet transform time-frequency map into the fifth modulation recognition module, thereby obtaining five independent prediction results; each prediction result includes class probability, confidence level, and intermediate features; The intermediate features are used for subsequent fusion.

10. The method for identifying the modulation mode of a multi-model fusion signal based on a time-frequency diagram according to claim 1, characterized in that: In step 6, the comprehensive decision maker adopts a fusion method of the hybrid mechanism of Transformer and MoE expert; The comprehensive decision process of the comprehensive decision device is as follows: Step 6.1, preprocessing and feature mapping: The class probability, confidence, and intermediate features are concatenated into a set of vectors and uniformly mapped to a 256-dimensional feature space through a fully connected layer to ensure that the output data of the five models are in the same format. Step 6.2, Transformer encoding: The 256-dimensional feature vectors are combined into a 5×256-dimensional matrix and input into a two-layer multi-head self-attention Transformer for encoding. This is used to capture the complementary information between the five models and enhance the feature expression capability. The output of the Transformer encoding is still a 5×256-dimensional matrix, representing five sets of enhanced features. Step 6.3, adaptive temperature and confidence estimation: Calculate the temperature coefficient T of the five models using a three-layer perceptron k and confidence C k The structure of the three-layer perceptron includes an input layer of 64 neurons, a hidden layer of 32 neurons, and an output layer of 1 neuron; the temperature coefficient T k Activate by softplus and ensure positive value, temperature coefficient T k Dynamic adjustment based on prediction entropy; confidence level C k Activated by sigmoid, the value is between (0,1); Step 6.4, joint feature construction: The enhanced features and temperature coefficient T output by Transformer k and confidence C k Spliced into a 258-dimensional joint feature vector h k , h k ∈R 258 ; Then 5 h k Spliced into a 1290-dimensional global fusion vector R 258 Represents a 258-dimensional vector space; R 1290 Represents a 1290-dimensional vector space; Step 6.5, MoE Expert Mixing: The global fusion vector The input is sent to the MoE module, which includes a gating network and eight expert networks. The outputs of the eight expert networks are weighted and fused according to the gating weights to obtain a comprehensive judgment result. The gating network consists of a linear layer and a softmax layer, and the weight matrix of the linear layer of the gating network is 1290×4. The gating network generates expert weights. Each of the eight expert networks uses a two-layer MLP to generate candidate logits. The first layer of the two-layer MLP fully connected mapping compresses the 1290-dimensional input to 512 dimensions, and the second layer mapping further compresses it to 8 dimensions, matching the number of modulation categories of 8. Step 6.6, output: Use softmax operation on the logits after MoE expert network fusion to output the final modulation category probability distribution; the logits are unnormalized prediction scores, representing the model's original confidence in each category; the softmax operation is a normalized exponential function, which maps the logits to a probability distribution in the interval [0,1].

Images