Modulation signal identification method combining CNN and Vision Transformer
By combining CNN and Vision Transformer in a hybrid recognition method, the problems of traditional CNN's difficulty in modeling long-range dependencies and ViT's hardware resource challenges are solved. A high-precision, low-complexity modulation signal recognition model is constructed, which significantly improves the recognition performance of modulation signals.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHENGDU AEROSPACE COMM EQUIP CO LTD
- Filing Date
- 2025-11-20
- Publication Date
- 2026-04-10
AI Technical Summary
Traditional methods for identifying modulated signals based on convolutional neural networks (CNNs) struggle to model long-range inter-sample dependencies in non-stationary modulated signals. Furthermore, the ViT model faces challenges in terms of hardware resources and tokenization methods, leading to classification confusion when identifying complex modulated signals.
By combining CNN and Vision Transformer in a hybrid recognition method, a high-precision, low-complexity model that takes into account both local details and global context is constructed through multi-scale shallow feature extraction, fusion of dual attention downsampling, tokenized feature extraction, MHSA global feature extraction, and multilayer perceptron classification.
While ensuring recognition accuracy, the model complexity is effectively controlled, which improves the recognition performance of complex modulated signals. In particular, it significantly improves accuracy, precision and recall under different signal-to-noise ratio conditions.
Smart Images
Figure CN121834489A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of modulation signal recognition technology, and more specifically, to a modulation signal recognition method that combines CNN and VisionTransformer. Background Technology
[0002] Traditional modulation signal recognition methods based on convolutional neural networks (CNNs) focus on extracting features from the local receptive field of the signal. While they can effectively capture the signal structure in the short time domain, they struggle to model the long-range inter-sample dependencies (i.e., global correlations) that are widespread in non-stationary modulation signals. This limitation restricts the model's ability to recognize complex modulation patterns (such as high-order QAM). Furthermore, CNNs often use pooling or stride convolution for feature downsampling. While this process improves the level of feature abstraction, it inevitably loses some subtle spatial (or time-frequency) structural information. This loss of information is particularly detrimental to distinguishing modulation signals that are highly similar in time-frequency distribution (such as QPSK and 8PSK), easily leading to classification confusion.
[0003] In recent years, the Vision Transformer (ViT) model has been introduced into the field of modulation recognition, effectively compensating for the shortcomings of CNNs in global semantic feature extraction through its self-attention mechanism. However, the ViT model itself has significant drawbacks: First, its large number of parameters and computational complexity place high demands on hardware resources, posing challenges to its deployment in practical communication devices; second, the performance of ViT is highly dependent on the tokenization method of the input sequence, and how to design a token embedding scheme that can retain key discriminative information for IQ modulated signals remains a current research challenge.
[0004] Therefore, exploring effective combinations of CNN and ViT to construct a hybrid recognition model that can balance local details and global context, while achieving both high accuracy and low complexity, has become a cutting-edge direction for improving the performance of Automatic Modulation Recognition (AMR). The core problem this invention aims to solve is how to design an intrinsic fusion mechanism that allows the local features extracted by CNN to serve as effective input to ViT, and how to optimize model efficiency in the process. Summary of the Invention
[0005] The present invention aims to solve at least one of the aforementioned technical problems existing in the prior art.
[0006] Therefore, this invention provides a modulation signal recognition method that combines CNN and Vision Transformer.
[0007] The modulation signal recognition method combining CNN and Vision Transformer proposed in this invention includes: Obtain the input IQ signal representing the modulation signal; Shallow features are extracted by performing convolution operations on the input IQ signal by combining convolution kernels of different sizes, and the extracted shallow features are fused to construct a multi-scale shallow feature map. The multi-scale shallow features after fusion are downsampled using a convolutional neural network to obtain a downsampled feature map; channel attention and spatial attention are introduced after downsampling. The downsampled feature map extracted by the convolutional neural network is converted into a sequence form that ViT can process. That is, the downsampled feature map is tokenized to obtain tokenized features. In this process, IQ samples are linearly projected and flattened. At the same time, a learnable classification representation is added to the beginning of the sequence, and learnable positional encoding is embedded for all tokens to preserve their spatial order information. Global features are extracted using MHSA based on tokenized features. Specifically, the correlation values between the query matrix and the key matrix are obtained through a multi-head self-attention mechanism, and the self-attention score is obtained using the Softmax function, thereby realizing the interaction between the class token and the patch token. A multilayer perceptron is used to perform feature fusion perception of global features through a fully connected layer, thereby realizing the recombination, transformation and enhancement of classification features; The output features of the multilayer perceptron are fully connected to several types of modulated signals, and the classification probability is obtained through the Softmax function to complete the identification and classification of the modulated signals.
[0008] The modulation signal recognition method combining CNN and Vision Transformer according to the above-described technical solution of the present invention may also have the following additional technical features: In the above technical solution, the step of performing convolution operations on the input IQ signal by combining convolution kernels of different sizes to extract shallow features, and fusing the extracted shallow features to construct a multi-scale shallow feature map, includes: The input IQ signal is processed by several convolutional kernels of different sizes to extract features under different receptive fields. Then use The function is nonlinearly activated, and through The convolution kernel completes channel adjustment and information integration; Finally, residual connections are introduced to alleviate gradient vanishing. The shallow features output by each convolutional kernel are represented as follows:
[0009] in, Indicates the first Shallow features output by each convolutional kernel When performing shallow feature extraction, the input features are represented. Indicates the input IQ signal; This represents the convolution operation; Represents convolution kernels of different sizes; Indicates the size is The convolution kernel; Indicates the activation function; After shallow feature extraction is completed, feature fusion is performed by channel-wise concatenation to obtain a multi-scale shallow feature map, represented as follows:
[0010] in, Represents shallow feature maps at multiple scales; This indicates a splicing operation.
[0011] In the above technical solution, when performing shallow feature extraction, four convolutional kernels of different sizes are used to extract features under different receptive fields, i.e. The sizes of the four convolutional kernels are respectively , , , .
[0012] In the above technical solution, the step of downsampling the fused multi-scale shallow features based on a convolutional neural network to obtain a downsampled feature map includes: Three convolutional neural networks are used to perform three downsampling processes on the fused multi-scale shallow features. The input of the first convolutional neural network is the multi-scale shallow feature map, and the input of the latter two convolutional neural networks is the output of the previous convolutional neural network. Each convolutional neural network performs convolution operations, normalization, non-linear activation, max pooling, channel attention extraction, and spatial attention extraction on the input features; The features extracted by the three convolutional neural networks are represented as follows:
[0013]
[0014]
[0015] in, This represents the downsampled feature map of the output of the first convolutional neural network; This represents the downsampled feature map output by the second convolutional neural network; This represents the downsampled feature map output by the third convolutional neural network; Indicates the size is The convolution kernel; This indicates a batch normalization operation; This indicates that a max pooling operation is performed with a step size of 2. This indicates the channel attention extraction operation; This represents the spatial attention extraction operation.
[0016] In the above technical solution, the channel attention extraction includes: For the input features, the first channel feature statistics are obtained by global average pooling, and the second channel feature statistics are obtained by global max pooling. The first channel feature statistics and the second channel feature statistics are passed through a dense layer and a nonlinear mapping layer, respectively, to obtain the corresponding first output feature and second output feature, which are expressed as follows:
[0017]
[0018] in, This represents the characteristic statistics of the first channel. , Indicates the number of channels; This represents the characteristic statistics of the second channel. ; Represents a dimension-reduced matrix; Represents an upgraded matrix; Indicates the compression ratio; express Activation function; express function; Indicates the first output feature; Indicates the second output feature; The first and second output features are fused, and the channel dimension weights are updated to obtain the channel attention feature extraction results, including:
[0019] in, This indicates the result of channel attention feature extraction; This indicates multiplication by channel; This represents the input features for channel attention extraction. .
[0020] In the above technical solution, the spatial attention extraction includes: For the input features, the first spatial feature statistics are obtained by average pooling by channel, and the second spatial feature statistics are obtained by max pooling by channel. After feature fusion of the first and second spatial feature statistics, a nonlinear mapping is performed, and finally, the spatial dimension weights are updated to obtain the spatial attention feature extraction results, including:
[0021] in, This represents the result of spatial attention feature extraction; This indicates multiplication by channel; The input features represent spatial attention extraction. ; Represents the first spatial feature statistics; This represents the second spatial characteristic statistics.
[0022] In the above technical solution, the step of converting the downsampled feature map extracted by the convolutional neural network into a sequence form that ViT can process, i.e., tokenizing the downsampled feature map to obtain tokenized features, includes:
[0023] in, Representation layer normalization; Represents feature embedding; Indicates tokenization characteristics; The feature embedding includes: Given input features Linear projection and flattening are achieved through convolution operations, where filters... Size is Step size is The number of filter inputs is The number of filter outputs is Output patch representation The quantity is , length is The patch indicates Represented as:
[0024] Then, a classification representation is introduced. The patch representation and the category representation are concatenated row by row, and finally a learnable positional code is added. To obtain the final feature embedding results, including:
[0025] in, This represents the feature embedding result.
[0026] In the above technical solution, the step of extracting global features based on tokenized features to obtain global features includes: After performing layer normalization, multi-head self-attention feature extraction, nonlinear mapping, and classification head extraction on the input features, the global feature representation obtained after global feature extraction of the tokenized features is as follows:
[0027] in, This indicates multi-head self-attention feature extraction. This indicates the extraction of classification head features; The multi-head self-attention feature extraction includes:
[0028]
[0029]
[0030] Specifically, the dimensions of the query matrix, key matrix, and value matrix are set to be equal. , , For the number of heads; Indicates the first Self-attention score based on height; , , Representing input features After the first The projection matrix corresponding to each head , and The resulting query matrix, key matrix, and value matrix; This indicates that normalization is performed row-wise.
[0031] In the above technical solution, the use of a multilayer perceptron to perform feature fusion perception of global features through fully connected layers, thereby realizing the recombination, transformation, and enhancement of classification features, includes:
[0032] in, Indicates the first linear layer; Indicates the second linear layer; This represents the output features of the multilayer perceptron, i.e., the classification features.
[0033] In the above technical solution, the step of fully connecting the output features of the multilayer perceptron with several modulation signal types and obtaining the classification probability through the Softmax function to complete the identification and classification of the modulation signal includes:
[0034] in, Indicates a fully connected layer; This indicates normalization by channel; P represents the classification probability.
[0035] In summary, due to the adoption of the above-mentioned technical features, the beneficial effects of the present invention are: This invention provides a modulation signal recognition method that combines CNN and Vision Transformer (ViT). This method constructs a hybrid neural network architecture that effectively controls model complexity while ensuring high recognition accuracy by synergistically utilizing the local feature extraction advantages of CNN and the global relationship modeling capabilities of ViT.
[0036] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description
[0037] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a general framework diagram of a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention.
[0038] Figure 2 This is a flowchart of a dataset generation process according to an embodiment of the present invention; Figure 3 This is a dataset category distribution diagram according to an embodiment of the present invention; Figure 4 This is a flowchart of a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention; Figure 5 This is a block diagram illustrating the channel attention extraction principle in a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention. Figure 6 This is a block diagram illustrating the spatial attention extraction principle in a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention. Figure 7 This is a block diagram illustrating the feature embedding principle in a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention. Figure 8 This is a block diagram of the multi-head self-attention (MHSA) principle in a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention. Figure 9This is a diagram of the CNN network model architecture in a modulation signal recognition method combining CNN and Vision Transformer according to an embodiment of the present invention. Figure 10 This is a schematic diagram comparing the signal recognition and classification curves of the method of this invention and the traditional method; Figure 11 This is a schematic diagram comparing the signal recognition and classification confusion matrix of the method of this invention with that of traditional methods. Detailed Implementation
[0039] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0040] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0041] The following reference Figures 1 to 11 This describes a modulation signal recognition method combining CNN and VisionTransformer, provided according to some embodiments of the present invention.
[0042] Some embodiments of this application provide a modulation signal recognition method that combines CNN and Vision Transformer.
[0043] like Figure 1 As shown, the first embodiment of this invention proposes a modulation signal recognition method combining CNN and Vision Transformer. This embodiment combines the advantages of CNN's local feature extraction with the characteristics of Vision Transformer's global semantic extraction to complete modulation signal recognition. The overall method includes: multi-scale shallow feature extraction, CNN downsampling feature extraction with dual attention, tokenized feature extraction, MHSA global feature extraction, class token-based MLP feature extraction, and classification output. Its overall architecture diagram is attached. Figure 1 As shown.
[0044] In one specific embodiment, the environment for the algorithm proposed in this disclosure is set up. Specifically, the algorithm is implemented based on an existing simulation platform, and the network model is constructed using the toolbox configured on the simulation platform. To meet the algorithm's real-time processing requirements for large-scale data, the computer platform is configured with 32GB of RAM to ensure efficient loading of the training dataset, and the core computing tasks are deployed on the GPU using the parallel computing capabilities of the graphics card, significantly improving the model training and inference process. The algorithm and the aforementioned hardware and software environment functionally support each other, together constituting a complete method for implementing modulation signal recognition and processing.
[0045] It should be noted that the modulation signal recognition method established in this disclosure is itself a modulation signal recognition model. The process of generating the dataset for training this model is as follows: Using the RadioML2016.10A dataset as a reference standard, eight digital modulation schemes are generated, including BPSK, QPSK, 8PSK, 16QAM, 64QAM, BFSK, CPFSK, and PAM4. Three open-source analog modulation signals are also read, including SB-FM, AM-SSB, and AM-DSB. To meet the RadioML2016.10A dataset standard, this disclosure passes the generated signal through a hybrid channel containing a Gaussian channel, a Ricean channel, and a center frequency offset and sampling time offset caused by clock offset. The modulated signal passing through the hybrid channel is then identified and classified. The dataset generation process is as follows: Figure 2 As shown in the figure. The Gaussian channel is configured with an SNR range of -20 to +20; the Ricean multipath channel is configured with delay distributions of 0, 1.8, and 3.4, average path gain of 0, -2, and -10, and a K-factor of 4. The digital signal center frequency is set to 902MHz, and the analog signal center frequency is set to 100MHz to set the signal frequency offset. The signal remains a baseband signal. The clock shift is set to 5ppm. Each symbol contains 8 samples (i.e., SPS = 8), and each frame contains 1024 samples (i.e., SPF = 1024), therefore each frame contains 128 symbols. The baseband signal sampling rate is set to 200kHz (digital signal bandwidth not exceeding 50kHz, analog signal bandwidth 30kHz). Finally, 10,000 frames are generated for each modulation type, with 80% used for training, 10% for validation, and 10% for testing. The data distribution is shown in the figure. Figure 3 As shown. During the training phase, training and validation sets are used for training and validation; during the testing phase, the classification accuracy of the algorithm is tested using the test set.
[0046] It is understood that the above dataset generation process is only illustrative, and those skilled in the art can adjust the dataset generation process as needed, which will not be elaborated here.
[0047] In some embodiments, the specific execution process of the algorithm model corresponding to the modulation signal recognition method combining CNN and Vision Transformer proposed in this disclosure (based on the trained algorithm model) is as follows: Figure 4 As shown, it includes the following steps S1-S7.
[0048] S1. Obtain the input IQ signal representing the modulation signal.
[0049] S2. By combining convolution kernels of different sizes, the input IQ signal is convolved to extract shallow features, and the extracted shallow features are fused to construct a multi-scale shallow feature map.
[0050] Specifically, step S2 corresponds to multi-scale shallow feature extraction. This step is used to extract robust local features from the original modulated signal IQ data. Four different scale convolution kernels are used in parallel to perform convolution operations on the input signal to construct a multi-scale shallow feature map. This design aims to overcome the limitations of a single receptive field on feature extraction, ensuring the capture of a variety of signal characteristics from subtle to macroscopic.
[0051] In some embodiments, step S2 includes: The input IQ signal is processed by several convolutional kernels of different sizes to extract features within different receptive fields. Specifically, for shallow feature extraction, features within different receptive fields can be extracted using four convolutional kernels of different sizes. The sizes of the four convolutional kernels are respectively , , , .
[0052] Then use The function is nonlinearly activated, and through The convolution kernel completes channel adjustment and information integration; Finally, residual connections are introduced to alleviate gradient vanishing. The shallow features output by each convolutional kernel are represented as follows:
[0053] in, Indicates the first Shallow features output by each convolutional kernel When performing shallow feature extraction, the input features are represented. Indicates the input IQ signal; This represents the convolution operation; Represents convolution kernels of different sizes; Indicates the size is The convolution kernel; This represents the activation function; it should be noted that, unless otherwise specified, in this disclosure, the activation function is used in each step. This represents the input features / input vector for the corresponding step.
[0054] After shallow feature extraction is completed, feature fusion is performed by channel-wise concatenation to obtain a multi-scale shallow feature map, represented as follows:
[0055] in, Represents shallow feature maps at multiple scales; This indicates a splicing operation.
[0056] S3. Based on the convolutional neural network, the fused multi-scale shallow features are downsampled to obtain the downsampled feature map; among them, channel attention and spatial attention are introduced after downsampling.
[0057] Specifically, step S3 corresponds to CNN downsampling feature extraction with dual attention. After multi-scale shallow feature extraction, this step downsamples the fused multi-scale features to expand the receptive field and compress the data dimensionality. Simultaneously, to mitigate the loss of spatial information caused by downsampling, channel attention (CA, enhancing channel interaction) and spatial attention (SA, enhancing spatial representation) are introduced after downsampling.
[0058] In some embodiments, step S3 includes: Three convolutional neural networks (CNNs) are used to perform three downsampling processes on the fused multi-scale shallow features. The input to the first CNN is the multi-scale shallow feature map, and the inputs to the latter two CNNs are the outputs of the previous CNN. In essence, this step divides the CNN downsampling feature extraction into three downsampling stages: CNN1, CNN2, and CNN3. Each stage has a downsampling factor of 2, and the three CNN layers are designed identically, differing only in the stage they are in. Of course, the number of CNNs is not a limitation of this disclosure and can be selected according to the actual situation.
[0059] Each convolutional neural network performs convolution operations, normalization, non-linear activation, max pooling, channel attention extraction, and spatial attention extraction on the input features; The features extracted by the three convolutional neural networks are represented as follows:
[0060]
[0061]
[0062] in, This represents the downsampled feature map of the output of the first convolutional neural network; This represents the downsampled feature map output by the second convolutional neural network; This represents the downsampled feature map output by the third convolutional neural network; Indicates the size is The convolution kernel; This indicates a batch normalization operation; This indicates that a max pooling operation is performed with a step size of 2. This indicates the channel attention extraction operation; This represents the spatial attention extraction operation.
[0063] In one specific embodiment, such as Figure 5 As shown, the channel attention extraction includes: For the input features, the first channel feature statistics are obtained by global average pooling, and the second channel feature statistics are obtained by global max pooling. Specifically, and They can be represented as:
[0064]
[0065] in, and This represents the height and width of the input feature map in channel attention extraction; Then, in the input feature map, with and The element corresponding to the index.
[0066] To simulate the interaction between channels, the feature statistics of the first channel and the feature statistics of the second channel are passed through a dense layer and a nonlinear mapping layer, respectively, to obtain the corresponding first output feature and second output feature, which are expressed as:
[0067]
[0068] in, This represents the characteristic statistics of the first channel. , Indicates the number of channels; This represents the characteristic statistics of the second channel. ; Represents a dimension-reduced matrix; Represents an upgraded matrix; Indicates the compression ratio; express Activation function; express function; Indicates the first output feature; The second output feature is represented; the first and second output features are fused, and the channel dimension weights are updated to obtain the channel attention extraction feature results, including:
[0069] in, This indicates the result of channel attention feature extraction; This indicates multiplication by channel; This represents the input features for channel attention extraction. .
[0070] In some embodiments, such as Figure 6 As shown, the spatial attention extraction includes: For the input features, the first spatial feature statistics are obtained by average pooling across channels, and the second spatial feature statistics are obtained by max pooling across channels; that is:
[0071]
[0072] in, This indicates average pooling by channel dimension. This indicates max pooling by channel dimension.
[0073] After feature fusion of the first and second spatial feature statistics, a nonlinear mapping is performed, and finally, the spatial dimension weights are updated to obtain the spatial attention feature extraction results, including:
[0074] in, This represents the result of spatial attention feature extraction; This indicates multiplication by channel; The input features represent spatial attention extraction. ; Represents the first spatial feature statistics; This represents the second spatial characteristic statistics.
[0075] S4. Convert the downsampled feature map extracted by the convolutional neural network into a sequence form that ViT can process, that is, tokenize the downsampled feature map to obtain tokenized features; wherein, the IQ sample points are linearly projected and flattened, and a learnable classification representation is added to the beginning of the sequence, and learnable position encoding is embedded for all tokens to preserve their spatial order information.
[0076] Specifically, step S4 corresponds to tokenized feature extraction. This step converts the feature map extracted by the CNN into a sequence format that ViT can process. Since the model input is an IQ signal, the algorithm performs linear projection and flattening operations on each IQ sample point. At the same time, a learnable classification representation class token is added to the beginning of the sequence, and learnable positional encodings are embedded for all tokens to preserve their spatial order information.
[0077] In some implementations, in step S4, the signal features after feature extraction via 3 layers of CNN downsampling are: To meet the input requirements of subsequent multi-head self-attention MHSA, the features need to be... Tokenization is then performed. This part mainly includes: convolutional processing, layer normalization, feature embedding (Patch Embedding) and non-linear mapping, as well as the final dropout layer. The output features after this step are... It can be represented as:
[0078] in, Representation layer normalization; Represents feature embedding; Indicates tokenization characteristics; In some embodiments, such as Figure 7 As shown, the feature embedding includes: Given input features Linear projection and flattening are achieved through convolution operations, where filters... Size is Step size is The number of filter inputs is The number of filter outputs is Output patch representation The quantity is , length is ; Then the patch indicates Represented as:
[0079] Then, a classification representation is introduced. The patch representation and the category representation are concatenated row by row, and finally a learnable positional code is added. To obtain the final feature embedding results, including:
[0080] in, This represents the feature embedding result.
[0081] S5. Extract global features from MHSA based on tokenized features to obtain global features; among them, the correlation values between the query matrix and the key matrix are obtained through a multi-head self-attention mechanism, and the self-attention score is obtained by using the Softmax function to realize the interaction between the class token and the patch token.
[0082] Specifically, step S5 corresponds to MHSA global feature extraction. This step takes tokenized features as input, obtains the correlation values between the Query and Key through a multi-head self-attention mechanism (MHSA), and uses Softmax to obtain the self-attention score, realizing the interaction between the class token and the patch token. Therefore, the class token aggregates the global information of the patch token, and multi-layer perceptron (MLP) feature extraction is performed on the class token. This method reduces the computational cost of the MLP model while ensuring algorithm performance.
[0083] In some embodiments, step S5 includes: The input features are subjected to layer normalization, multi-head self-attention feature extraction, nonlinear mapping, and classification head extraction. In other words, MHSA global feature extraction consists of layer normalization, multi-head self-attention mechanism, nonlinear mapping, classification head extraction, and dropout layer.
[0084] The global feature representation obtained after tokenization feature extraction is as follows:
[0085] in, This indicates multi-head self-attention feature extraction. This indicates the extraction of classification head features; In some embodiments, such as Figure 8 As shown, given input features There are a total of Each attention head is used for feature extraction (the number of attention heads is for illustrative purposes only). Each head independently computes a self-attention score for the projected features and obtains the correlation between the label vector at each location and the label vectors at other locations. Then... The self-attention scores of each head are concatenated and then linearly projected to obtain the global relevance features of the input information. This process can be represented as:
[0086]
[0087]
[0088] Specifically, the dimensions of the query matrix, key matrix, and value matrix are set to be equal. , , For the number of heads; Indicates the first Self-attention score based on height; , , Representing input features After the first The projection matrix corresponding to each head , and The resulting query matrix, key matrix, and value matrix; This indicates that normalization is performed row-wise.
[0089] S6. A multilayer perceptron is used to perform feature fusion perception of global features through full connectivity, thereby realizing the recombination, transformation and enhancement of classification features.
[0090] Specifically, step S6 corresponds to MLP feature extraction based on class tokens. In this step, the classification representation class tokens that have undergone multi-head attention mechanism are processed through a fully connected layer (first increasing the dimensionality and then reducing the dimensionality, i.e., the hidden layer is 4 times the size of the input layer and the output layer is 1 / 4 the size of the hidden layer) to perform feature fusion perception, thereby completing feature recombination, transformation and enhancement.
[0091] In some embodiments, step S6 includes: Using a multilayer perceptron (MLP) For global feature fusion sensing, the Multilayer Perceptron (MLP) mainly consists of two fully connected layers and one nonlinear layer, and its output features are represented as follows:
[0092] in, Indicates the first linear layer; Indicates the second linear layer; This represents the output features of the multilayer perceptron, i.e., the classification features.
[0093] S7. Connect the output features of the multilayer perceptron to several types of modulation signals, and obtain the classification probability through the Softmax function to complete the identification and classification of the modulation signals.
[0094] Specifically, step S7 corresponds to the classification output. This step fully connects the output features of the MLP multilayer perceptron to 11 modulation signals and obtains the classification probability through Softmax, thus completing the identification and classification of the modulation signal.
[0095] In some embodiments, step S7 is represented as:
[0096] in, Indicates a fully connected layer; This indicates normalization by channel; P represents the classification probability.
[0097] In one specific embodiment, the main hyperparameter configurations for training the model are as follows: Optimizer: Adam optimizer; Maximum epochs: 100; Learning rate: 0.001; Minimum batch size: 256; Gradient decay factor: 0.95; Mean squared gradient decay factor: 0.999; Validation frequency: 343; Validation tolerance epochs: 10; Learning rate decay epochs: 5; Learning rate decay factor: 0.8; L2 regularization factor: 0.0001; Execution environment: GPU.
[0098] In one specific embodiment, the network model corresponding to the modulation signal recognition method combining CNN and Vision Transformer proposed in this invention is evaluated.
[0099] Specifically, the network model sets the SNR range to -20 to +20 based on the Gaussian channel, and the algorithm model hyperparameters are configured according to the above embodiment. The baseline algorithm model architecture (traditional algorithms, CNN1-CNN5 all use max pooling, CNN6 uses average pooling) is as follows: Figure 9 As shown in the figure. A comparison of the performance curves of the baseline algorithm and the algorithm proposed in this patent for identifying modulated signals is illustrated in the figure. Figure 10 As shown, the comparison of the modulation signal recognition confusion matrices of the two under several typical signal-to-noise ratios is illustrated in the figure below. Figure 11 As shown. Its performance metrics include accuracy, precision, recall, and F1 score. This indicates a true positive (the prediction is true, and the sample is a positive class). True negative (predicted as true, sample is negative). False positive (predicted as false, but the sample is of the positive class). False negatives (predicting a false negative, the sample is a negative class). The performance metrics are calculated as follows:
[0100]
[0101]
[0102]
[0103] Compared to the baseline algorithm, the performance improvement of the algorithm proposed in this disclosure is as follows: At a signal-to-noise ratio of -20, the modulation signal recognition accuracy improved by 3.78%, precision by 2.79%, recall by 3.78%, and F1 score by 3.24%. At a signal-to-noise ratio of -10, the modulated signal recognition accuracy improved by 5.81%, precision by 8.74%, recall by 5.81%, and F1 score by 7.27%. At a signal-to-noise ratio of 0, the modulated signal recognition accuracy improved by 5.06%, precision by 6.03%, recall by 5.06%, and F1 score by 5.54%. At a signal-to-noise ratio of 10, the modulated signal recognition accuracy improved by 6.03%, precision by 6.38%, recall by 6.03%, and F1 score by 6.21%. At a signal-to-noise ratio of 20, the modulated signal recognition accuracy improved by 7.09%, precision by 7.07%, recall by 7.09%, and F1 score by 7.08%.
[0104] The test results of the two algorithms are compared in Table 1: Table 1 Test Results
[0105] The test results are as follows: This invention discloses a hybrid model method for modulated signal recognition combining CNN and Vision Transformer, with a total model parameter count of 286k. The method includes multi-scale shallow feature extraction, CNN downsampling feature extraction incorporating a dual attention mechanism, and global semantic modeling based on Vision Transformer (encompassing tokenized feature extraction, MHSA global feature extraction, and class token-based MLP feature extraction). Finally, it achieves the recognition of 11 classes of modulated signals through classification output. Compared to the baseline CNN model (98.3k parameters), although this method increases the number of parameters by 187.7k, it significantly improves the comprehensive evaluation metrics under 21 signal-to-noise ratio conditions: overall average recognition accuracy increases by 6.01%, overall average recognition precision increases by 6.39%, overall average recognition recall increases by 6.01%, and overall average recognition F1 score increases by 6.20%. This method achieves efficient synergy between the local perception capabilities of CNNs and the global modeling advantages of Vision Transformers, providing an innovative solution to the problem of balancing local feature extraction and global dependency modeling in lightweight models. It also provides a new paradigm for model design in the field of intelligent sensing in communication and is of great significance for promoting the practical engineering implementation of related algorithms on resource-constrained embedded platforms.
[0106] In this specification, the illustrative expressions of the terms used do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0107] Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention shall be included within the scope of protection of this invention.
Claims
1. A method for identifying modulated signals combining CNN and Vision Transformer, characterized in that, include: Obtain the input IQ signal representing the modulation signal; Shallow features are extracted by performing convolution operations on the input IQ signal by combining convolution kernels of different sizes, and the extracted shallow features are fused to construct a multi-scale shallow feature map. The multi-scale shallow features after fusion are downsampled using a convolutional neural network to obtain a downsampled feature map; channel attention and spatial attention are introduced after downsampling. The downsampled feature map extracted by the convolutional neural network is converted into a sequence form that ViT can process. That is, the downsampled feature map is tokenized to obtain tokenized features. In this process, IQ samples are linearly projected and flattened. At the same time, a learnable classification representation is added to the beginning of the sequence, and learnable positional encoding is embedded for all tokens to preserve their spatial order information. Global features are extracted using MHSA based on tokenized features. Specifically, the correlation values between the query matrix and the key matrix are obtained through a multi-head self-attention mechanism, and the self-attention score is obtained using the Softmax function, thereby realizing the interaction between the class token and the patch token. A multilayer perceptron is used to perform feature fusion perception of global features through a fully connected layer, thereby realizing the recombination, transformation and enhancement of classification features; The output features of the multilayer perceptron are fully connected to several types of modulated signals, and the classification probability is obtained through the Softmax function to complete the identification and classification of the modulated signals.
2. The modulation signal recognition method combining CNN and Vision Transformer according to claim 1, characterized in that, The process of extracting shallow features by performing convolution operations on the input IQ signal using convolution kernels of different sizes, and then fusing the extracted shallow features to construct a multi-scale shallow feature map, includes: The input IQ signal is processed by several convolutional kernels of different sizes to extract features under different receptive fields. Then use The function is nonlinearly activated, and through The convolution kernel completes channel adjustment and information integration; Finally, residual connections are introduced to alleviate gradient vanishing. The shallow features output by each convolutional kernel are represented as follows: in, Indicates the first Shallow features output by each convolutional kernel When performing shallow feature extraction, the input features are represented. Indicates the input IQ signal; This represents the convolution operation; Represents convolution kernels of different sizes; Indicates the size is The convolution kernel; Indicates the activation function; After shallow feature extraction is completed, feature fusion is performed by channel-wise concatenation to obtain a multi-scale shallow feature map, represented as follows: in, Represents shallow feature maps at multiple scales; This indicates a splicing operation.
3. The modulation signal recognition method combining CNN and Vision Transformer according to claim 2, characterized in that, When performing shallow feature extraction, four convolutional kernels of different sizes are used to extract features under different receptive fields, i.e. The sizes of the four convolutional kernels are respectively , , , .
4. The modulation signal recognition method combining CNN and Vision Transformer according to claim 2, characterized in that, The process of downsampling the fused multi-scale shallow features based on a convolutional neural network to obtain a downsampled feature map includes: Three convolutional neural networks are used to perform three downsampling processes on the fused multi-scale shallow features. The input of the first convolutional neural network is the multi-scale shallow feature map, and the input of the latter two convolutional neural networks is the output of the previous convolutional neural network. Each convolutional neural network performs convolution operations, normalization, non-linear activation, max pooling, channel attention extraction, and spatial attention extraction on the input features; The features extracted by the three convolutional neural networks are represented as follows: in, This represents the downsampled feature map of the output of the first convolutional neural network; This represents the downsampled feature map output by the second convolutional neural network; This represents the downsampled feature map output by the third convolutional neural network; Indicates the size is The convolution kernel; This indicates a batch normalization operation; This indicates that a max pooling operation is performed with a step size of 2. This indicates the channel attention extraction operation; This represents the spatial attention extraction operation.
5. The modulation signal recognition method combining CNN and Vision Transformer according to claim 4, characterized in that, The channel attention extraction includes: For the input features, the first channel feature statistics are obtained by global average pooling, and the second channel feature statistics are obtained by global max pooling. The first channel feature statistics and the second channel feature statistics are passed through a dense layer and a nonlinear mapping layer, respectively, to obtain the corresponding first output feature and second output feature, which are expressed as follows: in, This represents the characteristic statistics of the first channel. , Indicates the number of channels; This represents the characteristic statistics of the second channel. ; Represents a dimension-reduced matrix; Represents an upgraded matrix; Indicates the compression ratio; express Activation function; express function; Indicates the first output feature; Indicates the second output feature; The first and second output features are fused, and the channel dimension weights are updated to obtain the channel attention feature extraction results, including: in, This indicates the result of channel attention feature extraction; This indicates multiplication by channel; This represents the input features for channel attention extraction. .
6. The modulation signal recognition method combining CNN and Vision Transformer according to claim 4, characterized in that, The spatial attention extraction includes: For the input features, the first spatial feature statistics are obtained by average pooling by channel, and the second spatial feature statistics are obtained by max pooling by channel. After feature fusion of the first and second spatial feature statistics, a nonlinear mapping is performed, and finally, the spatial dimension weights are updated to obtain the spatial attention feature extraction results, including: in, This represents the result of spatial attention feature extraction; This indicates multiplication by channel; The input features represent spatial attention extraction. ; Represents the first spatial feature statistics; This represents the second spatial characteristic statistics.
7. The modulation signal recognition method combining CNN and Vision Transformer according to claim 4, characterized in that, The process of converting the downsampled feature map extracted by the convolutional neural network into a sequence format that ViT can process, i.e., tokenizing the downsampled feature map to obtain tokenized features, includes: in, Representation layer normalization; Represents feature embedding; Indicates tokenization characteristics; The feature embedding includes: Given input features Linear projection and flattening are achieved through convolution operations, where filters... Size is Step size is The number of filter inputs is The number of filter outputs is Output patch representation The quantity is , length is ; Then the patch indicates Represented as: Then, a classification representation is introduced. The patch representation and the category representation are concatenated row by row, and finally a learnable positional code is added. To obtain the final feature embedding results, including: in, This represents the feature embedding result.
8. The modulation signal recognition method combining CNN and Vision Transformer according to claim 7, characterized in that, The step of extracting global features from MHSA based on tokenized features includes: After performing layer normalization, multi-head self-attention feature extraction, nonlinear mapping, and classification head extraction on the input features, the global feature representation obtained after global feature extraction of the tokenized features is as follows: in, This indicates multi-head self-attention feature extraction. This indicates the extraction of classification head features; The multi-head self-attention feature extraction includes: Specifically, the dimensions of the query matrix, key matrix, and value matrix are set to be equal. , , For the number of heads; Indicates the first Self-attention score based on height; , , Representing input features After the first The projection matrix corresponding to each head , and The resulting query matrix, key matrix, and value matrix; This indicates that normalization is performed row-wise.
9. The modulation signal recognition method combining CNN and Vision Transformer according to claim 8, characterized in that, The method employs a multilayer perceptron to perform feature fusion perception of global features through a fully connected layer, thereby achieving the recombination, transformation, and enhancement of classification features, including: in, Indicates the first linear layer; Indicates the second linear layer; This represents the output features of the multilayer perceptron, i.e., the classification features.
10. The modulation signal recognition method combining CNN and Vision Transformer according to claim 9, characterized in that, The step of performing a full connection between the output features of the multilayer perceptron and several modulation signal types, and obtaining the classification probability through the Softmax function to complete the identification and classification of the modulation signal includes: in, Indicates a fully connected layer; This indicates normalization by channel; P represents the classification probability.