A multi-modal and CNN-based overlapping signal modulation mode recognition method

Abstract

The application discloses a kind of based on multimodal and CNN's overlapping signal modulation mode identification method, comprising the following steps: S1.given multiple potential modulation modes needing identification;S2.according to potential modulation mode and its combination, generate multiple overlapping signals, and carry out multimodal feature extraction and modulation mode marking to each overlapping signal, constitute the sample set of overlapping signal;S3.build multi-scale neural network, and carry out the training of multi-scale neural network based on sample set;S4.utilize the neural network of training and carry out overlapping signal modulation mode identification.The application is based on multimodal data and multi-scale CNN, can complete the accurate identification of multiple modulation modes contained in overlapping signal.

Description

Technical Field

[0001] This invention relates to the identification of overlapping signal modulation patterns, and in particular to a method for identifying overlapping signal modulation patterns based on multimodal and CNN. Background Technology

[0002] In the realm of overlapping automatic modulation recognition (AMR) tasks, where mixed or multiple signal modulations exist, the application of pulse descriptor techniques remains feasible. However, a fundamental prerequisite for successful implementation is the introduction of blind signal separation methods. On the other hand, deep learning methods are emerging and gradually becoming the dominant approach in this field. This shift aligns with increased computational power and the superior ability of neural networks to extract latent information from complex input modalities.

[0003] More specifically, intra-pulse modulated signals essentially consist of a series of identical pulses, although each pulse itself is modulated. These non-stationary signals exhibit stable intra-pulse characteristics in the I / Q domain, the time-frequency domain, and other conversion domains. Significant progress has been made in AMR by applying these intra-pulse characteristics.

[0004] Implementing existing AMR technology in real-world scenarios and constructing a dataset containing every possible combination of signal modulation types is computationally impossible, especially considering intelligent waveform design techniques. Therefore, a certain proportion of unseen modulation combinations is expected in the test dataset. Thus, a good modulation identification technique should be as robust as possible to unseen modulation combinations. Furthermore, the joint consideration of the AMR problem is less frequent under non-ideal SNR and AR conditions. However, in reality, such extreme cases (low SNR and high AR) can be common, and this must be taken into account in the classifier design. An implicit assumption in past work is that different modulated signals have the same pulse width. This assumption simplifies the automatic AMR task but deviates from common waveform design patterns. Another implicit assumption concerns the alignment of the intercepted signals. In a scenario where an auxiliary antenna listens for a period of time and passes the collected signals to the signal processing and identification algorithm, the initial state of the signals (signals of different modulation types) is naturally unknown. However, past work implicitly assumes correct signal alignment, i.e., that the first frame of each signal is aligned with the first frame of the intercepted signal, which is inconsistent in practice. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for identifying overlapping signal modulation patterns based on multimodal and CNN, which can accurately identify the multiple modulation patterns contained in overlapping signals.

[0006] The objective of this invention is achieved through the following technical solution: a method for identifying overlapping signal modulation patterns based on multimodal and CNN methods, comprising the following steps:

[0007] S1. Given a variety of potential modulation schemes that need to be identified;

[0008] S2. Based on the potential modulation schemes and their combinations, generate multiple overlapping signals, and perform multi-mode feature extraction and modulation scheme labeling on each overlapping signal to form a sample set of overlapping signals;

[0009] S3. Construct a multi-scale neural network and train the multi-scale neural network based on the sample set;

[0010] S4. Use the trained neural network to identify overlapping signal modulation patterns.

[0011] Preferably, if the number of potential modulation schemes is N, then the number of modulation combinations is 2. N -1; The number of modulation methods N is 8, including: linear frequency modulation, sinusoidal frequency modulation, hyperbolic frequency modulation, step frequency modulation, Costas-coded frequency modulation, binary phase modulation, Barker-coded phase modulation, and Frank-coded phase modulation;

[0012] For all modulation methods, the center frequency of all signals is the same;

[0013] For all types of frequency modulation signals, the bandwidth range remains consistent. Frequency modulation signals refer to signals with linear frequency modulation, sinusoidal frequency modulation, hyperbolic frequency modulation, and step frequency modulation.

[0014] In step S2, multiple overlapping signals need to be generated for each combination of modulation methods.

[0015] In step S2, the method for multimodal feature extraction of any overlapping signal is as follows:

[0016] Representing the analytic signal of the overlapping signal at n time-domain sampling points as x[n], and performing short-time Fourier transform and WVD transform on the overlapping signal, we obtain:

[0017]

[0018]

[0019] Where w represents frequency, m is time window parameter, and * represents conjugate;

[0020] For STFM(m, w), take amplitude A1 and phase P1 respectively, and for WVD(m, w), take amplitude A2. Then the results constitute the feature vector [A1, P1, A2] of the multimode of the overlapping signal.

[0021] In step S2, the modulation schemes for the overlapping signals are labeled as: K1, K2, ..., K N Where N represents the number of modulation schemes; let K n The label represents the nth modulation scheme, where n = 1, 2, ..., N; if the overlapping signal contains the nth modulation scheme, then K n =1, if the overlapping signal does not contain the nth modulation scheme, then K n =0.

[0022] In step S2, the feature vector of any overlapping signal is used as a sample feature, and the label of the overlapping signal is used as a sample label. The sample feature and the sample label constitute the signal sample of the overlapping signal.

[0023] By adding all the signal samples of overlapping signals into the same set, a sample set of overlapping signals is formed.

[0024] Preferably, for each combination of modulation methods, a sample set (training set) can be constructed using signal samples from partially overlapping signals for training, and then a test set and a validation set can be constructed using signal samples from another part of the overlapping signals to complete the testing and validation of the trained model and ensure the accuracy of the model; for example, 20 samples are generated for each combination of modulation methods, of which 10 samples are used for training, and the remaining 10 samples are divided equally and used for validation and testing respectively.

[0025] The multi-scale neural network constructed in step S3 includes an input layer, four hidden layers, and an output layer;

[0026] The four hidden layers serve as feature fusion layers, and the output layer integrates the fused features to output the prediction result. Each layer is processed using convolutional connections, and the information block size between each layer is different to ensure that feature information at each scale is fused together through connections.

[0027] The beneficial effects of this invention are: based on multimodal data and multi-scale CNN, this invention can accurately identify multiple modulation methods contained in overlapping signals. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a schematic diagram of the structure of a neural network. Detailed Implementation

[0030] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0031] like Figure 1 As shown, a method for recognizing overlapping signal modulation patterns based on multimodal and CNN methods includes the following steps:

[0032] S1. Given a variety of potential modulation schemes that need to be identified;

[0033] S2. Based on the potential modulation schemes and their combinations, generate multiple overlapping signals, and perform multi-mode feature extraction and modulation scheme labeling on each overlapping signal to form a sample set of overlapping signals;

[0034] S3. Construct a multi-scale neural network and train the multi-scale neural network based on the sample set;

[0035] S4. Use the trained neural network to identify overlapping signal modulation patterns.

[0036] In embodiments of this application, the dataset contains eight types of modulation, such as linear frequency modulation (LFM), sinusoidal frequency modulation (SinFM), hyperbolic frequency modulation (HyperFM), step frequency modulation (StepFM), Costas-coded frequency modulation (Costas), binary phase modulation (BPSK), Barker-coded phase modulation (Barker), and Frank-coded phase modulation (Frank). All signals share the same center frequency, and the bandwidth range remains consistent for each type of frequency modulation signal.

[0037] In the embodiments of this application, when there are a total of eight candidate modulation types, the dataset is generated by MATLAB simulation based on the analytical expressions of different modulation schemes. Each sample may contain multiple overlapping modulation signals. The dataset contains all 2^8–1 = 255 combinations of the eight modulation schemes. The constructed dataset contains a total of Nsample = 10,200 samples, of which 50% are randomly selected for training, and the remainder are equally distributed for validation and testing.

[0038] The generated samples used three modalities as neural network inputs: the amplitude of the STFT, the phase of the STFT, and the amplitude of the WVD. The sample labels were multi-labels, such as 01000111, where 0 indicates that the modulation scheme does not exist in the sample, and 1 indicates the opposite.

[0039] In the embodiments of this application, the proposed neural network overview is as follows: Figure 2As shown, the captured raw I / Q signal (also known as the analytic signal) is processed into three time-frequency representations. These three data representations are then used as input to form a so-called "multimodal" input. The input data is passed through cascaded "multi-scale" blocks to extract hidden information from the input and reassemble the latent information by splicing. Finally, the output layer follows the generated feature vectors to derive the final multi-label prediction. Notably, the proposed neural network does not include linear projection, i.e., it only applies convolutional functions. This ensures that features extracted from different input channels and different scales are fully fused. The proposed process employs a multi-label learning strategy and leverages a binary cross-entropy loss function. Unlike multi-class learning, where each modulation combination is treated as a new class, multi-label classification reduces to a binary classification of each modulation type.

[0040] Specifically, such as Figure 2 As shown, a multi-scale neural network consists of an input layer, four hidden layers, and an output layer:

[0041] The information input from the input layer is used as the initial information block and is fed into the first hidden layer for processing (convolution, normalization, and processing using the Leaky ReLU activation function). The processing result is then transmitted to the second hidden layer. The first hidden layer has two convolutional kernels: the first convolutional kernel has the same number of output channels as the input channels, and is used to process the initial information block. After normalization and processing using the Leaky ReLU activation function, it forms the first information block output by the first hidden layer. The second convolutional kernel has twice the number of output channels as the input channels, and is used to process the initial information block. After normalization and processing using the Leaky ReLU activation function, it forms the second information block output by the first hidden layer.

[0042] In the second hidden layer, there are three convolutional kernels. The first convolutional kernel has the same number of output channels as its input channels. It processes the initial information block and the first information block output from the first hidden layer, then normalizes and processes it using the Leaky ReLU activation function to form the first information block output from the second hidden layer. The second convolutional kernel has the same number of output channels as its input channels. It processes the second information block output from the first hidden layer, then normalizes and processes it using the Leaky ReLU activation function to form the second information block output from the second hidden layer. The third convolutional kernel has twice the number of output channels as its input channels. The second hidden layer first concatenates all the information blocks output from the first hidden layer, then feeds them into the third convolutional kernel for processing, then normalizes and processes them using the Leaky ReLU activation function to form the third information block output from the second hidden layer.

[0043] In the third hidden layer, there are four convolutional kernels. The first kernel has the same number of output channels as its input channels. It processes the initial information block and the first information block output from the first two hidden layers, then normalizes and applies the Leaky ReLU activation function to form the first information block output from the third hidden layer. The second kernel has the same number of input channels as its output channels. It processes the second information block output from the first two hidden layers, then normalizes and applies the Leaky ReLU activation function to form the second information block output from the third hidden layer. The third kernel has the same number of output channels as its input channels. It processes the third information block output from the second hidden layer, then normalizes and applies the Leaky ReLU activation function to form the third information block output from the third hidden layer. The fourth kernel has twice the number of output channels as its input channels. The third hidden layer first concatenates all information blocks output from the second hidden layer, then feeds them into the fourth kernel for processing, normalization, and Leaky ReLU activation. After ReLU processing, the fourth information block is generated from the output of the third hidden layer;

[0044] The fourth hidden layer contains five convolutional kernels. The first kernel has the same number of output and input channels. It processes the initial information block and the first information block output from the first three hidden layers, then normalizes and applies the Leaky ReLU activation function to form the first information block output from the fourth hidden layer. The second kernel has the same number of input and output channels. It processes the second information block output from the first three hidden layers, then normalizes and applies the Leaky ReLU activation function to form the second information block output from the fourth hidden layer. The third kernel has the same number of output and input channels. It processes the third information block output from the second and third hidden layers, then normalizes and applies the Leaky ReLU activation function to form the third information block output from the fourth hidden layer. The fourth kernel has the same number of output and input channels. It processes the fourth information block output from the third hidden layer, then normalizes and applies the Leaky ReLU activation function. After ReLU processing, the fourth information block output by the fourth hidden layer is formed; the number of output channels of the fifth convolutional kernel is twice the number of input channels. The fourth hidden layer first concatenates all the information blocks output by the third hidden layer, then feeds them into the fifth convolutional kernel for processing, then normalizes them and uses the Leaky ReLU activation function to form the fifth information block output by the fourth hidden layer.

[0045] The output layer includes two convolutional layers;

[0046] In the output layer, all information blocks output from the fourth hidden layer are first concatenated and then fed into the first convolutional kernel of the output layer for processing. After normalization and processing using the Leaky ReLU activation function, global feature mapping is completed and then fed into the second convolutional kernel. The number of input channels of the first convolutional kernel in the output layer is the same as the number of channels of the concatenated information blocks, and the number of output channels is the preset number. The number of input channels of the second convolutional kernel is the same as the number of output channels of the first convolutional kernel, and the number of output channels of the second convolutional kernel is the same as the number of modulation schemes, which is used to predict the final output.

[0047] In the multi-scale neural network, the number of channels in the input layer is the same as the number of channels in the sample features. All convolutional kernels in the hidden layer of the multi-scale neural network use the two-dimensional convolutional function conv2d. The first convolutional kernel of the output layer of the multi-scale neural network uses the two-dimensional convolutional function conv2d, and the second convolutional kernel of the output layer of the multi-scale neural network uses the one-dimensional convolutional function conv1d.

[0048] The three time-frequency inputs for multimodal processing are the amplitude of the short-time Fourier transform, the phase of the short-time Fourier transform, and the WVD (Wigner-Ville distribution). The calculation formulas are as follows:

[0049]

[0050]

[0051] To accurately identify short and long pulses, it is necessary to comprehensively evaluate both small-scale and large-scale features in the data representation simultaneously. Therefore, a multi-scale structure is employed to achieve this. The entire multi-scale structure consists of four blocks, each with an almost identical architecture, differing only in kernel size and number of channels. The key is the concatenation of small-scale features (e.g., 3x3 blocks) with large-scale features (e.g., 18x18 blocks). Repeating this operation across the four cascaded blocks fuses information from both small-scale and large-scale features, enabling the simultaneous classification of short and long pulses. The output layer includes a 2D convolutional layer for global feature mapping and a 1D convolutional layer for predicting the final output. This constitutes the "multi-scale" neural network.

[0052] In the embodiments of this application, to compare the performance of the proposed method, Table 1 compares the test results of the proposed process and the baseline by displaying the average recognition accuracy and variance of the entire performance cube. The proposed process achieves the highest average recognition accuracy and the lowest variance, indicating that it has significantly consistent AMR performance under different SNR, AR, and various modulation types. Although SABLNet and ResNet lead under some AMR conditions, the proposed method still leads by approximately 0.1 in average AMR accuracy.

[0053] Table 1: Comparison of recognition results of the proposed method and the baseline method on AMR

[0054]

[0055] The computational complexity results for the proposed neural network and the baseline are shown in Table 2. These results demonstrate that SABLNet has the lowest FLOPs, thanks to the low computational complexity of its self-attention layers and LSTM architecture. However, compared to neural networks processing inputs similar to 2D images, the proposed network achieves state-of-the-art AMR performance at near-minimum computational cost. This is likely a result of the rational design of the multimodal, multi-scale architecture and the strategic utilization of deep learning tools.

[0056] Table 2: Comparison of network complexity of the proposed method and baseline methods on AMR

[0057]

[0058] The above description represents preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technical or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for recognizing overlapping signal modulation patterns based on multimodal and CNN, characterized in that: Includes the following steps: S1. Given a variety of potential modulation schemes that need to be identified; S2. Based on the potential modulation schemes and their combinations, generate multiple overlapping signals, and perform multimodal feature extraction and modulation scheme labeling on each overlapping signal to form a sample set of overlapping signals; In step S2, the method for multimodal feature extraction of any overlapping signal is as follows: Representing the analytic signal of the overlapping signal at n time-domain sampling points as x[n], and performing short-time Fourier transform and WVD transform on the overlapping signal, we obtain: ； Where w represents frequency, m is time window parameter, and * represents conjugate; For STFT(m, w), take amplitude A1 and phase P1 respectively, and for WVD(m, w), take amplitude A2. Then the results are used to form the feature vector [A1, P1, A2] of the multimode of the overlapping signal. S3. Construct a multi-scale neural network and train the multi-scale neural network based on the sample set; The multi-scale neural network constructed in step S3 includes an input layer, four hidden layers, and an output layer; Among them, four hidden layers serve as feature fusion layers, and the output layer integrates the fused features to output the prediction result. Each layer is processed using convolutional connections, and the information block size between each layer is different to ensure that feature information at each scale is fused together through connections. There are two convolutional kernels in the first hidden layer, three convolutional kernels in the second hidden layer, four convolutional kernels in the third hidden layer, and five convolutional kernels in the fourth hidden layer. In the multi-scale neural network, the number of channels in the input layer is the same as the number of channels in the sample features. All convolutional kernels in the hidden layer of the multi-scale neural network use the two-dimensional convolutional function conv2d. The first convolutional kernel in the output layer of the multi-scale neural network uses the two-dimensional convolutional function conv2d, and the second convolutional kernel in the output layer of the multi-scale neural network uses the one-dimensional convolutional function conv1d. The input data is passed through four cascaded hidden layers to extract hidden information, and the latent information is reassembled by splicing. Finally, the output layer follows the generated feature vector to obtain the final multi-label prediction. S4. Use the trained multi-scale neural network to identify overlapping signal modulation patterns.

2. The method for recognizing overlapping signal modulation patterns based on multimodal and CNN according to claim 1, characterized in that: Let N be the number of potential modulation schemes, then the number of modulation combinations is: -1.

3. The method for recognizing overlapping signal modulation patterns based on multimodal and CNN according to claim 2, characterized in that: The number of modulation methods N is 8, including: linear frequency modulation, sinusoidal frequency modulation, hyperbolic frequency modulation, step frequency modulation, Costas-coded frequency modulation, binary phase modulation, Barker-coded phase modulation, and Frank-coded phase modulation. For all modulation methods, the center frequency of all signals is the same; For all types of frequency modulation signals, the bandwidth range remains consistent. Frequency modulation signals refer to signals with linear frequency modulation, sinusoidal frequency modulation, hyperbolic frequency modulation, and step frequency modulation.

4. The method for recognizing overlapping signal modulation patterns based on multimodal and CNN according to claim 1, characterized in that: In step S2, multiple overlapping signals need to be generated for each combination of modulation methods.

5. The method for recognizing overlapping signal modulation patterns based on multimodal and CNN according to claim 1, characterized in that: In step S2, the modulation schemes for the overlapping signals are labeled as: K1, K2, ..., K N Where N represents the number of modulation schemes; let K n The label represents the nth modulation scheme, where n = 1, 2, ..., N; if the overlapping signal contains the nth modulation scheme, then K n =1, if the overlapping signal does not contain the nth modulation scheme, then K n =0.

6. The method for recognizing overlapping signal modulation patterns based on multimodal and CNN according to claim 1, characterized in that: In step S2, the feature vector of any overlapping signal is used as a sample feature, and the label of the overlapping signal is used as a sample label. The sample feature and the sample label constitute the signal sample of the overlapping signal. Adding all the signal samples of overlapping signals to the same set constitutes the sample set of overlapping signals.

Images