Multi-modal feature fusion modulation identification method based on signal-to-noise ratio guidance

Abstract

The invention discloses a modulation identification method for multi-modal feature fusion based on signal-to-noise ratio guidance, and belongs to the technical field of signal processing driven by artificial intelligence. In order to solve the problems that in an existing modulation recognition technology, single-mode features are not universal, noise is sensitive, performance is reduced in a low signal-to-noise ratio environment, and utilization of complementary information among modes is insufficient due to coarseness of a multi-mode fusion strategy, an original I / Q signal is converted into three modes including a time-frequency graph, a three-dimensional constellation point cloud and an I / Q time sequence; an input signal is dynamically evaluated by utilizing a pre-trained signal-to-noise ratio estimation model, the fusion weight of each modal feature is intelligently adjusted by taking an estimated signal-to-noise ratio value as guide information, adaptive weighted fusion is realized, and finally a modulation type is identified through a classifier. According to the method, the accuracy and robustness of modulation identification in a complex electromagnetic environment are effectively improved, and the method is suitable for wireless communication, frequency spectrum monitoring and Internet of Things equipment authentication.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence driven signal processing, and particularly relates to a modulation recognition method based on multi-modal feature fusion guided by signal-to-noise ratio. BACKGROUND

[0002] With the rapid growth of the number of wireless communication devices, modern communication systems face severe challenges in spectrum management and communication monitoring, and it is urgent to improve the performance of modulation recognition technology to ensure spectrum utilization efficiency. Data-driven deep learning technology has become a research hotspot in this field. Based on a large amount of labeled data, researchers have widely explored methods for modulation recognition using single modal features, such as using IQ time series data, time-frequency graphs or constellation graphs as input, combining convolutional neural networks, recurrent neural networks and other models for feature learning and classification, and have made certain achievements. However, in actual application scenarios, especially in complex electromagnetic environments with multipath effects, signal attenuation and strong noise, the limitations of the above single modal methods are increasingly evident. The features of modulation signals are essentially multi-dimensional, covering amplitude, phase, frequency and other aspects, and single-perspective features are difficult to fully and stably reflect their inherent characteristics.

[0003] Specifically, single modal features have the problem of non-universality, limited ability to distinguish different modulation types, and are extremely sensitive to noise interference, resulting in a significant decline in recognition performance under low signal-to-noise ratio conditions. To overcome the shortcomings of single modal, multi-modal deep learning technology is introduced, aiming to learn more discriminative feature representations by fusing complementary information from different modalities. Existing research attempts to fuse real-time features, statistical features, spectral features and I / Q signals, or combine cyclic spectrograms and constellation graphs, etc., which has improved performance to some extent, but their fusion strategies are usually simple, such as directly concatenating or averaging weighted after independent feature extraction. This coarse-grained fusion method has inherent defects, as it fails to consider the dynamic changes in the reliability of each modal feature under different signal-to-noise ratio conditions. In low signal-to-noise ratio scenarios, the effective information of some modalities is easily overwhelmed by noise, and its contribution cannot be adaptively suppressed in a fixed weight fusion strategy, while the advantages of other modalities that are relatively robust in noisy environments are not effectively highlighted, resulting in the failure to fully exploit the synergistic advantages between modalities, limiting further improvement in recognition performance.

[0004] Therefore, how to design a method that can intelligently evaluate signal quality and dynamically adjust the fusion strategy in complex and variable actual electromagnetic environments, especially in the face of dynamic fluctuations in signal-to-noise ratio, has become a key problem in improving the robustness and accuracy of modulation recognition systems. SUMMARY

[0005] To address the aforementioned technical problems, this invention proposes a modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion, thereby resolving the issues present in the prior art.

[0006] In a first aspect, to achieve the above objectives, the present invention provides a modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion, comprising the following steps:

[0007] S1. Input the original I / Q signal into the pre-trained signal-to-noise ratio estimation model to obtain the corresponding signal-to-noise ratio estimate;

[0008] S2. Preprocess the original I / Q signal to construct constellation point cloud mode, time-frequency map mode and I / Q timing mode respectively;

[0009] S3. Input the constellation point cloud mode, time-frequency map mode and I / Q timing mode into the corresponding encoder model respectively, and extract the features of each mode;

[0010] S4. Using the signal-to-noise ratio estimate as guiding information, the extracted modal features are weighted and fused to obtain fused features;

[0011] S5. Input the fused features into the classifier to perform modulation category prediction.

[0012] Optionally, in S1, the pre-training process of the signal-to-noise ratio estimation model includes:

[0013] Construct a model consisting of a one-dimensional convolutional neural network and a long short-term memory network;

[0014] The model is trained by minimizing the mean absolute error between the predicted and true signal-to-noise ratios, using the original I / Q signals as input and the true signal-to-noise ratio as the label.

[0015] Optionally, in S2, the process of constructing the time-frequency diagram mode includes:

[0016] Convert the in-phase and quadrature components of the original I / Q signal into a complex signal;

[0017] A time-frequency coefficient matrix is ​​generated by applying continuous wavelet transform to the complex signal;

[0018] The modulus of the time-frequency coefficient matrix is ​​taken as the time-frequency graph and enhanced.

[0019] The process of constructing constellation point cloud modes includes:

[0020] Oversampling interpolation and symbol timing synchronization are performed on the original I / Q signal;

[0021] Convert each symbol point to amplitude and phase in polar coordinates;

[0022] Normalize and compress the amplitude;

[0023] The polar coordinates are mapped to a three-dimensional Cartesian coordinate system to form a point cloud.

[0024] Optionally, in S3, the encoder model includes:

[0025] A point cloud processing encoder for constellation point cloud modes, used to extract spatial geometric features;

[0026] A two-dimensional convolutional neural network encoder for time-frequency plot modes, used to extract joint time-frequency features;

[0027] One-dimensional convolutional neural network or long short-term memory network encoder for I / Q temporal modalities, used to extract temporal features.

[0028] Optionally, in S4, the weighted fusion process includes:

[0029] The signal-to-noise ratio estimate is normalized to a preset range;

[0030] The normalized signal-to-noise ratio value is input into the fully connected layer for feature embedding;

[0031] The embedded features are input into a multilayer perceptron to generate fusion weight vectors for each modality;

[0032] The weight vector is normalized using the Softmax function; the fused features are obtained by multiplying each modal feature by its corresponding weight and then summing the results.

[0033] Optionally, in S5, the classifier is a fully connected neural network; the classification process includes: inputting the fused features into the fully connected neural network, outputting the predicted scores for each modulation category; and applying the Softmax function to convert the predicted scores into a probability distribution.

[0034] Secondly, the present invention also provides a modulation recognition system based on signal-to-noise ratio-guided multimodal feature fusion, for implementing a modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion, the system comprising:

[0035] The signal-to-noise ratio (SNR) estimation module is used to input the original I / Q signal into the pre-trained SNR estimation model and obtain the corresponding SNR estimate.

[0036] The mode construction module is used to preprocess the original I / Q signal to construct constellation point cloud modes, time-frequency map modes, and I / Q timing modes, respectively.

[0037] The feature extraction module is used to input the constellation point cloud mode, time-frequency map mode, and I / Q time series mode into the corresponding encoder model respectively, and extract the features of each mode;

[0038] The feature fusion module is used to use the signal-to-noise ratio estimate as guiding information to perform weighted fusion of the extracted modal features to obtain fused features;

[0039] The classification module is used to input the fused features into the classifier for modulation category prediction.

[0040] Thirdly, the present invention also provides a computer terminal device, comprising:

[0041] One or more processors;

[0042] A memory, coupled to the processor, for storing one or more programs;

[0043] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion in the first aspect described above.

[0044] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion in the first aspect described above.

[0045] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion described in the first aspect above.

[0046] Compared with the prior art, the present invention has the following advantages and technical effects:

[0047] This invention provides a modulation recognition method based on signal-to-noise ratio (SNR)-guided multimodal feature fusion. By constructing a three-modal representation of the signal in the time, frequency, and spatial domains, and employing a staged dual-modal fusion mechanism based on SNR estimation, robust feature extraction and classification under strong noise environments are achieved. This method fully utilizes the complementary information of multimodal features, solving the problems of non-generality and noise sensitivity of single-modal features. Through SNR-guided dynamic weight allocation, adaptive adjustment of the importance differences of each mode under different noise conditions is achieved, overcoming the coarse-grained defects of existing multimodal fusion strategies. Ultimately, this effectively improves the discrimination ability, generalization performance, and anti-interference ability of modulation recognition in complex electromagnetic environments. Attached Figure Description

[0048] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0049] Figure 1 This is a flowchart illustrating the overall method of an embodiment of the present invention;

[0050] Figure 2 This is a diagram illustrating the model training framework of an embodiment of the present invention. Detailed Implementation

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0053] Example 1

[0054] like Figure 1 As shown, this embodiment provides a modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion, including:

[0055] S1. Input the original I / Q signal into the pre-trained signal-to-noise ratio estimation model to obtain the corresponding signal-to-noise ratio estimate;

[0056] S2. Preprocess the original I / Q signal to construct constellation point cloud mode, time-frequency map mode and I / Q timing mode respectively;

[0057] S3. Input the constellation point cloud mode, time-frequency map mode and I / Q timing mode into the corresponding encoder model respectively, and extract the features of each mode;

[0058] S4. Using the signal-to-noise ratio estimate as guiding information, the extracted modal features are weighted and fused to obtain fused features;

[0059] S5. Input the fused features into the classifier to perform modulation category prediction.

[0060] like Figure 2 As shown in the diagram, this embodiment provides a model training framework diagram. The present invention specifically includes the following steps:

[0061] S1: Input the original I / Q signal into the pre-trained signal-to-noise ratio (SNR) estimation model to obtain the corresponding SNR estimate;

[0062] S2: Preprocess the raw I / Q signals to construct three modal data: constellation point cloud, time-frequency diagram, and I / Q time series.

[0063] S3: Input the three modal data into their respective encoder models, and optimize the parameters of each modal encoder through training to achieve efficient modal feature extraction;

[0064] S4: Using the SNR estimate obtained in S1 as guiding information, the modal features extracted in step S3 are weighted and fused to achieve multimodal feature fusion guided by signal-to-noise ratio.

[0065] S5: Input the fused features into the classifier to predict the modulated class, calculate the loss based on the prediction results, and optimize the parameters of the classifier and the overall model.

[0066] As one implementation method in this embodiment, in S1, the pre-training process of the signal-to-noise ratio estimation model includes:

[0067] Construct a model consisting of a one-dimensional convolutional neural network and a long short-term memory network;

[0068] The model is trained by minimizing the mean absolute error between the predicted and true signal-to-noise ratios, using the original I / Q signals as input and the true signal-to-noise ratio as the label.

[0069] Specifically: S1.1: Pre-train the signal-to-noise ratio (SNR) estimation model using raw I / Q data. A lightweight convolutional neural network model is constructed using the raw I / Q signals to perform regression estimation of the signal's SNR value. Let the first... The true SNR of each sample is The model predicts the value. The training objective is to minimize the mean absolute error (MAE) between the predicted and true values, and its loss function is defined as follows:

[0070] (1)

[0071] Where N represents the number of training samples.

[0072] As one implementation method in this embodiment, in S2, the process of constructing the time-frequency diagram mode includes:

[0073] Convert the in-phase and quadrature components of the original I / Q signal into a complex signal;

[0074] A time-frequency coefficient matrix is ​​generated by applying continuous wavelet transform to the complex signal;

[0075] The modulus of the time-frequency coefficient matrix is ​​taken as the time-frequency graph and enhanced.

[0076] The process of constructing constellation point cloud modes includes:

[0077] Oversampling interpolation and symbol timing synchronization are performed on the original I / Q signal;

[0078] Convert each symbol point to amplitude and phase in polar coordinates;

[0079] Normalize and compress the amplitude;

[0080] The polar coordinates are mapped to a three-dimensional Cartesian coordinate system to form a point cloud.

[0081] Furthermore, step S2 includes the following:

[0082] S2.1: Convert the raw I / Q data into a time-frequency graph. Perform continuous wavelet transform (CWT) on the raw I / Q signal to generate a time-frequency graph (scalogram), and further perform enhancement and normalization processing. The specific steps are as follows:

[0083] (1) Generate complex signals:

[0084] Convert the input real number IQ data into complex form:

[0085] (2)

[0086] Where, I∈ and Q∈ These represent the in-phase and quadrature components of the input signal, respectively.

[0087] (2) Calculation of time-frequency diagram using continuous wavelet transform (CWT):

[0088] For complex signals Using Morlet wavelets for CWT, the time-frequency coefficient matrix C is generated:

[0089] (3)

[0090] Where a is the scaling factor and b is the translation factor. This is the mother wavelet function. The final result is the modulus of the coefficients. As the original time-frequency plot, some enhancement operations are then performed on the time-frequency plot.

[0091] S2.2 converts the raw I / Q data into a 3D point cloud. First, it uses the I / Q data in complex form as described above. Then, it performs symbol timing synchronization to eliminate the influence of sampling bias on the constellation structure. First, it performs oversampling interpolation on the signal (e.g., 16x upsampling), and then uses Gardner decision error to recover the symbol timing.

[0092] (4)

[0093] in, It is the first Complex signal values ​​at each sampling point; These are the complex signal values ​​of adjacent sampling points; This refers to complex conjugate operations; Take the real part of the complex number; It refers to the first The symbol timing error value for each sampling point.

[0094] Then, the standardized constellation points are mapped from the two-dimensional complex plane to three-dimensional space to form a point cloud representation:

[0095] (1) Polar coordinate transformation to obtain amplitude With angle for:

[0096] (5)

[0097] in, It refers to the first The amplitude of each sampling point; It refers to the first Phase angle of each sampling point; It refers to the arctangent function in the four quadrants.

[0098] (2) Amplitude compression and pruning (weakening the influence of outliers):

[0099] (6)

[0100] in, This refers to the 95th percentile of the amplitude value; It refers to the normalized amplitude value, which is restricted to the interval [0,1].

[0101] (3) Convert polar coordinates to three-dimensional coordinate representation:

[0102] (7)

[0103] in, These are the coordinate values ​​of the 3D point cloud.

[0104] As one implementation method in this embodiment, in S3, the encoder model includes:

[0105] A point cloud processing encoder for constellation point cloud modes, used to extract spatial geometric features;

[0106] A two-dimensional convolutional neural network encoder for time-frequency plot modes, used to extract joint time-frequency features;

[0107] One-dimensional convolutional neural network or long short-term memory network encoder for I / Q temporal modalities, used to extract temporal features.

[0108] Furthermore, step S3 includes the following:

[0109] S3.1: Modal feature extraction. For constellation point cloud modalities, the spatial distribution features are encoded using a point cloud processing model to preserve the spatial arrangement pattern of the modulation pattern on the constellation diagram.

[0110] For time-frequency map modes, local joint features in the time and frequency domains are extracted using a two-dimensional convolutional network (such as a lightweight ResNet).

[0111] For I / Q time-series modes, one-dimensional convolution or time-series modeling structures (such as CNN1D, LSTM, etc.) are used to encode the amplitude and phase changes in relation to time.

[0112] The feature extraction process can be formally represented as:

[0113] (8)

[0114] in, This represents the input signal for the m-th mode; This indicates the encoder structure corresponding to the modality; This represents the extracted modal features.

[0115] As one implementation method in this embodiment, in S4, the weighted fusion process includes:

[0116] The signal-to-noise ratio estimate is normalized to a preset range;

[0117] The normalized signal-to-noise ratio value is input into the fully connected layer for feature embedding;

[0118] The embedded features are input into a multilayer perceptron to generate fusion weight vectors for each modality;

[0119] The weight vector is normalized using the Softmax function; the fused features are obtained by multiplying each modal feature by its corresponding weight and then summing the results.

[0120] Specifically: S4.1: Signal SNR estimation, using a pre-trained SNR regression model. Estimate the original IQ signal and output the predicted SNR value for each sample:

[0121] (9)

[0122] in, This refers to the input IQ signal data.

[0123] S4.2: SNR Feature Embedding and Modal Weight Generation. The normalized SNR values ​​are input into a linear layer for high-dimensional embedding to generate feature vectors. The input is then fed into a multilayer perceptron (MLP) to generate modal fusion weights. :

[0124] (10)

[0125] The softmax operation is used to ensure that the sum of all modal weights is 1.

[0126] S4.3: Modal feature weighting. Based on the above fusion weights, the modal features are weighted and summed to obtain the final fused feature vector:

[0127] (11)

[0128] in, Indicates multiplication. , These are the characteristics after fusion.

[0129] As one implementation method in this embodiment, in S5, the classifier is a fully connected neural network; the classification process includes: inputting the fused features into the fully connected neural network, outputting the predicted scores of each modulation category; and applying the Softmax function to convert the predicted scores into a probability distribution.

[0130] Furthermore, step S5 includes the following:

[0131] S5.1: Input the fused multimodal feature vectors into a fully connected neural network classifier, and output the prediction scores for each modulation category:

[0132] (12)

[0133] A linear layer maps the features into a vector representing the number of categories, yielding the predicted score for each category. This vector can then be used as input to the logits function for subsequent inference or loss calculation.

[0134] Used in the inference stage or for cross-entropy loss calculation:

[0135] (13)

[0136] Where p is the predicted probability and C is the number of categories.

[0137] This concludes the flowchart of the present invention. Further implementation details are provided below with specific examples:

[0138] Taking the RML2016.10a radio modulation recognition dataset as an example, this dataset contains three modality data:

[0139] I / Q timing mode: Each sample has 128 complex sampling points and a sampling rate of 1MHz;

[0140] Time-frequency mode: obtained from the I / Q signal through continuous wavelet transform. Grayscale image;

[0141] Constellation map 3D point cloud modality: Each sample contains 128 symbol points, and each symbol point is converted into 3D coordinates. express.

[0142] The dataset contains 11 modulation schemes (BPSK, QPSK, 8PSK, QAM16, QAM64, AM-DSB, AM-SSB, FM, GFSK, CPFSK, PAM4), with an SNR range of [missing information]. to The step size is 2dB.

[0143] Step 1: Data Preprocessing

[0144] Hyperparameter settings: Wavelet transform scaling parameters Morlet wavelet mother function center frequency ;

[0145] 3D point cloud of constellation diagram: interpolated to 128 symbol points, symbol timing recovery and phase correction performed; converted to polar coordinates. Then, the amplitude is normalized to [0,1], and a nonlinear compression function is used. Ultimately, it is mapped to three-dimensional spatial coordinates.

[0146] Time-frequency graph: The time-frequency coefficient matrix is ​​obtained by continuous wavelet transform, the amplitude spectrum is logarithmically enhanced, then Gaussian smoothed, and finally subjected to global and local double normalization to adjust to... size.

[0147] Step 2: Pre-training of the signal-to-noise ratio estimation module:

[0148] Network structure: 4 layers of 1D convolution (kernel size 3, stride 1, number of channels 32, 64, 128, 256 respectively) + residual connection + two layers of LSTM (256 hidden units per layer).

[0149] Loss function: Mean Absolute Error (MAE);

[0150] Optimizer and training parameters: Adam optimizer, initial learning rate 0.001, batch size 256, training epochs 50.

[0151] The original I / Q signal is used as input, and the predicted SNR value of the output signal is used. After training, the parameters are fixed as the guiding module.

[0152] Step 3: Multimodal feature extraction:

[0153] PointNet extracts spatial geometric features of constellation point clouds (input 128 points, output feature dimension 256).

[0154] LiteResNet extracts frequency domain features from time-frequency maps (16 / 32 / 64 convolutional channels, 3 residual units, and 256 output feature dimensions).

[0155] 1D-CNN extracts I / Q temporal features (kernel size 3, number of channels 32 / 64 / 128, output feature dimension 256).

[0156] The output feature vectors of the three encoders have a uniform dimension of 256.

[0157] Step 4: Signal-to-noise ratio guided multimodal feature fusion:

[0158] Use the SNR estimation module in step 2 to obtain the SNR value of the current sample. Normalize to [0,1]; Input to a fully connected network (64 hidden units), output a three-modal weight vector Normalized by Softmax;

[0159] Fusion formula:

[0160] (14)

[0161] in For the first The feature vectors of each modality.

[0162] Step 5: Classification Decision:

[0163] The classifier consists of layer normalization + fully connected layers (256→128→11) + Dropout (0.5); it is trained using the cross-entropy loss function, the Adam optimizer, the initial learning rate of 0.0005, and the batch size of 128; the output layer is converted into the probability distribution of 11 modulation modes by Softmax.

[0164] The above describes an application example of this invention for modulation recognition using multimodal fusion of IQ signals in a practical situation. This invention proposes a signal-to-noise ratio (SNR)-guided multimodal fusion radio modulation recognition method. It innovatively combines three modal features: time-domain I / Q timing, frequency-domain time-frequency plot, and spatial-domain constellation point cloud. Through a pre-trained SNR estimation model, it dynamically guides the fusion weights, achieving adaptive weighted fusion of multimodal features. This method fully mines and fuses multi-source information from radio signals, effectively improving the discrimination ability and generalization performance of modulation recognition, and is particularly suitable for complex and variable electromagnetic environments. Validation on public datasets demonstrates the advantages of this method in terms of recognition accuracy and robustness. This invention has broad application prospects and is applicable to fields such as wireless communication spectrum monitoring, radar signal recognition, and IoT device authentication.

[0165] Example 2

[0166] In this embodiment, a computer terminal device is provided, including:

[0167] One or more processors;

[0168] A memory, coupled to the processor, for storing one or more programs;

[0169] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion described above.

[0170] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion described above.

[0171] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion described above.

[0172] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion described above.

[0173] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0174] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0175] This embodiment provides such a device or system. The system, referred to as a modulation recognition system based on signal-to-noise ratio-guided multimodal feature fusion, includes:

[0176] The signal-to-noise ratio (SNR) estimation module is used to input the original I / Q signal into the pre-trained SNR estimation model and obtain the corresponding SNR estimate.

[0177] The mode construction module is used to preprocess the original I / Q signal to construct constellation point cloud modes, time-frequency map modes, and I / Q timing modes, respectively.

[0178] The feature extraction module is used to input the constellation point cloud mode, time-frequency map mode, and I / Q time series mode into the corresponding encoder model respectively, and extract the features of each mode;

[0179] The feature fusion module is used to use the signal-to-noise ratio estimate as guiding information to perform weighted fusion of the extracted modal features to obtain fused features;

[0180] The classification module is used to input the fused features into the classifier for modulation category prediction.

[0181] As one implementation method in this embodiment, the signal-to-noise ratio estimation module includes a one-dimensional convolutional neural network unit and a long short-term memory network unit; the one-dimensional convolutional neural network unit is used to extract local features of the original I / Q signal; the long short-term memory network unit is used to capture the temporal dependency of the original I / Q signal; the signal-to-noise ratio estimation model is composed of the one-dimensional convolutional neural network unit and the long short-term memory network unit.

[0182] As one implementation method in this embodiment, the modality construction module includes a time-frequency diagram construction unit and a constellation point cloud construction unit. The time-frequency diagram construction unit is used to convert the original I / Q signal into a complex signal, apply continuous wavelet transform to generate a time-frequency coefficient matrix, and perform enhancement processing by taking the modulus value to generate a time-frequency diagram. The constellation point cloud construction unit is used to perform oversampling interpolation and symbol timing synchronization on the original I / Q signal, convert each symbol point into amplitude and phase in polar coordinates, normalize and compress the amplitude, and map it to a three-dimensional Cartesian coordinate system to form a point cloud.

[0183] As one implementation method in this embodiment, the feature extraction module includes a point cloud encoder unit, a time-frequency map encoder unit, and a time sequence encoder unit; the point cloud encoder unit is used to process constellation point cloud modes and extract spatial geometric features; the time-frequency map encoder unit is used to process time-frequency map modes and extract time-frequency joint features; the time sequence encoder unit is used to process I / Q time sequence modes and extract time sequence features.

[0184] As one implementation method in this embodiment, the feature fusion module includes a normalization unit, a feature embedding unit, a weight generation unit, and a weighted summation unit; the normalization unit is used to normalize the signal-to-noise ratio estimate to a preset range; the feature embedding unit is used to input the normalized signal-to-noise ratio value into a fully connected layer for feature embedding; the weight generation unit is used to input the embedded features into a multilayer perceptron to generate fusion weight vectors for each modality, and uses the Softmax function to normalize the weight vectors; the weighted summation unit is used to multiply each modality feature by its corresponding weight and then sum them to obtain the fusion feature.

[0185] As one implementation method in this embodiment, the classification module includes a fully connected neural network unit and a probability transformation unit; the fully connected neural network unit is used to map the fused features to the predicted score of the modulation category; the probability transformation unit is used to apply the Softmax function to convert the predicted score into a probability distribution.

[0186] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0187] The above implementation method solves the problem of modulation recognition based on signal-to-noise ratio-guided multimodal feature fusion in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0188] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A modulation recognition method based on signal-to-noise ratio-guided multimodal feature fusion, characterized in that, Includes the following steps: S1. Input the original I / Q signal into the pre-trained signal-to-noise ratio estimation model to obtain the corresponding signal-to-noise ratio estimate; S2. Preprocess the original I / Q signal to construct constellation point cloud mode, time-frequency map mode and I / Q timing mode respectively; S3. Input the constellation point cloud mode, time-frequency map mode and I / Q timing mode into the corresponding encoder model respectively, and extract the features of each mode; S4. Using the signal-to-noise ratio estimate as guiding information, the extracted modal features are weighted and fused to obtain fused features; S5. Input the fused features into the classifier to perform modulation category prediction.

2. The method according to claim 1, characterized in that, In S1, the pre-training process of the signal-to-noise ratio estimation model includes: Construct a model consisting of a one-dimensional convolutional neural network and a long short-term memory network; The model is trained by minimizing the mean absolute error between the predicted and true signal-to-noise ratios, using the original I / Q signals as input and the true signal-to-noise ratio as the label.

3. The method according to claim 1, characterized in that, In S2, the process of constructing the time-frequency diagram mode includes: Convert the in-phase and quadrature components of the original I / Q signal into a complex signal; A time-frequency coefficient matrix is ​​generated by applying continuous wavelet transform to the complex signal; The modulus of the time-frequency coefficient matrix is ​​taken as the time-frequency graph and enhanced. The process of constructing constellation point cloud modes includes: Oversampling interpolation and symbol timing synchronization are performed on the original I / Q signal; Convert each symbol point to amplitude and phase in polar coordinates; Normalize and compress the amplitude; The polar coordinates are mapped to a three-dimensional Cartesian coordinate system to form a point cloud.

4. The method according to claim 1, characterized in that, In S3, the encoder model includes: A point cloud processing encoder for constellation point cloud modes, used to extract spatial geometric features; A two-dimensional convolutional neural network encoder for time-frequency plot modes, used to extract joint time-frequency features; One-dimensional convolutional neural network or long short-term memory network encoder for I / Q temporal modalities, used to extract temporal features.

5. The method according to claim 1, characterized in that, In S4, the weighted fusion process includes: The signal-to-noise ratio estimate is normalized to a preset range; The normalized signal-to-noise ratio value is input into the fully connected layer for feature embedding; The embedded features are input into a multilayer perceptron to generate fusion weight vectors for each modality; The weight vector is normalized using the Softmax function; the fused features are obtained by multiplying each modal feature by its corresponding weight and then summing the results.

6. The method according to claim 1, characterized in that, In S5, the classifier is a fully connected neural network; The classification process includes: inputting the fused features into the fully connected neural network and outputting the prediction score for each modulation category; The Softmax function is applied to convert the predicted scores into a probability distribution.

7. A modulation recognition system based on signal-to-noise ratio-guided multimodal feature fusion, characterized in that, The system for implementing the method of any one of claims 1-6 comprises: The signal-to-noise ratio (SNR) estimation module is used to input the original I / Q signal into the pre-trained SNR estimation model and obtain the corresponding SNR estimate. The mode construction module is used to preprocess the original I / Q signal to construct constellation point cloud modes, time-frequency map modes, and I / Q timing modes, respectively. The feature extraction module is used to input the constellation point cloud mode, time-frequency map mode, and I / Q time series mode into the corresponding encoder model respectively, and extract the features of each mode; The feature fusion module is used to use the signal-to-noise ratio estimate as guiding information to perform weighted fusion of the extracted modal features to obtain fused features; The classification module is used to input the fused features into the classifier for modulation category prediction.

8. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images