A signal modulation recognition method based on graph neural network and time-frequency network fusion

Abstract

The application discloses a signal modulation recognition method based on fusion of a graph neural network and a time-frequency network, and belongs to the technical field of signal processing and artificial intelligence. The method comprises the following steps: acquiring signal sample data sets, and pre-processing the signal sample data sets to finally generate a training set, a verification set and a test set; constructing a signal modulation recognition model; training the signal modulation recognition model based on the training set and the verification set; and testing the trained signal modulation recognition model based on the test set. The method simultaneously utilizes topological structure information between signal samples and time sequence dynamic information of the signal samples through a double-branch architecture, and realizes automatic extraction and deep adaptive fusion of multi-modal features through advanced modules such as a formulaically defined graph structure, a time sequence feature extraction structure and a bidirectional cross attention, so that the method exhibits excellent performance, strong robustness and good generalization capability in a signal modulation recognition task through end-to-end joint training and a precise optimization strategy.

Description

Technical Field

[0001] This invention relates to the fields of signal processing and artificial intelligence, specifically to a signal modulation recognition method based on the fusion of graph neural networks and time-frequency networks. Background Technology

[0002] With the rapid development of wireless communication technology, signal modulation identification plays an important role in fields such as spectrum monitoring, electronic reconnaissance, and cognitive radio. Traditional signal modulation identification methods mainly rely on manual feature extraction, such as instantaneous features, statistical features, and transform domain features. These methods heavily depend on the prior knowledge of domain experts and have poor robustness under different signal-to-noise ratio conditions.

[0003] Modulation recognition technology has roughly gone through three stages of development. Initially, methods based on decision theory were used for recognition through hypothesis testing and likelihood ratio calculation. Although theoretically sound, these methods were computationally complex and required known channel parameters, limiting their practicality. Subsequently, feature extraction-based pattern recognition methods became mainstream. These methods combined manually designed feature extractors (such as calculating higher-order cumulants and spectral features of the signal) with traditional classifiers (such as support vector machines and decision trees) for classification, reducing reliance on prior knowledge. However, the robustness and generalization ability of the features remained bottlenecks.

[0004] In recent years, deep learning technology has been widely used in signal modulation recognition tasks. Convolutional Neural Networks (CNNs) can automatically extract local features of signals, but they have difficulty capturing temporal dependencies when processing long sequences of signals. Recurrent Neural Networks (RNNs) and their variants can process sequential data, but their serial computation characteristics lead to low training efficiency and the problems of vanishing and exploding gradients. More importantly, existing deep learning-based recognition methods still suffer from several inherent defects when dealing with signals in complex electromagnetic environments, severely limiting their performance ceiling: First, existing models (including CNN, RNN, and Transformer) typically treat each signal sample as an independent entity, lacking the ability to model the potential topological relationships between signal samples and ignoring the structural information of the "sample domain." Second, in the "time domain" processing, while the Transformer model based on self-attention can capture long-range dependencies, its computational complexity increases quadratically with the sequence length, resulting in huge computational overhead when processing long, highly sampled signals. RNN, on the other hand, is limited by its serial computation, making parallelization difficult and inefficient. Third, existing methods mostly process a single mode. Even when research attempts to fuse multiple features, it often employs shallow fusion strategies such as simple concatenation or weighted summation, failing to achieve deep interaction and adaptive complementarity between the structural features of the signal's "sample domain" and the dynamic features of the "time domain." Furthermore, existing architectures lack specialized and efficient modeling mechanisms for the complex periodicity and non-stationary characteristics commonly found in communication signals. Therefore, existing single-modal deep learning methods often cannot simultaneously utilize the structural and temporal information of signals, and have significant shortcomings in computational efficiency, feature depth fusion, and periodicity mining, which limits further improvement in recognition performance.

[0005] To address the aforementioned issues, there is an urgent need for a signal modulation recognition method based on the fusion of graph neural networks and time-frequency networks, which can solve the problems existing in traditional methods. Summary of the Invention

[0006] The purpose of this invention is to provide a signal modulation recognition method based on the fusion of graph neural networks and time-frequency networks. By using a dual-branch architecture, it simultaneously utilizes the topological structure information between signal samples and the temporal dynamic information of the signal itself. Through advanced modules such as a formulaically defined graph structure, a time-frequency feature extraction structure, and bidirectional cross-attention, it achieves automatic extraction and deep adaptive fusion of multimodal features. Through end-to-end joint training and precise optimization strategies, it exhibits excellent performance, strong robustness, and good generalization ability in signal modulation recognition tasks.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A signal modulation recognition method based on the fusion of graph neural networks and time-frequency networks includes:

[0009] Step 1: Obtain the signal sample dataset and preprocess it to generate the training set, validation set, and test set;

[0010] Step 2: Construct a signal modulation recognition model, which includes a graph neural network branch, a time-frequency network branch, and a feature fusion module;

[0011] Step 3: Train the signal modulation recognition model based on the training set and validation set;

[0012] Step 4: Test the trained signal modulation recognition model based on the test set.

[0013] Further, in step 1, the signal sample dataset is obtained and preprocessed to ultimately generate a training set, a validation set, and a test set, specifically as follows:

[0014] Step 101: Obtain the signal sample dataset, label each signal sample, and obtain the modulation type label corresponding to each signal sample;

[0015] Step 102: Perform multimodal processing on the signal sample dataset, construct graph structure data and original IQ signal time series data respectively, and divide the graph structure data, original IQ signal time series data and corresponding modulation type labels according to a preset ratio to obtain training set, validation set and test set.

[0016] Further, in step 102, the signal sample dataset undergoes multimodal processing to construct graph-structured data, specifically as follows:

[0017] Each signal sample in the signal sample dataset is converted into a graph structure representation. The node feature matrix contains five dimensions of statistical features, the edge index matrix is ​​constructed using the K-nearest neighbor algorithm to define the spatial connection relationship between nodes, and the batch index information is used to distinguish the node sets of different samples, thus obtaining the graph structure data.

[0018] Further, in step 102, multimodal processing is performed on the signal sample dataset to construct the original IQ signal time series data, specifically as follows:

[0019] Each signal sample in the signal sample dataset is represented in a three-dimensional tensor form. The first dimension represents the batch size, supporting batch processing; the second dimension represents the number of signal channels, which is fixed at 2 channels, corresponding to the in-phase component I and the quadrature component Q respectively; the third dimension represents the time series length, which is fixed at 128 time steps through resampling to ensure the uniformity of the input size.

[0020] Furthermore, the graph neural network branch consists of three consecutive graph convolutional layers.

[0021] Furthermore, the time-frequency network branch composite architecture includes a time-domain signal feature extraction module, a frequency-domain signal feature extraction module, and a cross-attention module. The time-domain signal feature extraction module is used to extract time-series features by embedding and positionally encoding the IQ signal and then using a multi-head self-attention mechanism. The frequency-domain signal feature extraction module is used to detect the main period of the IQ signal through fast Fourier transform and reshape the time-series data according to the detected period, and extract features within the period through two-dimensional convolutional blocks. The cross-attention module is used for the output of the time-domain signal feature extraction module and the frequency-domain signal feature extraction module.

[0022] Furthermore, the feature fusion module adopts a bidirectional cross-attention mechanism, including a first-direction cross-attention module and a second-direction cross-attention module. The first-direction cross-attention module is used to perform feature fusion with temporal features as queries and graph features as keys and values. The second-direction cross-attention module is used to perform feature fusion with graph features as queries and temporal features as keys and values.

[0023] Furthermore, in step 3, the training process employs a cross-entropy loss function and the Adam optimizer, along with an early stopping mechanism and an adaptive learning rate scheduling strategy based on validation set performance.

[0024] In summary, the present invention has at least one of the following beneficial technical effects:

[0025] 1. This invention innovatively proposes a dual-branch parallel architecture of graph neural networks and temporal networks, which simultaneously captures the structural correlation characteristics and temporal dynamic characteristics of signals through multimodal learning, thus solving the problem of incomplete feature representation of single-modal models.

[0026] 2. This invention designs a feature fusion mechanism based on bidirectional cross-attention, which realizes adaptive interaction and information complementarity between heterogeneous features through formulaic attention calculation, significantly improving the discriminative ability of fused features.

[0027] 3. This invention adopts the principle of graph isomorphic feature extraction and frequency domain period detection algorithm. Through mathematical modeling, it enhances the model's ability to understand the essential features of the signal and improves the theoretical basis of feature extraction.

[0028] 4. This invention achieves complete end-to-end learning from data preprocessing to final classification, and avoids the suboptimal problem caused by the separation of feature engineering and classifier design in traditional methods through joint optimization.

[0029] 5. By introducing a multi-granularity evaluation system and an adaptive training strategy, this invention ensures the model's strong robustness and good generalization ability in complex electromagnetic environments, providing reliable technical support for practical applications. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention;

[0031] Figure 2 This is a schematic diagram of the signal modulation recognition model structure;

[0032] Figure 3 This is a schematic diagram of the cross-attention mechanism;

[0033] Figure 4 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0035] like Figure 1 and Figure 4 As shown, this invention provides a signal modulation recognition method based on the fusion of graph neural networks and time-frequency networks, comprising:

[0036] Step 1: Obtain the signal sample dataset and preprocess it to generate the training set, validation set, and test set;

[0037] Step 2: Construct a signal modulation recognition model, which includes a graph neural network branch, a time-frequency network branch, and a feature fusion module;

[0038] Step 3: Train the signal modulation recognition model based on the training set and validation set;

[0039] Step 4: Test the trained signal modulation recognition model based on the test set.

[0040] Next, the present invention will be described in detail with reference to specific embodiments.

[0041] In step 1, the signal sample dataset is obtained and preprocessed to generate training, validation, and test sets, specifically:

[0042] Step 101: Obtain the signal sample dataset, label each signal sample, and obtain the modulation type label corresponding to each signal sample;

[0043] Step 102: Perform multimodal processing on the signal sample dataset, constructing graph structure data and original IQ signal time series data respectively. Divide the graph structure data, original IQ signal time series data, and corresponding modulation type labels according to a preset ratio to obtain training set, validation set, and test set, specifically:

[0044] 1. Convert each signal sample in the signal sample dataset into graph-structured data. The graph-structured data adopts the standard data format of the PyTorch Geometric framework and contains three core components:

[0045] (1) The node feature matrix part: each node represents a local segment of the signal or a feature dimension. The node features are five statistical features extracted from the original signal, constituting the dimension as follows: The matrix, where This represents the total number of nodes in a sample graph.

[0046] (2) The edge index matrix adopts the COO (Coordinate Format) sparse format with a dimension of It defines the connection relationships (edges) between nodes in the graph. The connection relationships can be constructed based on the K-Nearest Neighbors of signal samples in the feature space or fixed topological rules.

[0047] (3) The batch index part is a one-dimensional vector used to map nodes in the merged large graph back to their original samples during batch processing.

[0048] 2. Represent each signal sample in the signal sample dataset as a three-dimensional tensor with dimensions of . The first dimension is the batch size; the second dimension is fixed at 2, corresponding to the in-phase component (I) and quadrature component (Q) channels respectively; the third dimension is the time series length, which is unified to 128 time steps through interpolation or truncation and resampling to maintain the consistency of the input size.

[0049] 3. Randomly divide all processed signal samples and their corresponding modulation type labels into training set, validation set and test set in a ratio of 80%:10%:10%.

[0050] The constructed signal modulation recognition model is as follows Figure 2 As shown, the signal modulation recognition model includes a graph neural network branch, a time-series network branch, and a feature fusion module. The time-frequency network branch composite architecture will be described in detail below:

[0051] 1. Graph Neural Network Branch

[0052] The graph neural network branch uses a graph isomorphic network, and the forward propagation formula of its core layer is as follows:

[0053] (1)

[0054] In the formula, Indicates the first Layer nodes Features It is a multilayer perceptron consisting of two linear layers, batch normalization (BatchNorm), and ReLU activation function. It is a learnable scalar parameter. It is a node The neighbor set, in this embodiment, the branch contains three consecutive graph convolutional layers, each with an input / output dimension of 64. After three convolutional layers, global add pooling is used to aggregate the node-level features into graph-level features, as follows:

[0055] (2)

[0056] in, It is the set of all nodes in the graph. To preserve multi-scale information, the graph-level features output by the three layers are... The features are concatenated to form a 192-dimensional feature vector, and then reduced to 128 dimensions by a fully connected layer, denoted as . .

[0057] 2. Temporal network branches

[0058] The temporal network branch adopts a composite architecture, including a time-domain signal feature extraction module, a frequency-domain signal feature extraction module, and a cross-attention module, which are described in detail below:

[0059] (1) Time-domain signal feature extraction module

[0060] In the time-domain signal feature extraction module, the input IQ signal is first processed. Batch normalization is performed, followed by embedding through a depthwise separable convolution to map the 2D channel signal to a 16D feature space, resulting in... Next, we introduce sinusoidal positional encoding. To provide positional information for the sequence, as follows:

[0061] (3)

[0062] (4)

[0063] In the formula, It's about location. It is a dimensional index. The embedded features are added to the positional encoding and then input into a 4-head multi-head self-attention (MSA) module. The calculation of single-head attention is as follows:

[0064] (5)

[0065] The output of multi-head attention is processed through layer normalization (LayerNorm) and a feedforward network (FFN) to obtain the output features of temporal module feature extraction. .

[0066] (2) Frequency domain signal feature extraction module

[0067] In the frequency domain signal feature extraction module, the 2D IQ signal is first mapped to a 16D feature space through an embedding layer. Then, the signal passes through two cascaded period extraction modules. Each period extraction module first processes the input sequence... To detect the main period, a one-dimensional Fast Fourier Transform (FFT) is performed as follows:

[0068] (6)

[0069] (7)

[0070] Select amplitude spectrum The three largest frequency components Its corresponding period is For each cycle Reshaping the original sequence into The two-dimensional tensor is used to extract intra-period features through an Inception-style two-dimensional convolutional block. Features from all periods are weighted and fused according to their energy intensity, and then output through residual connections and layer normalization. The final output of the frequency domain signal feature extraction module is compressed into a 16-dimensional global temporal feature vector through global average pooling, denoted as . ;

[0071] (3) Cross-attention module

[0072] In the temporal branch internal fusion model, the output features of the temporal signal feature extraction module Output features of the (preserving timing structure) and frequency domain signal feature extraction module (Global representation) is fused through a cross-attention module to achieve... For query. It enhances the representation capability of time-series features by providing keys and values.

[0073] 3. Feature Fusion Module

[0074] The feature fusion module is responsible for integrating graph branch features. Temporal branching features To achieve deep fusion, firstly, the two are mapped to a unified 256-dimensional space through two independent linear projection layers:

[0075] (8)

[0076] (9)

[0077] The projected features are then subjected to layer normalization, and subsequently, as follows: Figure 3 As shown, a bidirectional cross-attention mechanism is used for deep interaction, which specifically includes a first-direction cross-attention module and a second-direction cross-attention module. The process is as follows:

[0078] (1) First-order cross-attention module: based on temporal features As a query , graph features As a key Sum ,for:

[0079] (10)

[0080] (2) Second-direction cross-attention module: based on graph features As a query Temporal characteristics As a key Sum for:

[0081] (11)

[0082] Each cross-attention module contains 8 attention heads with residual connections and layer normalization. It concatenates two attention-enhanced 256-dimensional feature vectors along their feature dimensions to obtain a 512-dimensional fused feature. This feature is then fused and dimensionality-reduced using a two-layer multilayer perceptron (MLP), resulting in:

[0083] (12)

[0084] The first layer maps 512 dimensions to 256 dimensions, and the second layer maps 256 dimensions to 128 dimensions. Finally, a linear classifier maps the 128-dimensional fused features to an 11-dimensional output space, corresponding to a modulation type of 1.

[0085] In step 3, the signal modulation recognition model is trained based on the training set and validation set, specifically as follows:

[0086] The model is trained using an end-to-end joint optimization approach. The loss function is multi-class cross-entropy loss, which is:

[0087] (13)

[0088] in, It refers to the batch size. It is the number of categories. It is a unique hot code for the real label. It is the probability distribution predicted by the model after being processed by Softmax;

[0089] The optimizer uses the Adam algorithm, with an initial learning rate set to... The training process employs the following strategies:

[0090] Learning rate scheduling: The ReduceLROnPlateau scheduler, based on validation set performance, is used. If the validation set accuracy does not improve for 5 consecutive epochs (patience value = 5), the learning rate is multiplied by a factor of 0.5 to decay.

[0091] Early stopping mechanism: If the accuracy of the validation set does not improve for 20 consecutive epochs (patience value = 20), training is terminated early, and the model parameters at the point of optimal performance on the validation set are restored.

[0092] Regularization: Dropout is used after the fully connected layers and attention mechanism of the model, with the dropout rate set to 0.1 to prevent overfitting.

[0093] In step 4, the trained signal modulation recognition model is tested based on the test set, specifically as follows:

[0094] After training, the best-performing model is used to infer new signal samples. The model receives preprocessed graph data and IQ time-series data, outputs a vector after forward propagation, and converts it into probabilities (confidence scores) for each category using the Softmax function. The category with the highest probability is taken as the final modulation scheme identification result.

[0095] To each Figures 1-3 Explain in both Chinese and English. Figure 1In this context, Graph Construction and ConvEncoder are used. Figure 2 In this context, CNN (Convolutional Neural Network) and cross-attention are mentioned. Figure 3 In this context, the technologies used include Attention, Linear, Batch Normalization (BN), and Fully Convolutional Network (FCN).

[0096] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A signal modulation recognition method based on the fusion of graph neural networks and time-frequency networks, characterized in that, include: Step 1: Obtain the signal sample dataset and preprocess it to generate the training set, validation set, and test set; Specifically: Step 101: Obtain the signal sample dataset, label each signal sample, and obtain the modulation type label corresponding to each signal sample; Step 102: Perform multimodal processing on the signal sample dataset to construct graph-structured data and original IQ signal time-series data respectively. Divide the graph-structured data, original IQ signal time-series data, and corresponding modulation type labels according to a preset ratio to obtain training set, validation set, and test set; specifically, perform multimodal processing on the signal sample dataset to construct graph-structured data as follows: Each signal sample in the signal sample dataset is converted into a graph structure representation. The node feature matrix contains five dimensions of statistical features, the edge index matrix is ​​constructed using the K-nearest neighbor algorithm to define the spatial connection relationship between nodes, and the batch index information is used to distinguish the node sets of different samples, thus obtaining the graph structure data. Step 2: Construct a signal modulation recognition model, which includes a graph neural network branch, a time-frequency network branch, and a feature fusion module; Step 3: Train the signal modulation recognition model based on the training set and validation set; Step 4: Test the trained signal modulation recognition model based on the test set.

2. The signal modulation recognition method based on the fusion of graph neural network and time-frequency network according to claim 1, characterized in that, In step 102, multimodal processing is performed on the signal sample dataset to construct the original IQ signal time series data, specifically as follows: Each signal sample in the signal sample dataset is represented in a three-dimensional tensor form. The first dimension represents the batch size, supporting batch processing; the second dimension represents the number of signal channels, which is fixed at 2 channels, corresponding to the in-phase component I and the quadrature component Q respectively; the third dimension represents the time series length, which is fixed at 128 time steps through resampling to ensure the uniformity of the input size.

3. The signal modulation recognition method based on the fusion of graph neural network and time-frequency network according to claim 2, characterized in that, The graph neural network branch consists of three consecutive graph convolutional layers.

4. The signal modulation recognition method based on the fusion of graph neural network and time-frequency network according to claim 3, characterized in that, The time-frequency network branch adopts a dual-branch composite architecture, extracting time-domain signal features and frequency-domain signal features respectively, as well as a cross-attention module for feature fusion. The time-domain signal feature extraction module is used to extract time-domain features after embedding and position encoding the IQ signal, using a multi-head self-attention mechanism. The frequency-domain signal feature extraction module is used to detect the period of the IQ signal through fast Fourier transform, and reshape the frequency-domain data according to the detected period, extracting features within the period through two-dimensional convolutional blocks. The cross-attention module is used to fuse the outputs of the time-domain signal feature extraction module and the frequency-domain signal feature extraction module.

5. The signal modulation recognition method based on the fusion of graph neural network and time-frequency network according to claim 4, characterized in that, The feature fusion module adopts a bidirectional cross-attention mechanism, including a first-direction cross-attention module and a second-direction cross-attention module. The first-direction cross-attention module is used to perform feature fusion with time-frequency domain fusion features as queries and graph features as keys and values. The second-direction cross-attention module is used to perform feature fusion with graph features as queries and time-frequency domain fusion features as keys and values.

6. The signal modulation recognition method based on the fusion of graph neural network and time-frequency network according to claim 5, characterized in that, Step 3: The training process employs the cross-entropy loss function and the Adam optimizer, along with an early stopping mechanism and an adaptive learning rate scheduling strategy based on validation set performance.

Images