A spectrum prediction method based on dynamic correlation matrix
By constructing a Transformer-based dynamic correlation matrix model and combining dynamic and static correlation multi-head attention mechanisms, the problem of insufficient prediction accuracy of existing spectrum prediction methods in complex radio environments is solved, and higher-precision spectrum prediction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING TECH UNIV
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-23
AI Technical Summary
Existing spectrum prediction methods struggle to effectively capture the multidimensional characteristics and long-term dependencies between channels when faced with complex and dynamic radio environments, resulting in insufficient prediction accuracy.
A dynamic correlation matrix model based on Transformer is adopted. By constructing encoder and decoder modules and combining dynamic and static correlation multi-head attention mechanisms, the temporal characteristics and channel correlation of the spectral signal are fused using the attention mechanism of the Transformer model to perform spectrum prediction.
It significantly improves prediction accuracy in real radio environments, better captures the complex dependencies and changing characteristics between channels, and improves the accuracy of spectrum prediction.
Smart Images
Figure CN122268514A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of wireless communication technology, and in particular to a spectrum prediction method based on a dynamic correlation matrix. Background Technology
[0002] Currently, the demand for radio resources is expanding rapidly. Early static spectrum allocation methods only allowed primary users (licensed users) to use the allocated spectrum for extended periods, resulting in significant waste of spectrum resources. Dynamic spectrum allocation, introduced in cognitive radio theory, effectively alleviated this problem. Dynamic spectrum allocation allows secondary users (aware users) to access and use the spectrum when the primary user is idle, without affecting the primary user's communication. Spectrum prediction technology utilizes hidden features in historical spectrum data, combined with relevant information from other channels within the same service, to obtain parameters such as future spectrum channel occupancy status, duty cycle, and power spectrum. This plays a crucial role in optimizing spectrum utilization, spectrum resource allocation, and supporting spectrum sharing.
[0003] Spectrum prediction can be divided into two approaches: data-driven and model-driven. Before the advent of deep learning, mathematical models were used for spectrum prediction. This paper was among the first to apply autoregressive models to spectrum prediction. Autoregressive models are a common method in statistics that models the dependencies of historical spectrum data on time series to predict future spectrum states. However, autoregressive models often assume a linear relationship, which is impossible to reconcile with actual spectrum and channel conditions. The channel environment is complex and dynamic, and this method has limited ability to capture long-term dependencies and non-stationary changes when dealing with long sequences. Furthermore, many people use Markov models for spectrum prediction. Markov models are probability-based statistical models that discretize the spectrum state and predict future changes through state transition probabilities. The core idea is that the current state of the spectrum depends only on the previous state. However, in real-world environments, the current spectrum state depends not only on the previous state but also on long-term channel relationships. Moreover, in continuous prediction tasks like spectrum prediction, the prediction accuracy of Markov models is severely limited by the granularity of state partitioning.
[0004] With the rapid development of neural networks, deep learning has received increasing attention in the field of spectrum prediction. Existing technologies rely on convolutional neural networks (CNNs) to analyze and predict the occupancy of surrounding spectra. CNNs are suitable for processing gridded data and can automatically extract local features for prediction through convolutional and pooling layers. However, in sequence prediction, due to the difficulty in effectively modeling long-term sequences and capturing temporal dependencies, they perform poorly in long-term sequence prediction tasks. Therefore, to address this issue, Long Short-Term Memory (LSTM) networks, which are better suited for handling time series, have been used in spectrum prediction tasks. LSTM, with its unique gate mechanism (input gate, forget gate, and output gate), is particularly effective at handling time series tasks and can learn long-term dependencies within sequences. The forget gate determines which parts of past information to discard, reducing the number of transmitted parameters; the input gate evaluates the current input to determine which new information needs to be stored and updated in the cell state; and the output gate adjusts the final hidden state output. Thus, LSTM is better suited for time series tasks and can learn long-term dependencies within sequences. However, when faced with multi-channel spectrum prediction tasks, LSTM often fails to capture the multidimensional characteristics of spectrum data, and its feature extraction between different channels is insufficient. Furthermore, its internal serial data processing results in a significant time requirement for model training. There is also the CNN-LSTM composite model, which uses the CNN model to extract local features of the spectrum, while the LSTM model extracts long-term temporal dependencies of the spectrum data, thus compensating for the shortcomings of CNN. However, due to the limitation of CNN convolutional kernel size, it cannot fully capture the non-stationary, multi-scale joint dynamic characteristics of the spectrum.
[0005] Currently, various deep learning models are emerging one after another, and the accuracy of spectrum prediction has been improved as a result. However, it can be found that most traditional spectrum prediction algorithms focus more on the temporal characteristics of the spectrum sequence, and the inherent physical correlation between channels is not explicitly modeled, and the feature mining between channels is not sufficient. Summary of the Invention
[0006] The present invention aims to at least partially solve one of the technical problems existing in the related art.
[0007] The purpose of this invention is to provide a dynamic correlation model based on Transformer, which utilizes the attention mechanism of the Transformer model to fuse the temporal characteristics of the spectrum signal with the channel correlation, thereby improving the prediction accuracy in real radio environments.
[0008] To achieve the above objectives, this invention provides a spectrum prediction method based on a dynamic correlation matrix, comprising the following steps:
[0009] S1. Obtain historical spectrum measurement data, preprocess it, and construct a spectrum prediction training set;
[0010] S2. Based on the Transformer architecture, a spectrum prediction network based on a dynamic correlation matrix is constructed. The spectrum prediction network includes an encoder module and a decoder module.
[0011] The encoder module includes four encoders, each of which includes a dynamic-static correlation combined multi-head attention module and a feedforward network module, with each module followed by a residual connection normalization layer; the dynamic-static correlation combined multi-head attention module receives the incoming sequence data, the static correlation matrix, and the dynamic correlation matrix generated by the dynamic correlation module at different times;
[0012] The decoder module includes four decoders, each of which includes a mask multi-head attention module, an encoder-decoder multi-head attention module, and a feedforward network module, with each module followed by a residual connection normalization layer;
[0013] S3. Train the spectrum prediction network based on the constructed spectrum prediction training set;
[0014] S4. Use the trained spectrum prediction network to perform spectrum prediction.
[0015] A further preferred embodiment of the present invention is that the preprocessing in step S1 specifically involves: performing a Fourier transform on the autocorrelation function of the spectrum measurement data according to the Wiener-Khinchin theorem to obtain the power spectral density of the spectrum measurement data, as specifically expressed by the formula:
[0016]
[0017]
[0018] in, It is an autocorrelation function. Indicates the expected value. It is a wide-stable random process as input. yes The complex conjugate, Due to time lag, It is the power spectral density. It's frequency. It is the Fourier transform operator. It is a complex exponential kernel.
[0019] Preferably, the implementation process of the dynamic-static correlation combined with the multi-head attention module is as follows:
[0020]
[0021]
[0022]
[0023] in, The composite correlation matrix is obtained by superimposing the static correlation matrix and the dynamic correlation matrix. R is the static correlation matrix, and D is the dynamic correlation matrix. It is a dynamic weighting coefficient. It is a normalization function. It is the strength coefficient. It is the projection of the final linear layer.
[0024]
[0025] in, , This indicates the dimension of the model input.
[0026] Preferably, the process of obtaining the dynamic correlation matrix is as follows:
[0027] (1) Feature extraction: Based on the original spectrum data, feature mapping is performed through pooling, flattening and MLP network to obtain the high-dimensional feature vector of the spectrum data;
[0028] (2) Generation of dynamic correlation matrix: The obtained high-dimensional feature vectors are used to obtain dynamic weight coefficients and dynamic correlation matrix through feature reconstruction, nonlinear transformation and Sigmoid function mapping and element-wise operation.
[0029] As a preferred method, the feature extraction process in obtaining the dynamic correlation matrix specifically involves:
[0030] The original spectral data is pooled using a downsampling operation, with the specific calculation formula as follows:
[0031]
[0032] Where 'c' is the channel number and 'b' is the batch number. It is the sequence length;
[0033] The pooled data is flattened to convert it into two-dimensional data.
[0034] Based on the transformed two-dimensional data, a three-layer MLP network is used to map the original channel features to a high-dimensional correlation representation space, generating a high-dimensional feature vector:
[0035]
[0036] in, It is an activation function. , , These are learnable weights. It is the input after being flattened. , and It is a bias term. It is the generated high-dimensional feature vector.
[0037] As a preferred embodiment, the generation of the dynamic correlation matrix in the process of obtaining the dynamic correlation matrix is specifically as follows:
[0038] By reshaping the dimensions, the high-dimensional feature vectors are restored to a standard matrix structure, and a preliminary channel correlation matrix is constructed.
[0039] The matrix elements are nonlinearly transformed by the hyperbolic tangent activation function, and the values of each element are constrained to the interval [-1, 1].
[0040] By introducing learnable and adjustable parameters, dynamic weight coefficients are obtained through mapping using the Sigmoid function:
[0041]
[0042] in, These are trainable scalar parameters. It is the Sigmoid function. These are dynamic weight parameters;
[0043] Based on the initial channel correlation matrix after element constraints, an identity matrix is created, and a corrected dynamic correlation matrix is generated using element-wise operations.
[0044]
[0045] in, This is the corrected dynamic correlation matrix. The dynamic correlation matrix that needs to be corrected. This represents element-wise multiplication. It is the identity matrix;
[0046] Let the MLP network be The final generated dynamic correlation matrix is:
[0047]
[0048] in, It is the raw data after pooling.
[0049] Preferably, the implementation process of the feedforward network is as follows:
[0050]
[0051] in, and It is a weight matrix. and It is a bias term. The original input for each sublayer, Activation function.
[0052] Preferably, the implementation process of the residual connection normalization layer is as follows:
[0053]
[0054]
[0055] in, For residual output, Indicates sub-level functions, It is the input feature vector. The mean value over the feature dimension. The variance is the variance along the feature dimension. It is a numerically stable term that prevents division by zero. It is a learnable scaling parameter. It is a learnable bias.
[0056] Preferably, the implementation process of the encoder-decoder attention module is as follows:
[0057]
[0058]
[0059] in, It is a query vector. It is a key vector. It is a value vector. For vector dimensions, This is a trainable weight matrix.
[0060] Preferably, the implementation process of the mask multi-head attention module is as follows:
[0061]
[0062] in, It is the attention score matrix after masking. It is a mask matrix.
[0063] Beneficial effects: This invention constructs a Transformer model to acquire historical spectral data, obtains long-term dependencies of sequences through an attention mechanism, and uses a static correlation matrix as prior information to obtain channel correlation. The correlation matrix is updated in real time through an MLP network to capture constantly changing channel characteristics. This, combined with the attention mechanism of the Transformer model, significantly improves the model's prediction accuracy. Attached Figure Description
[0064] Figure 1 This is a cognitive radio network model.
[0065] Figure 2 Modeling process for spectrum occupancy status.
[0066] Figure 3 This is a schematic diagram of the spectrum prediction network structure based on the dynamic-static correlation of Transformer proposed in this invention.
[0067] Figure 4 This is a schematic diagram of the dynamic correlation module structure proposed in this invention.
[0068] Figure 5 This is a schematic diagram of the static correlation matrix proposed in this invention.
[0069] Figure 6 This diagram illustrates a comparison of the prediction accuracy of the present invention's method and existing methods for the same spectrum under different step sizes.
[0070] Figure 7 This is a schematic diagram comparing the F1 score prediction of the same spectrum by the method of the present invention and existing methods. Detailed Implementation
[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0072] The following is combined Figures 1-7 This paper describes a spectrum prediction method based on a dynamic correlation matrix.
[0073] In the field of cognitive radio, licensed users refer to communication entities that possess legal licenses to use specific frequency bands. They can communicate through base stations and use the allocated frequency bands. Unlicensed users refer to devices that do not have fixed spectrum allocations but possess cognitive capabilities. They are only allowed to access and communicate when licensed users are idle, and once a licensed user reappears, the unlicensed user must immediately log out to avoid interfering with the primary user's communication.
[0074] Figure 1This represents a cognitive radio network model. Spectrum prediction forecasts future states using historical spectrum data. First, the spectrum data is modeled; this paper uses power spectral density (PSD) as the prediction data. Power spectral density describes the statistical distribution of random signal power in the frequency domain, representing the power intensity at different frequencies. According to the Wiener-Khinchin theorem, the autocorrelation function of a signal and its power spectral density are Fourier transform pairs, as shown in the following formula:
[0075]
[0076]
[0077] in It is an autocorrelation function. Indicates the expected value. It is a wide-stable random process as input. yes The complex conjugate, The time lag is the independent variable of the autocorrelation function. It is the power spectral density, which is obtained by taking the Fourier transform of the autocorrelation function.
[0078] By training on historical PSD data, predictions of future states are achieved. Multiple channels are divided based on different sensors, and data values are measured in time slots, transforming spectrum prediction into a multivariate time series prediction problem. The measured PSD values are compared with preset threshold values to obtain channel occupancy status data. The specific comparison process is as follows:
[0079]
[0080] in, Indicates the nth channel at The occupancy status at any given moment. Indicates the nth channel at time... PSD value, This represents the threshold value, which is the average noise threshold plus a certain tolerance value. Figure 2 Modeling process for spectrum occupancy status.
[0081] Example 1: As Figure 3 As shown, this embodiment provides a spectrum prediction method based on a dynamic correlation matrix, including the following steps:
[0082] S1. Obtain historical spectrum measurement data, preprocess it, and construct a spectrum prediction training set;
[0083] The preprocessing specifically involves performing a Fourier transform on the autocorrelation function of the spectrum measurement data according to the Wiener-Khinchin theorem to obtain the power spectral density of the spectrum measurement data.
[0084] S2. Based on the Transformer architecture, a spectrum prediction network based on a dynamic correlation matrix is constructed. The spectrum prediction network includes an encoder module and a decoder module.
[0085] The encoder module includes four encoders, each of which includes a dynamic-static correlation combined multi-head attention module and a feedforward network module, with each module followed by a residual connection normalization layer; the dynamic-static correlation combined multi-head attention module receives the incoming sequence data, the static correlation matrix, and the dynamic correlation matrix generated by the dynamic correlation module at different times;
[0086] The decoder module consists of four decoders, each of which includes a mask multi-head attention module, an encoder-decoder multi-head attention module, and a feedforward network module, with each module followed by a residual connection normalization layer.
[0087] S3. Train the spectrum prediction network based on the constructed spectrum prediction training set;
[0088] S4. Use the trained spectrum prediction network to perform spectrum prediction.
[0089] The spectrum prediction method based on the dynamic correlation matrix proposed in this embodiment has the following overall algorithm structure: Figure 3 As shown, the prediction algorithm is based on the Transformer architecture and includes encoder and decoder parts. For the encoder module, each encoder consists of a multi-head attention module, a residual connection normalization layer, and a feedforward network. The dynamic-static correlation multi-head attention module receives the input sequence data, the prior correlation matrix, and the dynamic correlation matrix generated by the dynamic correlation module at different times. The decoder part employs a masked multi-head attention mechanism. In the spectrum prediction task, this mechanism prevents the model from accessing spectral information at future time steps, ensuring that the prediction is based solely on the historical sequence to maintain the temporal order of the autoregressive generation. The model input is (batch_size, seq_len, num_channels). The encoder input is obtained from linear projection and positional encoding, while the decoder uses a learnable start vector and positional encoding as input. Under the constraint of a causal mask, it generates spectrum predictions for multiple future time steps in parallel.
[0090] Encoder Decoder:
[0091] Encoder: The input is first mapped to a continuous vector space through a linear projection layer. Compared to the standard Transformer model, this linear projection can be viewed as an embedding layer. Because the spectral data itself is continuous, word embedding is not required beforehand; instead, the dimensionality can be adjusted directly through a linear transformation. The positional information injection formula is as follows:
[0092]
[0093]
[0094] in, Indicates location, Representing dimension, For feature dimensions.
[0095] The processed data enters the encoder module. The encoder module consists of two sub-modules: a dynamic-static combined attention mechanism and a feedforward network. Each sub-module is surrounded by residual connections and layer normalization. The feedforward network, composed of fully connected network layers, serves as the feature enhancement module in the Transformer model. First, the attention mechanism captures the relationships between different positions. Then, the feedforward network performs nonlinear transformations and depth processing on the information at each position, significantly improving the model's representational ability. Using fully connected network layers allows for independent processing of features at each time step of the sequence with parameter sharing, resulting in a simple and efficient structure, as shown in the following formula:
[0096]
[0097] in and It is a weight matrix. and It is a bias term. The original input for each sublayer, Activation functions are used to introduce nonlinearity.
[0098] In feedforward networks, both the input and output pass through residual connections and normalization layers. Network degradation is a common phenomenon in deep learning; that is, while the model converges normally after its initial construction, its performance degrades as the network depth increases. The residual connection module in the encoder allows the original input to be directly passed to the next layer. The network output represents the superposition of a non-linear transformation of the input and a linear relationship with the input, thus avoiding information loss and solving the gradient vanishing problem in deep models. The formula is:
[0099]
[0100] in, For residual output, This represents the sub-layer function, i.e., the output of a self-attention network or feedforward network. It is the input feature vector.
[0101] A normalization layer normalizes all feature dimensions of each sample, mapping the input data or the output data of a neural network layer to a standard range. Layer normalization is frequently used when normalizing sequence data to ensure the output distribution has a stable mean and variance. Layer normalization normalizes the output after residual connections and then uses learnable parameters to scale and shift the normalized output. This maintains both the stability of the data distribution and a degree of flexibility. The mathematical formula for layer normalization is:
[0102]
[0103] in It is the input feature vector. The mean value over the feature dimension. The variance is the variance along the feature dimension. It is a numerically stable term to prevent division by zero, and is usually taken as an extremely small number. It is a learnable scaling parameter. It is a learnable bias.
[0104] The encoder module transforms the input sequence into a feature representation, acting as a temporal feature extractor. Through an attention mechanism combined with other modules and multiple overlapping layers, it learns the global dependencies of historical spectral sequences, capturing complex patterns in both the frequency and time domains and converting them into deep feature representations. These features effectively model the long-term patterns of spectral changes, providing strong contextual support for subsequent prediction modules and thus improving the accuracy of spectral prediction.
[0105] Decoder: The decoder is primarily responsible for generating the output sequence. In this paper, it generates a spectral prediction sequence for future time steps based on the encoder's analysis of historical spectra. It mainly includes a masked attention layer, an encoder-decoder attention layer, a feedforward network, and residual connections and normalization layers.
[0106] The attention mechanism, as the core of the Transformer model, is implemented in the encoder-decoder attention layer as cross-attention. This module is crucial for connecting the encoder and decoder in the decoder. The cross-attention mechanism is as follows:
[0107]
[0108] in It is a query vector, used to ask a question, indicating the specific content that needs to be focused on in the current input vector; It is the key vector, whose function is to match it with the query vector and determine which input vectors need to be considered; It is a value vector used to store information and generate new representations; Let X be the vector dimension. In actual calculations, given the input vector X, we can use the following formula:
[0109]
[0110] This is a trainable weight matrix. In standard attention mechanisms, , , They all come from the same sequence and are only used to capture the internal relationships within the sequence itself. In the encoder-decoder attention layer, cross-attention... Output from the decoder mask attention layer, and The output from the last layer of the encoder. This allows the decoder to selectively obtain and integrate the raw sequence information provided by the encoder at each position when generating the target sequence.
[0111] Masked attention, a module unique to the decoder, primarily functions to restrict the model's access to past information while masking future information, thus simulating an autoregressive process and preventing information leakage. Specifically, in spectrum prediction tasks, the model can only use current and past spectrum information to predict the next spectrum state, without anticipating future information. In implementation, this is achieved by analyzing the current spectrum information... and future spectrum information Masking is performed so that the model can only compute attention scores based on the generated sequences. In the formula, a mask matrix is used to prevent the model from accessing future positions when computing attention scores. The dimension of the mask matrix is the same as the input matrix of the spectral data, and the values for future positions in the matrix are set to negative infinity. The attention score matrix after masking is:
[0112]
[0113] in It is a mask matrix in which past and present information is set to 1, and future information is set to 0, thereby masking future information.
[0114] Like the encoder, the decoder's feedforward network layer is connected to residual connections and layer normalization, achieving the same functionality. As the generative module in spectrum prediction, the decoder is responsible for progressively transforming the deep features extracted by the encoder into future spectrum sequences. It employs an autoregressive approach, combining the global context provided by the encoder and the local state of the generated sequence at each time step, and fusing the temporal and frequency-dimensional dependencies through an attention mechanism, ultimately outputting the spectrum prediction results generated by the model.
[0115] Dynamic correlation module:
[0116] This section will introduce the dynamic channel correlation module designed in this paper. For multi-channel spectrum prediction, the correlation between each channel can be measured using the Pearson correlation coefficient. The Pearson correlation coefficient is a statistical indicator used to measure the strength and direction of the linear relationship between two continuous variables. Its value ranges from -1 to 1, where -1 represents a strong negative correlation and 1 represents a strong positive correlation. For two variables... and Its formula is:
[0117]
[0118] in It is a variable The One observation value; It is a variable The One observation value, and These are the averages of the two variables. In actual prediction tasks, let the number of channels be... Then the correlation matrix It is The matrix:
[0119]
[0120] in It is the first The first channel and the first Pearson correlation coefficient between channels, diagonal A value of 1 always indicates the degree of correlation with itself. The static correlation matrix is as follows: Figure 5 As shown.
[0121] Strong correlation between channels indicates that they are coupled or share the same driving source in a specific frequency band; weak correlation indicates that the two channels are relatively independent. In practical spectrum prediction tasks, the correlation between channels is not constant; it changes rapidly over time. A single initial correlation matrix cannot effectively capture this time-varying characteristic. To address this issue, this paper designs a dynamic correlation matrix generator.
[0122] like Figure 4 As shown, this module extracts features from the original input sequence. First, the input enters a pooling layer. The pooling layer reduces the dimensionality of the feature map through downsampling in the neural network, thereby improving computational efficiency, reducing the number of parameters, and preventing overfitting. For each channel... and each batch :
[0123]
[0124] in This refers to the sequence length. After pooling, the data is flattened, transforming it into a two-dimensional output to match the input of subsequent linear networks. The processed data is then fed into a multilayer perceptron (MLP) network for training. A three-layer MLP network maps the original channel features to a high-dimensional correlation representation space.
[0125]
[0126] in , , For learnable weights, The input is flattened. GELU was chosen as the activation function instead of the traditional ReLU because GELU provides a smooth non-linear transformation, making it more suitable for modeling probabilistic correlations. Then, the dynamic correlation matrix is generated, restoring the high-dimensional feature vectors from the previous stage to a standard matrix structure through dimension reshaping. The magnitude of the correlation coefficient is determined to construct a preliminary representation of the inter-channel correlation. Then, the Tanh (hyperbolic tangent activation function) is applied to perform a non-linear transformation on the matrix elements, constraining the values of each element to the interval [-1, 1]. This ensures that the output conforms to the mathematical definition of the correlation coefficient, and its smooth gradient characteristics enhance the stability of model training. Subsequently, the module introduces a learnable dynamic intensity adjustment parameter; we define a trainable scalar parameter... (Initialized to 0.1), dynamic weight coefficients are obtained through Sigmoid function mapping. :
[0127]
[0128] This coefficient is then used to adjust the final output of the dynamic correlation matrix, enabling the model to dynamically adjust the strength of the generator matrix on the final attention score based on the characteristics of the current input data, thereby achieving an adaptive balance mechanism between static prior knowledge and dynamic observation information.
[0129] Finally, the matrix is corrected by creating an identity matrix with dimensions matching the generated matrix as a baseline template. Element-wise operations are then used to force the diagonal elements of the matrix to be 1 while leaving the off-diagonal elements unchanged.
[0130]
[0131] This is the corrected dynamic correlation matrix. The dynamic correlation matrix that needs to be corrected. This represents element-wise multiplication. Let be the identity matrix. Let the MLP network be... The final dynamic correlation matrix generated by this module is as follows:
[0132]
[0133] in, It is the raw data after pooling.
[0134] Channel correlation is primarily used to measure the correlation index between multiple channels. In real-world communication environments, each channel exhibits a certain correlation due to interference from other channels caused by factors such as location, noise, and magnetic fields. Mining these correlations is crucial for achieving efficient and accurate spectrum prediction. Traditional spectrum prediction models fail to capture the rapidly changing dependencies between channels, leading to a sharp decline in prediction performance when the wireless environment undergoes sudden changes. The dynamic channel correlation module proposed in this paper can infer and output a correlation matrix that matches the current environmental state in real time, thus better capturing rapidly changing channel correlations.
[0135] A multi-head attention mechanism combining movement and stillness:
[0136] In spectrum prediction tasks, there are complex interdependencies between different channels. The traditional Transformer attention mechanism obtains the interaction relationship between elements of the sequence through dot product operations. However, this approach has two main limitations in spectrum prediction scenarios: first, the inherent physical correlation between channels is not explicitly modeled; second, the time-varying correlation between channels cannot be effectively captured.
[0137] This paper proposes a dynamic-static combined multi-head attention mechanism. Based on standard scaled dot product attention, it introduces a relevance-guided term to form a composite attention score. The comprehensive relevance matrix generated by the dynamic-static relevance module is as follows:
[0138]
[0139] By superimposing static and dynamic correlations, a comprehensive correlation matrix is obtained, and the final attention is:
[0140]
[0141] The function compresses vector values to the range [0, 1], and is a normalization function. This is the intensity coefficient, set to 0.5. The formula for the multi-head attention mechanism is:
[0142]
[0143] in, , Here , Indicates the dimensionality of the model input. This is the projection of the final linear layer. This multi-head attention is computed in parallel, which reduces time consumption.
[0144] As the core of the Transformer, the attention mechanism guides the model to adaptively focus on the most relevant temporal and channel features in the input spectrum for the current prediction target in spectral prediction tasks, capturing complex long-term dependencies and channel correlations in the spectral signal. The attention mechanism gives more attention to channels with strong correlations, and vice versa. Static correlations reflect the inherent physical connections between channels, while dynamic correlations capture the changing patterns of data-driven temporal and channel associations. Combining the two allows for the effective integration of domain prior knowledge while maintaining the Transformer's generalization capabilities.
[0145] To demonstrate the feasibility and advantages of the proposed method, experiments were conducted on the generated simulation dataset.
[0146] The dataset used in this experiment comes from the publicly available spectrum sharing website ElectroSense. The data originates from spectrum measurements in the 600–700 MHz band collected by sensors in Madrid, Spain. The dataset spans from June 1st to June 8th, 2021, with a time resolution of 1 minute, generating 10081 time steps. In this experiment, 70% of the spectrum data was used as the training set, 15% as the validation set, and the remaining 15% as the test set. The learning rate was 0.001, the number of Transformer layers was 4, and the time step was 16.
[0147] The evaluation metrics used are mean absolute error (MAE), mean squared error (MSE), and root mean squared error (RMSE), and their expressions are as follows:
[0148]
[0149]
[0150]
[0151] in, Represents the total predicted points. Represents the predicted value. This is the true value. Furthermore, we will compare it with some common spectrum prediction models, including CNN, LSTM, CNN-LSTM, and the standard Transformer; we will name our model CTRAS.
[0152] Table 1. Comparison of results from the CTRAS model with other baseline methods
[0153]
[0154] As shown in Table 1, our proposed method achieved the best predictive performance on all evaluation metrics in this study. Based on the experimental results, we can conclude that incorporating a relevance-guided attention mechanism can significantly enhance the model's predictive performance. Furthermore, dividing the input data into busy and idle states according to time and then making predictions using this method can yield results such as... Figure 6 The accuracy of the spectrum predictions shown.
[0155] from Figure 6 It can be observed that the prediction accuracy gradually decreases as the prediction step size increases. Even when predicting up to 50 steps into the future, CTRAS still maintains an accuracy of 85.27%, demonstrating the model's potential in multi-channel, multi-step spectrum prediction tasks. Furthermore, the F1 function can also serve as a significant indicator of performance in predicting idle or busy states.
[0156]
[0157] The F1 score is the harmonic mean of precision and recall. Precision is the proportion of samples where both the predicted and actual values are 1 out of all samples with a predicted value of 1. Recall is the proportion of samples where both the predicted and actual values are 1 out of all samples with a actual value of 1. In practical applications, especially when class distributions are imbalanced, relying solely on precision or recall can lead to a misunderstanding of model performance. The F1 score, by balancing these two metrics, makes model evaluation more fair and comprehensive. Figure 7 This is a schematic diagram comparing the F1 score prediction of the same spectrum by the method of the present invention and the existing method in the above experiment.
[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A spectrum prediction method based on a dynamic correlation matrix, characterized in that, Includes the following steps: S1. Obtain historical spectrum measurement data, preprocess it, and construct a spectrum prediction training set; S2. Based on the Transformer architecture, a spectrum prediction network based on a dynamic correlation matrix is constructed. The spectrum prediction network includes an encoder module and a decoder module. The encoder module includes four encoders, each of which includes a dynamic-static correlation combined multi-head attention module and a feedforward network module, with each module followed by a residual connection normalization layer; the dynamic-static correlation combined multi-head attention module receives the incoming sequence data, the static correlation matrix, and the dynamic correlation matrix generated by the dynamic correlation module at different times; The decoder module includes four decoders, each of which includes a mask multi-head attention module, an encoder-decoder multi-head attention module, and a feedforward network module, with each module followed by a residual connection normalization layer; S3. Train the spectrum prediction network based on the constructed spectrum prediction training set; S4. Use the trained spectrum prediction network to perform spectrum prediction.
2. The spectrum prediction method based on dynamic correlation matrix according to claim 1, characterized in that, The preprocessing in step S1 specifically involves: performing a Fourier transform on the autocorrelation function of the spectrum measurement data according to the Wiener-Khinchin theorem to obtain the power spectral density of the spectrum measurement data, as expressed by the following formula: ； ； in, It is an autocorrelation function. Indicates the expected value. It is a wide-range stationary random process as input. yes The complex conjugate, Due to time lag, It is the power spectral density. It's frequency. It is the Fourier transform operator. It is a complex exponential kernel.
3. The spectrum prediction method based on dynamic correlation matrix according to claim 2, characterized in that, The implementation process of the dynamic-static correlation combined with the multi-head attention module is as follows: ； ； ； in, The composite correlation matrix is obtained by superimposing the static correlation matrix and the dynamic correlation matrix. R is the static correlation matrix, and D is the dynamic correlation matrix. It is a dynamic weighting coefficient. It is a normalized Hafe number. It is the strength coefficient. It is the projection of the final linear layer. ； in, , This indicates the dimension of the model input.
4. The spectrum prediction method based on dynamic correlation matrix according to claim 3, characterized in that, The process of obtaining the dynamic correlation matrix is as follows: (1) Feature extraction: Based on the original spectrum data, feature mapping is performed through pooling, flattening and MLP network to obtain the high-dimensional feature vector of the spectrum data; (2) Generation of dynamic correlation matrix: The obtained high-dimensional feature vectors are used to obtain dynamic weight coefficients and dynamic correlation matrix through feature reconstruction, nonlinear transformation and Sigmoid function mapping and element-wise operation.
5. The spectrum prediction method based on dynamic correlation matrix according to claim 4, characterized in that, The feature extraction process in obtaining the dynamic correlation matrix is as follows: The original spectral data is pooled using a downsampling operation, with the specific calculation formula as follows: ； Where 'c' is the channel number and 'b' is the batch number. It is the sequence length; The pooled data is flattened to convert it into two-dimensional data. Based on the transformed two-dimensional data, a three-layer MLP network is used to map the original channel features to a high-dimensional correlation representation space, generating high-dimensional feature vectors: ； in, It is an activation function. , , These are learnable weights. It is the input after being flattened. , and It is a bias term. It is the generated high-dimensional feature vector.
6. The spectrum prediction method based on dynamic correlation matrix according to claim 5, characterized in that, The generation of the dynamic correlation matrix during the acquisition process is as follows: By reshaping the dimensions, the high-dimensional feature vectors are restored to a standard matrix structure, and a preliminary channel correlation matrix is constructed. The matrix elements are nonlinearly transformed by the hyperbolic tangent activation function, and the values of each element are constrained to the interval [-1, 1]. By introducing learnable and adjustable parameters, dynamic weight coefficients are obtained through mapping using the Sigmoid function: ； in, These are trainable scalar parameters. It is the Sigmoid function. These are dynamic weight parameters; Based on the initial channel correlation matrix after element constraints, an identity matrix is created, and a corrected dynamic correlation matrix is generated using element-wise operations. ； in, This is the corrected dynamic correlation matrix. The dynamic correlation matrix that needs to be corrected. This represents element-wise multiplication. It is the identity matrix; Let the MLP network be The final generated dynamic correlation matrix is: ； in, It is the raw data after pooling.
7. The spectrum prediction method based on a dynamic correlation matrix according to claim 6, characterized in that, The implementation process of the feedforward network is as follows: ； in, and It is a weight matrix. and It is a bias term. The original input for each sublayer, Activation function.
8. The spectrum prediction method based on dynamic correlation matrix according to claim 7, characterized in that, The implementation process of the residual connection normalization layer is as follows: ； ； in, For residual output, Indicates sub-level functions, It is the input feature vector. The mean value over the feature dimension. The variance is the variance along the feature dimension. It is a numerically stable term that prevents division by zero. It is a learnable scaling parameter. It is a learnable bias.
9. The spectrum prediction method based on dynamic correlation matrix according to claim 8, characterized in that, The implementation process of the encoder-decoder attention module is as follows: ； ； in, It is a query vector. It is a key vector. It is a value vector. For vector dimensions, This is a trainable weight matrix.
10. The spectrum prediction method based on a dynamic correlation matrix according to claim 9, characterized in that, The implementation process of the mask multi-head attention module is as follows: ； in, It is the attention score matrix after masking. It is a mask matrix.