Spectrum prediction method based on time-frequency fusion attention network
By constructing a time-frequency fusion attention network and combining time and frequency features, the problem of insufficient feature utilization in existing spectrum prediction methods is solved, and spectrum prediction with higher accuracy and robustness is achieved.
Patent Information
- Application Number
- CN202411155742.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2025-10-10
- Estimated Expiration
- 2044-08-22
AI Technical Summary
Existing spectrum prediction methods fail to fully utilize the characteristic information of the frequency dimension when dealing with complex and dynamic wireless communication environments, and ignore the influence of time periodicity, resulting in insufficient prediction accuracy and weak model robustness.
A method based on time-frequency fusion attention network is adopted. The time domain features are extracted by time convolution network, the frequency domain features are extracted by graph convolution network, and the attention mechanism is used for dynamic weighting to construct a spectrum prediction model that comprehensively characterizes the time-frequency correlation.
It improves the accuracy of spectrum prediction and the robustness of the model, can better capture the multi-dimensional characteristics of spectrum data, and improves the efficiency of spectrum resource management and utilization.
Smart Images
Figure CN119155802B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electromagnetic spectrum technology, and in particular to a spectrum prediction method based on a time-frequency fusion attention network. Background Art
[0002] With the development of wireless networks and communication technologies, the number of frequency-using devices and users continues to increase, and the demand for spectrum resources in wireless communications continues to grow rapidly. Traditional static spectrum allocation strategies are unable to cope with the dynamic spectrum demand in this complex environment, resulting in low utilization and further exacerbating spectrum resource shortages. Spectrum prediction technology leverages the hidden characteristics of historical spectrum data and combines it with information from other channels in the same service to determine parameters such as future spectrum channel occupancy, duty cycle, and power spectrum. It plays a vital role in optimizing spectrum utilization, allocating spectrum resources, and supporting spectrum sharing. Analyzing historical spectrum data changes, summarizing spectrum usage patterns, and predicting future user spectrum usage can better implement efficient spectrum resource management and intelligent spectrum switching strategies.
[0003] Traditional model-driven approaches include autoregressive analysis, hidden Markov models, and Bayesian models. However, in real-world spectrum environments, state transition probabilities may vary over time, limiting the applicability of these models. Spectrum utilization patterns exhibit stochastic properties in the spatial, frequency, and temporal dimensions, depending on factors such as the wireless propagation environment, transmit power, and user mobility. Traditional spectrum prediction methods face challenges such as uneven spectrum resource allocation and dynamic changes. Research is needed on spectrum prediction methods that adapt to complex and dynamic spectrum environments.
[0004] Because deep learning can capture the complex dependencies inherent in nonlinear spectrum data, neural networks, such as convolutional neural networks, recurrent neural networks, and long short-term memory networks, are increasingly being applied to spectrum prediction tasks. In actual spectrum utilization, the periodic nature of user behavior causes spectrum data to exhibit distinct characteristics over different time periods, resulting in distinct periodic patterns and significant periodicity in spectrum occupancy. Existing deep learning methods for spectrum prediction face two specific challenges: some methods focus solely on extracting features in the temporal dimension and fail to fully utilize the important features contained in the frequency dimension. While they can capture temporal variations, they lack consideration of frequency-domain features and are unable to fully reflect the dynamic complexity of spectrum data. On the other hand, some methods, while attempting to combine the time and frequency dimensions for prediction, ignore the impact of temporal periodicity, limiting the model's ability to accurately capture data variations. Summary of the Invention
[0005] In response to the above problems, the purpose of the present invention is to provide a spectrum prediction method based on a time-frequency fusion attention network, to construct a spectrum prediction model that comprehensively characterizes the correlation between the time domain and the frequency domain, and to optimize the problem of insufficient prediction accuracy of the spectrum prediction model based on deep learning.
[0006] The technical solution adopted by the present invention is: a spectrum prediction method based on time-frequency fusion attention network, comprising:
[0007] Step S01: constructing a spectrum data prediction network model, setting initial learning parameters, inputting the collected spectrum data into the spectrum data prediction network model, performing periodic analysis on the spectrum data, and determining the time period attribute of the spectrum data;
[0008] Step S02: using a temporal convolutional network to extract the time domain features of the spectrum data, and using a gated linear unit to pass the spectrum data to a graph convolutional layer network based on the time domain features;
[0009] Step S03: extracting frequency domain features of the spectrum data using a graph convolutional network;
[0010] Step S04: dynamically weighting the extracted time domain features and the extracted frequency domain features using an attention mechanism to obtain predicted spectrum data;
[0011] Step S05: Calculate the mean absolute error based on the predicted spectrum data and the collected spectrum data;
[0012] Step S06: reversely training the spectrum data prediction network model according to the mean absolute error, optimizing the learning parameters, and completing the training of the spectrum data prediction network model;
[0013] Step S07: inputting the spectrum data to be processed into the trained spectrum data prediction network model to obtain a prediction result.
[0014] Preferably, the method for determining the time period attribute in step S01 includes: determining the minimum historical data length corresponding to the same evolution trend according to the evolution trend of the spectrum data in different time slots, and determining the time period attribute of the spectrum data according to the time period corresponding to the historical data length.
[0015] Preferably, the temporal convolutional network in step S02 includes three dilated causal convolutions, and the three dilated causal convolutions are respectively used to receive spectrum data of different time period attributes.
[0016] Preferably, the time period attributes include proximity trend, daily trend and weekly trend.
[0017] Preferably, the gated linear unit in step S02 includes a gating mechanism, and the gating mechanism is used to perform information filtering and selective transmission on the spectrum data.
[0018] Preferably, the method for extracting the frequency domain features in step S03 includes: constructing a correlation coefficient matrix based on the frequency band correlation of the spectrum data, and utilizing the ability of the graph convolutional network to aggregate node features to extract the frequency domain features.
[0019] Preferably, the dynamic weighting process in step S04 includes: summing the time domain features and frequency domain features corresponding to the spectrum data of different time period attributes, and using an attention layer to assign different weights to the time domain features and frequency domain features corresponding to different spectrum data.
[0020] Preferably, the predicted spectrum data in step S04 is output through an output layer, and the output layer includes two convolutional layers for deepening the depth of the spectrum data prediction network model, and adjusting and controlling the output dimension.
[0021] Preferably, the learning parameters in step S01 include the number of iterations, the learning rate and the batch size.
[0022] Preferably, when the number of iterations set in the learning parameters is completed, the training of the spectrum data prediction network model is completed.
[0023] Beneficial effects of the above technical solution:
[0024] This invention discloses a spectrum prediction method based on a time-frequency fusion attention network, which solves the problems of insufficient consideration of data features, weak model robustness, and insufficient prediction accuracy in existing frequency domain prediction algorithms. First, based on a convolutional network, an expansion factor is introduced to increase the modeling scope, allowing the network layer to consider longer-term temporal dependencies and fully extract time domain features. Then, the ability of graph convolutional networks to aggregate node features is utilized to fully extract frequency domain features. Finally, an attention mechanism is used to dynamically weight time domain features and frequency domain features, thereby improving the prediction accuracy of the model while fully considering the multidimensional characteristics of the spectrum data. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 This is a flow chart of a spectrum prediction method based on a time-frequency fusion attention network provided by an embodiment of the present invention;
[0026] Figure 2 is a proximity trend graph of spectrum data provided by an embodiment of the present invention;
[0027] Figure 3 This is a trend chart of spectrum data transformation based on a daily cycle, provided by an embodiment of the present invention;
[0028] Figure 4is a spectrum data transformation trend graph provided by the embodiment of the present application, which is periodical with a week as a cycle;
[0029] Figure 5 is a spectrum prediction network framework graph provided by the embodiment of the present application;
[0030] Figure 6 is a comparison graph of the spectrum prediction method provided by the embodiment of the present application and other existing spectrum prediction methods;
[0031] Figure 7 is a training convergence speed comparison graph of the spectrum prediction method provided by the embodiment of the present application and other existing spectrum prediction methods;
[0032] Figure 8 is a training speed comparison graph of the spectrum prediction method provided by the embodiment of the present application and other existing spectrum prediction methods. DETAILED DESCRIPTION
[0033] The embodiments of the present application are described in further detail below. Obviously, the described embodiments are only part of the embodiments of the present application, and are not exhaustive of all embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0034] The terms "first", "second", and the like (if any) in the specification and claims are used to distinguish similar objects, and do not necessarily describe a particular order or chronological sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units, is not necessarily limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0035] It should be understood that the term "and / or" used in the present application only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.
[0036] Figure 1 is a spectrum prediction method flow chart based on a time-frequency fusion attention network implemented by the present application. In combination with Figure 1 , the method of the present application comprises:
[0037] Step S01: constructing a spectrum data prediction network model, setting initial learning parameters, inputting the collected spectrum data into the spectrum prediction network model, performing periodic analysis on the spectrum data, and determining the time period attribute of the spectrum data;
[0038] Step S02: using a temporal convolutional network to extract the time domain features of the spectrum data, and using a gated linear unit to pass the spectrum data to a graph convolutional layer network based on the time domain features;
[0039] Step S03: extracting frequency domain features of the spectrum data using a graph convolutional network;
[0040] Step S04: dynamically weighting the extracted time domain features and the extracted frequency domain features using an attention mechanism to obtain predicted spectrum data;
[0041] Step S05: Calculate the mean absolute error based on the predicted spectrum data and the collected spectrum data;
[0042] Step S06: reversely training the spectrum data prediction network model according to the mean absolute error, optimizing the learning parameters, and completing the training of the spectrum data prediction network model;
[0043] Step S07: inputting the spectrum data to be processed into the trained spectrum data prediction network model to obtain a prediction result.
[0044] The above spectrum prediction method is further described below with reference to the accompanying drawings.
[0045] Step 1: Construct a spectrum data prediction network model, collect spectrum data, input the spectrum data into the spectrum data prediction network model, perform periodic analysis on the spectrum data, and determine the time period attribute of the spectrum data.
[0046] The radio spectrum state of a frequency band exhibits both regularity and randomness in its temporal evolution. Spectrum states can be predicted using historical data because they are correlated in both the time and frequency domains. Various time-period attributes can be observed from collected, measured spectrum data, including proximity trends, daily trends, and weekly trends. The following formula is used to determine the input vector for different time-period attributes:
[0047]
[0048] Among them, I W , I D , I C are the characteristic vectors of weekly trend, daily trend and proximity trend respectively, W t 、D t 、C tare the lengths of historical data corresponding to the trend, daily trend, and proximity trend, t is the current moment, I t is the spectrum data at time t, p is the spectrum sensing frequency, X W ∈{1,...,XW} is an integer representing the week. X d ∈{1,...,XD} represents the number of days.
[0049] Perform time domain analysis on the measured data, such as Figure 2 This is a trend chart of the proximity of spectrum data, that is, the SNR ratio curve. The x-axis is the spectrum sensing time slot interval, and the y-axis is the average ratio between spectrum data sensed at different time intervals. It can be seen that the data ratio increases with the increase of the time interval, and the similarity between the data decreases accordingly. Figure 3 This is a daily trend chart of spectrum data changes, showing the perception data of all time slots in a week. The spectrum data shows obvious periodic characteristics with a daily cycle. Figure 4 The figure shows the transformation trend of spectrum data with a weekly period. The transformation trend of spectrum data with a weekly period is given. The data has the same trend in the same time period.
[0050] Step 2: Use the temporal convolutional network to obtain the time domain features of the spectrum data.
[0051] The time domain input vector obtained in step 1 is input into the temporal convolutional network. The dilated causal convolution is used to combine causality and dilatation, so that the spectrum data prediction network model can effectively extract long-term dependent features without violating the time order. Unlike traditional spectrum prediction methods that only consider the closest historical data as prediction dependencies, the spectrum prediction method proposed in the present invention uses three dilated causal convolutions (DCC) to receive the time domain feature vectors of proximity trend, daily trend, and periodic trend respectively, such as Figure 5 The calculation process is as follows:
[0052]
[0053] Among them, W i is the weight of the convolution kernel, K is the parameter of the convolution kernel (i.e. the number of weights), d is the expansion factor used to control the sampling interval, x t-(i-1)×d Represents the value of the input sequence at time t-(i-1)×d.
[0054] Use gated linear units to control information transmission. The gated linear unit (GLU) selectively transmits and filters information by introducing a gating mechanism, thereby improving the model's expressiveness and training efficiency. The extracted time domain features are passed to the next layer of the network through the gated linear unit. The calculation process is as follows:
[0055] h T =(W1*x+b1)⊙σ(W2*x+b2)
[0056] Among them, W1, W2, b1, b2 are learnable parameters, x is the input feature, ⊙ is the element-by-element product between matrices, σ is the sigmoid activation function, h T is the output of the gated linear unit.
[0057] Step 3: Use graph convolutional network to obtain frequency domain features of spectral data.
[0058] The correlation coefficient between two random variables can be expressed by the Pearson coefficient:
[0059]
[0060] Among them, x n 、x m They are spectrum data of different frequency bands, P∈R N×N is the correlation coefficient matrix, cov(,) is the covariance function, and σ(·) is the standard deviation. The larger the correlation coefficient, the stronger the correlation.
[0061] Therefore, we use the ability of the Graph Convolutional Network (GCN) to aggregate node features and use the frequency band correlation to extract frequency domain features. The calculation process is as follows
[0062]
[0063] Among them, h G is the output of the graph convolutional network, X is the feature matrix, representing the output data of GLU. θ is the learnable convolution kernel. E node ∈R N×d represents a learnable graph node embedding matrix, I N is the identity matrix.
[0064] Using E node and Multiply them together to get the adaptive adjacency matrix A G , A G Instead of the widely used similarity adjacency matrix, nodes can automatically obtain the main dependencies. Figure 5 As shown in Figure 2, the output of the gated linear unit is used as the input of the graph convolutional network.
[0065] Step 4: Dynamically weight the prediction output through the attention mechanism.
[0066] like Figure 5 As shown in the figure, the attention mechanism is introduced to dynamically weight the extracted time domain features and frequency domain features to accelerate the convergence process. Preferably, all time domain features and frequency domain features in each time-frequency block (TF Block) are summed up and then different weights are assigned to multiple features through an attention layer. The calculation process is as follows:
[0067]
[0068] Among them, q is the query vector, α(q,k i ) is the query vector q and the key vector k i The similarity score between i is the output of the gated linear unit or graph convolution, and h is the weighted sum calculated by the attention mechanism.
[0069] Use the normalized weights to compare the corresponding values h i Perform weighted summation to obtain the final representation h.
[0070] Sum all the outputs h obtained from different time-frequency blocks to get h out , through an output layer containing two convolutional layers, which is used to increase the depth of the model, thereby improving the generalization ability of the spectrum data prediction network model, and gradually adjusting and controlling the output dimension. The calculation process is as follows:
[0071]
[0072] Where W c1 、W c2 、b c1 、b c2 are the learnable weights and biases in the convolutional layer, and ReLU is the activation function.
[0073] Step 5: Calculate the mean absolute error based on the predicted spectrum data and the collected spectrum data.
[0074] First, all learnable parameters in the spectral data prediction network model are randomly initialized, and the number of iterations, learning rate, and batch size are set. Adam is selected as the optimizer. The initial number of iterations is 1, the maximum number of iterations is 500, the learning rate is 0.0001, and the batch size is 64. The training dataset is input into the spectral data prediction network model in batches for training. The training error of each batch is backpropagated to optimize all learnable parameters. The error loss is calculated as the mean absolute error (MAE) between the predicted value and the true value. The calculation process is as follows:
[0075]
[0076] Where N is the number of data points, Y i is the i-th true value, is the i-th predicted value, is the absolute error of the i-th data.
[0077] When all the data in the training data have completed forward output and back propagation, an iteration is completed.
[0078] Step 6: Use the training set data to train the spectrum data prediction network model.
[0079] Determine whether the current number of iterations has reached the maximum number of iterations. If so, complete the training and output the prediction results. If not, add 1 to the number of iterations and return to continue training the model.
[0080] Step 7: Input the test data set into the trained spectrum data prediction network model and output the prediction results.
[0081] The effects of the present invention will be further described below in conjunction with simulation experiments.
[0082] 1. Simulation conditions and parameter settings
[0083] The simulation experiments of the present invention were conducted on a simulation platform using Python 3.7 and Pytorch 1.9. The computer CPU model was an Intel(R) Xeon(R) CPU E5-2680 v4, equipped with an Nvidia GeForce RTX 3060 independent graphics card.
[0084] The maximum number of iterations of the network is 500, the learning rate is 0.0001, the Adam optimization algorithm is selected as the network training optimizer, and the data size of each batch is 64.
[0085] 2. Simulation content
[0086] Figure 6A comparison chart of the spectrum prediction method according to an embodiment of the present invention and other existing spectrum prediction methods. TFFAN refers to the present invention; the method without the attention network is referred to as TFFN. All other parameter settings are the same. The first 10 time steps are used to predict the input, and the output is the last 5 time steps. To cover the length of the input sequence, the dilation factors of the dilated causal convolution are set to 1, 2, 1, 2, 1, 2.
[0087] Figure 6 The horizontal axis represents different models, and the vertical axis represents the root mean square error and mean absolute error of the prediction. It can be seen from the figure that compared with the TFFN method that adds an attention network, the present invention has improved MAE and RMSE by 5.6% and 5.4% respectively, which verifies the effectiveness of the proposed scheme in considering time diversity and frequency domain features. Compared with the traditional Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM) and Gated Recurrent Unit (Gated Recurrent Unit), the MAE has improved by 25.9%, 22% and 5.7% respectively.
[0088] Table 1 lists the prediction error values (RMSE, MAE, MAPE) of the five models.
[0089] Table 1 Numerical results of TFFAN model prediction error
[0090]
[0091] Table 1 shows that all models have relatively small prediction errors. However, compared with other methods, the proposed TFFAN method exhibits the best prediction performance across all evaluation metrics. Multi-step time series prediction requires a model with good long-term memory to capture long-term dependencies in the time series. Due to the lack of a gating mechanism, RNNs cannot selectively update their states across multiple time steps. This can lead to the neglect of important information or the retention of unnecessary information when processing complex time series patterns. While LSTMs have a strong ability to capture long-term dependencies, their complex gating mechanism can lead to excessive filtering of information or the propagation of redundant information. The GRU structure is simpler, making it easier to find the optimal parameter configuration during training, thereby reducing the accumulation of prediction errors. The TFFN method simply sums the extracted time and frequency features and then passes the predicted value through the output layer. Without the attention mechanism, the model loses its dynamic weighting capability, resulting in insufficient information extraction and a reduced ability to capture complex patterns. In general, the proposed method has a MAE 5.6% to 25.9% lower, a RMSE 5.5% to 28.2% lower, and a MAPE 4.7% to 25.4% lower than the baseline algorithm, with average improvements of 15.8%, 16.9%, and 15.1%, respectively.
[0092] In order to further verify the training stability and training efficiency of the model, this paper records the loss value changes and total training time of each model training process, such as Figure 7 As shown in the figure, the loss of the TFFAN network with the attention network decreases significantly faster than that of the TFFN network. However, it is worth noting that the TFFAN network converges more slowly than other methods. This is because it considers both temporal and frequency features, requiring adaptation and adjustment of the weights and relationships between the two types of features. However, the MAE continues to decline, demonstrating the model's ability to continuously optimize.
[0093] like Figure 8 The figure shows a training efficiency comparison. It shows that the TFFN without the attention mechanism only improves training efficiency by 6.4% compared to the proposed TFFAN method, but its prediction accuracy decreases by 5.7%. Compared to the GRU and LSTM, the TFFAN model's training efficiency increases by 36.8% and 48.1%, respectively. Furthermore, while the RNN's training efficiency decreases by only 7.3% compared to TFFAN, its prediction accuracy decreases by 25.9%.
[0094] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. Any obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.
Claims
1. A spectrum prediction method based on time-frequency fusion attention network, characterized in that: include: Step S01: constructing a spectrum data prediction network model, setting initial learning parameters, inputting the collected spectrum data into the spectrum data prediction network model, performing periodic analysis on the spectrum data, and determining the time period attribute of the spectrum data; Step S02: Using a temporal convolutional network to extract the time domain features of the spectrum data, and based on the time domain features, using a gated linear unit to pass the spectrum data to a graph convolutional layer network, the temporal convolutional network includes three dilated causal convolutions, each of which is used to receive spectrum data of different time period attributes, including proximity trend, daily trend, and weekly trend. Step S03: Utilize a graph convolutional network to extract frequency domain features of the spectrum data. The frequency domain feature extraction method includes: constructing a correlation coefficient matrix based on the frequency band correlation of the spectrum data, and utilizing the graph convolutional network's ability to aggregate node features to extract the frequency domain features. The calculation process is as follows: Among them, h G is the output of the graph convolutional network, X is the feature matrix, representing the output data of GLU, θ is the learnable convolution kernel, E node ∈R N×d represents a learnable graph node embedding matrix, I N is the identity matrix; Step S04: Dynamically weighting the extracted time domain features and frequency domain features using an attention mechanism to obtain predicted spectrum data; the dynamic weighting process includes: summing the time domain features and frequency domain features corresponding to spectrum data of different time period attributes, and using an attention layer to assign different weights to the time domain features and frequency domain features corresponding to different spectrum data; the predicted spectrum data is output through an output layer, which includes two convolutional layers for deepening the depth of the spectrum data prediction network model and adjusting and controlling the output dimension; Step S05: Calculate the mean absolute error based on the predicted spectrum data and the collected spectrum data; Step S06: reversely training the spectrum data prediction network model according to the mean absolute error, optimizing the learning parameters, and completing the training of the spectrum data prediction network model; Step S07: inputting the spectrum data to be processed into the trained spectrum data prediction network model to obtain a prediction result.
2. The spectrum prediction method based on time-frequency fusion attention network according to claim 1 is characterized in that The method for determining the time period attribute in step S01 includes: determining the minimum historical data length corresponding to the same evolution trend according to the evolution trend of the spectrum data in different time slots, and determining the time period attribute of the spectrum data according to the time period corresponding to the historical data length.
3. The spectrum prediction method based on time-frequency fusion attention network according to claim 1 is characterized in that The gated linear unit in step S02 includes a gating mechanism, which is used to filter and selectively transmit information on spectrum data.
4. The spectrum prediction method based on time-frequency fusion attention network according to claim 1 is characterized in that The learning parameters in step S01 include the number of iterations, the learning rate, and the batch size.
5. The spectrum prediction method based on time-frequency fusion attention network according to claim 1 is characterized in that When the number of iterations set in the learning parameters is completed, the training of the spectrum data prediction network model is completed.