Dynamic spectrum access algorithm for evaluating spectrum stability in cognitive vehicle-to-everything (V2X)

Abstract

This invention claims protection for a dynamic spectrum access algorithm for evaluating spectrum stability in cognitive vehicle-to-everything (V2X) networks. The paper proposes a dynamic spectrum access algorithm for evaluating spectrum stability. First, based on signal-to-noise ratio (SNR), received signal strength, and bandwidth parameters, a long short-term memory (LSTM) neural network is used to predict the values ​​of these parameters at multiple time points within a future period, and the rate of change of each parameter over one period is calculated. The results are used as an evaluation index for spectrum stability. Second, the K-Means algorithm is used to cluster the rate of change vectors to construct a stability evaluation model. Third, the state space and reward function are reconstructed based on the stability evaluation results, and a reinforcement learning-based dynamic spectrum access algorithm is proposed. Finally, experimental results show that the proposed algorithm can meet the stability requirements of different vehicle terminal services, improve the utilization rate of spectrum resources, and reduce the probability of collisions during spectrum access.

Description

Technical Field

[0001] This invention belongs to the field of mobile communications, specifically to the dynamic spectrum access algorithm in cognitive vehicle networking. Background Technology

[0002] With the widespread adoption of vehicle-to-everything (V2X) applications, the demand for spectrum resources from vehicle terminals is rapidly increasing. Cognitive V2X, combining cognitive radio technology, has shown great potential as an emerging network environment. However, in this network, the stability of spectrum resources is affected by the environment and motion state of the vehicle terminals, and channel parameters change rapidly and require real-time monitoring, making spectrum stability difficult to assess. Accessing spectrum with poor stability can severely impact the efficiency of vehicle terminal transmission services, thereby affecting the overall network performance. Furthermore, high-concurrency vehicle terminal access requests intensify competition for spectrum resources, increasing the probability of collisions during access. Therefore, in cognitive V2X, how to assess the stability of spectrum resources, effectively utilize spectrum with varying stability, improve spectrum utilization, and reduce the probability of access collisions has become a hot research topic in this field.

[0003] Currently, research on cognitive vehicle-to-everything (V2X) environments mainly focuses on four aspects: wireless access, spectrum sensing, routing protocols, and security issues. Among these, few studies have quantitatively evaluated spectrum stability to improve its performance. In the paper [Chuang MC. Cooperation-Assisted Spectrum Handover Mechanism in Vehicular Ad Hoc Networks[A]. 2021 Wireless Telecommunications Symposium (WTS)[C]. CA, USA: IEEE, 2021: 1–7.], the authors abstract the number of available spectrum resources into a numerical value, calculate its variance after three rounds of spectrum scanning, and use the result as a spectrum stability index. This paper uses the variance of a single parameter to measure stability and introduces the concept of a stability index. References [N. Cheng, N. Zhang, N. Lu, X. Shen, JW Mark, and F. Liu, “Opportunistic spectrum access for CR-VANETs: A game-theoretic approach,” IEEE Trans. Veh. Technol., vol. 63, no. 1, pp. 237–251, Jan. 2014.] analyze the impact of primary user channel usage behavior and vehicle mobility on channel availability. Based on this, a congestion game model is used to model the spectrum access process of vehicle terminals. References [Niyato D, Hossain E, Wang P. Optimal Channel Access Management with QoS Support for Cognitive Vehicular Networks[J]. IEEE Transactions on Mobile Computing, 2011, 10(4): 573–591.] analyze the impact of spectrum status on access results from multiple perspectives, including abnormal spectrum perception results, packet transmission status, and weighted scheduling priority. This paper only measures spectrum availability through the single-moment state of the spectrum.The literature [Xiang P, Shan H, Wang M,. Multi-Agent RL Enables Decentralized Spectrum Access in Vehicular Networks[J].IEEE Transactions on Vehicular Technology,2021,70(10):10750–10762.] considers the different latency and throughput requirements of spectrum for vehicle-to-infrastructure users and vehicle-to-vehicle users, and maximizes network performance by setting different reward functions to meet the quality of service of users.

[0004] The aforementioned access algorithms analyze the impact of spectrum status or availability on spectrum access from different perspectives. Different models are used to model the dynamic spectrum access process, aiming to maximize spectrum utilization and minimize spectrum collision probability. However, most of the literature, when evaluating spectrum status, does not consider the long-term performance of multiple spectrum parameters and their impact on access, nor does it address the varying spectrum stability requirements of services transmitted by different vehicle terminals. To solve these problems, how to comprehensively evaluate spectrum stability and achieve rational utilization of spectrum with different stability levels becomes the core issue addressed in this paper. Therefore, this paper proposes a spectrum stability evaluation model and designs a dynamic spectrum access algorithm based on reinforcement learning. Summary of the Invention

[0005] This invention aims to solve the problems of the prior art mentioned above. It proposes a dynamic spectrum access algorithm for cognitive vehicle-to-everything (V2X) networks. The technical solution of this invention is as follows:

[0006] An access algorithm for cognitive vehicle-to-everything (V2X) networks that considers spectrum stability includes the following steps:

[0007] First, based on signal-to-noise ratio, received signal strength, and bandwidth parameters, a stability parameter prediction model is established using a long short-term memory neural network. This model predicts the parameter values ​​at multiple future time points and uses the rate of change of the parameter within a prediction period as an evaluation index for spectral stability. Second, the K-Means algorithm is used to cluster the predicted rate of change vectors to achieve stability evaluation, thus constructing a stability evaluation model. Third, combining the spectral stability index and channel throughput, a dynamic spectrum access algorithm based on reinforcement learning is proposed.

[0008] Furthermore, the construction of the stability parameter prediction model specifically includes: firstly, using historical and real-time data of bandwidth, signal-to-noise ratio, and received signal strength as inputs to a long short-term memory neural network. Through a stepwise prediction algorithm, the predicted values ​​of these parameters at multiple time points are obtained, thus constructing a stability parameter prediction model in the above manner.

[0009] Furthermore, the described process of predicting stability parameters mainly involves analyzing historical spectrum data and real-time data collected to obtain predicted spectrum data for multiple future time points, and calculating the rate of change within one period. This mainly includes the following three steps:

[0010] 1) The initial input matrix is ​​composed of three features that together form the input and output vectors of the Long Short-Term Memory Neural Network. This matrix consists of real-time data at time t and historical data from time t-1 to time tT-1.

[0011] 2) Next, matrix X t After inputting the data into the prediction model, the predicted data y at time t+1 is obtained. t+1 The historical data from time t-1 to time tT, the real-time data at time t, and the predicted data at time t+1 are combined to form a new input matrix. The predicted data y at time t+2 is then obtained through model prediction. t+2 .

[0012] 3) Repeat the process in the previous step to obtain the final predicted value Y. T =[y t+1 ,y t+2 ,....y t+T This matrix consists of the predicted data from time t+1 to time t+T.

[0013] Furthermore, the signal-to-noise ratio, bandwidth, and received signal strength specifically include:

[0014] Signal-to-noise ratio:

[0015]

[0016] Among them, P t i,j This indicates that at time t, vehicle terminal i detected the transmit power of channel j. μ represents the channel gain of channel j accessed by vehicle terminal i. t This represents background noise with a mean of 0 and a standard deviation of σ. This represents the distance between vehicle terminal i and the base station belonging to channel j at time t.

[0017] Received signal strength:

[0018]

[0019] Among them, P t i,j This indicates that at time t, vehicle terminal i detected the transmit power of channel j. μ represents the distance between vehicle terminal i and the base station belonging to channel j at time t.t This represents background noise with a mean of 0 and a standard deviation of σ.

[0020] bandwidth:

[0021]

[0022] Where high(f) t i,j ) represents the highest frequency of channel j accessed by vehicle terminal i at time t, low(f t i,j ) represents the lowest frequency of channel j accessed by vehicle terminal i at time t.

[0023] Furthermore, the K-Means algorithm is used in the stability assessment model to complete the stability assessment. This paper assumes a K value of 3, meaning there are ultimately three clusters classified by stability. Three channels are randomly selected from the channel set M as initial centroids based on their rate of change vectors. According to the stability parameter prediction model above, the predicted value vectors of each parameter for each channel at time t+T are obtained. The rate of change vector of each channel from time t to time t+T is then calculated. The rate of change vector of the channel i to be evaluated is then... As input to the stability assessment model, three clusters are obtained, distinguished by stability, thus completing the stability assessment.

[0024] Furthermore, the calculated rate of change of signal-to-noise ratio, rate of change of received signal strength, and rate of change of bandwidth specifically include:

[0025] Signal-to-noise ratio change rate:

[0026]

[0027] in, This represents the real-time signal-to-noise ratio of channel j measured by vehicle terminal i at time t. This represents the predicted signal-to-noise ratio (SNR) of channel j at time t+T, output by the LSTM. The calculation of the rate of change of received signal strength and the rate of change of bandwidth is similar.

[0028] Further stability evaluation using the K-Means algorithm involves the following steps:

[0029] 1) Normalization of Predicted Value Change Rate: Since the rate of change of the channel's signal-to-noise ratio, the rate of change of the received signal strength, and the rate of change of bandwidth vary within different intervals, this paper uses the Min-Max normalization method to normalize these parameters. The normalization calculation of the rate of change of the signal-to-noise ratio is shown in the following formula:

[0030]

[0031] in This represents the rate of change of the signal-to-noise ratio of channel j within time step T. Indicates time

[0032] The rate of change of bandwidth and the rate of change of received signal strength are normalized using the same method.

[0033] 2) Stability Clustering: Calculate the Euclidean distance between the rate of change vector of each channel and each centroid, and assign the channel's rate of change vector to the cluster containing the centroid with the smallest Euclidean distance. Randomly select three channels from the channel set M, using their rate of change vectors as initial centroids. Assume channel m is one of the initial centroids, and the Euclidean distance between channel j and centroid m is dis. m,j The calculation formula is as follows:

[0034]

[0035] 3) Update the centroid: For each cluster, calculate the average of the rate of change vectors of all its channels, and use this average as the new centroid. If channels j+1, j+2, and j+3 are assigned to the same cluster, the new centroid is represented as:

[0036]

[0037] 4) Complete stability evaluation: After repeated iterations until the centroid no longer changes, the final three clusters and centroid are obtained. Assume the final centroid is centroid. i The norm of i∈{1,2,3} is:

[0038]

[0039] ω 1 ω 2 ω 3 Sort by size from smallest to largest; a larger value indicates higher stability within the cluster. If ω 1 <ω 2 <ω 3 Then the centroid 1 Centroid 2 Centroid 3 The stability of the clusters they belong to are low, medium, and high, respectively. If channel i belongs to the centroid... 1 The stability index of the cluster it belongs to.

[0040] Furthermore, in this paper's network scenario, the service requests from vehicle terminals are categorized into two types: high-stability services and low-stability services. For a vehicle terminal, the goal is to access a channel that meets its stability and throughput requirements while competing with other vehicle terminals for access channels. Therefore, this paper proposes an access algorithm based on reinforcement learning, which redesigns the state space and reward function according to the service type requested by the terminal and the spectrum stability assessment results.

[0041] Furthermore, in constructing the state space, this paper aims to enable the vehicle terminal to effectively interact with channels of varying stability, thereby obtaining effective samples for reinforcement learning training. Therefore, this chapter selects channel state, stability index, and throughput to construct the reinforcement learning state space. This allows the vehicle terminal to iteratively select the most suitable channel for data transmission based on the channel stability and throughput requirements of its transmitted services, thus achieving rational utilization of spectrum resources with different stability levels. In summary, the state space of this paper is... The state space is composed of the channel state vector, stability index vector, and throughput vector of vehicle terminal 1 to vehicle terminal i, respectively.

[0042] Furthermore, the channel occupancy vector This represents the state of channel 1 to channel m detected by vehicle terminal i at time t. This indicates that at time t, vehicle terminal i detects that channel j is occupied. This indicates that at time t, vehicle terminal i detects that channel j is idle.

[0043] Furthermore, the stability index vector This represents the stability index of channels 1 to m as evaluated by the system at time t.

[0044] Furthermore, throughput vector This represents the throughput that vehicle terminal i can obtain by accessing channels 1 to m at time t, where the throughput calculation formula is as follows:

[0045]

[0046] Furthermore, This is the action space of this paper, including the channel selection behavior of vehicle terminal 1 to vehicle terminal n. This indicates that at time t, vehicle terminal i did not choose to access any channel. This indicates that at time t, vehicle terminal i selects to access channel j.

[0047] Furthermore, considering the different requirements for spectrum stability and throughput between high-stability and low-stability services, this paper uses a tradeoff coefficient λ to differentiate the reward function of vehicle terminals transmitting these two types of services. Its expression is as follows:

[0048]

[0049] Wherein, if the service transmitted by vehicle terminal i at time t is a high-stability service, then λ > 0.5; if the service transmitted is a low-stability service, then λ < 0.5. Let represent the channel j accessed by vehicle terminal i at time t, and the normalized stability index. The normalization method is the same as before, using the Min-Max method. Let represent the normalized throughput of channel j accessed by vehicle terminal i at time t.

[0050] Furthermore, this paper uses an experience playback protocol to store the experience tuples used by the vehicle terminal at each time step. The data is stored in the experience replay pool. Simultaneously, the vehicle terminal uses a mini-batch algorithm to randomly extract L experience tuples from the experience replay pool each time, and inputs them into a neural network composed of a Gated Recurrent Unit (GRU) and a Dueling Network for stochastic gradient descent to update the neural network parameters θ. t This allows the access algorithm in this paper to converge faster.

[0051] Furthermore, this paper uses a GRU as the first hidden layer because channel parameters are typically time-series data, and channel states change continuously over time. The GRU, through its internal loop structure, can memorize and utilize historical information, thereby effectively tracking the changing trends of channel parameters. Therefore, the introduction of a GRU addresses the problem of rapid changes in channel parameters in cognitive vehicular networks.

[0052] Furthermore, this paper uses a Dueling Network as the second hidden layer. The Dueling Network divides the Q-value function into state value functions. With advantage function This approach addresses the overestimation of Q-values, accelerating convergence. Furthermore, it accurately estimates the value of each channel selection behavior under different environments. For instance, a low-stability channel will have a lower Q-value in an environment where most high-stability channels are idle. Conversely, in environments where high-stability channels are mostly occupied and few idle channels are available, the Q-value should be higher to encourage vehicle terminals to access the network. Dueling Network enables vehicle terminals to dynamically adjust their access strategies based on different environments, ensuring the rational utilization of channels with varying stability.

[0053] Furthermore, the reinforcement learning-based access algorithm proposed in this paper continuously adjusts the loss function L... t (θ t Gradient descent is used to update the neural network parameters, thereby continuously bringing the estimated value of the Q-value function closer to the actual value, ultimately yielding an optimal θ. t This allows the access strategies of each vehicle terminal to converge to the optimal level. The loss function L... t (θ t ) is represented as:

[0054]

[0055] in This article will The value of Q is used as the temporal difference error (TDR) θ, and the Q value is updated by continuously learning from the error. The Q value is updated in the following way:

[0056]

[0057] Furthermore, this paper employs an ε-strategy for channel selection, meaning that the action with the largest Q-value is selected with a probability of 1-ε each time. This strategy is defined as:

[0058]

[0059] The advantages and beneficial effects of this invention are as follows:

[0060] 1. Based on historical and real-time data of signal-to-noise ratio, received signal strength, and bandwidth parameters, the traditional stepwise prediction algorithm is improved. LSTM is used to predict channel parameters at multiple times within a single period, and the rate of change within the prediction period is used as the spectral stability evaluation index. The K-Means algorithm is used to cluster the stability of the rate of change vector, thus constructing a spectral stability evaluation model.

[0061] 2. Based on the stability assessment results and the stability requirements of the services transmitted by the vehicle terminal, the state space, action space and reward function of reinforcement learning were redesigned, and an access algorithm based on reinforcement learning was proposed to achieve reasonable utilization of spectrum with different stability. Attached Figure Description

[0062] Figure 1 Here is the algorithm flowchart;

[0063] Figure 2 The neural network structure for the prediction model;

[0064] Figure 3 Prediction results for the bandwidth training set;

[0065] Figure 4 For bandwidth test set prediction results;

[0066] Figure 5 The training set loss function curve;

[0067] Figure 6 The accuracy curve for the test set;

[0068] Figure 7 For the Q-value update process;

[0069] Figure 8 A simulation diagram of urban cognitive vehicle networking;

[0070] Figure 9 The results show the comparison of spectrum utilization.

[0071] Figure 10 The results show the comparison of collision probabilities;

[0072] Figure 11 It is the cumulative stability index;

[0073] Figure 12 For cumulative throughput;

[0074] Figure 13 Comparison of convergence speeds under different numbers of vehicles and channels; Detailed Implementation

[0075] The technical solutions of the present invention will now be described clearly and in detail with reference to the accompanying drawings of the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention.

[0076] The technical solution of the present invention is as follows:

[0077] First, based on signal-to-noise ratio, received signal strength, and bandwidth parameters, a long short-term memory neural network is used to predict the values ​​of these parameters at multiple time points within a future period, and the rate of change of each parameter over one period is calculated. The results are used as an evaluation index for spectrum stability. Second, the K-Means algorithm is used to cluster the rate of change vectors to construct a stability evaluation model. Third, the state space and reward function are reconstructed based on the stability evaluation results, and a dynamic spectrum access algorithm based on reinforcement learning is proposed. Finally, experimental results show that the proposed algorithm can meet the stability requirements of different vehicle terminal services, improve the utilization rate of spectrum resources, and reduce the probability of collisions during spectrum access.

[0078] The access algorithm for evaluating stability proposed in this invention includes the following steps:

[0079] Step 1: Use historical and real-time data on bandwidth, signal-to-noise ratio, and received signal strength as input to the LSTM. Through a stepwise prediction algorithm, obtain predicted values ​​for these parameters at multiple points within a future prediction period. A drop-out algorithm is introduced to balance the influence of historical and real-time data on the prediction results.

[0080] Step 2: Use the rate of change of the predicted values ​​from the start of the prediction period to the end of a prediction cycle as input to the K-Means algorithm. After steps such as normalizing the rate of change of predicted values, allocating samples, and updating centroids, stability clustering is achieved. A stability assessment model is then constructed to perform stability evaluation.

[0081] Step 3: Based on the spectrum stability assessment results, the state space, action space, and reward function of the reinforcement learning were redesigned to enable vehicle terminals transmitting services with different stability requirements to access channels that meet their needs. The problems of overestimation and utilization of historical channel information were addressed by introducing a Dueling Network and a GRU neural network layer.

[0082] To evaluate the performance of the proposed algorithm, a network simulation scenario was conducted using Matlab to model a 3000m long and 30m wide one-way three-lane road in a core urban area. In this simulation, two 5G macro base stations (5GM) and several vehicles were deployed. It was assumed that each base station had a coverage range of 40m and the vehicles traveled at a speed of 10-15m / s. To simplify the simulation setup, the number of vehicles and channels were set to 10 and 5, respectively.

[0083] In the simulation experiment, the proposed dynamic spectrum access algorithm for evaluating spectrum stability was compared with the following access algorithms: Q-learning-based algorithm [Chen L, Zhao Q, Fu K, et al. Multi-user reinforcement learning based multi-reward for spectrum access in cognitive vehicular networks[J]. Telecommunication Systems, 2023, 83(1):51-65.], Deep Q-learning-based algorithm [Chen L, Wang Z, Zhao X, et al. A dynamic spectrum access algorithm based on deep reinforcement learning with novel multi-vehicle reward functions in cognitive vehicular networks[J]. Telecommunication Systems, 2024:1-25.], and time-slot Aloha algorithm [K. Kar, S. Sarkar, and L. Tassiulas, Achieving proportional fairness using local information in aloha networks,[J]. IEEE A comparison was made with [Trans.Autom.Control, 2004, 49(10): 1858–1863].

[0084] Figure 9The graphs show the spectrum utilization changes of four algorithms during 100,000 iterations of reinforcement learning for the vehicle terminal. Performance is calculated every 5,000 iterations. Vehicle terminals using the slotted Aloha protocol randomly select channels for access. Therefore, their utilization stabilizes around 0.4 as the number of iterations increases. Q-learning, Deep Q-learning, and the proposed algorithm are all reinforcement learning-based algorithms. In the initial state, the neural network parameters of these algorithms are random, leading to a low initial utilization. By continuously learning from the environment and iteratively updating the neural network parameters, the channel access strategy is continuously optimized. Therefore, the utilization of vehicle terminals using these three algorithms increases with the number of iterations. Compared to Q-learning and Deep Q-learning algorithms, the proposed algorithm shows the most significant improvement in spectrum utilization, converging to around 0.9 after 20 iterations. This is because the algorithm introduces a GRU layer and an experience replay protocol, enabling the vehicle terminal to maximize the utilization of the access experience accumulated during the iteration process. Figure 9 As can be seen, the utilization rate of the Deep Q-learning algorithm is higher than that of the Q-learning algorithm at all stages. Since the Q-learning algorithm updates Q-values ​​by maintaining a Q-table, the overhead of maintaining the Q-table increases exponentially with the number of iterations and the increasing complexity of the environmental information, thus its performance is lower than that of the Deep Q-learning algorithm.

[0085] Figure 10 The diagram shows the collision probability changes of four algorithms during 100,000 reinforcement learning iterations for the vehicle terminal. Performance results are calculated every 5,000 iterations. Figure 10As can be seen, except for the collision probability of the time-slot Aloha algorithm, which fluctuates around 0.6, the collision probabilities of other algorithms are relatively high in the initial training stage, but gradually decrease and converge with the increase of iterations. This is because reinforcement learning algorithms like Q-learning, Deep Q-learning, and the algorithm proposed in this paper involve each vehicle terminal exploring a random channel each time in the initial training stage, which easily leads to multiple terminals accessing the same channel. However, as the number of iterations increases, the probability gradually decreases to 0, and the access strategy of each vehicle terminal gradually reaches its optimal level. Each terminal is highly likely to choose the channel with the highest Q-value for access, thus gradually reducing the collision probability. The algorithm proposed in this paper decreases the collision probability faster than Q-learning and Deep Q-learning algorithms, and basically converges to 0 after 12 iterations. This is because the algorithm in this paper differentiates the reward function according to the service type of the vehicle terminal. Thus, as the number of iterations increases, vehicle terminals transmitting high-stability services are more inclined to access idle high-stability channels. Vehicle terminals transmitting low-stability services tend to access medium- or low-stability channels, thus their collision probability converges to 0 faster than other algorithms.

[0086] Figure 11 The presentation shows the cumulative stability index changes when multiple vehicle terminals select different algorithms to transmit high-stability services. Figure 11 It can be seen that the cumulative stability index of the three algorithms gradually increases with the increase of the number of iterations. This indicates that during the training process, when the vehicle terminal is transmitting high-stability services, it tends to access channels with high stability. The cumulative stability index obtained by the vehicle terminal of the algorithm proposed in this paper is much higher than that of the other two algorithms. The reason is that this paper makes the tradeoff coefficient λ in formula (10) > 0.5, so that the vehicle terminal accessing the high-stability channel can obtain more rewards.

[0087] Figure 12 This paper presents the cumulative throughput curves of multiple vehicle terminals transmitting low-stability services using different algorithms. As the number of iterations increases, the cumulative throughput of all three algorithms gradually rises, but the algorithm proposed in this paper achieves a significantly higher throughput for the vehicle terminal compared to the other two algorithms. This is because this paper utilizes a GRU layer and an experience replay protocol to maximize the use of dynamic spectrum access experience during the training process. With increasing iterations, vehicle terminals transmitting low-stability services no longer compete for high-stability channels but instead access medium-to-low-stability channels. While these channels have lower stability, they still guarantee basic throughput. This improves channel utilization, reduces collision probability, and ensures sufficient throughput.

[0088] Figure 13 The convergence speed of the proposed algorithm was compared with two other comparative algorithms under different numbers of vehicle terminals and channels. The numbers of vehicle terminals were 3, 5, 7, and 9, and the number of channels were 2, 3, 4, and 5, respectively. Figure 13 It can be observed that as the number of vehicle terminals increases, the number of iterations required for algorithm convergence also increases accordingly. The algorithm proposed in this paper, by introducing a Dueling Network, correctly estimates the value of each channel selection behavior under different environments, solves the problem of overestimation of Q-values, and accelerates the convergence speed.

[0089] In summary, this paper is more suitable for cognitive vehicle networks with a large number of vehicles, and its convergence speed in small-scale networks is not much slower than that of the comparison algorithms.

[0090] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0091] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0092] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. An access method for evaluating spectrum stability in a cognitive vehicle network, characterized in that, Includes the following steps:

101. Based on three types of parameters—signal-to-noise ratio, received signal strength, and bandwidth—a long short-term memory neural network is used to predict the values ​​of the three types of parameters at multiple moments within a future prediction period. The rate of change of the three types of parameters from the current moment to the end of the prediction period is calculated, and the rate of change of the three types of parameters is combined into a rate of change vector as an evaluation index of spectrum stability.

102. Based on the rate of change vector obtained in step 101, the rate of change of signal-to-noise ratio, the rate of change of received signal strength, and the rate of change of bandwidth in the rate of change vector are first normalized. Then, the normalized rate of change vector is used as the input of the K-Means algorithm to calculate the Euclidean distance between the rate of change vector of each channel and each centroid. Each channel is assigned to the cluster containing the centroid with the smallest Euclidean distance. The centroid is updated according to the average value of the rate of change vector within each cluster to obtain clusters distinguished by stability. The centroid of the cluster to which the channel to be evaluated belongs is used as the stability evaluation result of the channel. The stability index vector is output to form a stability evaluation model.

103. Perform dynamic spectrum access based on the stability index vector obtained in step 102, specifically including: using the stability index vector, throughput vector, and channel occupancy vector as input states for the reinforcement learning method, selecting a channel from the channel set for access as an action; and constructing a reward function based on the weighted sum of the normalized stability index and the normalized throughput. The vehicle terminal selects an access channel under the current input state, receives a reward after performing the access action, and trains a neural network based on an experience tuple containing the state, action, reward, and next state to update the channel access strategy and complete dynamic spectrum access.

2. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 1, characterized in that, The construction of the stability assessment model specifically includes:

201. Use historical and real-time data on bandwidth, signal-to-noise ratio, and received signal strength as inputs to a long short-term memory neural network; 202. Based on the input in step 201, the predicted values ​​of the above parameters at multiple times in a future prediction period are obtained through a stepwise prediction algorithm.

203. The rate of change of the predicted value calculated in step 202 is used as an evaluation index of spectral stability.

3. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 2, characterized in that, By analyzing historical and real-time spectrum data, the spectrum status at multiple future points in time is predicted, and the rate of change within one period is calculated and used as a stability assessment indicator.

301. Vehicle terminal At any moment Channel detected signal-to-noise ratio, Vehicle terminal At any moment Channel detected Received signal strength Vehicle terminal At any moment Channel detected The bandwidth, together with the three features that form the input and output vectors of the long short-term memory neural network, constitute the initial input matrix. The matrix is ​​composed of Real-time data at any given moment, and Time's up The composition of historical data at any given moment; 302. Based on the input matrix in step 301 ,get Predicted data for each moment ,Will Time's up Historical data of the moment Real-time data and The predicted data at each time point form a new input matrix, which is then used by the model to obtain... Predicted data for each moment ; 303. By repeatedly performing step 302, the final predicted value matrix is ​​obtained. The matrix is ​​composed of Time's up The composition of the predicted data at any given time.

4. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 2, characterized in that... Stability is comprehensively evaluated using three parameters related to spectral stability, specifically: Signal-to-noise ratio: ， in, express Time vehicle terminal Channel detected The transmission power, Indicates vehicle terminal Accessed Channel Channel gain, This indicates that the mean is 0 and the standard deviation is 0. Background noise, express Time vehicle terminal to channel The distance between the affiliated base stations; Received signal strength: ， in, express Time vehicle terminal Channel detected The transmission power, express Time vehicle terminal to channel The distance between the base stations This indicates that the mean is 0 and the standard deviation is 0. Background noise, Indicates the path loss index; bandwidth: ， in express Time vehicle terminal Accessed Channel The highest frequency, express Time vehicle terminal Accessed Channel The lowest frequency.

5. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 4, characterized in that, The calculation of the rate of change of signal-to-noise ratio, the rate of change of bandwidth, and the rate of change of received signal strength specifically includes: Signal-to-noise ratio change rate: ， in, Indicates channel exist Time is determined by the vehicle terminal The measured real-time signal-to-noise ratio, This represents the output of the Long Short-Term Memory (LSTM) neural network. Time Channel The predicted signal-to-noise ratio; Bandwidth change rate: ， in, Indicates channel exist Time is determined by the vehicle terminal The measured real-time bandwidth This represents the output of the Long Short-Term Memory (LSTM) neural network. Time Channel The bandwidth prediction value; Rate of change of received signal strength: ， in, Indicates channel exist Time is determined by the vehicle terminal The measured real-time received signal strength, This represents the output of the Long Short-Term Memory (LSTM) neural network. Time Channel The predicted value of the received signal strength; 501. Based on the stability parameter prediction model above, the values ​​of each channel in the model are obtained. The predicted value vectors of each parameter at each time point are obtained by calculation from each channel. Time's up The rate of change vector at time t, in the channel to be evaluated rate of change vector As input to the stability assessment model, three clusters are obtained according to stability, and the stability assessment is completed. The main steps include the following four steps: Normalization of the predicted value change rate: Since the channel signal-to-noise ratio change rate, received signal strength change rate, and bandwidth change rate vary in different intervals, this paper uses the Min-Max normalization method to normalize the above parameters.

502. Stability Clustering: Calculate the Euclidean distance between the rate of change vector of each channel and each centroid, and assign the rate of change vector of the channel to the cluster containing the centroid with the smallest Euclidean distance.

503. Update centroid: For each cluster, calculate the average of the rate of change vectors of all its channels and use this average as the new centroid.

6. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 5, characterized in that... Channel state, stability index, and throughput were selected to construct the state space for reinforcement learning. This allows the vehicle terminal to select the most suitable channel for data transmission through continuous iterative training, based on the channel stability and throughput requirements of its transmitted services. This achieves the rational utilization of spectrum resources with different stability levels. Therefore, the state space in this paper is: This state space is divided into vehicle terminal 1 and vehicle terminal 2 respectively. It consists of the channel state vector, stability index vector, and throughput vector; Where the channel occupancy vector Indicates in Time vehicle terminal Detected channel 1 to channel state, , Indicates in Time vehicle terminal Channel detected Occupied Indicates in Time vehicle terminal Channel detected idle; Where the stability index vector Indicates in The system-side evaluation of channel 1 to channel 2 at any given time Stability index; Where throughput vector Indicates in Time vehicle terminal Access channel 1 to channel 2 respectively The achievable throughput, where the throughput calculation formula is as follows: ， Considering the different requirements of high-stability and low-stability services for spectrum stability and throughput, this paper uses a trade-off coefficient. To differentiate the reward functions of vehicle terminals transmitting these two types of services Its expression is as follows: ， Among them, if in Time vehicle terminal The transmitted service belongs to the category of high-stability services. If the transmitted service is of low stability, then , express Time vehicle terminal Accessed channels The normalized stability index is obtained by using the Min-Max method, as described above. express Time vehicle terminal Accessed channels The normalized throughput.

7. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 6, characterized in that... This paper uses an experience replay protocol to store the experience tuples used by the vehicle terminal at each time step into an experience replay pool. Simultaneously, the vehicle terminal uses a mini-batch algorithm to randomly extract data from the experience replay pool each time. The empirical tuples are input into a neural network consisting of a Gated Recurrent Unit (GRU) and a Dueling Network to perform stochastic gradient descent, thereby updating the neural network parameters and enabling the access method in this paper to converge faster.

8. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 7, characterized in that... This paper uses GRU as the first hidden layer because channel parameters are usually time-series data and the channel state changes continuously over time. GRU can remember and utilize historical information through its internal loop structure, thereby effectively tracking the changing trend of channel parameters. Therefore, GRU is introduced to solve the problem of rapid changes in channel parameters in cognitive vehicle networks.

9. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 8. This paper uses a Dueling Network as the second hidden layer. The Dueling Network divides the Q-value function into state value functions. With advantage function In this way, on the one hand, the problem of Q-value overestimation is solved, accelerating the convergence speed; on the other hand, the value of each channel selection behavior is correctly estimated in different environments. Low-stability channels have lower Q-values ​​in environments where most high-stability channels are idle, while in environments where high-stability channels are all occupied and there are few idle channels, their Q-values ​​should be higher. This encourages vehicle terminals to access the network. The Dueling Network enables vehicle terminals to dynamically adjust their access strategies according to different environments, ensuring that channels of varying stability are utilized rationally. The Q-value is expressed as: 。 10. The access method for evaluating spectrum stability in a cognitive vehicle network as described in claim 9, by continuously adjusting the loss function... Gradient descent is used to update the neural network parameters, thereby continuously bringing the estimated value of the Q-value function closer to the actual value, ultimately yielding an optimal value. This allows the access strategies for each vehicle terminal to converge to the optimal value, where the loss function... Represented as: ， in This article will The value is used as the Temporal Difference Error (TDR). It continuously learns from errors to update the Q value, where the Q value is updated in the following way: ， This article uses channel selection. Strategy, that is, each time with The strategy that selects the action with the highest probability Q-value is defined as follows: 。

Images