A collaborative edge caching method based on dynamic content popularity

Abstract

The present invention specifically relates to an edge collaborative caching method based on dynamic content popularity, which solves the technical problems that the existing Internet of Vehicles edge caching technology has low caching efficiency, and the efficient edge caching strategy does not simultaneously consider users' different content popularity preferences and collaborative caching between multiple devices. The present invention includes the following steps: 1) determining a training set and a test set; 2) establishing an LSTM-Attention network model; 3) obtaining a dynamic content popularity model; 4) obtaining cluster centers and the number of clusters; 5) obtaining an edge caching decision plan; 6) when requesting content, determining whether the requested content is cached. If it is cached, the edge servers of the vehicle nodes and roadside units with caching capabilities directly feed back the requested content to the remaining vehicle nodes without caching capabilities; otherwise, triggering a remote server, which directly transmits the requested content to the remaining vehicle nodes without caching capabilities, thereby completing the edge collaborative caching of dynamic content popularity.

Description

Technical Field

[0001] The present invention relates to an intelligent transportation system caching method, and in particular to an edge collaborative caching method based on dynamic content popularity. Background Art

[0002] The Internet of Vehicles (IoV) is the foundational paradigm of 5G networks. The Vehicle Edge Network (VEN), comprised of a large number of vehicles and roadside units (RSUs), provides comfortable and reliable services to vehicle users in the Intelligent Transportation System (ITS), such as intelligent route planning, in-vehicle multimedia services, and multi-vehicle collaborative autonomous driving. These diverse in-vehicle applications require large amounts of internet data, and model applications are highly sensitive to latency. Excessive data traffic often results in high latency and high network load.

[0003] In order to solve the problems of degraded user experience caused by excessive data traffic and high latency in Internet of Vehicles transmission, Mobile Edge Computing (MEC) has been gradually introduced into ITS. MEC is a new distributed computing method based on mobile communication networks. It extends the functionality of cloud computing by bringing it to the edge of the network. It can also be regarded as an edge server running at the edge of the mobile communication network and performing specific tasks. Therefore, MEC is one of the key technologies of 5G communication and can dynamically obtain and process real-time data.

[0004] In the connected vehicle (IoV) system, user-required content often needs to be pre-deployed in roadside equipment rather than relying on the real-time nature of edge computing. This reduces the computing pressure on edge devices, making edge caching a more reliable way to improve user experience. Edge caching is a key emerging technology in edge computing. It caches computing resources from the ITS closer to the user, pre-caching content in the RSU, thereby reducing transmission costs between vehicles and the cache. Edge caching can alleviate high backhaul bandwidth demands by providing storage capabilities on the wireless access network, significantly reducing the frequent communication between multiple devices in the scenario and lowering the latency of content transmission between them.

[0005] Existing edge caching technologies, limited by the storage capacity of edge servers, cannot store much valid content, nor guarantee the timeliness of information transmission between multiple devices. For RSU content caching, existing approaches often formulate caching strategies based on historical vehicle data or content request frequency. However, this can result in some user-preferred content being overlooked, leading to uneven service.

[0006] Given these issues, it's essential to consider how to improve caching efficiency. Multi-device collaborative caching can effectively improve caching efficiency. Specifically, cooperating caching devices can jointly serve users, and all user request information can be forwarded between multiple devices. Using appropriate algorithms can significantly improve the performance of multi-device collaborative caching. Therefore, overcoming hardware limitations and employing software optimization algorithms to improve edge caching efficiency is crucial for enhancing user experience.

[0007] At present, there have been many related works on more efficient edge caching strategies. Taking active caching as an example, Su et al. proposed a collaborative caching scheme for vehicles and RSUs in "An edge caching scheme to distribute content in vehicular networks" [in IEEE Trans. Veh. Technol., vol. 67, no. 6, pp. 5346-5356, 2018]. For the content in the two nodes, the requesting user mainly calculates the node location based on the content acquisition delay, and considering the limited cache capacity of the node, a dynamic caching strategy based on cross entropy is proposed for RSU; Zhang et al. considered probabilistic caching based on random geometry to improve the offloading rate of the backhaul link in "Optimal probabilistic caching in heterogeneous IoT networks" [in IEEE Internet Things J., vol. 7, no. 4, pp. 3404-3414, 2020], and proposed an improved cache probability conversion algorithm to obtain a closed-form solution. In "Popularity Tracking for Proactive Content Caching with Dynamic Factor Analysis" (in 2019 IEEE / CIC International Conference on Communications in China (ICCC), Changchun, China, 2019), Mehrizi et al. developed a probabilistic dynamic model to predict content popularity and minimize network costs using cache replacement. This model uses the traditional Zipf distribution to cache content, but does not consider user preferences for different content popularity levels or collaborative caching across multiple devices. Summary of the Invention

[0008] The purpose of the present invention is to solve the technical problems that the existing Internet of Vehicles edge caching technology has low caching efficiency, and the efficient edge caching strategy does not simultaneously consider users' different content popularity preferences and collaborative caching among multiple devices. It provides an edge collaborative caching method based on dynamic content popularity.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] An edge collaborative caching method based on dynamic content popularity includes the following steps:

[0011] 1) Using the data in Movielens as a training set, and reprocessing and integrating the user request content collected by the vehicle as a test set; the data types and dimensions of the training set and the test set are the same;

[0012] 2) Establish an LSTM-Attention network model;

[0013] 3) Use the training set to train the LSTM-Attention network model, and then use the test set to test the trained LSTM-Attention network model to obtain a dynamic content popularity model;

[0014] 4) Cluster the vehicles to obtain the cluster center and the number of clusters. The cluster center is used as the vehicle node with caching capability, and the number of clusters is used as the number of vehicle nodes with caching capability.

[0015] 5) Using a dynamic content popularity model to obtain content vectors, the content vectors are used as cache content repositories. Caching-capable vehicle nodes and roadside unit edge servers are used as agents. These agents use the content vectors in the cache content repositories to perform multi-agent reinforcement learning to obtain edge caching decision solutions.

[0016] 6) Input the edge caching decision plan into the vehicle nodes with caching capabilities and the remaining vehicle nodes without caching capabilities respectively;

[0017] When a vehicle node with caching capabilities requests content, the remote server directly feeds the requested content back to the vehicle node with caching capabilities, completing edge collaborative caching of dynamic content popularity.

[0018] When the remaining vehicle nodes without caching capabilities request content, the vehicle nodes with caching capabilities and the edge servers of the roadside units determine whether the requested content has been cached. If it has been cached, the requested content is directly fed back to the remaining vehicle nodes without caching capabilities, completing the edge collaborative caching of dynamic content popularity; otherwise, the vehicle nodes with caching capabilities or the edge servers of the roadside units immediately trigger the remote server, and the remote server directly transmits the requested content to the remaining vehicle nodes without caching capabilities, completing the edge collaborative caching of dynamic content popularity.

[0019] Furthermore, step 5) is specifically as follows:

[0020] 5.1. Utilize a dynamic content popularity model to generate content vectors, which are then used as cached content repositories. Vehicle nodes and roadside unit edge servers with caching capabilities are used as intelligent agents.

[0021] 5.2. The agent uses the content vector in the cache content library to perform multi-agent reinforcement learning; the agent is represented by state and action; its state s g Expressed as:

[0022] s g ={h,f i ,l,c r}

[0023] Where h represents the cache content of the g-th agent, f i is the cache content library, i is the content number in the cache content library, l is the location of the agent in the entire environment, c r is the remaining cache capacity;

[0024] Its action a g In binary, it is represented as:

[0025] a g ={m,c}

[0026] Where m = 1 means that the agent obtains the requested content through transmission between cluster heads, m = 0 means that the agent obtains the requested content through transmission between cluster heads and roadside units; c = 1 means that the requested content has been cached, c = 0 means that the requested content has not been cached;

[0027] 5.3. Use the Actor-Critic network to train all agents and enable each agent to make independent decisions;

[0028] The cache action A taken by the agent's Actor network g Denoted as:

[0029] A g =π(Og θ g )

[0030] Where, O g is the observation vector on the central controller, π is the decision set of the agent, θ g is the decision parameter of the agent;

[0031] The state-action function of the agent's Critic network Denoted as:

[0032]

[0033] 5.4. According to step 5.2, the agent is in state s g Take action a g Combined with the decision made in step 5.3, we get the reward function Υ t [s g ,a g ]for:

[0034]

[0035] Where G is the total delay reduction of cache content acquisition, plt is a negative constant, and c i is the size of the i-th content in the cache content library, r i is the decision vector, C max is the maximum cache capacity;

[0036] 5.5. Iterate the cumulative reward function to obtain the maximum reward value R, and use the maximum reward value R to obtain the edge cache decision plan;

[0037]

[0038] Where, δ t is the discount factor of the reward function at different stages, and maxE is the maximum mathematical expectation of the reward value.

[0039] Furthermore, in step 5.3, the parameter update method of the Actor-Critic network is the policy gradient algorithm, and the gradient of the policy gradient algorithm is for:

[0040]

[0041] Where, E s,a is the cumulative expected reward strategy gradient value, M is the experience replay pool, is the state action-function policy gradient of the Actor network, is the state-action function of the Actor network, μ g is the decision set.

[0042] Furthermore, step 2) is specifically as follows:

[0043] 2.1. Introduce the RNN model and introduce four gating units into the RNN model to obtain the LSTM model; the four gating units are the forget gate, input gate, state gate and output gate;

[0044] 2.2. Introduce the Attention model and calculate each past hidden state h in the Attention model based on the user's request content at different times t j With the current hidden state h t The similarity score e tj :

[0045] e tj =tanh(W j h j +U h h t-1 +b att )

[0046] Where, time j = 1, 2, ..., t-1; W j , U h with b att These are all parameters of the Attention model;

[0047] 2.3, Use the softmax function to score the similarity e tj Normalize and get the attention weight a tj :

[0048] a tj =softmax(e tj );

[0049] 2.4. According to the attention weight a tj Hidden state h j Perform weighted summation to obtain the context vector

[0050]

[0051] 2.5. Context vector Input into the LSTM model to obtain the LSTM-Attention network model.

[0052] Furthermore, step 3) is specifically as follows:

[0053] 3.1. Use the stochastic gradient descent algorithm to optimize the LSTM-Attention network model;

[0054] 3.2. Input the training set data into the optimized LSTM-Attention network model according to the five dimensions;

[0055] 3.3. Use the test set to test the optimized LSTM-Attention network model to obtain the dynamic content popularity model.

[0056] Furthermore, step 4) is specifically as follows:

[0057] 4.1. K-means clustering optimized by genetic algorithm;

[0058] 4.1.1. Based on vehicle speed and location, determine the vehicle nodes with caching capabilities and the remaining vehicle nodes, and minimize the sum of the distances from each vehicle to its cluster center to obtain the initial k clusters and k cluster centers.

[0059] The total distance E between the vehicle and the cluster center is:

[0060]

[0061] Where x k is the kth cluster position, x c is the cluster center location;

[0062] 4.1.2. The performance index of the genetic algorithm is measured by the fitness function, and its fitness value f s for:

[0063]

[0064] Where n represents the number of vehicles, τ V2V is the one-way communication delay between vehicles, τ V2I is the one-way communication delay between the vehicle and the roadside unit;

[0065] 4.1.3. The k cluster centers are used as individuals in the genetic algorithm population. After encoding and selecting the individuals in the population, a new population is obtained. The individuals in the new population are then selected, crossover, and mutated until the maximum number of iterations is reached or the fitness function reaches the optimal value, completing the K-means clustering optimization.

[0066] The individual x in the population i The selection probability and crossover probability P c , mutation probability P m They are:

[0067]

[0068] In the above formula, f(x k) is the fitness function value of individuals in the population, k1 is the crossover coefficient, k2 is the coefficient of variation, f s is the fitness value, f max For a given maximum fitness value, f s ′ is the maximum fitness value of individuals in the population;

[0069] 4.2. Based on vehicle speed and location, the optimized K-means clustering is used to obtain the cluster center and the number of clusters. The cluster center is used as the vehicle node with caching capability, and the number of clusters is used as the number of vehicle nodes with caching capability.

[0070] Furthermore, in step 3), the LSTM-Attention network model adopts the Adam optimizer.

[0071] Beneficial effects of the present invention:

[0072] 1. The present invention provides an edge collaborative caching method based on dynamic content popularity, which improves the user's experience of requesting content services in the Internet of Vehicles scenario. User-requested content does not rely on remote content servers. Instead, the requested content is pre-deployed in the edge servers equipped with roadside units. When users pass through the coverage area of ​​the corresponding roadside units, they can directly obtain the required content, which can effectively shorten transmission delays and save caching costs.

[0073] 2. The present invention proposes an edge collaborative caching method based on dynamic content popularity. The proposed K-means clustering algorithm based on genetic algorithm optimization can effectively improve the defect of the traditional K-means algorithm that cannot quickly determine the cluster center. This method has certain universality in multi-node scenario division, and the algorithm convergence speed is improved by 78% compared with the K-means algorithm.

[0074] 3. The present invention proposes an edge collaborative caching method based on dynamic content popularity, which takes into account the preferences of different users. The proposed dynamic content popularity prediction model can improve the problem that traditional content popularity cannot meet the content request needs of a wide range of Internet of Vehicles users. It considers data mining in historical request content information and effectively predicts user preferences, thereby providing better services.

[0075] 4. The present invention proposes an edge collaborative caching method based on dynamic content popularity and a dynamic content popularity model, which considers the vehicle nodes with caching capabilities and the edge servers of the roadside units as intelligent entities (i.e., caching nodes), which can better utilize the performance of the caching equipment within the limited caching capacity. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flow chart of an embodiment of an edge collaborative caching method based on dynamic content popularity of the present invention;

[0077] Figure 2 Schematic diagram of establishing an LSTM-Attention network model in an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiment is only a model embodiment of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0079] like Figure 1 As shown, an edge collaborative caching method based on dynamic content popularity includes the following steps:

[0080] 1) The data in Movielens is used as the training set, and the user request content collected by the vehicle is reprocessed and integrated as the test set; the data type and dimension of the training set and the test set are the same;

[0081] In this embodiment, the training set is mainly based on Movielens, which is often used in recommendation systems and data prediction scenarios. In this embodiment, the training set mainly uses the public data set of Movielens1M, which contains 1 million rating data of more than 6,000 users for 18 types of request content. The data set is processed to finally obtain a set of input data with five dimensions: user number, gender, age, timeline, and content type. The test set is mainly based on the user request content collected by the vehicle. The user request content collected by the vehicle is reprocessed and integrated as a test set with the same dimensions as the training set; wherein, the training set and test set account for 80% and 20% respectively, and the n vehicles in the training set and the test set are assigned random data for simulation. During the simulation, the loss function is measured by MSE (Mean-Square Error) and MAE (Mean Absolute Error), and its expression is:

[0082]

[0083] Among them, y pre Represents the predicted data, y tru Represents real data, and i represents the number of the user's requested content.

[0084] 2) If Figure 2 As shown, the LSTM-Attention network model is established, specifically:

[0085] Traditional caching strategies often use the Zipf model, derived from historical request content data, to estimate user preferences. However, as the number of users grows exponentially, content popularity changes rapidly. Because each user has their own preferences, estimating based on historical request content data will inevitably result in some user requests not being met, leading to increased network overhead. Therefore, we can predict user request content and convert the content they are likely to request into a Zipf-distributed content popularity. This information can then be cached on the device in advance, reducing network overhead.

[0086] In recent years, machine learning has shown absolute advantages in data or text prediction methods. Obviously, user needs are time-dependent and have different needs in different time periods. Therefore, it can be regarded as a data prediction problem on a time series.

[0087] Numerous studies have demonstrated that the RNN model (Recurrent Neural Network) exhibits excellent performance in time series prediction. The core of the RNN model lies in the introduction of a memory mechanism, enabling it to process sequence data and retain memory of past information. Specifically, the RNN model can share information between different time steps, thereby capturing contextual relationships and modeling time series data. However, while the RNN model performs well when processing sequence data of a certain length, it may suffer from long-term dependencies due to gradient issues when processing long-term series data.

[0088] In order to solve the defects in the RNN model, this embodiment adopts the LSTM (Long Short-Term Memory Neural Network) model to introduce four gate units into the RNN model to obtain the LSTM model; the LSTM model is a specially optimized RNN model, and its basic structure at time t is composed of the forget gate f t 、Input gate t , State Gate C t and output gate out t The expression of the optimized RNN model is as follows:

[0089] f t =σ(b ft +U ft +W ft h t-1 )

[0090] in t =σ(x t W Ct +b Ct +U Ct b t-1 )

[0091] C t =f t C t-1 +in t [tanh(b Ct +U Ct x t +W Ct h t-1 )]

[0092] o t =σ(b out +U out x t +W out h t-1 )

[0093] h t =out t tanh(C t )

[0094] Where b is the bias vector, h is the hidden layer vector, U is the weight matrix, W is the cyclic weight matrix, σ and tanh are activation functions, x is the search history vector, in is the input gate vector, C is the state gate vector, and out is the output gate vector.

[0095] The above introduces the basic structure of the LSTM model after improving the RNN model. However, for predicting user request content, the user's historical information has social attributes and can reflect the high correlation characteristics between user demand content in certain time periods. Although the LSTM model has good performance in long data sequences, it still lacks attention to certain highly correlated features in the data sequence. Therefore, the present invention introduces the Attention model to compensate for this shortcoming, extracting more important features from the sequence data to achieve higher prediction accuracy.

[0096] The working principle of the Attention model is as follows: First, calculate each past hidden state h in the Attention model j , where j = 1, 2, ..., t-1, and then calculate each past hidden state h j With the current hidden state h t The similarity score e tj :e tj =tanh(W j h j +U h h t-1 +b att ), where time j = 1, 2, ..., t-1; W j , U h with batt are all parameters of the Attention model; the similarity score etj is normalized by the softmax function to obtain the attention weight atj, which is expressed as atj = softmax(etj), and the past hidden state hj is weighted and summed to obtain the context vector Its expression is Finally, the context vector output by the Attention model Input into the LSTM model, specifically, modify the input gate to Update the LSTM model and obtain the LSTM-Attention network model.

[0097] 3) Use the training set to train the LSTM-Attention network model, and then use the test set to test the trained LSTM-Attention network model to obtain a dynamic content popularity model;

[0098] In this embodiment, the input vector of the LSTM-Attention model is represented as F = [F1, F2, ..., Fi], and the Stochastic Gradient Descent (SGD) algorithm is used to optimize the LSTM-Attention network model. The expression of stochastic gradient descent is: Where θ t is the parameter of the LSTM-Attention network model for the tth iteration model, η is the learning rate of the LSTM-Attention network model, is the training gradient of the loss function L; where the loss function L is expressed as the mean square error N is the number of data in the dataset.

[0099] In the LSTM-Attention network model, the first step is the input layer, which inputs the training set data according to five dimensions. The input data is weighted by the attention layer, and then the neural network is trained through the LSTM model, and finally the prediction result is output. The output vector after training is represented as the user's preference content vector f i =[f1,f2,...,f i ], its distribution is expressed as Zipf ε is the popularity coefficient of the preferred content, For the preferred content f i Hot spot coefficient.

[0100] In this embodiment, the training and testing of the LSTM-Attention model is to divide the data set in step 1) into 100 consecutive time periods according to the time series, with a total of 50,000 user rating information. The LSTM-Attention network model uses the Adam optimizer, and the regularization parameter in the LSTM-Attention network model is 10 -5 , the dropout parameter is 0.2, the number of hidden layer nodes is 50, the data feature dimension is 5, the batch size is 20, and the number of cross-validations is 10. The prediction effect is evaluated using MSE and MAE. Finally, the MSE and MAE evaluation prediction results are compared with the RNN model and the LSTM model without attention weights, see Table 1:

[0101] Table 1

[0102]

[0103] To verify whether integrating the Attention model into the LSTM-Attention network model improves prediction performance, we randomly selected data with request type number 5 to predict the number of content requests over a time series (i.e., Event 1) and predicted the types of content requests from different users over a fixed time period (i.e., Event 2). The LSTM-Attention network model was trained and validated. As shown in Table 1, for Event 1, the LSTM-Attention network model achieved an accuracy of 73.75%, an improvement of 8.67% and 3.78% over the basic RNN model and LSTM model, respectively. For Event 2, the LSTM-Attention network model achieved an 86% prediction success rate, an improvement of 26% and 17% over the RNN model and LSTM model, respectively. This is because the time series in large datasets are long, and the Attention model can extract the influence of different factors on the results in long sequence data, thereby obtaining key information from the long sequence data. Table 2 shows the MAE and MSE comparisons of different models for Events 1 and 2.

[0104] Table 2

[0105] Model MSE MAE RNN model (Event 1) 86.45 10.67 RNN model (Event 2) 109.98 1.11 LSTM model (Event 1) 37.71 4.91 LSTM model (Event 2) 45.17 0.68 LSTM-Attention Network Model (Event 1) 5.4 1.72 LSTM-Attention Network Model (Event 2) 11.84 0.22

[0106] The optimized LSTM-Attention model is tested on the test set to obtain the dynamic content popularity model in this embodiment.

[0107] 4) Cluster the vehicles to obtain the cluster center and the number of clusters. The cluster center is used as the vehicle node with caching capability, and the number of clusters is used as the number of vehicle nodes with caching capability.

[0108] The traditional K-means clustering algorithm requires a predetermined number of clusters, k. However, in practice, scenarios often vary randomly, making it quite difficult to determine the right number of clusters, k. Furthermore, the K-means algorithm requires random initial cluster centers, and different initial cluster centers can lead to completely different clustering results. Therefore, simply using the K-means algorithm can lead to illogical results.

[0109] To improve the above phenomenon, the vehicle clustering in the present invention adopts an improved K-means clustering algorithm, namely, K-means clustering optimized by genetic algorithm. According to the vehicle speed and position, the vehicle nodes with caching capability (Caching Vehicles, CVs) and the remaining vehicle nodes in the cluster (Requesting Vehicles, RVs) are determined, and the sum of the distances from the vehicle to its cluster center is minimized to obtain the initial k clusters and k cluster centers.

[0110] In this embodiment, the K-means clustering optimization goal of the genetic algorithm is to calculate the sample x k With cluster center x c Total distance E:

[0111]

[0112] Where x k is the kth vehicle position, x c is the cluster center location;

[0113] Considering that the main indicator in the clustering scenario is the rationality of communication performance, the fitness function in the genetic algorithm is designed based on the communication delay between the vehicle and its corresponding roadside unit (RSU). If the vehicle does not communicate with the RSU, the delay is 2τ V2V , otherwise, 2τ V2V +τ V2I Considering that the fitness function needs to take a larger value, the performance index of the genetic algorithm is used to measure the fitness function, and its fitness value is:

[0114]

[0115] Where n represents the number of vehicles, τ V2V is the one-way communication delay between vehicles, τ V2I is the one-way communication delay between the vehicle and the roadside unit;

[0116] The specific K-means clustering algorithm improved based on genetic algorithm is as follows:

[0117] First, the initial cluster center is randomly selected and the first K-means clustering is performed. Then, the difference in the number of clusters in the first K-means clustering is compared with the threshold ψ1. If the difference in the number of clusters is less than or equal to the threshold ψ1, the clustering result is maintained. Otherwise, the K-means clustering is continued until the number of clusters k is obtained.

[0118] Set the maximum number of initialization iterations, population size, and crossover probability P c , mutation probability P m and iteration threshold ψ2; encode individual chromosomes in the population to generate the initial population;

[0119] Calculate the total distance function and recalculate the cluster centers;

[0120] Calculate the fitness function of individuals in the initial population based on the total distance function and cluster center;

[0121] Determine whether the fitness value of each individual is greater than or equal to the threshold ψ2. If so, terminate the iteration, that is, select the best individual in the population as the cluster center; otherwise, recalculate the total distance function and cluster center, perform selection, crossover, and mutation operations, and iterate the above steps until the maximum number of iterations is reached or the fitness function reaches the optimal value, completing the K-means clustering optimization.

[0122] The k cluster centers are individuals in the genetic algorithm population. The individuals in the population are coded and their fitness is calculated. The individuals in the population are selected, and the selection probability p(x k ), crossover probability P c and mutation probability P m They are:

[0123]

[0124] In the above formula, f(x k ) is the fitness function value of individuals in the population, k1 is the crossover coefficient, k2 is the coefficient of variation, f s is the fitness value, f max For a given maximum fitness value, f s ′ is the maximum fitness value of individuals in the population;

[0125] According to the above steps, the cluster center and the number of clusters are obtained by optimizing K-means clustering based on the vehicle speed and position. The cluster center is used as the vehicle node with caching capability, and the number of clusters is used as the number of vehicle nodes with caching capability.

[0126] 5) Using a dynamic content popularity model to obtain content vectors, the content vectors are used as cache content repositories. Caching-capable vehicle nodes and roadside unit edge servers are used as agents. These agents use the content vectors in the cache content repositories to perform multi-agent reinforcement learning to obtain edge caching decision solutions.

[0127] The multi-agent reinforcement learning method mainly includes the state space, action space, reward function, actor network and critic network of the agent. The strategy of each agent depends not only on its own strategy and environmental feedback, but also on the actions and cooperation of other agents. Different agents will cause certain changes in the entire environment. In this embodiment, an actor-critic network (Actor-Critic) is adopted. The actor network of each agent is independent, and each actor network corresponds to a critic network. All critic networks are in a central controller. After the central controller centrally trains all critic networks, each agent makes independent decisions to obtain an actor-critic network. Its core principle is to allow all agents to learn cooperation and competition to achieve mutual benefit.

[0128] The specific process of multi-agent reinforcement learning is:

[0129] The content vector is obtained by using the dynamic content popularity model and used as the cache content library; the vehicle node and the edge server of the roadside unit with caching capability are used as the intelligent agent; the intelligent agent can only observe its own request content and the cache status of the surrounding RSUs and CVs, and its state s g It can be expressed as:

[0130] s g ={h,f i ,l,c r}

[0131] Where h represents the cache content of the g-th agent, f i is the cache content library, i is the content number in the cache content library, l is the location of the agent in the entire environment, c r is the remaining cache capacity;

[0132] Its action a g In binary, it is represented as:

[0133] a g ={m,c}

[0134] Where m = 1 means that the agent obtains the requested content through transmission between cluster heads, m = 0 means that the agent obtains the requested content through transmission between cluster heads and roadside units; c = 1 means that the requested content has been cached, and c = 0 means that the requested content has not been cached.

[0135] Use the Actor-Critic network to train all agents and make each agent make decisions independently; the cache action A taken by the agent in the Actor network g Denoted as:

[0136] A g =π(O g θ g )

[0137] Where, O g is the observation vector on the central controller, π is the decision set of the agent, θ g is the decision parameter of the agent;

[0138] The state-action function of the agent's critic network Denoted as:

[0139]

[0140] The state-action output of the Critic network is then sent to the corresponding agent's Actor network for parameter training. During training, the Critic network collects global observations and actions. During execution, the Actor network independently makes the optimal action without reference to the actions or states of other agents.

[0141] The agent is in state s g Take cache action a g After that, the reward function is obtained to reflect whether the total delay is reduced after caching, which is expressed as t [s g ,a g ]for:

[0142]

[0143] Where G is the total delay reduction of cache content acquisition, plt is a negative constant, and c i is the size of the i-th content in the cache content library, r i is the decision vector, C max is the maximum cache capacity;

[0144] The maximum reward value R is obtained by iteratively accumulating the reward function, and the edge cache decision plan is obtained using the maximum reward value R;

[0145]

[0146] Where, δ t is the discount factor of the reward function at different stages, and maxE is the maximum mathematical expectation of the reward value.

[0147] The maximum reward value R is used to obtain the edge cache decision solution. In this embodiment, the parameter update method of the Actor-Critic network is the policy gradient algorithm, and the cumulative expected reward is expressed as the policy gradient

[0148]

[0149] Where, E s,a is the cumulative expected reward strategy gradient value, M is the experience replay pool, is the state action-function policy gradient of the Actor network, is the state-action function of the Actor network, μ g is the decision set.

[0150] The Actor-Critic network minimizes the loss function To update the parameters, the target reward function value in, is the state-action function of the Critic network, r g is the initial reward function value of the agent, and γ is the discount factor of the Critic network.

[0151] 6) Input the edge caching decision plan into the vehicle nodes with caching capabilities and the remaining vehicle nodes without caching capabilities respectively;

[0152] When a vehicle node with caching capabilities requests content, the remote server directly feeds the requested content back to the vehicle node with caching capabilities, completing edge collaborative caching of dynamic content popularity.

[0153] When the remaining vehicle nodes without caching capabilities request content, the vehicle nodes with caching capabilities and the edge servers of the roadside units determine whether the requested content has been cached. If it has been cached, the requested content is directly fed back to the remaining vehicle nodes without caching capabilities, completing the edge collaborative caching of dynamic content popularity; otherwise, the vehicle nodes with caching capabilities or the edge servers of the roadside units immediately trigger the remote server, and the remote server directly transmits the requested content to the remaining vehicle nodes without caching capabilities, completing the edge collaborative caching of dynamic content popularity.

[0154] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention shall be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. An edge collaborative caching method based on dynamic content popularity, characterized in that , including the following steps: 1) Using the data in Movielens as a training set, and reprocessing and integrating the user request content collected by the vehicle as a test set; the data types and dimensions of the training set and the test set are the same; 2) Establish an LSTM-Attention network model; 3) Use the training set to train the LSTM-Attention network model, and then use the test set to test the trained LSTM-Attention network model to obtain a dynamic content popularity model; 4) Cluster the vehicles to obtain the cluster center and the number of clusters. The cluster center is used as the vehicle node with caching capability, and the number of clusters is used as the number of vehicle nodes with caching capability. 5) Using a dynamic content popularity model to obtain content vectors, the content vectors are used as cache content repositories. Caching-capable vehicle nodes and roadside unit edge servers are used as agents. These agents use the content vectors in the cache content repositories to perform multi-agent reinforcement learning to obtain edge caching decision solutions. 6) Input the edge caching decision plan into the vehicle nodes with caching capabilities and the remaining vehicle nodes without caching capabilities respectively; When a vehicle node with caching capabilities requests content, the remote server directly feeds the requested content back to the vehicle node with caching capabilities, completing edge collaborative caching of dynamic content popularity. When the remaining vehicle nodes without caching capabilities request content, the vehicle nodes with caching capabilities and the edge servers of the roadside units determine whether the requested content has been cached. If it has been cached, the requested content is directly fed back to the remaining vehicle nodes without caching capabilities, completing the edge collaborative caching of dynamic content popularity; otherwise, the vehicle nodes with caching capabilities or the edge servers of the roadside units immediately trigger the remote server, and the remote server directly transmits the requested content to the remaining vehicle nodes without caching capabilities, completing the edge collaborative caching of dynamic content popularity.

2. According to claim 1, a method for edge collaborative caching based on dynamic content popularity is characterized in that ,Step 5) is specifically as follows: 5.

1. Utilize a dynamic content popularity model to generate content vectors, which are then used as cached content repositories. Vehicle nodes and roadside unit edge servers with caching capabilities are used as intelligent agents. 5.

2. The agent uses the content vector in the cache content library to perform multi-agent reinforcement learning; the agent is represented by state and action; its state s g Expressed as: s g ={h,f i ,l,c r } Where h represents the cache content of the g-th agent, f i is the cache content library, i is the content number in the cache content library, l is the location of the agent in the entire environment, c r is the remaining cache capacity; Its action a g In binary, it is represented as: a g ={m,c} Where m = 1 means that the agent obtains the requested content through transmission between cluster heads, m = 0 means that the agent obtains the requested content through transmission between cluster heads and roadside units; c = 1 means that the requested content has been cached, c = 0 means that the requested content has not been cached; 5.

3. Use the Actor-Critic network to train all agents and enable each agent to make independent decisions; The cache action A taken by the agent's Actor network g Denoted as: A g =π(O g ;θ g ) Where, O g is the observation vector on the central controller, π is the decision set of the agent, θ g is the decision parameter of the agent; The state-action function of the agent's Critic network Denoted as: 5.

4. According to step 5.2, the agent is in state s g Take action a g Combined with the decision made in step 5.3, we get the reward function Υ t [s g ,a g ]for: Where G is the total delay reduction of cache content acquisition, plt is a negative constant, and c i is the size of the i-th content in the cache content library, r i is the decision vector, C max is the maximum cache capacity; 5.

5. Iterate the cumulative reward function to obtain the maximum reward value R, and use the maximum reward value R to obtain the edge cache decision plan; Where, δ t is the discount factor of the reward function at different stages, and maxE is the maximum mathematical expectation of the reward value.

3. The edge collaborative caching method based on dynamic content popularity according to claim 2, characterized in that: In step 5.3, the parameter update method of the Actor-Critic network is the policy gradient algorithm, and the gradient of the policy gradient algorithm is for: Where, E s,a is the cumulative expected reward strategy gradient value, M is the experience replay pool, is the state action-function policy gradient of the Actor network, is the state-action function of the Actor network, μ g is the decision set.

4. The edge collaborative caching method based on dynamic content popularity according to claim 3 is characterized in that ,Step 2) is specifically as follows: 2.

1. Introduce the RNN model and introduce four gating units into the RNN model to obtain the LSTM model; the four gating units are the forget gate, input gate, state gate and output gate; 2.

2. Introduce the Attention model and calculate each past hidden state h in the Attention model based on the user's request content at different times t j With the current hidden state h t The similarity score e tj : e tj =tanh(W j h j +U h h t-1 +b att ) Where, time j = 1, 2, ..., t-1; W j , U h with b att These are all parameters of the Attention model; 2.3, Use the softmax function to score the similarity e tj Normalize and get the attention weight a tj : the tj =softmax(e tj ); 2.

4. According to the attention weight a tj Hidden state h j Perform weighted summation to obtain the context vector 2.

5. Context vector Input into the LSTM model to obtain the LSTM-Attention network model.

5. The edge collaborative caching method based on dynamic content popularity according to claim 1 is characterized in that ,Step 3) is specifically as follows: 3.

1. Use the stochastic gradient descent algorithm to optimize the LSTM-Attention network model; 3.

2. Input the training set data into the optimized LSTM-Attention network model according to the five dimensions; 3.

3. Use the test set to test the optimized LSTM-Attention network model to obtain the dynamic content popularity model.

6. The edge collaborative caching method based on dynamic content popularity according to claim 1 is characterized in that ,Step 4) is specifically as follows: 4.

1. K-means clustering optimized by genetic algorithm; 4.1.

1. Based on vehicle speed and location, determine the vehicle nodes with caching capabilities and the remaining vehicle nodes, and minimize the sum of the distances from each vehicle to its cluster center to obtain the initial k clusters and k cluster centers. The total distance E between the vehicle and the cluster center is: Where x k is the kth cluster position, x c is the cluster center location; 4.1.

2. The performance index of the genetic algorithm is measured by the fitness function, and its fitness value f s for: Where n represents the number of vehicles, τ V2V is the one-way communication delay between vehicles, τ V2I is the one-way communication delay between the vehicle and the roadside unit; 4.1.

3. The k cluster centers are used as individuals in the genetic algorithm population. After encoding and selecting the individuals in the population, a new population is obtained. The individuals in the new population are then selected, crossover, and mutated until the maximum number of iterations is reached or the fitness function reaches the optimal value, completing the K-means clustering optimization. The individual x in the population i The selection probability and crossover probability P c , mutation probability P m They are: In the above formula, f(x k ) is the fitness function value of individuals in the population, k1 is the crossover coefficient, k2 is the coefficient of variation, f s is the fitness value, f max For a given maximum fitness value, f s ′ is the maximum fitness value of individuals in the population; 4.

2. Based on vehicle speed and location, the optimized K-means clustering is used to obtain the cluster center and the number of clusters. The cluster center is used as the vehicle node with caching capability, and the number of clusters is used as the number of vehicle nodes with caching capability.

7. The edge collaborative caching method based on dynamic content popularity according to claim 1, characterized in that: In step 3), the LSTM-Attention network model uses the Adam optimizer.

Images