A collaborative edge caching method based on asynchronous federation and deep reinforcement learning

Abstract

The present invention discloses a collaborative edge caching method based on asynchronous federation and deep reinforcement learning. The method first uses an asynchronous federated learning framework to cache edge content, so that vehicle users' data can be trained locally, thereby reducing the privacy risks of vehicle users, reducing communication costs, and adapting to the highly dynamic vehicle network environment. Secondly, an autoencoder model is used to predict content popularity. Each vehicle user uses a content popularity prediction algorithm based on an autoencoder to learn the potential correlations in the data from the vehicle user's historical request content and context information, thereby improving the caching performance of each edge device. Finally, a dueling DQN reinforcement learning algorithm is used to learn the requested content data of the vehicle user in each edge device. Based on the content popularity predicted by the autoencoder model, the dueling DQN can make the optimal caching decision, reduce the average request latency of the vehicle, and improve the caching performance of each edge device.

Description

Technical Field

[0001] The present invention belongs to the field of deep reinforcement learning technology for vehicle network federation, and specifically relates to a vehicle edge caching method based on asynchronous federation and deep reinforcement learning. Background Art

[0002] Autonomous vehicles support a range of in-vehicle applications, such as automated cruise control, image recognition, and multimedia entertainment. These applications require users to request data, videos, web pages, and other content. Due to the high speeds of vehicles, users must obtain the requested content quickly. Traditionally, users first communicate with a base station, then access the data center storing the data through the core backbone network. The data center then transmits the requested data back to the user. This request method suffers from long end-to-end latency, limited backhaul bandwidth, and inefficient redundant transmission. Vehicle Edge Computing (VEC) effectively addresses these issues. This technology caches data in the roadside unit (RSU) closest to the user. The RSU acts as an edge server, providing computing and storage services. Users can then communicate with the RSU within a single hop to obtain the requested content. This reduces backhaul network burden and service latency, thereby improving the application's Quality of Service (QoS).

[0003] Due to the limited storage capacity of edge servers, caching the most popular content is a major challenge for edge caching. Traditional caching solutions cache content based on user requests, meaning that content is cached only after being requested by a user. However, in the complex, high-speed, in-vehicle environment, content previously requested by users quickly becomes outdated, and content popularity fluctuates dynamically, making traditional request-based caching solutions incapable of meeting user needs. Machine learning (ML) technology enables edge servers to extract hidden features from user data to effectively predict content popularity and cache predicted popular content. This allows even content not requested by users to be cached, thereby achieving dynamic management of cached content. Based on an RL framework, the network can dynamically and efficiently determine the optimal caching strategy based on the content information requested by in-vehicle users and the content popularity at each RSU, thereby optimizing content delivery. RSUs collect a large amount of in-vehicle user data, update the model using training data, and use the converged model to predict content popularity. However, data in in-vehicle applications often involves personal privacy information, and users are often reluctant to share this data with each other, making data collection difficult for RSUs.

[0004] Federated learning (FL) can address privacy challenges. Specifically, each user trains on their local data to update the local model. The RSU then aggregates each user's local model to update the global model. Each user then uses the updated global model to update the local model. This process iterates continuously until the global model converges. Compared to other ML methods, RSU aggregates users' local models instead of collecting user data. This allows the global model to be updated while avoiding the sharing of data containing private information among multiple users. The RSU needs to wait for all users to send their updated models before aggregation. If any user delays transmitting the updated model to the RSU, such users are called stragglers, which will increase the convergence time of the global model. When a user's update time exceeds the maximum time threshold, the RSU is forced to abandon the user's model. Even if the RSU receives too few user models, it must abandon the received model and restart the next round of updates. These situations may ultimately make it impossible to obtain an accurate global model. In a vehicular environment, vehicles move at high speeds and pass through the coverage of the RSU at a very fast speed. Once the vehicle leaves the coverage area, the model updated by the onboard user will become invalid, which can easily form stragglers, making it difficult to obtain an accurate model based on synchronous federated learning methods. Summary of the Invention

[0005] Purpose of the invention: In order to overcome the shortcomings of the existing technology, the present invention provides a vehicle edge caching method based on asynchronous federation and deep reinforcement learning, which solves the above-mentioned technical problems.

[0006] Technical Solution: The vehicle edge caching method based on asynchronous federation and deep reinforcement learning described in the present invention uses a communication network architecture: the top layer is the core network and macro base stations (MBSs), the middle layer includes multiple roadside units (RSUs) that act as edge servers to cache content, and the bottom layer includes vehicles and vehicle request content. The method includes the following steps:

[0007] (1) There are multiple vehicles within the coverage of an RSU. A motion model based on vehicle speed is calculated in each communication round. In the motion model, all vehicles are moving in the same direction. When a vehicle passes through the coverage of different RSUs, it connects to the RSU it is communicating with at the current moment and then transfers to the next adjacent RSU.

[0008] (2) Obtain the wireless transmission rate between the RSU and the mobile vehicles within its coverage at time t based on the channel gain of each vehicle in a communication round, where the mobile vehicles are served by the MBS;

[0009] (3) The vehicle obtains the requested content, thereby obtaining the content request delay of all vehicles in a certain communication round, and describes the collaborative caching strategy problem based on the request delay;

[0010] (4) Applying an asynchronous federated learning framework to aggregate the local models from each vehicle in a highly dynamic IoV environment to obtain a global model, and then training an efficient and convergent global model through multiple iterations;

[0011] (5) Use the global model to predict the popular content of each vehicle user and perform aggregate comparison to obtain multiple contents with the highest content popularity;

[0012] (6) Dueling DQN is used to solve the collaborative caching strategy problem and determine which of the multiple most popular contents should be proactively cached in the appropriate RSU.

[0013] Further, including:

[0014] The step (1) specifically includes:

[0015] In the rth communication round, there are N r Mobile vehicles Assume that the arrival of vehicles follows a Poisson distribution, and its arrival rate λ v ; In each communication round, the speed of the vehicle follows an independent and identical distribution, which is represented by the set express;

[0016] Consider the speed limit at [U min ,U max ], that is, the vehicles V within the coverage area of the RSU i r Driving speed is subject to minimum or maximum speed limits, i.e. Assume that in the rth communication round, the vehicle's speed Generated from a truncated Gaussian distribution:

[0017]

[0018] Among them, σ 2 is the variance, μ(-∞<μ<+∞) is the mean, and erf() is the Gaussian error function.

[0019] Further, including:

[0020] Step (2) specifically includes:

[0021] In the rth round of communication, the vehicle V i r The channel gain is modeled as:

[0022]

[0023] in, and represent channel path loss and path shadowing, respectively;

[0024] Since the vehicles V i r Different distances from the RSU will cause different channel impairments for vehicles, resulting in different data transmission rates at different times. Therefore, in r communication rounds, the communication between the RSU and the mobile vehicles V within its coverage area is i r The wireless transmission rate between

[0025]

[0026] in, Is the vehicle V i r allocated bandwidth, Is the vehicle V i r The transmission energy, is the noise energy, if V i r MBS provides services and defines the wireless transmission rate at time t as

[0027]

[0028] Among them, p T is the transmission energy of MBS.

[0029] Further, including:

[0030] The step (3) comprises the following steps:

[0031] (31) The content requested by the vehicle is obtained in the following three ways: first, the vehicle obtains the cached content from the local RSU serving it;

[0032] If the content requested by the vehicle is cached in the local RSU, the local RSU directly delivers the content to the vehicle, allowing represents the delay for all vehicles to obtain the requested content of all vehicles from the local RSU in the rth round of communication;

[0033]

[0034] Where Nr is the number of vehicles participating in content request in the rth round of communication, F i Is the vehicle V ir The number of requested contents, is a binary variable indicating whether the vehicle obtains the requested content from the local RSU serving it in the rth round of communication. Indicates that in the rth round of communication, the vehicle obtains the requested content from the local RSU, otherwise, In the rth round of communication, each RSU takes the content transmission delay transmitting the requested content from the local cache to the vehicle;

[0035]

[0036] s f is the size of the content f;

[0037] Second, the vehicle obtains the requested content from other adjacent RSUs;

[0038] If the content requested by the vehicle is not cached in the local RSU, the vehicle needs to obtain the requested content from other adjacent RSUs;

[0039] In the rth round of communication, the delay time for all vehicles to obtain the request content from the adjacent RSU is express:

[0040]

[0041] in, is a binary variable indicating whether the vehicle obtains the requested content from the neighboring RSUs in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the neighboring RSUs, otherwise, is the transmission delay between two RSUs:

[0042]

[0043] Among them, R R-R represents the transmission rate between two RSUs;

[0044] Finally, the vehicle obtains the requested content from the MBS: If the requested content is not cached in the local RSU and neighboring RSUs, the vehicle needs to obtain the requested content from the MBS; In the rth round of communication, the time delay for all vehicles to obtain the requested content from the MBS is:

[0045]

[0046] in, is a binary variable indicating whether the vehicle obtains the requested content from the MBS in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the MBS, otherwise,

[0047] In the communication round t, V i r The transmission delay between MBS To indicate that:

[0048]

[0049] in, It is V i r and the transmission rate between MBS;

[0050] (32) In the rth communication round, for a given cache state in the local RSU, the content request latency of all vehicles can be expressed as:

[0051]

[0052] (33) Considering the limited storage space of RSU and the integer characteristics of cache decision variables, the collaborative caching problem is described as follows:

[0053]

[0054]

[0055]

[0056] The constraint is to ensure that the amount of content cached locally in each RSU cannot exceed its maximum storage capacity.

[0057] Further, including:

[0058] The step (4) specifically includes the following steps:

[0059] (41) Vehicle selection: The factor considered in the vehicle selection process is the time the vehicle stays within the RSU coverage area during the driving process. In the tth communication round, the time each connected vehicle stays within the current RSU coverage area is as follows:

[0060]

[0061] Among them, P i r In the tth communication round, vehicle V i r Distance from RSU entrance, L s is the coverage size of RSUs.

[0062] Assume that the average training time and test time per round of communication is T training and T inference ,if This indicates that the vehicle meets the requirements for participating in asynchronous federated learning and has been selected for asynchronous federated learning model training.

[0063] (42) Global model download: In the rth round of communication, the selected vehicle participates in the asynchronous federated learning model training; the selected vehicle downloads the global model from the current RSU. In each communication round, the RSU will save the model of the previous communication vehicle participating in the asynchronous federated learning training and update the model based on it;

[0064] (43) Asynchronous Federated Learning Model Training: Definition The data stored by the selected vehicles in each round of communication, N r is the total number of vehicles within the RSU coverage area in the rth communication round, It is The data of a vehicle is of length d is the sum of the data stored in all selected vehicles, that is, Among them, d i Is the vehicle V i r The length of the stored data;

[0065] The minimization global objective function F(ω) is defined as:

[0066]

[0067] Among them, f i (ω i ) is the vehicle V i r The local loss function is:

[0068]

[0069] Among them, l i (x k ;ω i ) is the empirical local loss function, x k is a data sample, ω i are the local model parameters of the vehicle; a gradient-based update method with a regularized loss function is used, defined as follows:

[0070]

[0071] Among them, ρ is the regularization parameter, ω r is the global model parameter for the rth communication round, ω iIs the vehicle V i r The local model parameters are optimized using stochastic gradient descent:

[0072]

[0073] in, is the local learning rate of the local parameters in the rth communication round, and

[0074]

[0075] Among them, η l is the initial value of the local learning rate;

[0076] When aggregating local models, using the rth communication round, after aggregation V i r Local Model:

[0077]

[0078] in, Is the vehicle V i r trained local model parameters, Is the vehicle V i r The previous local model parameters, β is the attenuation coefficient;

[0079] (44) Upload the updated model: After the local model training is completed, the local model of the vehicle is uploaded from the vehicle user to the RSU server;

[0080] (45) The RSU server and the vehicle perform asynchronous updates, that is, in the rth round of communication, the RSU server receives the data from the vehicle V i r The local model of , and update the global model by weighted averaging,

[0081]

[0082] χ i Is the vehicle V i r Parameters for weight aggregation:

[0083]

[0084] Among them, μ1+μ2=1, is the vehicle V in the rth communication round i r The position weight depends on the vehicle V i r location; is the vehicle V in the rth communication round i r The data transmission weight depends on the vehicle V i r The data transmission rate is 100%, and after multiple iterative updates, a more efficient and convergent global model is trained.

[0085] Further, including:

[0086] Predicting the popular content of a certain vehicle user in step (5) specifically includes the following steps:

[0087] (51) Data preprocessing: Based on the content request history of each connected vehicle user, a vehicle request content scoring matrix X is established;

[0088] (52) Data feature expression: The score matrix X is used as the input data of the autoencoder. The autoencoder finds the potential features of the correlation between vehicle users and request contents. Considering the potential features and vehicle user information, the similarity matrix of vehicle users and file content is calculated respectively.

[0089] Construction of historical request matrix: Assuming that the current vehicle is an active vehicle, based on the similarity matrix of the current vehicle user, determine the K adjacent vehicle users of the vehicle user. The historical request contents of the K selected adjacent vehicle users are combined with the historical request contents of the current vehicle to construct a historical retrieval matrix K * ;

[0090] (53) Obtaining similarity: The historical request matrix of the current vehicle user is called A * , calculate A through the document similarity matrix * With K * The mean of similarities between

[0091] (54) Content popularity aggregation: Fc contents with the highest similarity are selected as recommended contents for the vehicle user. Then, the aggregation method of predicted file popularity is executed to generate a recommendation list of popular files for content caching. Each connected vehicle user uploads their recommendation list to the RSU server. After receiving the recommendation list, the RSU server aggregates and compares the recommendation lists of all vehicle users who uploaded the list and selects Fc contents with the highest content popularity.

[0092] Further, including:

[0093] Step (6) includes: After obtaining the Fc-popular content, the next challenge is to determine which of the Fc contents should be actively cached in the appropriate RSU before each time period. Based on DQN, dueling DQN decomposes the action-state value function Q into a state value function V and an action advantage function A, forming a dueling DQN neural network. The state value function V is independent of the action, and the action advantage function A is related to the action. It is the average return for executing action a(t) in state s(t) to solve the reward bias problem. The action Q-value function is written as:

[0094] Q(s(t), a(t); θ) = V(s(t); θ) + A(s(t), a(t); θ)

[0095] The action advantage is usually set as the single action advantage function minus the average of all action advantage functions in a specific state. The action Q-value function is calculated as follows:

[0096]

[0097] Therefore, dueling DQN is used to solve the content caching problem. The state space, action space, and reward function are described as follows:

[0098] State space s(t): The state space s(t) is defined as s(t) = (s1, s2,..., s c ), where s(t) represents the specific content cached in the local RSU, and c is the cache capacity of the RSU;

[0099] Action space a(t): The action space represents whether the cached content in the RSU needs to be replaced. The popular content that is not cached in the local RSU belongs to the set κ. If a(t) = 1, k (k < c) contents are randomly selected from the set κ and exchanged with the k contents with the lowest index cached in the local RSU, and then the content indices in the state space are sorted in descending order based on content popularity to obtain s(t + 1). In this way, it can be ensured that at time t, the content replaced in the local RSU is the least popular content. a(t) = 0 means that the content cached in the local RSU will not be replaced;

[0100] Reward function r(t): When the local RSU cache state is s(t), the reward function r(t) will be obtained after executing action a(t); To minimize the total content request delay and system reward, the reward function is designed as follows:

[0101]

[0102] At time t, vehicle Vi r Request a reward for content f;

[0103]

[0104] λ1+λ2+λ3=1,λ1<λ2≤λ3, is the reward for the vehicle to obtain the requested content f from the local RSU; is the reward for the vehicle to obtain the requested content f from the neighboring RSU; is the reward for the vehicle to obtain the requested content f from the MBS. The RSU optimizes the caching strategy π* by maximizing the reward function r(t) based on the local content popularity and the vehicle’s requested content information.

[0105] Beneficial Effects: Compared with existing technologies, this invention offers the following significant advantages: 1. It uses an asynchronous federated learning framework for edge content caching, enabling local training of vehicle user data, thereby reducing privacy risks, lowering communication costs, and adapting to highly dynamic vehicle network environments. 2. It uses an autoencoder model to predict content popularity. Each vehicle user uses an autoencoder-based content popularity prediction algorithm to learn potential correlations in data from historical vehicle user requests and contextual information, thereby improving the caching performance of each edge device. 3. It uses a dueling DQN reinforcement learning algorithm to learn the vehicle user's requested content data on each edge device. Based on the content popularity predicted by the autoencoder model, the dueling DQN can make optimal caching decisions, reducing average vehicle request latency and improving the caching performance of each edge device. 4. The collaborative caching scheme (CAFR) based on asynchronous federation and deep reinforcement learning proposed in this invention significantly improves edge caching performance, protects user privacy, reduces average vehicle request latency, and reduces communication costs. Experimental results show that CAFR outperforms other baseline caching schemes in cache hit rate and average vehicle request latency in highly dynamic vehicle network environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 Schematic diagram comparing the cache hit rate of the caching method of the present invention and the other three algorithms when the RSU cache capacity is 50-400;

[0107] Figure 2 This is a comparative diagram of the average request delay per vehicle when the RSU cache capacity is 50-400 when the caching method of the present invention and the other three algorithms are used;

[0108] Figure 3 A schematic diagram comparing the cache hit rate and the average request latency per vehicle of the cache method of the present invention under different vehicle densities;

[0109] Figure 4 A graph showing the relationship between cache hit rate and communication rounds for the caching method of the present invention and a typical synchronous federated learning training (FedAVG);

[0110] Figure 5 A schematic diagram comparing the training time of each communication round between the caching method of the present invention and FedAVG;

[0111] Figure 6 This is a graph showing the relationship between the cache hit rate of each episode and the average vehicle request latency in the cache replacement strategy based on dueling DQN in the caching method of the present invention;

[0112] Figure 7 Schematic diagram showing a comparison of cache hit rates between the caching method of the present invention and the caching strategy after removing deep reinforcement learning (DRL);

[0113] Figure 8 A schematic diagram comparing request latency between the caching method of the present invention and the caching strategy after removing DRL;

[0114] Figure 9 This is an architectural diagram of the caching method described in the present invention. DETAILED DESCRIPTION

[0115] The collaborative edge caching solution for asynchronous federation and deep reinforcement learning of the present invention is further described in detail below in conjunction with the drawings and examples of the specification. The implementation methods of the present invention include but are not limited to the following examples.

[0116] The present invention is based on a collaborative edge caching solution based on asynchronous federation and deep reinforcement learning. The specific steps are as follows:

[0117] Step (1): The solution is a three-tier architecture, such as Figure 9 As shown in the figure, the top layer includes the core network and MBS; the middle layer includes RSUs, which act as edge servers to cache content; and the bottom layer includes vehicles and vehicle-requested content. These three layers are based on an asynchronous federation and deep reinforcement learning framework. Vehicles download the global model from the RSU server, calculate local models using local data, and send them to the RSU. After receiving the local model from a vehicle, the RSU aggregates and updates the global model without waiting for other vehicles to complete model training. The RSU edge server trains a converged global model through multiple iterations. The content requested by the local vehicles participating in the training serves as the input of the global model, and the output is the predicted content popularity.

[0118] Sort by predicted content popularity and get F cThe Dueling DQN network determines which content to proactively cache on appropriate edge devices before each time period based on the vehicle's requested content and the predicted content popularity at each RSU. However, the high mobility of vehicles may result in insufficient time to complete the above steps due to their high speed and limited RSU coverage. Therefore, the vehicle may require assistance from neighboring RSUs to obtain the requested content from the edge device. Otherwise, the vehicle must retrieve the requested content from the MBS, which increases vehicle request latency and network loss.

[0119] Therefore, we developed a mobile-aware, efficient caching strategy based on asynchronous federation and deep reinforcement learning models to address potential issues. MBS dynamically updates and manages the cache of each RSU by understanding the motion characteristics of connected vehicles and the cache content of RSUs within their coverage area. When a vehicle moves from the current RSU to the next RSU's coverage area, MBS delivers the vehicle's predicted popular content to the next RSU.

[0120] Step (2): Consider the vehicle motion model in each communication round. In the rth communication round, there are N vehicles within the RSU coverage area. r Mobile vehicles Assume that the arrival of vehicles follows a Poisson distribution, and its arrival rate λ v In each communication round, the speed of the vehicle follows an independent and identical distribution, which is represented by the set In order to characterize the real highway environment, consider the speed limit in [U min ,U max ], that is, the vehicles V within the coverage area of the RSU i r Driving speed is subject to minimum or maximum speed limits, i.e. Assume that in the rth communication round, the vehicle's speed Generated from a truncated Gaussian distribution:

[0121]

[0122] where σ 2 is the variance, μ(-∞<μ<+∞) is the mean, and erf() is the Gaussian error function.

[0123] Vehicle speed follows a truncated Gaussian distribution. Compared to a standard Gaussian distribution or a fixed value, vehicle movement is more flexible and more consistent with real-world dynamic vehicle network environments. In our model, all vehicles travel in the same direction. Due to vehicle mobility, as a vehicle passes through the coverage area of different RSUs, it connects to the RSU it is communicating with at the current moment and then transfers to the next adjacent RSU.

[0124] Step (3): The vehicle communication network considered in this invention is based on Mode 4 defined in the 3GPP C-V2X architecture. RSUs exchange information with vehicles within their coverage area via V2R links, and MBSs exchange information with RSUs and vehicles within their coverage area via V2B links. An RSU in the vehicle communication network is considered a fixed node that is connected to other RSUs via wired links at the edge of the network, allowing information about cached contents in different RSUs to be exchanged.

[0125] In the rth round of communication, we have vehicle V i r The channel gain is modeled as:

[0126]

[0127] in, and Represent channel path loss and path shadow respectively. Since in different communication rounds, vehicles V i r Different distances from the RSU will cause different channel impairments for vehicles, resulting in different data transmission rates at different times. Therefore, in r communication rounds, the RSU and the mobile vehicles V within its coverage area are i r The wireless transmission rate between

[0128]

[0129] in, Is the vehicle V i r allocated bandwidth, Is the vehicle V i r The transmission energy, is the noise energy. If V i r Provided by MBS, we define the wireless transmission rate at time t as

[0130]

[0131] Among them, pT is the transmission energy of MBS.

[0132] Step (4): There are three ways for the vehicle to obtain the requested content:

[0133] 1) The vehicle obtains cached content from the local RSU serving it. If the content requested by the vehicle is cached in the local RSU, the local RSU directly delivers the content to the vehicle. Represents the delay for all vehicles to obtain all vehicle request contents from the local RSU in the rth communication round.

[0134]

[0135] Where Nr is the number of vehicles participating in content request in the rth round of communication, F i Is the vehicle V i r The number of requested contents, is a binary variable indicating whether the vehicle obtains the requested content from the local RSU serving it in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the local RSU, otherwise, In the rth round of communication, each RSU takes the content transmission delay Deliver the requested content from the local cache to the vehicle.

[0136]

[0137] s f is the size of the content f.

[0138] 2) The vehicle obtains the requested content from other adjacent RSUs. If the content requested by the vehicle is not cached in the local RSU, the vehicle needs to obtain the requested content from other adjacent RSUs. In the rth round of communication, the delay for all vehicles to obtain the requested content from the adjacent RSU is express:

[0139]

[0140] in, is a binary variable indicating whether the vehicle obtains the requested content from the neighboring RSUs in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the neighboring RSUs, otherwise, is the transmission delay between two RSUs:

[0141]

[0142] Among them, RR-R Represents the transmission rate between two RSUs.

[0143] 3) The vehicle obtains the requested content from the MBS. If the requested content is not cached in the local RSU and neighboring RSUs, the vehicle needs to obtain the requested content from the MBS. In the rth round of communication, the time delay for all vehicles to obtain the requested content from the MBS is:

[0144]

[0145] in, is a binary variable indicating whether the vehicle obtains the requested content from the MBS in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the MBS, otherwise, In the communication round t, V i r The transmission delay between MBS To indicate that:

[0146]

[0147] in, It is V i r and the transmission rate between MBS.

[0148] In the rth communication round, for a given cache status in the local RSU, the content request latency of all vehicles can be expressed as:

[0149]

[0150] The goal is to find the optimal caching strategy π* to achieve the goal of minimizing the content request latency in the network. Therefore, considering the limited storage space of RSU and the integer nature of the caching decision variable, the collaborative caching problem is described as follows:

[0151]

[0152]

[0153]

[0154] The constraint is to ensure that the amount of content cached locally in each RSU cannot exceed its maximum storage capacity, which improves the diversity of the content cached locally in each RSU.

[0155] In the cooperative cache network we are discussing, cache hit rate is considered to be a good indicator for evaluating network performance. The cache hit rate of each RSU is calculated as follows:

[0156]

[0157] The cache hit rate represents the probability of obtaining the content requested by a vehicle from the local cache of the RSU that provides services to the vehicle.

[0158] Step (5): In the RSU, we apply an asynchronous federated learning framework to aggregate the local models from each vehicle in a highly dynamic IoV environment to obtain a global model. The global model trained after multiple iterations is used to predict the content popularity of the mobile-aware caching solution. The asynchronous federated learning scheme we designed is as follows:

[0159] 1) Vehicle Selection: Due to the limited coverage of RSUs and the high speeds of vehicles on highways, some vehicles may not complete asynchronous federated learning training due to a short dwell time when passing through the current RSU. This situation results in an inefficient global model trained by asynchronous federated learning in the RSU, resulting in poor cache performance. Updating and aggregating high-quality vehicle models in each RSU server can train a more accurate global model. The selected vehicle will act as a node to calculate local data to update the global model.

[0160] The primary consideration in vehicle selection is the time a vehicle spends within the RSU's coverage area during travel, which is highly dependent on the connected vehicle's location and speed. Sufficient time within the RSU's coverage area allows for a complete training process and for the training results to be transferred to the vehicle. The time each connected vehicle spends within the current RSU's coverage area during the tth communication round can be calculated as follows:

[0161]

[0162] Among them, P i r In the tth communication round, vehicle V i r Distance from RSU entrance, L s is the coverage size of RSUs.

[0163] Assume that the average training time and test time per round of communication is T training and T inference , which depends on the size of the dataset and the deep learning model. This indicates that the vehicle meets the conditions for participating in asynchronous federated learning (FL) and is selected for asynchronous FL training.

[0164] 2) Global Model Download: In the rth communication round, the selected vehicle participates in asynchronous federated learning training. The selected vehicle downloads the global model, specifically its parameters, from the current RSU. During each communication round, the RSU saves the model of the previous communicating vehicle participating in FL training and updates the model based on it. Using the previous model improves model training efficiency and saves training time.

[0165] 3) Asynchronous Federated Learning Model Training: Definition The data stored for the selected vehicles in each round of communication. r is the total number of vehicles within the RSU coverage area in the rth communication round, It is The data of a vehicle is of length d is the sum of the data stored in all selected vehicles, that is, where d i Is the vehicle V i r The length of the stored data.

[0166] Similar to synchronous FL, the goal of our proposed asynchronous FL is also to minimize the global objective function F(ω):

[0167]

[0168] where f i (ω i ) is the vehicle V i r The local loss function is:

[0169]

[0170] l i (x k ;ω i ) is the empirical local loss function, where x k is a data sample, ω i are the vehicle’s local model parameters. In order to reduce the deviation between the vehicle’s local model and the global model, thereby improving the convergence of the asynchronous federated learning algorithm, a gradient-based update method with a regularized loss function is adopted, which is defined as follows:

[0171]

[0172] Among them, ρ is the regularization parameter, ω r is the global model parameter for the rth communication round, ω i Is the vehicle V i rWe use stochastic gradient descent (SGD) to optimize:

[0173]

[0174] in, is the local learning rate of the local parameters in the rth communication round:

[0175]

[0176] Among them, η l is the initial value of the local learning rate. In asynchronous FL, local gradients that have not been transmitted in the previous communication rounds are aggregated into the local gradients of the current communication round. Such local gradients that have not been transmitted in the previous rounds are called delayed local gradients. Devices with such delayed local gradients are called stragglers, which have an adverse effect on model convergence because the parameters used to calculate the delayed local gradients are different from the local parameters used to calculate the current local gradients. To solve this problem, we need to balance the previous local gradients and the current local gradients. When aggregating local models, we use β as the decay coefficient. At the rth communication round, after aggregation V i r Local Model:

[0177]

[0178] in, Is the vehicle V i r trained local model parameters, Is the vehicle V i r Previous local model parameters.

[0179] 4) Upload the updated model: After the local model training is completed, the local model of the vehicle is uploaded from the vehicle user to the RSU server. Uploading the local model of each vehicle user to the RSU server instead of local data greatly protects the privacy of the vehicle user and reduces the communication cost in the vehicle network.

[0180] 5) Asynchronous aggregation: RSU servers and vehicles perform asynchronous updates. In other words, as soon as the RSU server receives the local model, it will immediately update the global model. Therefore, the RSU server and the vehicle can update the model at any time in an asynchronous manner, which is very suitable for vehicle users in heterogeneous conditions. Specifically, in round r of communication, the RSU server receives the data from the vehicle V i r The local model of , and the global model is updated by weighted averaging, which is expressed as follows:

[0181]

[0182] Taking into account the mobility of vehicles and the transmission rate of V2R (vehicle to RSU), we improve the weight of global parameter aggregation. Then, we can rewrite the aggregation method as follows:

[0183]

[0184] χ i Is the vehicle V i r Parameters for weight aggregation:

[0185]

[0186] where μ1+μ2=1, is the vehicle V in the rth communication round i r The position weight depends on the vehicle V i r location; is the vehicle V in the rth communication round i r The data transmission weight depends on the vehicle V i r data transfer rate.

[0187] After multiple iterative updates, a relatively efficient and convergent global model is trained, and each RSU server uses the trained global model to predict content popularity.

[0188] Step (6): The autoencoder can mine the potential correlations in the data in this transformation chain and save them as a trainable parameter set in the model, which can be used to predict the popularity of content. We use the autoencoder to extract the features of vehicles and files to calculate similarity. Popular content is generated based on the request content of the active vehicle and the request content of the other K neighboring vehicles. The request content of the K neighboring vehicles represents the content that most vehicles may request within the coverage of the RSU. The similarity between vehicle users is calculated because the historical content requested by neighboring vehicle users also reflects the current vehicle user preferences to a certain extent. When the historical content requested by the vehicle user and the historical content requested by the neighboring vehicles are known, the popular content recommended by the vehicle user is generated based on the similarity between vehicle users and the similarity between the request content. In other words, the popularity of content is mainly predicted based on the degree of interest in the content and the personal information of the vehicle user. The process of predicting the popular content of a certain vehicle user performs the following five steps as follows:

[0189] 1) Data Preprocessing: Based on the content request history of each connected vehicle user, a vehicle request content scoring matrix X is constructed. The vehicle's rating of the content represents its preference for the content. The vehicle user's personal information, such as gender and age, is also taken into account.

[0190] 2) Data Feature Representation: The rating matrix X is used as input to the autoencoder. The autoencoder discovers potential features that indicate the correlation between vehicle users and request content. Taking these potential features into account along with vehicle user information, the autoencoder calculates similarity matrices for vehicle users and file content. Similarity matrices reflect the correlation between any two features. The vehicle similarity matrix and the file similarity matrix represent the distance between each vehicle and each file, respectively.

[0191] 3) Construction of historical request matrix: We assume that the current vehicle is an active vehicle. Based on the similarity matrix of the current vehicle user, the K adjacent vehicle users of the vehicle user are determined. The historical request contents of the K selected adjacent vehicle users are combined with the historical request contents of the current vehicle to construct a historical retrieval matrix K * .

[0192] 4) Obtaining similarity: The historical request matrix of the current vehicle user is called A * . Calculate A through the document similarity matrix * With K * The mean similarity between .

[0193] 5) Content Popularity Aggregation: The Fc most similar content items are selected as recommendations for the vehicle user. Then, an aggregation method based on predicted file popularity is applied to generate a recommended list of popular files for content caching. Each connected vehicle user uploads their recommendation list to the RSU server. Upon receiving the recommendation list, the RSU server aggregates and compares the recommendation lists of all uploaded vehicle users and selects the Fc most popular content items.

[0194] Step (7): After obtaining Fc popular contents, the next challenge is to determine which of the Fc contents should be actively cached in the appropriate RSU before each time period. We use dueling DQN to solve the content caching problem. Based on DQN, dueling DQN decomposes the action-state value function Q into the state value function V and the action advantage function A to form the dueling DQN neural network. The state value function V is independent of the action, and the action advantage function A is related to the action. It is the average return of performing action a(t) in state s(t) to solve the reward deviation problem. The action Q-value function is rewritten as:

[0195] Q(s(t), a(t); θ) = V(s(t); θ) + A(s(t), a(t); θ) (24)

[0196] In fact, the action advantage is usually set as the single action advantage function minus the average value of all action advantage functions in a specific state. The action Q-value function is calculated as follows:

[0197]

[0198] This can ensure that the dominant function of each action in this state remains relatively unchanged. The advantage of this method is that it reduces the Q-value range and eliminates redundant degrees of freedom, thereby improving the stability of the algorithm. Therefore, we use dueling DQN to solve the content caching problem. The state space, action space, and reward function are described as follows:

[0199] 1) State space s(t): The state space s(t) is defined as s(t) = (s1, s2,..., s c ), where s(t) represents the specific content cached in the local RSU, and c is the cache capacity of the RSU. We sort the content indices in the state space in descending order according to the content popularity, which can reduce the frequency of low-frequency content indices appearing in the state space.

[0200] 2) Action space a(t): The action space represents whether the cached content in the RSU needs to be replaced. The popular content that is not cached in the local RSU belongs to the set κ. If a(t) = 1, k (k < c) contents are randomly selected from κ and exchanged with the k contents with the lowest indices cached in the local RSU, and then the content indices in the state space are sorted in descending order according to the content popularity to obtain s(t + 1), which can ensure that at time t, the content replaced in the local RSU is the least popular content. a(t) = 0 means that the content cached in the local RSU will not be replaced.

[0201] 3) Reward function r(t): When the local RSU cache state is s(t), the reward function r(t) will be obtained after executing the action a(t). To minimize the total content request delay and system reward, we design the reward function as follows:

[0202]

[0203] is the reward for vehicle V i r requesting content f at time t.

[0204]

[0205] λ1+λ2+λ3=1,λ1<λ2≤λ3, is the reward for the vehicle to obtain the requested content f from the local RSU; is the reward for the vehicle to obtain the requested content f from the neighboring RSU; is the reward for the vehicle to obtain the requested content f from the MBS. The RSU optimizes the caching strategy π* by maximizing the reward function r(t) based on the local content popularity and the vehicle’s requested content information.

[0206] The pseudo codes for the content prediction algorithm based on asynchronous federated learning and the content cache placement algorithm based on dueling DQN are as follows:

[0207]

[0208]

[0209]

[0210] A simulation experiment was conducted on the caching method described in the present invention, and the results are as follows: Figure 1 The cache hit rate is described for RSU cache capacities ranging from 50 to 400 items. The random algorithm provides the worst cache hit rate. As the cache capacity increases, the cache hit rate of all algorithms increases. Our proposed algorithms outperform other caching algorithms. This is because the random and Thompson Sampling algorithms do not learn from the content requested by vehicle users in the past, while CAFR and c-ε-greedy determine the cache content by observing past request content. The CAFR algorithm outperforms c-ε-greedy because CAFR considers the user's contextual information, captures useful features from the data, and aggregates the data in the latent space. Figure 2 The average content request latency per vehicle is described for RSU cache capacities ranging from 50 to 400 contents. The random algorithm provides the longest content request latency. As the cache capacity increases, the content request time of all algorithms decreases. The average content request latency of vehicles in our proposed method is lower than that of other cache algorithms. Figure 1 ,2,It can be concluded that in terms of cache hit rate and request delay, the CAFR algorithm is superior to other cache algorithms.

[0211] Figure 3 The cache concentration rate of the proposed CAFR algorithm under different vehicle densities is shown under the condition that the RSU cache capacity is 100. Figure 3As can be seen, as vehicle density increases, the cache hit rate also increases. This is because as more vehicles enter the RSU's coverage area, these vehicles are able to train more data, thus improving the vehicle network's computing power and resulting in more accurate content predictions. Furthermore, as the vehicle density within the RSU's coverage area increases, the average vehicle request latency decreases. This is because when vehicle density increases, the cache hit rate also increases, allowing more vehicles to obtain content directly from the RSU.

[0212] Figure 4 This figure shows the relationship between the cache hit rate and communication rounds for our proposed CAFR algorithm and a typical synchronous federated learning training (FedAVG) algorithm using the MovieLens 1M dataset. In the simulation, 15 vehicles collaboratively participated in a global model training. The results show that the cache hit rate of our proposed CAFR method fluctuated between 22.5% and 24% over 30 communication rounds, while the cache hit rate of the FedAVG method fluctuated between 22% and 23.5% over 30 communication rounds. This demonstrates that the CAFR method outperforms the FedAVG method because it considers vehicle mobility (location and speed) to select vehicles and aggregate local models, thereby improving the accuracy of the global model.

[0213] Figure 5 This figure shows the training time per round for the CAFR and FedAVG methods when the vehicle density is 15 vehicles / km and the cache capacity of each RSU is 100 items. We can see that the training time for CAFR ranges from 1s to 2s per round, while the training time for FedAVG ranges from 22s to 24s per round. This indicates that the training time per round for CAFR is significantly shorter than that for FedAVG. This is because FedAVG aggregates the local models of all vehicles in each round, while CAFR only aggregates the local model of a single vehicle.

[0214] Figure 6 This study demonstrates the relationship between cache hit rate, average vehicle request latency, and episode length in a dueling DQN-based cache replacement strategy. In the experiment, 15 vehicles collaboratively trained a global model using asynchronous federated learning. After obtaining predicted content popularity, the RSUs used the dueling DQN's DRL algorithm to determine which content should be cached within the RSUs. As the number of episodes increased, the cache hit rate gradually improved, while the average vehicle request latency gradually decreased. The cache hit rate and average vehicle request latency converged after approximately 10 rounds.

[0215] Figure 7 and Figure 8The effectiveness of the cache replacement strategy based on dueling DQN is demonstrated. We compare the cache hit rate and average vehicle request latency of CAFR and CAFR without DRL. Figure 7 As shown in , the cache hit rate of CAFR is better than that of CAFR without DRL. Figure 8 As shown in Figure 2, the average vehicle request delay for CAFR is less than that for CAFR without DRL. Figure 7 and Figure 8 Experimental results show that the proposed cache replacement strategy based on dueling DQN can further improve the cache performance of the edge cache scheme in a highly dynamic Internet of Vehicles environment.

[0216] Any content not described in detail in this application belongs to the prior art known to those skilled in the art. The above embodiments are merely preferred embodiments of the present invention. It should be noted that those skilled in the art may make various improvements and equivalent substitutions without departing from the principles of the present invention. Such improvements and equivalent substitutions to the claims of the present invention fall within the scope of protection of the present invention.

Claims

1. A collaborative edge caching method based on asynchronous federation and deep reinforcement learning, characterized in that: The communication network architecture used in this method is as follows: the top layer is the core network and macro base stations (MBSs), the middle layer includes multiple roadside units (RSUs), which act as edge servers to cache content, and the bottom layer includes vehicles and vehicle request content. The method includes the following steps: (1) There are multiple vehicles within the coverage of an RSU. A motion model based on vehicle speed is calculated in each communication round. In the motion model, all vehicles are moving in the same direction. When a vehicle passes through the coverage of different RSUs, it connects to the RSU it is communicating with at the current moment and then transfers to the next adjacent RSU. (2) obtaining the wireless transmission rate between the RSU and the mobile vehicles within its coverage at a certain moment based on the channel gain of each vehicle in a communication round, wherein the mobile vehicles are served by the MBS; (3) The vehicle obtains the requested content, thereby obtaining the content request delay of all vehicles in a certain communication round, and describes the collaborative caching strategy problem based on the request delay; the method for the vehicle to obtain the requested content is: if the content requested by the vehicle is cached in the local RSU, the local RSU directly transmits the content to the vehicle; if the content requested by the vehicle is not cached in the local RSU, the vehicle obtains the requested content from other adjacent RSUs; if the requested content is not cached in the local RSU and adjacent RSUs, the vehicle obtains the requested content from the MBS; (4) Applying an asynchronous federated learning framework to aggregate the local models from each vehicle in a highly dynamic IoV environment to obtain a global model, and then training an efficient and convergent global model through multiple iterations; (5) Use the global model to predict the popular content of each vehicle user and perform aggregate comparison to obtain multiple contents with the highest content popularity; (6) Dueling DQN is used to solve the collaborative caching strategy problem and determine which of the multiple most popular contents are actively cached in the appropriate RSU. The method for determining which of the multiple most popular contents are actively cached in the appropriate RSU is as follows: Based on DQN, Dueling DQN decomposes the action-state value function Q into the state value function V and the action advantage function A, forming the Dueling DQN neural network. The state value function V is independent of the action, while the action advantage function A is related to the action. It is the average return of performing action a(t) in state s(t) to solve the reward bias problem. The action Q-value function is written as: Q(s(t),a(t);θ)=V(s(t);θ)+A(s(t),a(t);θ) Where θ is the neural network parameter of dueling DQN, the action advantage is usually set to a single action advantage function minus the average of all action advantage functions in a specific state, and the action Q-value function is calculated as follows: Therefore, dueling DQN is used to solve the content caching problem. The state space, action space, and reward function are described as follows: State space s(t): The state space s(t) is defined as s(t) = (s1, s2, ..., s c ), where s(t) represents the specific content of the local RSU cache, and c is the cache capacity of the RSU; Action space a(t): The action space represents whether the cached content in the RSU needs to be replaced. Popular content that is not cached locally in the RSU belongs to the set κ. If a(t) = 1, k (k < c) contents will be randomly selected from the set κ and exchanged with the k contents with the lowest index cached locally in the RSU. Then, the content indices in the state space are sorted in descending order based on content popularity to obtain s(t + 1). In this way, it can be ensured that at time t, the content replaced in the local RSU is the least popular content. a(t) = 0 means that the content cached in the local RSU will not be replaced; Reward function r(t): When the local RSU cache state is s(t), the reward function r(t) will be obtained after performing the action a(t); In order to minimize the total content request delay and system reward, the reward function is designed as follows: At time t, vehicle V i r Request a reward for content f; λ1+λ2+λ3=1,λ1<λ2≤λ3, is the reward for the vehicle to obtain the requested content f from the local RSU; is the reward for the vehicle to obtain the requested content f from the neighboring RSU; is the reward for the vehicle to obtain the requested content f from the MBS. The RSU optimizes the caching strategy π* by maximizing the reward function r(t) based on the local content popularity and the vehicle’s requested content information.

2. The collaborative edge caching method based on asynchronous federation and deep reinforcement learning according to claim 1 is characterized in that: The specific steps of step (1) include: In the rth communication round, there are N r Mobile vehicles Assume that the arrival of vehicles follows a Poisson distribution, and its arrival rate λ v ; In each communication round, the speed of the vehicle follows an independent and identical distribution, which is represented by the set express; Consider the speed limit at [U min ,U max ], that is, the vehicles V within the coverage area of the RSU i r Driving speed is subject to minimum or maximum speed limits, i.e. Assume that in the rth communication round, the vehicle's speed Generated from a truncated Gaussian distribution: Among them, σ 2 is the variance, μ(-∞<μ<+∞) is the mean, and erf() is the Gaussian error function.

3. The collaborative edge caching method based on asynchronous federation and deep reinforcement learning according to claim 2 is characterized in that: The specific steps of step (2) include: In the rth round of communication, the vehicle V i r The channel gain is modeled as: in, and represent channel path loss and path shadowing, respectively; Since vehicles in different communication rounds Different distances from the RSU will cause different channel impairments for vehicles, resulting in different data transmission rates at different times. Therefore, in r communication rounds, the communication between the RSU and the mobile vehicles V within its coverage area is i r The wireless transmission rate between in, Is the vehicle V i r allocated bandwidth, Is the vehicle V i r The transmission energy, is the noise energy, if V i r MBS provides services and defines the wireless transmission rate at time t as Among them, p T is the transmission energy of MBS.

4. The collaborative edge caching method based on asynchronous federation and deep reinforcement learning according to claim 3 is characterized in that: The step (3) includes the following steps: (31) Obtain the content requested by the vehicle in the following three ways: First, the vehicle obtains the cached content from the local RSU serving it; If the content requested by the vehicle is cached in the local RSU, the local RSU directly delivers the content to the vehicle, allowing represents the delay for all vehicles to obtain the requested content of all vehicles from the local RSU in the rth round of communication; Where Nr is the number of vehicles participating in content request in the rth round of communication, Is the vehicle V i r The number of requested contents, is a binary variable indicating whether the vehicle obtains the requested content from the local RSU serving it in the rth round of communication. Indicates that in the rth round of communication, the vehicle obtains the requested content from the local RSU, otherwise, In the rth round of communication, each RSU takes the content transmission delay transmitting the requested content from the local cache to the vehicle; s f is the size of the content f; Second, the vehicle obtains the requested content from other adjacent RSUs; If the content requested by the vehicle is not cached in the local RSU, the vehicle needs to obtain the requested content from other adjacent RSUs; In the rth round of communication, the delay time for all vehicles to obtain the request content from the adjacent RSU is express: in, is a binary variable indicating whether the vehicle obtains the requested content from the neighboring RSUs in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the neighboring RSUs, otherwise, is the transmission delay between two RSUs: R R-R represents the transmission rate between two RSUs; Finally, the vehicle obtains the requested content from the MBS: If the requested content is not cached in the local RSU and neighboring RSUs, the vehicle needs to obtain the requested content from the MBS; In the rth round of communication, the time delay for all vehicles to obtain the requested content from the MBS is: in, is a binary variable indicating whether the vehicle obtains the requested content from the MBS in the rth round of communication. Indicates that in the rth round of communication, the vehicle can obtain the requested content from the MBS, otherwise, In the communication round t, V i r The transmission delay between MBS To indicate that: in, It is V i r and the transmission rate between MBS; (32) In the r-th communication round, for a given cache state in the local RSU, the content request delays of all vehicles can be expressed as: (33) Considering the limited storage space of the RSU and the integer characteristics of the cache decision variables, the collaborative caching problem is described as follows: The constraint is to ensure that the amount of content cached locally in each RSU cannot exceed its maximum storage capacity.

5. The collaborative edge caching method based on asynchronous federation and deep reinforcement learning according to claim 4 is characterized in that: The specific steps of step (4) include the following steps: (41) Select vehicles: The factor considered in the vehicle selection process is the residence time of the vehicle during driving within the coverage of the RSU. In the t-th communication round, the residence time of each connected vehicle within the current RSU coverage is as follows: Among them, P i r In the tth communication round, vehicle V i r Distance from RSU entrance, L s is the coverage size of RSUs; Assume that the average training time and test time per round of communication is T training and T inference ,if This indicates that the vehicle meets the requirements for participating in asynchronous federated learning and has been selected for asynchronous federated learning model training. (42) Global model download: In the r-th communication round, the selected vehicles participate in asynchronous federated learning model training; The selected vehicles download the global model from the current RSU. Each communication round, the RSU saves the models of the previously communicating vehicles participating in asynchronous federated learning training and updates the model based on them; (43) Asynchronous Federated Learning Model Training: Definition The data stored by the selected vehicles in each round of communication, N r is the total number of vehicles within the RSU coverage area in the rth communication round, It is the Nth r The data of a vehicle is of length d is the sum of the data stored in all selected vehicles, that is, Among them, d i Is the vehicle V i r The length of the stored data; Minimizing the global objective function F(ω) is defined as: Among them, f i (ω i ) is the vehicle V i r The local loss function is: Among them, l i (x k ;ω i ) is the empirical local loss function, x k is a data sample, ω i are the local model parameters of the vehicle; Adopt a gradient-based update method using a regularized loss function, defined as follows: Among them, ρ is the regularization parameter, ω r is the global model parameter for the rth communication round, ω i Is the vehicle V i r The local model parameters are optimized using stochastic gradient descent: in, is the local learning rate of the local parameters in the rth communication round, and Among them, η l is the initial value of the local learning rate; When aggregating local models, using the rth communication round, after aggregation V i r Local Model: in, Is the vehicle V i r trained local model parameters, Is the vehicle V i r The previous local model parameters, β is the attenuation coefficient; (44) Upload the updated model: After the local model training is completed, upload the local model of the vehicle from the vehicle user to the RSU server; (45) The RSU server and the vehicle perform asynchronous updates, that is, in the rth round of communication, the RSU server receives the data from the vehicle V i r The local model of , and update the global model by weighted averaging, χ i Is the vehicle V i r Parameters for weight aggregation: Among them, μ1+μ2=1, is the vehicle V in the rth communication round i r The position weight depends on the vehicle V i r location; is the vehicle V in the rth communication round i r The data transmission weight depends on the vehicle V i r The data transmission rate is 100%, and after multiple iterative updates, a more efficient and convergent global model is trained.

6. The collaborative edge caching method based on asynchronous federation and deep reinforcement learning according to claim 5, characterized in that: The specific steps of predicting the popular content of a certain vehicle user in step (5) include the following steps: (51) Data preprocessing: Based on the content request history of each connected vehicle user, establish a vehicle request content scoring matrix X; (52) Data feature expression: Take the scoring matrix X as the input data of the autoencoder. The autoencoder discovers the potential features of the correlations between vehicle users and between requested contents. Considering the potential features and vehicle user information, calculate the similarity matrices of vehicle users and file contents respectively; Construction of historical request matrix: Assuming that the current vehicle is an active vehicle, based on the similarity matrix of the current vehicle user, determine the K adjacent vehicle users of the vehicle user. The historical request contents of the K selected adjacent vehicle users are combined with the historical request contents of the current vehicle to construct a historical retrieval matrix K * ; (53) Obtaining similarity: The historical request matrix of the current vehicle user is called A * , calculate A through the document similarity matrix * With K * The mean of similarities between (54) Content popularity aggregation: Fc contents with the highest similarity are selected as recommended contents for the vehicle user. Then, the aggregation method of predicted file popularity is executed to generate a recommendation list of popular files for content caching. Each connected vehicle user uploads their recommendation list to the RSU server. After receiving the recommendation list, the RSU server aggregates and compares the recommendation lists of all vehicle users who uploaded the list and selects Fc contents with the highest content popularity.

7. The collaborative edge caching method based on asynchronous federation and deep reinforcement learning according to claim 6, characterized in that: The step (6) includes: After obtaining Fc popular contents, the next challenge is to determine which contents among the Fc contents should be proactively cached in appropriate RSUs before each time period.

Images