Coupling method and system for computing unloading and content caching of Internet of Vehicles

Abstract

The invention provides a coupling method and system for computing unloading and content caching of the Internet of Vehicles, relates to the technical field of the Internet of Vehicles and vehicle-road cooperation, and can better adapt to scenes with relatively large computing task requirements in an environment compared with a single caching method. The method comprises the steps of firstly setting a deployment mode of a road side unit (RSU), then constructing an elite genetic algorithm and a multi-agent depth deterministic policy gradient (GAC-MADDPG) algorithm, and finally constructing a dual-time scale model to couple the elite genetic algorithm and the multi-agent depth deterministic policy gradient (GAC-MADDPG) algorithm, so that cache allocation of the RSU is updated according to historical request preference and information age of a cache task under a large time scale; and performing unloading calculation on the task of the task vehicle on a small time scale, and feeding back to a cache allocation algorithm for joint optimization according to a cache hit rate. The method is suitable for the field of Internet of Vehicles and vehicle-road collaboration, and is used for providing task calculation unloading and caching functions for vehicles in an urban road environment.

Description

Technical Field

[0001] The present invention relates to the field of vehicle networking and vehicle-road collaboration technology, and in particular to a method for coupling vehicle networking computing offloading and content caching. Background Art

[0002] With the continuous development of connected vehicle (IoV) technology, the amount of IoV data generated by onboard sensing devices is growing exponentially, leading to the demand for massive computationally intensive (such as virtual reality, augmented reality, and high-definition video), latency-sensitive (such as real-time road condition monitoring and updates), and energy-intensive tasks. Meeting the growing demands for communication, computing, and storage while addressing the limited computing, storage, and communication resources inherent in vehicles and improving user service quality has become a key challenge in the IoV field.

[0003] To address the resource constraints and high energy consumption of onboard devices, cloud computing was initially used to offload computing tasks to remote servers. While this improved computing and storage capabilities, the long distance from the vehicle resulted in high latency, increased bandwidth costs, and reduced communication reliability. Mobile edge computing (MEC), as an improved solution, deploys some computing and storage resources on roadside units (RSUs) close to the vehicle, significantly reducing transmission latency and improving task processing efficiency. However, MEC server deployment is costly and can waste resources during off-peak hours, making it difficult to sustainably meet the demands of large-scale dynamic tasks. To further improve resource utilization, researchers have proposed the vehicle edge computing (VEC) model, configuring some vehicles as computing nodes to support task offloading. However, due to the limited computing power and resource volatility of onboard nodes, task allocation requires more sophisticated management. When multiple users concurrently offload tasks, edge node resources may be insufficient, resulting in queuing or relegation to the cloud for processing, increasing latency and energy consumption, and compromising service quality. Furthermore, the large number of repetitive computing tasks in the connected vehicle network, such as those for video, mapping, and road status, can easily cause network congestion and waste resources. To this end, an edge caching mechanism has been introduced to cache the results of popular tasks on edge servers, directly returning the results in subsequent requests, significantly reducing computational overhead and transmission latency. However, edge node cache space is limited, and a large number of tasks can lead to a decrease in cache hit rate. Therefore, implementing efficient caching and replacement strategies for high-frequency content becomes a key issue. Task offloading and caching strategies are closely related, and joint optimization can significantly improve system performance, but also faces challenges such as complex model construction and difficult algorithm design. Further exploration of efficient collaborative optimization mechanisms adapted to the Internet of Vehicles environment is needed to achieve more intelligent and efficient edge computing systems. Summary of the Invention

[0004] The present invention provides a method for coupling vehicle network computing offloading and content caching, which is more adaptable to scenarios with greater computing task demands in the environment compared to a single caching method.

[0005] To achieve the above object, the present invention provides the following technical solutions: The present invention provides a method for coupling vehicle network computing offloading and content caching, the method comprising the following steps: Step S1: setting a deployment mode of a roadside unit (RSU); Step S2: construct a dual-time-scale model, which includes multiple large time scales and small time scales; Step S3: At the end of each large time scale cycle, the RSU uses the improved collaborative elite genetic algorithm to collaboratively update the cache decision based on the unloading request of the task vehicle and the popularity model of the task. In each small time scale time slot, the multi-agent deep deterministic policy gradient algorithm is used to select the appropriate unloading node according to the cache status and the wireless environment status to make the optimal unloading decision.

[0006] Furthermore, the deployment method of the above-mentioned roadside unit RSU is specifically as follows: Multiple RSUs are deployed on the roadside, and each RSU is equipped with a MEC server and a cache server for computing and cache query services respectively; A base station is set up at the center of multiple RSU groups to store the cache tasks of all RSUs in the area.

[0007] Furthermore, the construction method of the above dual time scale model is: The entire time system is divided into D A large time scale, and define the length of each large time scale period as , the set is ,use Indicates the A large time scale cycle; Then, in each large time period, it is divided into time slots, and the length of each time slot is , as a small time scale, and represented by an index set ,use Expressed as The first Hourly time scale period.

[0008] Furthermore, the improved collaborative elitist genetic algorithm is used to collaboratively update the cache decision as follows: Parameter encoding and initial population setting for the genetic algorithm include: initializing the population and generating a set of random solutions as the initial exploration points of the search space. One gene represents a cached task, and each individual represents the complete cache strategy solution for each RSU. Add a fitness function to the genetic algorithm to quantitatively evaluate the retention of each individual, and add overhead indicator optimization; An improved elite selection strategy is adopted, which keeps the best solution of each generation and passes it directly to the next generation until the optimal solution of caching decision is obtained.

[0009] Furthermore, the above improved elite selection strategy is specifically as follows: For the Xth generation population , find the current population The maximum fitness value ; Generate the next generation of intermediate temporary population , and find the temporary population The maximum fitness value ; like , then Medium fitness is greater than or equal to The number of individual copies is recorded as , then randomly generate Y New individuals are generated by combining the random parent with the original parent. N children, and then perform a replacement operation, using N Subsequent replacement The lowest fitness N individuals, and obtain a new generation of population ; like , then directly retain the temporary population For the new generation of population .

[0010] Furthermore, the network structure of the multi-agent deep deterministic policy gradient algorithm includes an Actor network and a Critic network.

[0011] Furthermore, the multi-agent deep deterministic policy gradient algorithm is used to make the optimal unloading decision as follows: A new cache-aware state space is constructed to first determine whether the requested content is in the RSU cache when the offload target is the RSU; Constructing the offloading decision action space, including offloading node selection decision and task offloading ratio ; Construct a cache collaborative reward function, set the hit reward as: directly use the cache content for incentives, and the miss penalty as: promote the improved collaborative elite genetic algorithm to optimize the cache strategy to make the best offloading decision.

[0012] The method for coupling vehicle network computing offloading and content caching described in the present invention can be fully implemented using computer software. Therefore, correspondingly, the present invention also provides a vehicle network computing offloading and content caching coupling system, which includes a storage device, and the storage device is used to execute the above-mentioned methods and steps.

[0013] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, any one of the above-mentioned methods for coupling vehicle network computing offloading and content caching is executed.

[0014] The present invention also provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes any one of the above-mentioned methods for coupling vehicle network computing offloading and content caching.

[0015] The beneficial effects of the present invention are: This invention provides a method for coupling computation offloading and content caching in connected vehicles. By employing an elite genetic algorithm and a multi-agent deep deterministic policy gradient (GAC-MADDPG) algorithm, a dual-timescale model is constructed to couple the two. On a large timescale, the cache allocation of RSUs is updated based on historical request preferences and the information age of cached tasks. On a small timescale, computation is offloaded to task vehicles, and the cache hit rate is fed back to the cache allocation algorithm for joint optimization. Compared to single-caching methods, this proposed method is more adaptable to scenarios with high computational demands.

[0016] The present invention is applicable to the fields of Internet of Vehicles and vehicle-road collaboration, and is used to provide task computing offloading and caching functions for vehicles in urban road environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 The dual time scale model of the present invention; Figure 2 The Actor-Critic network structure of the MADDPG algorithm described in the present invention; Figure 3 The training and execution process of the MADDPG algorithm described in the present invention; Figure 4 The joint scenario model of the MEC-based Internet of Vehicles computing offloading cache described in the present invention; Figure 5 Flowchart of the GAC-MADDPG algorithm under dual time scales according to the present invention; DETAILED DESCRIPTION In the following description, the specific implementation details (such as experimental equipment, operating procedures, data processing steps and example parameters) of "a method for coupling vehicle network computing offloading and content caching" provided in this specification are for illustrative purposes rather than restrictive definitions, and are intended to help those skilled in the art to thoroughly understand the principles and implementation of the present invention; however, those skilled in the art should be clear that these details only represent one of the feasible embodiments, and the core concept of the present invention can be fully implemented through other technical means or workarounds that are not fully described without departing from its spirit, and the omission of conventional experimental methods and device details known in the art in the specification is to avoid redundant information interfering with the understanding of the innovation points. This does not mean that these known technologies are not required for implementation, and technical personnel should be able to supplement and apply them on their own based on professional knowledge.

[0019] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details that obscure the description of the present application.

[0020] Implementation Method 1: Combination Figure 5 This embodiment describes a method for coupling computation offloading and content caching in the connected vehicle (IoV) environment. Specifically, it provides a method for offloading computational tasks and allocating caches for IoV vehicles. This method utilizes an elite genetic algorithm and a multi-agent deep deterministic policy gradient (GAC-MADDPG) algorithm to construct a dual-timescale model to couple the two. On a large timescale, cache allocation for RSUs is updated based on historical request preferences and the information age of cached tasks. On a small timescale, computational tasks are offloaded to task vehicles, and the cache hit rate is fed back to the cache allocation algorithm for joint optimization. Compared to single-caching methods, this method is more adaptable to scenarios with high computational demands.

[0021] like Figure 5 As shown, the method includes the following steps: Step S1: Setting the deployment mode of the roadside unit RSU; Step S2: construct a dual-time-scale model, which includes multiple large time scales and small time scales; Step S3: At the end of each large time scale cycle, the RSU uses the improved collaborative elite genetic algorithm to collaboratively update the cache decision based on the unloading request of the task vehicle and the popularity model of the task. In each small time scale time slot, the multi-agent deep deterministic policy gradient algorithm is used to select the appropriate unloading node according to the cache status and the wireless environment status to make the optimal unloading decision.

[0022] Implementation Method 2: Combination Figures 1 to 4 This embodiment specifically describes the method for coupling vehicle network computing offloading and content caching proposed in the first embodiment above. Step S1: Setting the deployment mode of the roadside unit RSU; Specifically: like Figure 4 As shown, RSUs are deployed roadside. Each RSU is equipped with an MEC server and a cache server, providing computation and cache query services, respectively. Each RSU can only serve vehicles within its coverage area, and the RSUs are interconnected via wired networks (such as fiber optic networks). At the center of the RSU cluster, a base station (BS) stores cache tasks for all RSUs in the area to facilitate subsequent cache retrieval. Computational assistance vehicles are also deployed to perform computational assistance functions for task vehicles.

[0023] Step S2: construct a dual-time-scale model, which includes multiple large time scales and small time scales; Specifically: In a VEC environment, the remaining computing space between the vehicle and RSU, the RSU cache status, and the vehicle's location all change dynamically. This implementation aims to reduce the overall system overhead of the connected vehicle (IoV) system and involves the joint optimization of cache allocation and offload computation. Cache allocation is typically based on content popularity patterns, which change more slowly than user request frequency. Frequent updates increase system overhead and cause system instability. Vehicle offload computation typically requires rapid processing feedback, ensuring low latency and high efficiency.

[0024] like Figure 1 As shown, the system includes two time scales, large and small. First, the entire time system is divided into D The large time period module is named large time scale, and defines the length of each large time scale period as , define the set as ,use Indicates the dIn each large time period, there are w time slots, and the length of each time slot is , named as small time scale, represented by index set ,use Expressed as d The first w Hourly time scale period. Figure 4 In the scenario shown, this implementation is set to the end of each large time scale period, and the RSU will collaboratively update the cache decision through the unloading request of the task vehicle and the popularity model of the task; in each small time scale time slot, the task vehicle will select the appropriate unloading node according to the cache status and the wireless environment status to make the optimal unloading decision.

[0025] Step S3: At the end of each large time scale cycle, the RSU uses the improved collaborative elite genetic algorithm to collaboratively update the cache decision based on the unloading request of the task vehicle and the popularity model of the task. In each small time scale time slot, the multi-agent deep deterministic policy gradient algorithm is used to select the appropriate unloading node according to the cache status and the wireless environment status to make the optimal unloading decision.

[0026] Specifically: Under large time scale, the improved collaborative elite genetic algorithm is used to collaboratively update cache decisions as follows: Parameter encoding and initial population setting for the genetic algorithm include: initializing the population and generating a set of random solutions as the initial exploration points of the search space. One gene represents a cached task, and each individual represents the complete cache strategy solution for each RSU. Add a fitness function to the genetic algorithm to quantitatively evaluate the retention of each individual, and add overhead indicator optimization; An improved elite selection strategy is adopted, which keeps the best solution of each generation and passes it directly to the next generation until the optimal solution of caching decision is obtained.

[0027] Among them, the improved collaborative elite genetic algorithm (GAC) is specifically: This implementation implements a GAC ​​cache update algorithm over a large time scale. Based on the traditional GA algorithm, an improved collaborative elitist genetic algorithm (GAC) is proposed. Unlike traditional GAs, this algorithm employs an elitist selection strategy and a distributed collaborative architecture with multiple RSUs. Ultimately, this algorithm aims to maximize cache hit rates and reduce system overhead within the constraints of limited cache capacity. The following are the algorithm details: (1) Parameter encoding and initial population setting First, the population is initialized and a set of random solutions are generated as the initial exploration point of the search space. A gene represents a cached task, and each individual represents the complete cache strategy solution of each RSU. Each RSU has its own cache manager, that is, each RSU runs its own genetic algorithm to optimize the cache content. The entire population consists of multiple individuals, and the entire group represents the potential solution space. It can be seen that the size of the potential solution space is Under energy- and latency-sensitive conditions, searching this vast space is impractical. Therefore, this implementation uses a Zipf distribution to generate content request probabilities, simulating the "long tail effect" of real-world scenarios. Each individual cache strategy is generated through probabilistic sampling, providing a better initial solution to reduce the solution space.

[0028] (2) Fitness function The fitness function reflects the adaptability of an individual to the environment during the evolution process. Such a quantitative evaluation can help GAC decide which individuals can be retained and which cannot. As far as this algorithm is concerned, the focus is on improving the cache hit rate and reducing the overhead of latency and energy consumption. Since the algorithm designed in this implementation is a joint optimization algorithm for cache and offload calculation, the overhead index optimization of latency and energy consumption is incorporated into the subsequent offload calculation algorithm. Therefore, the indicator that the current GAC algorithm is most concerned about is the improvement of the cache hit rate, and the age of information (AoI) is added to optimize and improve the freshness of the task. At the same time, the popularity model prediction item is introduced to provide long-term rules, which is expressed by the following formula:

[0029] in, Indicates the m The number of historical requests for RSU, Indicates the m The number of cache hits for an RSU. 、 and is the weight coefficient.

[0030] (3) Elite selection strategy In genetic algorithms, the selection operation is crucial, as it directly affects the evolution of individuals and the convergence speed of the algorithm. Generally speaking, individuals with higher fitness are more likely to be retained, thereby accelerating the search for the optimal solution. This embodiment adopts an improved elite selection strategy, the purpose of which is to retain the best solution of each generation and pass it directly to the next generation, avoiding the crossover and mutation processes from damaging the existing optimal solution. This embodiment makes certain improvements to the elite selection strategy, and the specific operations are as follows: 1) For the Xth generation population , find the fitness function value of all individual matrices and select the maximum value, which is recorded as .

[0031] 2) Generate the next generation of intermediate temporary population

[0032] In addition, this embodiment uses a tournament selection method. Compared with the roulette method, the tournament selection method performs better in a dynamic environment through a random competition mechanism. The specific process of the tournament selection method is as follows: first, random sampling is performed to randomly select n Individuals are sampled without replacement to ensure no duplicates within the group, increasing diversity and forming tournament groups. A fitness competition is then conducted to compare the fitness of each individual within the group. The winning individuals are retained, and the individuals with the highest fitness are selected to advance to the next generation. Repeated sampling is then performed until the size of the new population reaches a preset number. Finally, the tournament algorithm selects parent individuals suitable for crossover and mutation.

[0033] After the parent individuals are selected in the tournament, the crossover operation is performed. The crossover operation generates offspring through cooperation between parents, which is a key method to maintain species diversity. This implementation adopts the following design logic. First, through pairing, here we take the example of two paired parents, and , first traverse All genes of the individual are added to the offspring with a probability of 50%, which prevents the offspring from completely copying the parent and promotes the exploration of new combinations. It also has a certain historical advantage inheritance, because the parent is a high-quality individual screened by the selection operation, and its content is more likely to cover recent hot spots. Then shuffle The genes of the individual are traversed. If the offspring is not full and the content is not repeated, it is added to the offspring. If there is still space in the offspring, the high-popularity genes are preferentially selected to be added, and new content is introduced to explore potential hot spots to avoid falling into the local optimum. At this time, the offspring is obtained. Reversal and , perform the same crossover operation to obtain offspring By analogy, two parent individuals will produce two offspring individuals. Ultimately, the number of offspring individuals is the same as the number of parents. Here, this implementation method screens non-elite individuals and selects 80% of individuals with lower fitness in the parent generation for crossover (retaining the top 20% elites and not participating in the crossover).

[0034] After the crossover operation, the mutation operation is performed. Mutation is a key method for maintaining population diversity. By altering genes within parental gene fragments, it affects the genetic information of individuals, thereby minimizing the differences between offspring and parents in the multidimensional solution space. While the crossover operation primarily improves global search capabilities, the mutation operation focuses on enhancing local search capabilities. Each has its own emphasis.

[0035] First, set a 10% probability to perform mutation operations on individuals to avoid excessive destruction of high-quality strategies with a low probability, while introducing new content to cope with sudden requests. Using fair replacement, each cache location has an equal chance of being replaced to avoid bias towards a certain type of content. Even high-quality content may be replaced, forcing the algorithm to verify the new strategy. Use the available content pool to generate to ensure that the mutated new content is not in the current cache. Finally, return the new individual after mutation. Merge the offspring after crossover mutation with the top 20% elites that did not participate in the crossover to form a temporary population. .

[0036] 3) Improve elite selection strategy First calculate the maximum fitness value, the current The maximum fitness value is , temporary new generation temporary population The maximum fitness value is ,like (The new generation is not improved), then copy the old elite and Medium fitness is greater than or equal to The number of individual copies is recorded as Y . Then randomly generate Y New individuals are generated by combining the random parent with the original parent. N Then perform the replacement operation, using this N Subsequent replacement The lowest fitness N Then a new generation of population is obtained .like , directly retain the temporary population For the new generation of population .

[0037] At a small time scale, the optimal unloading decision is made using a multi-agent deep deterministic policy gradient algorithm as follows: A new cache-aware state space is constructed to first determine whether the requested content is in the RSU cache when the offload target is the RSU; Constructing the offloading decision action space, including offloading node selection decision and task offloading ratio ; Construct a cache collaborative reward function, set the hit reward to: directly use the cache content for incentives, and the miss penalty to: promote the GAC algorithm to optimize the cache strategy to make the best offloading decision.

[0038] The new cache-aware state space is as follows: When only considering computational offloading, the state indicators set are the channel allocation matrix , the channel quality between vehicle and RSU Communication channel quality with the vehicle , the remaining tasks Information age with tasks , and the computational load of RSU After the cache is introduced in this embodiment, the state space should change. When the offload target is RSU, it should first determine whether the cache hits, that is, whether the requested content is in the RSU cache. Indicates the information age of the task cached on the RSU In summary, the new state space is set to the following vector:

[0039] The specific construction of the offloading decision action space is as follows: Adopting uninstall decisions Select the task offloading node; and the task offloading ratio , select the task offloading ratio.

[0040]

[0041] The specific cache collaborative reward function is: The reward function of this embodiment should be set to a reward function that is consistent with the environment of this embodiment. In addition to considering the reward setting for the simple calculation offloading part, it also introduces cache-related rewards. Hit reward: directly use cache content for incentives, thereby reducing return overhead. Miss penalty: promote the GAC algorithm to optimize the cache strategy. The above can be expressed as an equation:

[0042] in, represents the reward for cache hit, 、 and is the reward weight.

[0043] The multi-agent deep deterministic policy gradient algorithm MADDPG is based on the DDPG algorithm. Each agent can share global information and make corresponding decisions based on its own state. It adopts the principle of centralized training and decentralized execution, effectively handles the competitive relationship in the environment, can accelerate the convergence speed and achieve accurate estimation.

[0044] (1) Multi-agent Actor-Critic Network Design In this embodiment, there are vehicles, RSUs, and cloud servers. The number of RSUs is mEach RSU represents an intelligent agent and executes actions. At the same time, the task vehicles in the environment receive actions and execute the corresponding computation offloading scheme after receiving the actions. They are jointly responsible for the management of the Actor and Critic networks and synchronously maintain the corresponding target network. Its specific structure is as follows Figure 2 As shown in Figure 2. The Actor network is used to determine the action strategy. It receives the environmental state observed by the current agent as input. Based on this state information, the Actor network outputs the unloading decision of all RSU nodes. The Critic network is based on the state-action value function. To evaluate the quality of the actions selected by the Actor network, the input is the state of all RSUs and the corresponding actions, and the output is a state-action value function Q ,Should Q The estimated value is fed back to the Actor network to adjust the parameters of its neural network.

[0045] (2) Multi-agent Actor-Critic network parameter update Unlike DDPG, MADDPG's Critic network needs to use information from other agents during training, while DDPG's Critic network only considers its own state and actions. Figure 3 As shown, for the intelligent agent m The Critic network, in addition to observing its own local state information and actions performed In addition, it is also necessary to obtain the action information of other agents And observe the status information of other agents , when each intelligent agent RSU takes action according to the state Finally, random noise is introduced to enhance the agent’s ability to explore the environment. When the environment state changes and moves to the next state , at this time the environment will give each agent a corresponding reward , then get experience value And store it in the experience pool, and then sample from the experience pool for training to improve the convergence speed of the algorithm.

[0046] Here the policy set of the intelligent agent RSU is defined as , and its corresponding parameter set is , Represents a set of observations of the environment. t Next, the Actor is in state The observation value is , according to the strategy Randomly select actions for:

[0047] The core of this algorithm is the optimization strategy , by updating its parameters , and finally get an optimal strategy To obtain the long-term expected return value of the intelligent agent. The return of RSU is expressed by the action-state value function .

[0048]

[0049] Each agent RSU will be updated according to its own strategy, where the optimal action is the strategy In the observed state The best choice under , is expressed as:

[0050] Represents the optimal action value function. In order to alleviate the overestimation problem of MADDPG, this implementation uses the target network to calculate the temporal difference (TD) target. For each Actor network, there is a corresponding target Actor, and each Critic network also corresponds to a target Critic network. m , the parameters of the Actor evaluation network and the parameters of the Critic network evaluation network , the parameters of the target Actor network and the target Critic network are respectively and express.

[0051] parameter and Real-time update, the update steps are: first randomly select a small batch of samples from the experience pool p , and input it to the Actor and Critic networks. During the training phase, the Actor and Critic networks update the parameters of the evaluation network based on these samples. The Critic network adjusts the parameters of its evaluation network by minimizing the loss. The loss function can be expressed as:

[0052]

[0053] Next, the goal of each Actor network is to maximize the reward and input the action generated by the decision into the Critic network for gradient update. The parameters of the Critic network can be adjusted by gradient descent. For each RSU agent, the gradient of its expected reward is expressed as:

[0054] Where D represents the experience pool. Subsequently, the parameters of the Actor network are optimized by minimizing the agent's policy gradient. The update direction is expressed as:

[0055] As the network parameters are evaluated and The parameters of the target network are continuously updated and Then, the evaluation network parameters are gradually approached through the soft update method. Finally, each agent updates its target actor and critic network according to the following formula, as shown in the following formula:

[0056]

[0057] in The smaller the value, the more obvious the soft update effect is. The value range is 0 to 1.

[0058] In summary, this embodiment provides a method for coupling computation offloading and content caching in connected vehicles. By employing an elite genetic algorithm and a multi-agent deep deterministic policy gradient (GAC-MADDPG) algorithm, a dual-timescale model is constructed to couple the two. On a large timescale, the cache allocation of RSUs is updated based on historical request preferences and the information age of cached tasks. On a small timescale, computational tasks are offloaded to task vehicles, and the cache hit rate is fed back to the cache allocation algorithm for joint optimization. Compared to single-caching methods, this proposed method is more adaptable to scenarios with high computational demands.

[0059] Implementation 3: The method for coupling vehicle network computing offloading and content caching proposed in any of the above implementations can be implemented entirely using computer software. Therefore, correspondingly, this implementation provides a vehicle network computing offloading and content caching coupling system, the system comprising: a storage device for setting a deployment mode of a roadside unit (RSU); Used to construct a dual-time-scale model, the model includes multiple storage devices of large time scales and small time scales; The system is used to collaboratively update cache decisions at the end of each large-scale cycle. The RSU uses the unloading request of the task vehicle and the popularity model of the task to adopt an improved collaborative elite genetic algorithm. In each small-scale time slot, the multi-agent deep deterministic policy gradient algorithm is used to select the appropriate unloading node according to the cache status and the wireless environment status to make the optimal unloading decision.

[0060] Implementation method 4: This implementation method provides a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes a method for coupling vehicle network computing offloading and content caching as described in any one of the above implementation methods.

[0061] Implementation method five: This implementation method provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for coupling vehicle network computing offloading and content caching as described in any one of the above implementation methods.

[0062] This embodiment provides a computer device, in which the hardware device of this part is a general model and is not shown in the form of a diagram. The system includes a processor and a memory, wherein the processor and the memory can be connected via a bus or other means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, and corresponding program instructions / modules. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, so as to realize a coupling method and steps of vehicle network computing offloading and content caching in the above-mentioned method embodiment.

[0063] The foregoing description is merely an embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of the claims.

Claims

1. A method for coupling computing offloading and content caching in an Internet of Vehicles, characterized in that: The method is: S1: Set the deployment mode of the roadside unit RSU; S2: Construct a dual-time-scale model, which includes multiple large and small time scales; S3: At the end of each large-scale cycle, the RSU uses an improved collaborative elite genetic algorithm to collaboratively update the cache decision based on the unloading request of the task vehicle and the popularity model of the task. In each small-scale time slot, a multi-agent deep deterministic policy gradient algorithm is used to select the appropriate unloading node according to the cache status and wireless environment status to make the optimal unloading decision.

2. The method for coupling computing offloading and content caching in an Internet of Vehicles according to claim 1, characterized in that: The specific deployment methods are: Multiple RSUs are deployed on the roadside, and each RSU is equipped with a MEC server and a cache server for computing and cache query services respectively; A base station is set up at the center of multiple RSU groups to store the cache tasks of all RSUs in the area.

3. The method for coupling computing offloading and content caching in an Internet of Vehicles according to claim 1, characterized in that: The dual time scale model is constructed as follows: The entire time system is divided into A large time scale, and define the length of each large time scale period as , the set is ,use Indicates the A large time scale cycle; Then, in each large time period, it is divided into time slots, and the length of each time slot is , as a small time scale, and represented by an index set ,use Expressed as The first Hourly time scale period.

4. The method for coupling computing offloading and content caching in an Internet of Vehicles according to claim 1, characterized in that: The improved collaborative elite genetic algorithm is used to collaboratively update cache decisions as follows: Parameter encoding and initial population setting for the genetic algorithm include: initializing the population and generating a set of random solutions as the initial exploration points of the search space. One gene represents a cached task, and each individual represents the complete cache strategy solution for each RSU. Add a fitness function to the genetic algorithm to quantitatively evaluate the retention of each individual, and add overhead indicator optimization; An improved elite selection strategy is adopted, which keeps the best solution of each generation and passes it directly to the next generation until the optimal solution of caching decision is obtained.

5. The method for coupling computing offloading and content caching in Internet of Vehicles according to claim 4, characterized in that: The improved elite selection strategy is as follows: For the Xth generation population , find the current population The maximum fitness value ; Generate the next generation of intermediate temporary population , and find the temporary population The maximum fitness value ; like , then Medium fitness is greater than or equal to The number of individual copies is recorded as , then randomly generate Y New individuals are generated by combining the random parent with the original parent. N children, and then perform a replacement operation, using N Subsequent replacement The lowest fitness N individuals, and obtain a new generation of population ; like , then directly retain the temporary population For the new generation of population .

6. The method for coupling computing offloading and content caching in an Internet of Vehicles according to claim 4, characterized in that: The network structure of the multi-agent deep deterministic policy gradient algorithm includes an Actor network and a Critic network.

7. The method for coupling computing offloading and content caching in an Internet of Vehicles according to claim 6, characterized in that: The optimal unloading decision is made using a multi-agent deep deterministic policy gradient algorithm as follows: A new cache-aware state space is constructed to first determine whether the requested content is in the RSU cache when the offload target is the RSU; Constructing the offloading decision action space, including offloading node selection decision and task offloading ratio ; Construct a cache collaborative reward function, set the hit reward as: directly use the cache content for incentives, and the miss penalty as: promote the improved collaborative elite genetic algorithm to optimize the cache strategy to make the best offloading decision.

8. A coupled system for vehicle network computing offloading and content caching, the system comprising a storage device configured to execute the method and steps described in claim 1.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the method for coupling vehicle network computing offloading and content caching as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for coupling vehicle network computing offloading and content caching as described in any one of claims 1 to 7.

Images