A heterogeneous resource allocation method for Internet of Vehicles based on cloud-edge-device collaborative computing

Abstract

The present invention discloses a method for allocating heterogeneous resources in an Internet of Vehicles (IoV) based on cloud-edge-end collaborative computing, and belongs to the field of mobile edge computing technology. The present invention combines collaborative filtering and attribute-based prediction to perform multi-objective optimized content popularity prediction, and the prediction results are used to guide the caching strategy of content in the cloud-edge-end network, reducing latency and improving cache hit rate; utilizing a multi-granularity task decomposition algorithm and a dual-depth Q network task scheduling algorithm to achieve fine-grained task decomposition and effective resource allocation, maximizing the weighted utility function of latency and energy consumption; in the vehicle edge network, each edge server can independently perform service migration and allocate resources for the service migration of other edge servers, utilizing a deep deterministic policy gradient algorithm to solve the model free Markov decision process problem, maximizing the total expected utility of services unloaded by the vehicle. The present invention can improve the overall performance and service quality of the network, and is particularly suitable for highly dynamic environments.

Description

Technical Field

[0001] The present invention belongs to the field of mobile edge computing technology, and specifically relates to a method for allocating heterogeneous resources in an Internet of Vehicles (IoV) based on cloud-edge-end collaborative computing. Background Art

[0002] With the rapid development of mobile communication and Internet of Things (IoT) technologies and the increasing demand for personalized services, the requirements for network quality of service (QoS) and user experience quality (QoE) are also increasing. The popularity of IoT devices has laid the foundation for the development of emerging IoT applications such as smart homes, smart cities, and virtual / augmented reality (VR / AR) in IoT networks, which has also led to a surge in mobile data traffic. To meet these demands, content pre-caching and efficient allocation of computing tasks at the network edge are required. Existing content caching strategies based on popularity prediction improve network service quality and user experience quality by pre-transmitting content at the network edge. However, jointly optimizing multiple network indicators remains a challenge, and traditional methods often have difficulty handling resource heterogeneity and network environment variability, especially in the highly dynamic Internet of Vehicles environment.

[0003] At the same time, emerging computing paradigms offer effective solutions to the conflict between the high computational demands of mobile devices and their limited capacity. This computing architecture has transitioned from a single-layer architecture to a multi-layer collaborative architecture, enhancing its flexibility to accommodate finer-grained computational offloading. Despite this, existing research on multi-layer computational offloading still faces a series of challenges, including inefficient resource awareness and task decomposition, and a lack of effective hierarchical task scheduling strategies within the multi-layer collaborative architecture.

[0004] The development of connected vehicle (IoV) technology is trending towards using widely distributed edge servers to support various latency-sensitive services within vehicles. However, the mobility and randomness of vehicles, as well as the heterogeneity of services, place higher demands on the quality of service required. Edge servers should migrate vehicle services to other edge servers based on the vehicle's mobility patterns. However, the randomness of vehicle mobility leads to randomness in the number and duration of computing services hosted by edge servers, which in turn affects service quality. Ensuring efficient dynamic resource allocation and service migration within edge servers is crucial. Summary of the Invention

[0005] The present invention provides a method for allocating heterogeneous resources in the Internet of Vehicles (IoV) based on cloud-edge-end collaborative computing. It combines collaborative filtering and attribute-based prediction to perform multi-objective optimized content popularity prediction. The prediction results are used to guide the caching strategy of content in the cloud-edge-end network, reducing latency and improving cache hit rate. It utilizes a multi-granularity task decomposition algorithm and a dual-depth Q network task scheduling algorithm to achieve fine-grained task decomposition and effective resource allocation, maximizing the weighted utility function of latency and energy consumption. In the vehicle edge network, each edge server can independently perform service migration and allocate resources for the service migration of other edge servers. It utilizes a deep deterministic policy gradient algorithm to solve the model free Markov decision process problem, maximizing the total expected utility of services unloaded by the vehicle, thereby solving at least one technical problem involved in the background technology.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] A method for allocating heterogeneous resources in an Internet of Vehicles (IoV) based on cloud-edge-device collaborative computing includes the following steps:

[0008] Step S1: Collect historical user behavior and use user-based collaborative filtering algorithms, item-based collaborative filtering algorithms, and attribute-based collaborative filtering algorithms to predict user product preferences. A multi-objective evolutionary algorithm is used to weight the prediction results of the three algorithms to determine the final prediction result. Based on the final prediction result, a cache replacement table is generated for the edge server and vehicle terminal.

[0009] Step S2: Collect the resource usage of all computing nodes in the edge server to form an input matrix, feed the input matrix into the LSTM model for training, and output the prediction result of the remaining resources of each computing node;

[0010] Step S3: Set a granularity level for the tasks to be offloaded by the vehicle terminal. Based on the granularity level, determine the resource threshold required for offloading each task. Determine whether the remaining resources of the computing node meet the resource threshold requirement. If so, offload the task to the corresponding computing node. Otherwise, decompose the task into smaller subtasks until the remaining resources of the computing node meet the resource threshold requirement for offloading the subtask, and then offload the subtask to the corresponding computing node.

[0011] Step S4: construct a dual-depth Q network model, optimize the computing node selection process for task offloading with the goal of maximizing the weighted utility function of delay and energy consumption, determine the optimal computing node for task offloading, and offload the task to the optimal computing node;

[0012] Step S5: construct a vehicle edge network within the target area. Within the vehicle edge network, each edge server independently performs service migration and allocates resources for the service migration of other edge servers. A Markov decision model is constructed with the optimization goal of maximizing the total expected utility of all services unloaded by vehicles in the vehicle edge network. A deep deterministic policy gradient algorithm is used to approximate the Markov decision model to determine the optimal solution for service migration and resource allocation for each edge server in the vehicle edge network.

[0013] As a preferred improvement, the prediction process of the multi-objective evolutionary algorithm is expressed as:

[0014]

[0015] Where, and Respectively represent user-based collaborative filtering, item-based collaborative filtering and attribute-based collaborative filtering as the predicted score of user u for item k at time τ; α τ and β τ Represents the weight coefficient.

[0016] As a preferred improvement, in the user-based collaborative filtering algorithm, the similarity between items k1 and k2 is expressed using the cosine similarity formula:

[0017]

[0018] Where, and Represents user groups The rating vector given to items k1 and k2; and Represent the ratings of user u on products k1 and k2 respectively;

[0019] The predicted rating of user u for item k Calculated by the following formula:

[0020]

[0021] Where N ei (k) represents the set of neighbors of item k, that is, the set of items with high similarity to item k; represents the sum of the actual ratings of user u on neighboring items k' related to item k; Represents the similarity between item k and item k'.

[0022] As a preferred improvement, in the item-based collaborative filtering algorithm, cosine similarity is used to represent the similarity between items:

[0023]

[0024] Where r uk represents the rating of user u on item k, represents the average rating of item k, and U(k1,k2) represents the set of users who have rated items k1 and k2 at the same time;

[0025] For a user u who has not rated item k, the predicted rating for item k is Calculated by the following formula:

[0026]

[0027] Where N(k) is the set of items closest to item k, r uk' is the actual rating of user u on item k'.

[0028] As a preferred improvement, in the collaborative filtering algorithm based on attribute prediction, for items k1 and k2, their corresponding feature vectors are x k1 and x k2 , the item feature vector represents the attribute of the item, and the cosine similarity is used to measure the angle between the two vectors to calculate the similarity between items:

[0029]

[0030] where x k1 ·x k2 Represents vector x k1 and x k2 The dot product of ||x k1 || and ||x k2 || respectively represent vector x k1 and x k2 The norm of

[0031] Then user u's predicted rating for item k is It can be calculated by the following formula:

[0032]

[0033] Where, P u represents the preference vector of user u, b u represents the bias term of user u, which is used to adjust the overall rating level; T represents the transposed matrix.

[0034] As a preferred improvement, the LSTM model is expressed as:

[0035] h t =σ(x t , h t-1 , c t-1 );

[0036] Where h t represents the hidden state of the LSTM model at time step t; c t-1 represents the state of the LSTM model at time step t-1; x t represents the slice of the input matrix MX at the same time step; σ(·) represents the sigmoid function;

[0037] The output Y of the LSTM model is expressed as:

[0038] Y=σ(M X );

[0039] The prediction result represents the hidden state h of the last time step t , we need to add a fully connected layer to represent it concretely, then the actual output Y′ of the LSTM model is expressed as:

[0040] Y′=W fc ×h t +b fc ;

[0041] Where W fc represents the weight matrix of the fully connected layer, b fc Represents the bias vector.

[0042] As a preferred improvement, the specific steps of the multi-granularity task decomposition algorithm are as follows:

[0043] Task graph structure modeling: The computing task is represented as a directed acyclic graph. Each node in the directed acyclic graph represents a computing operation, and each edge represents the dependency relationship between operations.

[0044] Multi-granularity task decomposition: Based on the results of LSTM resource prediction, modular decomposition is performed to decompose one or more nodes in the directed acyclic graph into a subtask;

[0045] Subtask mapping: defines a mapping for assigning subtasks to available compute nodes, so that a task can only be assigned one subtask to a compute node and each task can be deployed on a maximum of K nodes, where K is equal to the total number of subtasks.

[0046] As a preferred improvement, the dual-depth Q network optimization process specifically includes the following steps:

[0047] Initialization: Initialize the parameters of the estimation network Q and the target network Q′, which have the same structure but different parameters θ and θ′;

[0048] Interact with the environment to collect experience: At each time step t, according to the current state s of the agent t Select and perform action at , observe the new state s t+1 and reward r t , and then (s t , a t , r t , s t+1 ) is stored in the experience replay pool D; wherein the action space can be expressed as:

[0049]

[0050] Where, represents the action vector of the i-th node at time t; a i represents the offloading decision vector, indicating whether to offload the task to node i; o i Represents the task allocation vector, which indicates the distribution of tasks unloaded to node i; N nodes Represents the number of nodes; where a i =[a1(t),a2(t),...,a n (t)],a n (t)∈{0,1},a n (t) = 1 means unloading the task, a n (t) = 0 means not offloading the task;

[0051] Update the parameters of the estimated network Q: use the current state s t The Q value Q(s t ,a t ;θ) and the next state s t+1 The Q-value Q'(s) of the best action a' in t+1 ,a';θ') to calculate the target value r t +γQ'(s t+1 ,a';θ'), update the parameter θ of the estimation network Q so that it is adjusted in the direction of reducing the loss function. The loss function is defined as the mean square error between the estimated Q value and the true Q value;

[0052] Target network update: Every certain number of steps N, the parameters θ of the estimated network Q are copied to the target network Q';

[0053] Iteration: Continue the above steps until a stopping condition is met, such as reaching a predetermined number of training rounds or the environment's terminal state.

[0054] As a preferred improvement, the optimization process of the Markov decision model specifically includes the following steps:

[0055] Initialization state and actions: At the beginning of each time slot, the edge server's state includes: the number of all vehicle terminals in the current coverage area, the number of possible service migrations, and the amount of computing resources available; the actions taken by the edge server include: the number of bandwidth resource blocks and computing resource blocks allocated to each vehicle terminal, and the number of service migrations to be initiated;

[0056] State representation: Use vectors to represent the current state of the edge server;

[0057] Action selection: Based on the current state of the edge server, the policy network in the deep deterministic policy gradient algorithm is used to select an action;

[0058] Execute actions and observe rewards: Based on the selected actions, resource allocation and service migration are actually performed; based on the results of the executed actions, rewards are calculated; rewards are calculated based on service delays. If the service delay is lower than or equal to a threshold, the reward is positive; if the service delay exceeds the threshold, the reward gradually decreases until it becomes zero;

[0059] Update state, policy, and value function: Based on the executed actions and results, the edge server's state at the next moment is updated. The value network in the deep deterministic policy gradient algorithm is used to evaluate the value of the selected action and update the policy network accordingly. At the same time, experience replay is used to update the value network based on the actual actions executed and the observed rewards.

[0060] Loop iteration: Repeat the above steps until the predetermined training cycle is reached or the termination condition is met.

[0061] The beneficial effects of the present invention are:

[0062] (1) Content popularity prediction through multi-objective optimization provides an important prerequisite for task decomposition and hierarchical scheduling. By predicting the popularity of content, the cache location of popular content can be arranged in advance, so that the location information of this content can be taken into account when decomposing tasks, reducing latency and improving cache hit rate. This prediction can also help optimize the granularity of task decomposition, ensuring that tasks can be effectively assigned to the nodes that are most suitable for processing them;

[0063] (2) Task decomposition and hierarchical scheduling further utilize the results of content popularity prediction from multi-objective optimization to optimize resource allocation. Through efficient resource perception and task scheduling, it can ensure that popular content can be quickly accessed and computing resources can be efficiently utilized. This not only reduces network latency but also improves the overall system response speed and user experience.

[0064] (3) Service migration and resource allocation further optimizes the resource allocation and task processing process based on multi-objective optimization of content popularity prediction, task decomposition, and hierarchical scheduling. Through intelligent service migration decisions, it can ensure that the service always runs on the best edge server, thereby reducing latency and optimizing overall network performance.

[0065] (3) Through the organic combination of the above technologies, the present invention can significantly improve the performance and service quality of the cloud-edge-end collaborative computing network, and is particularly suitable for highly dynamic Internet of Vehicles environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart showing a method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] See also Figure 1 The present invention provides a method for allocating heterogeneous resources in an Internet of Vehicles (IoV) based on cloud-edge-device collaborative computing, comprising the following steps:

[0069] Step S1: Collect the user's historical behavior, use the user-based collaborative filtering algorithm, the item-based collaborative filtering algorithm, and the attribute-based collaborative filtering algorithm to predict the user's product preferences respectively, use the multi-objective evolutionary algorithm to weight the prediction results of the three algorithms, determine the final prediction result, and generate the cache replacement form for the edge server and vehicle terminal based on the final prediction result.

[0070] The basic idea of ​​user-based collaborative filtering algorithms is to recommend products to users that are similar to their interests and are preferred by other users. The basic idea of ​​item-based collaborative filtering algorithms is to recommend items similar to items previously liked by users. The basic idea of ​​attribute-based collaborative filtering algorithms is to recommend items similar to their own attributes or to items. The technical solution of the present invention simultaneously predicts user preferences from three perspectives and then weights the prediction results of the three algorithms. This improves prediction accuracy. The prediction results are used to guide cache replacement strategies, thereby improving cache hit rates and reducing delays caused by reloading due to mismatches between cache and user preferences.

[0071] In the user-based collaborative filtering algorithm, the similarity between items k1 and k2 is expressed using the cosine similarity formula:

[0072]

[0073] Where, and Represents user groups The rating vector given to items k1 and k2; and Represent the ratings of user u on products k1 and k2 respectively;

[0074] The predicted rating of user u for item k Calculated by the following formula:

[0075]

[0076] Where N ei (k) represents the set of neighbors of item k, that is, the set of items with high similarity to item k; represents the sum of the actual ratings of user u on neighboring items k' related to item k; Represents the similarity between item k and item k'.

[0077] The item-based collaborative filtering algorithm is similar to the user-based collaborative filtering algorithm, and the cosine similarity is used to represent the similarity between items:

[0078]

[0079] Where r uk represents the rating of user u on item k, represents the average rating of item k, and U(k1,k2) represents the set of users who have rated items k1 and k2 at the same time.

[0080] For a user u who has not rated item k, the predicted rating for item k is Calculated by the following formula:

[0081]

[0082] Where N(k) is the set of items closest to item k, r uk' is the actual rating of user u on item k'.

[0083] In the collaborative filtering algorithm based on attribute prediction, for two items k1 and k2, their corresponding feature vectors are x k1 and x k2 , the item feature vector represents the attribute of the item, and the cosine similarity is used to measure the angle between the two vectors to calculate the similarity between items:

[0084]

[0085] where x k1 ·x k2 Represents vector x k1 and x k2 The dot product of ||x k1 || and ||x k2 || respectively represent vector x k1 and x k2 The norm of .

[0086] Assume that user u’s preference can be represented by a vector P u Represents that user u’s predicted rating for item k is It can be calculated by the following formula:

[0087]

[0088] Where, P u is the preference vector of user u, b u is the bias term of user u, which is used to adjust the overall rating level; T represents the transposed matrix.

[0089] By building a multi-objective optimized linear weighted rating prediction model to integrate collaborative filtering and attribute-based prediction:

[0090]

[0091] Where, and Respectively represent user-based collaborative filtering, item-based collaborative filtering and attribute-based collaborative filtering as the predicted score of user u for item k at time τ; α τ and β τ represents the weight coefficients, which are optimized at each time point through a multi-objective evolutionary algorithm to discover the best combination weights suitable for the current Internet of Vehicles network environment.

[0092] Based on the different service capabilities and coordination states of edge and end nodes, and the predicted user preferences for items, a cache replacement list for edge and end nodes is generated. The combined weights of the three optimize the basic prediction algorithm to simultaneously enhance and balance optimization objectives.

[0093] In step S2, the resource usage of all computing nodes in the edge server is collected to form an input matrix, which is fed into the LSTM model for training and outputs the prediction result of the remaining resources of each computing node.

[0094] The LSTM model is represented as:

[0095] h t =σ(xt ,h t-1 ,c t-1 );

[0096] Where h t represents the hidden state of the LSTM model at time step t; c t-1 represents the state of the LSTM model at time step t-1; x t represents the slice of the input matrix MX at the same time step; σ(·) represents the sigmoid function;

[0097] The output Y of the LSTM model is expressed as:

[0098] Y=σ(M X ).

[0099] The prediction result represents the hidden state h of the last time step t , we need to add a fully connected layer to represent it concretely, then the actual output Y' of the LSTM model is expressed as:

[0100] Y'=W fc ×h t +b fc ;

[0101] Where W fc represents the weight matrix of the fully connected layer, b fc Represents the bias vector.

[0102] Step S3, set the granularity level for the tasks that need to be unloaded by the vehicle terminal, determine the resource threshold required for unloading each task based on the granularity level, and judge whether the remaining resources of the computing node meet the resource threshold requirements. If so, unload the task to the corresponding computing node; otherwise, decompose the task into smaller subtasks until the remaining resources of the computing node meet the resource threshold requirements for unloading the subtask, and unload the subtask to the corresponding computing node.

[0103] The specific steps of the multi-granularity task decomposition algorithm are as follows:

[0104] Task graph structure modeling: The computing task is represented as a directed acyclic graph (DAG), where each node of the DAG represents a computing operation and each edge represents the dependency relationship between operations. Based on modular decomposition, one or more nodes in the DAG can be decomposed into a module (i.e., subtask). If the computing nodes in the DAG are independent of other nodes, each node or multiple nodes can be decomposed into a subtask. On the contrary, if there is a dependency relationship, such as a subsequent computing node depends on the result of its previous node, these interdependent nodes will be decomposed into a subtask to ensure computing accuracy;

[0105] Multi-granularity task decomposition: Based on the results of LSTM resource prediction, the task is decomposed into k uneven subtasks. Since the remaining resources of the node will change dynamically and the amount will vary, the LSTM prediction results are used at time point X. t Break down the tasks.

[0106] Subtask mapping: defines the mapping that assigns subtasks to available computing nodes. Let x ik Indicates whether the kth subtask is assigned to node i, that is, x ik ∈{0,1};

[0107]

[0108] This means that a task can only assign one subtask to a computing node; and:

[0109]

[0110] This means that each task can be deployed on a maximum of K nodes, where K is equal to the total number of subtasks.

[0111] For each subtask k∈K, there is τ n'k,n ≥0, that is, the task start time must be non-negative, and

[0112]

[0113] Where, d k-1n',n represents the amount of data sent from the k-1th subtask, B k-1n',n represents the bandwidth from node n to its predecessor node n'.

[0114] Through the above steps, the task can be effectively decomposed into multiple subtasks according to the resource conditions of each node in the current system, and further scheduled to the nodes that best suit their respective resource requirements for processing.

[0115] Step S4: construct a dual-depth Q network model, optimize the computing node selection process for task offloading with the goal of maximizing the weighted utility function of delay and energy consumption, determine the optimal computing node for task offloading, and offload the task to the optimal computing node.

[0116] The principle of the dual-depth Q network is that at any time t, the agent takes action a t And act on the environment to achieve interaction with the environment. After the interaction is completed, the state of the agent is determined by s t Transformations t+1 , set the value function to evaluate action a t The value of r t , the agent is based on the value r tContinuously optimize the behavior trajectory and eventually learn the optimal behavior strategy. The value function includes the state value function and the action value function. The action value function (Q value) represents the action value of the agent under the guidance of the strategy π according to the current state s t , take action a t The expected return obtained. The strategy π represents the mapping from state to behavior, which is equivalent to the decision-making strategy of the agent, selecting different behaviors according to the different states of the agent.

[0117] The network structure of the dual-depth Q network consists of an estimation network Q and a target network Q'. The two networks have the same structure but different parameters. The estimation network Q has the latest network parameters, calculates the value of the current state-action pair, and regularly updates the parameters of the target network Q'. The action corresponding to the maximum action value function (Q value) is selected through the estimation network Q, and then the target network calculates the target Q value based on the corresponding action. Through the continuous interaction between the intelligent agent and the environment, experience tuples are generated and stored in the experience replay buffer. A priority experience replay mechanism is used to improve the efficiency of reusing historical experience. Every certain number of steps, the parameters of the estimation network Q are updated to the target network Q', keeping the structure the same but the parameters different. The estimated Q value of the estimation network Q and the true Q value of the target network Q' are calculated, and the loss function is calculated based on these two. The parameters of the estimation network Q are updated using the gradient descent method.

[0118] When offloading computing tasks, the main considerations include whether to offload the task and to which node. Therefore, the action space can be expressed as:

[0119]

[0120] Where, represents the action vector of the i-th node at time t; a i represents the offloading decision vector, indicating whether to offload the task to node i; o i Represents the task allocation vector, which indicates the distribution of tasks unloaded to node i; N nodes Represents the number of nodes; where a i =[a1(t), a2(t), ..., a n (t)],a n (t)∈{0,1},a n (t) = 1 means unloading the task, a n (t)=0 means not to offload the task.

[0121] The dual-depth Q network optimization process specifically includes the following steps:

[0122] Initialization: Initialize the parameters of the estimation network Q and the target network Q′, which have the same structure but different parameters θ and θ′;

[0123] Interact with the environment to collect experience: At each time step t, according to the current state s of the agent t Select and perform action a t , observe the new state s t+1 and reward r t , and then (s t , a t , r t , s t+1 ) is stored in the experience replay pool D;

[0124] Update the parameters of the estimated network Q: use the current state s t The Q value Q(s t , a t ;θ) and the next state s t+1 The Q-value Q′(s t+1 , a′; θ′) to calculate the target value r t +γQ′(s t+1 , a′; θ′), update the parameters θ of the estimation network Q so that it is adjusted in the direction of reducing the loss function. The loss function is defined as the mean square error (MSE) between the Q value estimate and the true value of the Q value;

[0125] Target network update: Every certain number of steps N, the parameters θ of the estimated network Q are copied to the target network Q′;

[0126] Iteration: Continue the above steps until a stopping condition is met, such as reaching a predetermined number of training rounds or the environment's terminal state.

[0127] Step S5: construct a vehicle edge network within the target area. Within the vehicle edge network, each edge server independently performs service migration and allocates resources for the service migration of other edge servers. A Markov decision model is constructed with the optimization goal of maximizing the total expected utility of all services unloaded by vehicles in the vehicle edge network. A deep deterministic policy gradient algorithm is used to approximate the Markov decision model to determine the optimal solution for service migration and resource allocation for each edge server in the vehicle edge network.

[0128] The optimization process of the Markov decision model specifically includes the following steps:

[0129] Initialization state and actions: At the beginning of each time slot, the state of the edge server includes: the number of all vehicle terminals in the current coverage area, the number of possible service migrations, and the amount of computing resources available; the actions taken by the edge server include: the number of bandwidth resource blocks (BRBs) and computing resource blocks (CRBs) allocated to each vehicle terminal, and the number of service migrations to be initiated;

[0130] State representation: Use vectors to represent the current state of the edge server;

[0131] Action selection: Based on the current state of the edge server, the policy network (Actor) in the Deep Deterministic Policy Gradient (DDPG) algorithm is used to select an action;

[0132] Execute actions and observe rewards: Based on the selected actions, resource allocation and service migration are actually performed; based on the results of the executed actions, rewards are calculated; rewards are calculated based on service delays. If the service delay is lower than or equal to a threshold, the reward is positive; if the service delay exceeds the threshold, the reward gradually decreases until it becomes zero;

[0133] Update state, policy, and value function: Based on the executed actions and results, the edge server's state at the next moment is updated. The value network (Critic) in the DDPG algorithm is used to evaluate the value of the selected action and update the policy network (Actor) accordingly. At the same time, experience replay is used to update the value network (Critic) based on the actual executed actions and observed rewards.

[0134] Loop iteration: Repeat the above steps until the predetermined training cycle is reached or the termination condition is met.

[0135] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A method for allocating heterogeneous resources in an Internet of Vehicles (IoV) based on cloud-edge-device collaborative computing, characterized in that: The steps include: Step S1: Collect historical user behavior and use user-based collaborative filtering algorithms, item-based collaborative filtering algorithms, and attribute-based collaborative filtering algorithms to predict user product preferences. A multi-objective evolutionary algorithm is used to weight the prediction results of the three algorithms to determine the final prediction result. Based on the final prediction result, a cache replacement table is generated for the edge server and vehicle terminal. Step S2: Collect the resource usage of all computing nodes in the edge server to form an input matrix, feed the input matrix into the LSTM model for training, and output the prediction result of the remaining resources of each computing node; Step S3: Set a granularity level for the tasks to be offloaded by the vehicle terminal. Based on the granularity level, determine the resource threshold required for offloading each task. Determine whether the remaining resources of the computing node meet the resource threshold requirement. If so, offload the task to the corresponding computing node. Otherwise, decompose the task into smaller subtasks until the remaining resources of the computing node meet the resource threshold requirement for offloading the subtask, and then offload the subtask to the corresponding computing node. Step S4: construct a dual-depth Q network model, optimize the computing node selection process for task offloading with the goal of maximizing the weighted utility function of delay and energy consumption, determine the optimal computing node for task offloading, and offload the task to the optimal computing node; Step S5: construct a vehicle edge network within the target area. Within the vehicle edge network, each edge server independently performs service migration and allocates resources for the service migration of other edge servers. A Markov decision model is constructed with the optimization goal of maximizing the total expected utility of all services unloaded by vehicles in the vehicle edge network. A deep deterministic policy gradient algorithm is used to approximate the Markov decision model to determine the optimal solution for service migration and resource allocation for each edge server in the vehicle edge network.

2. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 1 is characterized in that: The prediction process of the multi-objective evolutionary algorithm is expressed as: Where, and Respectively represent user-based collaborative filtering, item-based collaborative filtering and attribute-based collaborative filtering as the predicted score of user u for item k at time τ; α τ and β τ Represents the weight coefficient.

3. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 2 is characterized in that: In the user-based collaborative filtering algorithm, the similarity between items k1 and k2 is expressed using the cosine similarity formula: Where, and Represents user groups The rating vector given to items k1 and k2; and Represent the ratings of user u on products k1 and k2 respectively; The predicted rating of user u for item k Calculated by the following formula: Where N ei (k) represents the set of neighbors of item k, that is, the set of items with high similarity to item k; represents the sum of the actual ratings of user u on neighboring items k' related to item k; Represents the similarity between item k and item k'.

4. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 2 is characterized in that: In the item-based collaborative filtering algorithm, cosine similarity is used to represent the similarity between items: Where r uk represents the rating of user u on item k, represents the average rating of item k, and U(k1,k2) represents the set of users who have rated items k1 and k2 at the same time; For a user u who has not rated item k, the predicted rating for item k is Calculated by the following formula: Where N(k) is the set of items closest to item k, r uk' is the actual rating of user u on item k'.

5. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 2 is characterized in that: In the collaborative filtering algorithm based on attribute prediction, for items k1 and k2, their corresponding feature vectors are x k1 and x k2 , the item feature vector represents the attribute of the item, and the cosine similarity is used to measure the angle between the two vectors to calculate the similarity between items: where x k1 ·x k2 Represents vector x k1 and x k2 The dot product of ||x k1 || and ||x k2 || respectively represent vector x k1 and x k2 The norm of Then user u's predicted rating for item k is It can be calculated by the following formula: Where, P u represents the preference vector of user u, b u represents the bias term of user u, which is used to adjust the overall rating level; T represents the transposed matrix.

6. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 1 is characterized in that: The LSTM model is represented as: h t =σ(x t ,h t-1 ,c t-1 ); Where h t represents the hidden state of the LSTM model at time step t; c t-1 represents the state of the LSTM model at time step t-1; x t represents the slice of the input matrix MX at the same time step; σ(·) represents the sigmoid function; The output Y of the LSTM model is expressed as: Y=σ(M X ); The prediction result represents the hidden state h of the last time step t , we need to add a fully connected layer to represent it concretely, then the actual output Y' of the LSTM model is expressed as: Y'=W fc ×h t +b fc ; Where W fc represents the weight matrix of the fully connected layer, b fc Represents the bias vector.

7. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 6 is characterized in that: The specific steps of the multi-granularity task decomposition algorithm are as follows: Task graph structure modeling: The computing task is represented as a directed acyclic graph. Each node in the directed acyclic graph represents a computing operation, and each edge represents the dependency relationship between operations. Multi-granularity tasks Decomposition: Based on the results of LSTM resource prediction, decomposition is performed based on modularity, decomposing one or more nodes from the directed acyclic graph into a subtask; Subtask mapping: defines a mapping for assigning subtasks to available compute nodes, so that a task can only be assigned one subtask to a compute node and each task can be deployed on a maximum of K nodes, where K is equal to the total number of subtasks.

8. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 1 is characterized in that: The dual-depth Q network optimization process specifically includes the following steps: Initialization: Initialize the parameters of the estimation network Q and the target network Q', both of which have the same structure but different parameters θ and θ'; Interact with the environment to collect experience: At each time step t, according to the current state s of the agent t Select and perform action a t , observe the new state s t+1 and reward r t , and then (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool D; wherein the action space can be expressed as: Where, represents the action vector of the i-th node at time t; a i represents the offloading decision vector, indicating whether to offload the task to node i; o i Represents the task allocation vector, which indicates the distribution of tasks unloaded to node i; N nodes Represents the number of nodes; where a i =[a1(t),a2(t),...,a n (t)],a n (t)∈{0,1},a n (t) = 1 means unloading the task, a n (t) = 0 means not offloading the task; Update the parameters of the estimated network Q: use the current state s t The Q value Q(s t ,a t ;θ) and the next state s t+1 The Q-value Q'(s) of the best action a' in t+1 ,a';θ') to calculate the target value r t +γQ'(s t+1 ,a';θ'), update the parameter θ of the estimation network Q so that it is adjusted in the direction of reducing the loss function. The loss function is defined as the mean square error between the estimated Q value and the true Q value; Target network update: Every certain number of steps N, the parameters θ of the estimated network Q are copied to the target network Q'; Iteration: Continue the above steps until a stopping condition is met, such as reaching a predetermined number of training rounds or the environment's terminal state.

9. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 1 is characterized in that: The optimization process of the Markov decision model specifically includes the following steps: Initialization state and actions: At the beginning of each time slot, the edge server's state includes: the number of all vehicle terminals in the current coverage area, the number of possible service migrations, and the amount of computing resources available; the actions taken by the edge server include: the number of bandwidth resource blocks and computing resource blocks allocated to each vehicle terminal, and the number of service migrations to be initiated; State representation: Use vectors to represent the current state of the edge server; Action selection: Based on the current state of the edge server, the policy network in the deep deterministic policy gradient algorithm is used to select an action; Execute actions and observe rewards: Based on the selected actions, resource allocation and service migration are actually performed; based on the results of the executed actions, rewards are calculated; rewards are calculated based on service delays. If the service delay is lower than or equal to a threshold, the reward is positive; if the service delay exceeds the threshold, the reward gradually decreases until it becomes zero; Update state, policy, and value function: Based on the executed actions and results, the edge server's state at the next moment is updated. The value network in the deep deterministic policy gradient algorithm is used to evaluate the value of the selected action and update the policy network accordingly. At the same time, experience replay is used to update the value network based on the actual actions executed and the observed rewards. Loop iteration: Repeat the above steps until the predetermined training cycle is reached or the termination condition is met.

Images