Internet of vehicles heterogeneous resource allocation method based on cloud side-end cooperative computing

Abstract

The invention discloses an Internet of Vehicles heterogeneous resource allocation method based on cloud edge-end cooperative computing, and belongs to the technical field of mobile edge computing. According to the method, the content popularity prediction of multi-target optimization is carried out by combining collaborative filtering and attribute basis prediction, the prediction result is used for guiding a caching strategy of the content in a cloud-edge-end network, delay is reduced, and the cache hit rate is increased; a multi-granularity task decomposition algorithm and a dual-depth Q network task scheduling algorithm are utilized to realize fine-granularity decomposition of tasks and effective allocation of resources, and a weighted utility function of delay and energy consumption is maximized; in a vehicle edge network, each edge server can independently execute service migration and allocate resources for service migration of other edge servers, a deep deterministic policy gradient algorithm is utilized to solve a model free Markov decision process problem, and the total expected utility of services unloaded by a vehicle is maximized. The method can improve the overall performance and service quality of the network, and is especially suitable for a highly dynamic environment.

Description

Technical Field

[0001] The present invention belongs to the field of mobile edge computing technology, and specifically relates to a method for allocating heterogeneous resources in an Internet of Vehicles based on cloud-edge-end collaborative computing. Background Art

[0002] With the rapid development of mobile communication technology and Internet of Things (IoT) technology and the increasing demand for personalized services, the requirements for matching network quality of service (QoS) and quality of user experience (QoE) are also increasing. The popularity of IoT devices has provided a foundation for the development of emerging IoT applications such as smart homes, smart cities, and virtual / augmented reality (VR / AR) in IoT networks, which has also led to a surge in mobile data traffic. In order to meet these needs, content pre-caching and efficient allocation of computing tasks at the network edge are required. Existing content caching strategies based on popularity prediction improve network service quality and user experience quality by pre-transmitting content at the network edge, but jointly optimizing multiple network indicators remains a challenge, and for traditional methods, especially in highly dynamic Internet of Vehicles environments, it is often difficult to handle the heterogeneity of resources and the variability of the network environment.

[0003] At the same time, the emerging computing paradigm provides an effective solution to the conflict caused by the high computing demand of mobile terminals and their limited capacity. This computing architecture has transitioned from a single-layer architecture to a multi-layer collaborative architecture, thereby enhancing its flexibility to adapt to finer-grained computing offloading. Nevertheless, existing multi-layer computing offloading research still faces a series of problems such as inefficient resource perception and task decomposition, and the lack of effective hierarchical task scheduling strategies in multi-layer collaborative architectures.

[0004] For the Internet of Vehicles technology, the development of vehicle networks shows a trend of using its widely distributed edge servers to support various delay-sensitive services for vehicles. However, the mobility and randomness of vehicles, as well as the heterogeneity of services, put forward higher requirements on the quality of service required in reality. The edge server should migrate the vehicle's services to other edge servers according to the vehicle's mobility pattern. However, the vehicle's mobility pattern is random, which causes the number of computing services hosted by the edge server and the duration of the service to become random, which in turn affects the quality of service. How to ensure the effective allocation of dynamic resources and service migration of edge servers has become the key. Summary of the invention

[0005] The present invention provides a method for allocating heterogeneous resources in an Internet of Vehicles based on cloud-edge-end collaborative computing, which combines collaborative filtering and attribute-based prediction to perform multi-objective optimized content popularity prediction, and the prediction results are used to guide the caching strategy of content in the cloud-edge-end network, thereby reducing delay and improving cache hit rate; a multi-granularity task decomposition algorithm and a dual-depth Q network task scheduling algorithm are used to achieve fine-grained decomposition of tasks and effective allocation of resources, and maximize the weighted utility function of delay and energy consumption; in the vehicle edge network, each edge server can independently perform service migration and allocate resources for the service migration of other edge servers, and a deep deterministic policy gradient algorithm is used to solve the model free Markov decision process problem, thereby maximizing the total expected utility of services unloaded by the vehicle, thereby solving at least one technical problem involved in the background technology.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] A method for allocating heterogeneous resources in an Internet of Vehicles based on cloud-edge-device collaborative computing includes the following steps:

[0008] Step S1, collecting the user's historical behavior, using a user-based collaborative filtering algorithm, an item-based collaborative filtering algorithm, and an attribute-based collaborative filtering algorithm to respectively predict the user's preference for products, using a multi-objective evolutionary algorithm to weight the prediction results of the three algorithms, determining the final prediction result, and generating a cache replacement form for the edge server and the vehicle terminal based on the final prediction result;

[0009] Step S2, collecting the resource usage of all computing nodes in the edge server to form an input matrix, sending the input matrix to the LSTM model for training, and outputting the prediction result of the remaining resources of each computing node;

[0010] Step S3, setting the granularity level for the tasks to be unloaded by the vehicle terminal, determining the resource threshold required for unloading each task based on the granularity level, judging whether the remaining resources of the computing node meet the resource threshold requirement, if so, unloading the task to the corresponding computing node; otherwise, decomposing the task into smaller subtasks until the remaining resources of the computing node meet the resource threshold requirement for unloading the subtask, and unloading the subtask to the corresponding computing node;

[0011] Step S4, constructing a dual-depth Q network model, taking the weighted utility function of delay and energy consumption as the goal, optimizing the computing node selection process for task offloading, determining the optimal computing node for task offloading, and offloading the task to the optimal computing node;

[0012] Step S5, constructing a vehicle edge network within the target area, in which each edge server independently performs service migration and allocates resources for the service migration of other edge servers, and constructing a Markov decision model with the maximization of the total expected utility of all services unloaded by vehicles in the vehicle edge network as the optimization goal, and using a deep deterministic policy gradient algorithm to approximate the Markov decision model to determine the optimal solution for service migration and resource allocation for each edge server in the vehicle edge network.

[0013] As a preferred improvement, the prediction process of the multi-objective evolutionary algorithm is expressed as:

[0014]

[0015] In the formula, and They represent user-based collaborative filtering, item-based collaborative filtering, and attribute-based collaborative filtering, respectively, for the predicted score of user u for item k at time τ; α τ and β τ Represents the weight coefficient.

[0016] As a preferred improvement, in the user-based collaborative filtering algorithm, item k 1 and k 2 The similarity is expressed by the cosine similarity formula:

[0017]

[0018] In the formula, and Represents user groups For item k 1 and k 2 The given rating vector; and They represent user u’s preference for product k. 1 and k 2 Ratings;

[0019] The predicted rating of user u for item k Calculated by the following formula:

[0020]

[0021] Where N ei (k) represents the neighbor set of item k, that is, the set of items with high similarity to item k; represents the sum of the actual ratings of user u on neighboring items k' related to item k; Represents the similarity between item k and item k'.

[0022] As a preferred improvement, in the collaborative filtering algorithm based on items, cosine similarity is used to represent the similarity between items:

[0023]

[0024] In the formula, r uk represents the rating of user u on item k, represents the average score of item k, and U(k1,k2) represents the average score of item k at the same time 1 and k 2 The set of users who have made ratings;

[0025] For a user u who has not rated item k, the predicted score for item k is Calculated by the following formula:

[0026]

[0027] Where N(k) is the set of items closest to item k, r uk' is the actual rating of item k' by user u.

[0028] As a preferred improvement, in the collaborative filtering algorithm based on attribute prediction, for item k 1 and k 2 , and their corresponding eigenvectors are x k1 and x k2 , the item feature vector represents the attribute of the item, and the cosine similarity is used to measure the angle between the two vectors, which is suitable for calculating the similarity between items:

[0029]

[0030] where x k1 ·x k2 Represents vector x k1 and x k2 The dot product of ||x k1 || and ||x k2 || respectively represent vector x k1 and x k2 The norm of ;

[0031] Then the predicted rating of user u for item k is It can be calculated by the following formula:

[0032]

[0033] Where P u represents the preference vector of user u, b u represents the bias term of user u, which is used to adjust the overall rating level; T represents the transposed matrix.

[0034] As a preferred improvement, the LSTM model is expressed as:

[0035] h t =σ(x t ,h t-1 , c t-1 );

[0036] In the formula, h t represents the hidden state of the LSTM model at time step t; c t-1 represents the state of the LSTM model at time step t-1; x t represents the slice of the input matrix MX at the same time step; σ(·) represents the sigmoid function;

[0037] The output Y of the LSTM model is expressed as:

[0038] Y=σ(M X );

[0039] The prediction result represents the hidden state h of the last time step t , a fully connected layer needs to be added to represent it concretely, then the actual output Y′ of the LSTM model is expressed as:

[0040] Y′=W fc ×h t +b fc ;

[0041] Where W fc represents the weight matrix of the fully connected layer, b fc Represents the bias vector.

[0042] As a preferred improvement, the specific steps of the multi-granularity task decomposition algorithm are as follows:

[0043] Task graph structure modeling: The computing task is represented as a directed acyclic graph. Each node in the directed acyclic graph represents a computing operation, and each edge represents the dependency relationship between operations.

[0044] Multi-granularity task decomposition: Based on the results of LSTM resource prediction, one or more nodes in the directed acyclic graph are decomposed into a subtask based on modularization;

[0045] Subtask mapping: Defines the mapping of subtasks to available compute nodes, so that a task can only assign one subtask to a compute node and each task can be deployed on a maximum of K nodes, where K is equal to the total number of subtasks.

[0046] As a preferred improvement, the dual-depth Q network optimization process specifically includes the following steps:

[0047] Initialization: Initialize the parameters of the estimation network Q and the target network Q′, both of which have the same structure but different parameters θ and θ′;

[0048] Interact with the environment to collect experience: At each time step t, according to the current state s of the agent t Select and perform an action t , observe the new state s t+1 and reward r t , and then (s t , a t , r t ,s t+1 ) is stored in the experience replay pool D; where the action space can be expressed as:

[0049]

[0050] In the formula, represents the action vector of the i-th node at time t; a i represents the offloading decision vector, indicating whether to offload the task to node i; o i represents the task allocation vector, indicating the allocation of tasks unloaded to node i; N nodes represents the number of nodes; where a i =[a 1 (t),a 2 (t),...,a n (t)],a n (t)∈{0,1},a n (t) = 1 means unloading the task, a n (t) = 0 means not offloading the task;

[0051] Update the parameters of the estimation network Q: Use the current state s t The Q value Q(s t ,a t ;θ) and the next state s t+1 The Q-value Q'(s) of the best action a' in t+1 ,a';θ') to calculate the target value r t +γQ'(s t+1 ,a';θ'), update the parameter θ of the estimation network Q so that it is adjusted in the direction of reducing the loss function. The loss function is defined as the mean square error between the estimated Q value and the true Q value;

[0052] Target network update: At every certain number of steps N, the parameters θ of the estimated network Q are copied to the target network Q';

[0053] Loop iteration: Continue the above steps until the stopping condition is met, such as reaching a predetermined number of training rounds or the environment's terminal state.

[0054] As a preferred improvement, the optimization process of the Markov decision model specifically includes the following steps:

[0055] Initialization state and action: At the beginning of each time slot, the state of the edge server includes: the number of all vehicle terminals in the current coverage area, the number of possible service migrations, and the number of computing resources that can be provided; the actions taken by the edge server include: the number of bandwidth resource blocks and computing resource blocks allocated to each vehicle terminal, and the number of service migrations that need to be initiated;

[0056] State representation: Use a vector to represent the current state of the edge server;

[0057] Select action: Based on the current state of the edge server, use the policy network in the deep deterministic policy gradient algorithm to select an action;

[0058] Execute actions and observe rewards: According to the selected actions, actually perform resource allocation and service migration; according to the results of executing the actions, calculate rewards; rewards are calculated based on service delays. If the service delay is lower than or equal to a threshold, the reward is positive; if the service delay exceeds the threshold, the reward gradually decreases until it becomes zero;

[0059] Update state, policy, and value function: Update the state of the edge server at the next moment based on the executed actions and results, and use the value network in the deep deterministic policy gradient algorithm to evaluate the value of the selected action and update the policy network accordingly. At the same time, use experience replay to update the value network based on the actual actions executed and the observed rewards;

[0060] Loop iteration: Repeat the above steps until the predetermined training cycle is reached or the termination condition is met.

[0061] The beneficial effects of the present invention are:

[0062] (1) Content popularity prediction through multi-objective optimization provides an important prerequisite for task decomposition and hierarchical scheduling. By predicting the popularity of content, the cache location of popular content can be arranged in advance, so that the location information of these contents can be taken into account when decomposing tasks, reducing latency and improving cache hit rate. This prediction can also help optimize the granularity of task decomposition, ensuring that tasks can be effectively assigned to the nodes that are most suitable for processing them;

[0063] (2) Task decomposition and hierarchical scheduling further utilize the results of content popularity prediction of multi-objective optimization to optimize resource allocation. Through efficient resource perception and task scheduling, it can ensure that popular content can be quickly accessed and computing resources can be efficiently utilized, which not only reduces network latency but also improves the overall system response speed and user experience.

[0064] (3) Service migration and resource allocation is to further optimize the resource allocation and task processing process based on multi-objective optimization of content popularity prediction and task decomposition and hierarchical scheduling. Through intelligent service migration decisions, it can ensure that the service always runs on the best edge server, thereby reducing latency and optimizing overall network performance.

[0065] (3) Through the organic combination of the above-mentioned technologies, the present invention can significantly improve the performance and service quality of the cloud-edge-end collaborative computing network, and is particularly suitable for highly dynamic Internet of Vehicles environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart showing a method for allocating heterogeneous resources in an Internet of Vehicles based on cloud-edge-end collaborative computing provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] See also Figure 1 The present invention provides a method for allocating heterogeneous resources in an Internet of Vehicles based on cloud-edge-end collaborative computing, comprising the following steps:

[0069] Step S1, collect the user's historical behavior, use the user-based collaborative filtering algorithm, the item-based collaborative filtering algorithm and the attribute-based collaborative filtering algorithm to predict the user's preference for the product respectively, use the multi-objective evolutionary algorithm to weight the prediction results of the three algorithms, determine the final prediction result, and generate the cache replacement form of the edge server and the vehicle terminal based on the final prediction result.

[0070] The basic idea of ​​user-based collaborative filtering algorithm is to recommend products that are liked by other users with similar interests to the user; the basic idea of ​​item-based collaborative filtering algorithm is to recommend items that are similar to items that the user liked before; the basic idea of ​​attribute-based collaborative filtering algorithm is to recommend items that are similar to the user's own attributes or item attributes. In the technical solution of the present invention, user preferences are predicted from three perspectives at the same time, and then the prediction results of the three algorithms are weighted to improve the accuracy of the prediction. The prediction results are used to guide the cache replacement strategy, which can improve the cache hit rate and reduce the delay caused by reloading due to the mismatch between the cache and the user's preferences.

[0071] In the user-based collaborative filtering algorithm, item k 1 and k 2 The similarity is expressed by the cosine similarity formula:

[0072]

[0073] In the formula, and Represents user groups For item k 1 and k 2 The given rating vector; and They represent user u’s preference for product k. 1 and k 2 Ratings;

[0074] The predicted rating of user u for item k Calculated by the following formula:

[0075]

[0076] Where N ei (k) represents the neighbor set of item k, that is, the set of items with high similarity to item k; represents the sum of the actual ratings of user u on neighboring items k' related to item k; Represents the similarity between item k and item k'.

[0077] The item-based collaborative filtering algorithm is similar to the user-based collaborative filtering algorithm, and the cosine similarity is used to represent the similarity between items:

[0078]

[0079] In the formula, r uk represents the rating of user u on item k, represents the average score of item k, and U(k1,k2) represents the average score of item k at the same time 1 and k2 The collection of users who have made ratings.

[0080] For a user u who has not rated item k, the predicted score for item k is Calculated by the following formula:

[0081]

[0082] Where N(k) is the set of items closest to item k, r uk' is the actual rating of item k' by user u.

[0083] In the collaborative filtering algorithm based on attribute prediction, for two items k 1 and k 2 , and their corresponding eigenvectors are x k1 and x k2 , the item feature vector represents the attribute of the item, and the cosine similarity is used to measure the angle between the two vectors, which is suitable for calculating the similarity between items:

[0084]

[0085] where x k1 ·x k2 Represents vector x k1 and x k2 The dot product of ||x k1 || and ||x k2 || respectively represent vector x k1 and x k2 The norm of .

[0086] Assume that user u's preference can be represented by a vector P u Represents that user u’s predicted rating for item k It can be calculated by the following formula:

[0087]

[0088] Where P u is the preference vector of user u, b u is the bias term of user u, which is used to adjust the overall rating level; T represents the transposed matrix.

[0089] By building a multi-objective optimized linear weighted rating prediction model to integrate collaborative filtering and attribute-based prediction:

[0090]

[0091] In the formula, and They represent user-based collaborative filtering, item-based collaborative filtering, and attribute-based collaborative filtering, respectively, for the predicted score of user u for item k at time τ; α τ and β τ represents the weight coefficients, which are optimized through a multi-objective evolutionary algorithm at each time point to discover the excellent combination weights suitable for the current Internet of Vehicles network environment.

[0092] According to the different service capabilities and coordination states of edge nodes and terminal nodes, cache replacement forms for edge nodes and terminal nodes are generated based on the predicted user preferences for items. The combined weights of the three optimize the basic prediction algorithm to simultaneously enhance and balance the optimization objectives.

[0093] Step S2, collect the resource usage of all computing nodes in the edge server to form an input matrix, send the input matrix to the LSTM model for training, and output the prediction result of the remaining resources of each computing node.

[0094] The LSTM model is represented as:

[0095] h t =σ(x t ,h t-1 ,c t-1 );

[0096] In the formula, h t represents the hidden state of the LSTM model at time step t; c t-1 represents the state of the LSTM model at time step t-1; x t represents the slice of the input matrix MX at the same time step; σ(·) represents the sigmoid function;

[0097] The output Y of the LSTM model is expressed as:

[0098] Y=σ(M X ).

[0099] The prediction result represents the hidden state h of the last time step t , we need to add a fully connected layer to represent it concretely, then the actual output Y' of the LSTM model is expressed as:

[0100] Y'=W fc ×h t +b fc ;

[0101] Where W fc represents the weight matrix of the fully connected layer, b fc Represents the bias vector.

[0102] Step S3, set the granularity level for the tasks that need to be unloaded by the vehicle terminal, determine the resource threshold required for unloading each task based on the granularity level, and judge whether the remaining resources of the computing node meet the resource threshold requirements. If so, unload the task to the corresponding computing node; otherwise, decompose the task into smaller sub-tasks until the remaining resources of the computing node meet the resource threshold requirements for unloading the sub-task, and unload the sub-task to the corresponding computing node.

[0103] The specific steps of the multi-granularity task decomposition algorithm are as follows:

[0104] Task graph structure modeling: The computing task is represented as a directed acyclic graph (DAG), where each node of the DAG represents a computing operation, and each edge represents the dependency between operations. Based on modular decomposition, one or more nodes from the DAG can be decomposed into a module (i.e., subtask). If the computing nodes in the DAG are independent of other nodes, each node or multiple nodes can be decomposed into a subtask. On the contrary, if there is a dependency, such as a subsequent computing node relying on the result of its previous node, these interdependent nodes will be decomposed into a subtask to ensure the calculation accuracy;

[0105] Multi-granularity task decomposition: According to the results of LSTM resource prediction, the task is decomposed into k uneven subtasks. Since the remaining resources of the node will change dynamically and the quantity will vary, according to the LSTM prediction results at time point X t Break down the tasks.

[0106] Subtask mapping: defines the mapping of subtasks to available computing nodes. Let x ik Indicates whether the kth subtask is assigned to node i, that is, x ik ∈{0,1};

[0107]

[0108] This means that a task can only assign one subtask to a computing node; and:

[0109]

[0110] This means that each task can be deployed on at most K nodes, where K is equal to the total number of subtasks;

[0111] For each subtask k∈K, there is τ n'k,n ≥0, that is, the task start time must be non-negative, and

[0112]

[0113] Where, d k-1n',nrepresents the amount of data sent from the k-1th subtask, B k-1n',n represents the bandwidth from node n to its predecessor node n'.

[0114] Through the above steps, the task can be effectively decomposed into multiple subtasks according to the resource conditions of each node in the current system, and further scheduled to the nodes that best suit their respective resource requirements for processing.

[0115] Step S4, construct a dual-depth Q network model, optimize the computing node selection process for task offloading with the goal of maximizing the weighted utility function of delay and energy consumption, determine the optimal computing node for task offloading, and offload the task to the optimal computing node.

[0116] The principle of the dual-depth Q network is that at any time t, the agent takes action a t And act on the environment to achieve interaction with the environment. After the interaction is completed, the state of the agent is determined by s t Transformations t+1 , set the value function to evaluate action a t The value of t , the agent is based on the value r t Continuously optimize the behavior trajectory and eventually learn the optimal behavior strategy. The value function includes the state value function and the action value function. The action value function (Q value) represents the action value of the agent under the guidance of the strategy π according to the current state s t , take action a t The expected return obtained. The strategy π represents the mapping from state to behavior, which is equivalent to the decision-making strategy of the agent, and different behaviors are selected according to the different states of the agent.

[0117] The network structure of the dual-depth Q network consists of an estimation network Q and a target network Q', which have the same structure but different parameters. The estimation network Q has the latest network parameters, calculates the value of the current state-action pair, and regularly updates the parameters of the target network Q'. The action corresponding to the maximum action value function (Q value) is selected through the estimation network Q, and then the target network calculates the target Q value based on the corresponding action. Through the continuous interaction between the agent and the environment, experience tuples are generated and stored in the experience replay buffer. The priority experience replay mechanism is used to improve the efficiency of reusing historical experience. Every certain number of steps, the parameters of the estimation network Q are updated to the target network Q', keeping the structure of the two the same but with different parameters. The estimated Q value of the estimation network Q and the true Q value of the target network Q' are calculated, and the loss function is calculated based on the two. The parameters of the estimation network Q are updated using the gradient descent method.

[0118] When offloading computing tasks, the main considerations include whether to offload the task and to which node. Therefore, the action space can be expressed as:

[0119]

[0120] In the formula, represents the action vector of the i-th node at time t; a i represents the offloading decision vector, indicating whether to offload the task to node i; o i represents the task allocation vector, indicating the allocation of tasks unloaded to node i; N nodes represents the number of nodes; where a i =[a 1 (t), a 2 (t), ..., a n (t)],a n (t)∈{0,1},a n (t) = 1 means unloading the task, a n (t)=0 means not offloading the task.

[0121] The dual-depth Q network optimization process specifically includes the following steps:

[0122] Initialization: Initialize the parameters of the estimation network Q and the target network Q′, both of which have the same structure but different parameters θ and θ′;

[0123] Interact with the environment to collect experience: At each time step t, according to the current state s of the agent t Select and perform an action t , observe the new state s t+1 and reward r t , and then (s t , a t , r t ,s t+1 ) is stored in the experience replay pool D;

[0124] Update the parameters of the estimation network Q: Use the current state s t The Q value Q(s t , a t ;θ) and the next state s t+1 The Q-value Q′(s t+1 , a′; θ′) to calculate the target value r t +γQ′(s t+1 , a′; θ′), update the parameter θ of the estimation network Q so that it is adjusted in the direction of reducing the loss function. The loss function is defined as the mean square error (MSE) between the estimated Q value and the true Q value;

[0125] Target network update: Every certain number of steps N, the parameters θ of the estimated network Q are copied to the target network Q′;

[0126] Loop iteration: Continue the above steps until the stopping condition is met, such as reaching a predetermined number of training rounds or the environment's terminal state.

[0127] Step S5, constructing a vehicle edge network within the target area, in which each edge server independently performs service migration and allocates resources for the service migration of other edge servers, and constructing a Markov decision model with the maximization of the total expected utility of all services unloaded by vehicles in the vehicle edge network as the optimization goal, and using a deep deterministic policy gradient algorithm to approximate the Markov decision model to determine the optimal solution for service migration and resource allocation for each edge server in the vehicle edge network.

[0128] The optimization process of the Markov decision model specifically includes the following steps:

[0129] Initialization state and action: At the beginning of each time slot, the state of the edge server includes: the number of all vehicle terminals in the current coverage area, the number of possible service migrations, and the number of computing resources that can be provided; the actions taken by the edge server include: the number of bandwidth resource blocks (BRBs) and computing resource blocks (CRBs) allocated to each vehicle terminal, and the number of service migrations that need to be initiated;

[0130] State representation: Use a vector to represent the current state of the edge server;

[0131] Select action: Based on the current state of the edge server, use the policy network (Actor) in the Deep Deterministic Policy Gradient (DDPG) algorithm to select an action;

[0132] Execute actions and observe rewards: According to the selected actions, actually perform resource allocation and service migration; according to the results of executing the actions, calculate rewards; rewards are calculated based on service delays. If the service delay is lower than or equal to a threshold, the reward is positive; if the service delay exceeds the threshold, the reward gradually decreases until it becomes zero;

[0133] Update state, strategy, and value function: Update the state of the edge server at the next moment based on the executed actions and results, and use the value network (Critic) in the DDPG algorithm to evaluate the value of the selected action, and update the policy network (Actor) accordingly. At the same time, use experience replay (ExperienceReplay) to update the value network (Critic) based on the actual executed actions and observed rewards;

[0134] Loop iteration: Repeat the above steps until the predetermined training cycle is reached or the termination condition is met.

[0135] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for allocating heterogeneous resources in an Internet of Vehicles based on cloud-edge-device collaborative computing, characterized in that: The steps include: Step S1, collecting the user's historical behavior, using a user-based collaborative filtering algorithm, an item-based collaborative filtering algorithm, and an attribute-based collaborative filtering algorithm to respectively predict the user's preference for products, using a multi-objective evolutionary algorithm to weight the prediction results of the three algorithms, determining the final prediction result, and generating a cache replacement form for the edge server and the vehicle terminal based on the final prediction result; Step S2, collecting the resource usage of all computing nodes in the edge server to form an input matrix, sending the input matrix to the LSTM model for training, and outputting the prediction result of the remaining resources of each computing node; Step S3, setting the granularity level for the tasks to be unloaded by the vehicle terminal, determining the resource threshold required for unloading each task based on the granularity level, judging whether the remaining resources of the computing node meet the resource threshold requirement, if so, unloading the task to the corresponding computing node; otherwise, decomposing the task into smaller subtasks until the remaining resources of the computing node meet the resource threshold requirement for unloading the subtask, and unloading the subtask to the corresponding computing node; Step S4, constructing a dual-depth Q network model, taking the weighted utility function of delay and energy consumption as the goal, optimizing the computing node selection process for task offloading, determining the optimal computing node for task offloading, and offloading the task to the optimal computing node; Step S5, constructing a vehicle edge network within the target area, in which each edge server independently performs service migration and allocates resources for the service migration of other edge servers, and constructing a Markov decision model with the maximization of the total expected utility of all services unloaded by vehicles in the vehicle edge network as the optimization goal, and using a deep deterministic policy gradient algorithm to approximate the Markov decision model to determine the optimal solution for service migration and resource allocation for each edge server in the vehicle edge network.

2. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 1 is characterized in that: The prediction process of the multi-objective evolutionary algorithm is expressed as: In the formula, and They represent user-based collaborative filtering, item-based collaborative filtering, and attribute-based collaborative filtering, respectively, for the predicted score of user u for item k at time τ; α τ and β τ Represents the weight coefficient.

3. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 2 is characterized in that: In the user-based collaborative filtering algorithm, the similarity between items k1 and k2 is expressed by the cosine similarity formula: In the formula, and Represents user groups The rating vector given to items k1 and k2; and Respectively represent the ratings of user u on products k1 and k2; The predicted rating of user u for item k Calculated by the following formula: Where N ei (k) represents the neighbor set of item k, that is, the set of items with high similarity to item k; represents the sum of the actual ratings of user u on neighboring items k' related to item k; Represents the similarity between item k and item k'.

4. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 2 is characterized in that: In the collaborative filtering algorithm based on items, cosine similarity is used to represent the similarity between items: In the formula, r uk represents the rating of user u on item k, represents the average rating of item k, and U(k1,k2) represents the set of users who have rated items k1 and k2 at the same time; For a user u who has not rated item k, the predicted score for item k is Calculated by the following formula: Where N(k) is the set of items closest to item k, r uk' is the actual rating of item k' by user u.

5. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 2 is characterized in that: In the collaborative filtering algorithm based on attribute prediction, for items k1 and k2, their corresponding feature vectors are x k1 and x k2 , the item feature vector represents the attribute of the item, and the cosine similarity is used to measure the angle between the two vectors, which is suitable for calculating the similarity between items: where x k1 ·x k2 Represents vector x k1 and x k2 The dot product of ||x k1 || and ||x k2 || respectively represent vector x k1 and x k2 The norm of ; Then the predicted rating of user u for item k is It can be calculated by the following formula: Where P u represents the preference vector of user u, b u represents the bias term of user u, which is used to adjust the overall rating level; T represents the transposed matrix.

6. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 1 is characterized in that: The LSTM model is represented as: h t =σ(x t ,h t-1 ,c t-1 ); In the formula, h t represents the hidden state of the LSTM model at time step t; c t-1 represents the state of the LSTM model at time step t-1; x t represents the slice of the input matrix MX at the same time step; σ(·) represents the sigmoid function; The output Y of the LSTM model is expressed as: Y=σ(M X ); The prediction result represents the hidden state h of the last time step t , we need to add a fully connected layer to represent it concretely, then the actual output Y' of the LSTM model is expressed as: Y'=W fc ×h t +b fc ; Where W fc represents the weight matrix of the fully connected layer, b fc Represents the bias vector.

7. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 6 is characterized in that: The specific steps of the multi-granularity task decomposition algorithm are as follows: Task graph structure modeling: The computing task is represented as a directed acyclic graph. Each node in the directed acyclic graph represents a computing operation, and each edge represents the dependency relationship between operations. Multi-granularity tasks Decomposition: Based on the results of LSTM resource prediction, decomposition is performed based on modularity to decompose one or more nodes from the directed acyclic graph into a subtask; Subtask mapping: Defines the mapping of subtasks to available compute nodes, so that a task can only assign one subtask to a compute node and each task can be deployed on a maximum of K nodes, where K is equal to the total number of subtasks.

8. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-end collaborative computing according to claim 1 is characterized in that: The dual-depth Q network optimization process specifically includes the following steps: Initialization: Initialize the parameters of the estimation network Q and the target network Q', both of which have the same structure but different parameters θ and θ'; Interact with the environment to collect experience: At each time step t, according to the current state s of the agent t Select and perform an action t , observe the new state s t+1 and reward r t , and then (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool D; where the action space can be expressed as: In the formula, represents the action vector of the i-th node at time t; a i represents the offloading decision vector, indicating whether to offload the task to node i; o i represents the task allocation vector, indicating the allocation of tasks unloaded to node i; N nodes represents the number of nodes; where a i =[a1(t),a2(t),...,a n (t)],a n (t)∈{0,1},a n (t) = 1 means unloading the task, a n (t) = 0 means not offloading the task; Update the parameters of the estimation network Q: Use the current state s t The Q value Q(s t ,a t ;θ) and the next state s t+1 The Q-value Q'(s t+1 ,a';θ') to calculate the target value r t +γQ'(s t+1 ,a';θ'), update the parameter θ of the estimation network Q so that it is adjusted in the direction of reducing the loss function. The loss function is defined as the mean square error between the estimated Q value and the true Q value; Target network update: Every certain number of steps N, the parameters θ of the estimated network Q are copied to the target network Q'; Loop iteration: Continue the above steps until the stopping condition is met, such as reaching a predetermined training round or the environment's terminal state.

9. The method for allocating heterogeneous resources in the Internet of Vehicles based on cloud-edge-device collaborative computing according to claim 1 is characterized in that: The optimization process of the Markov decision model specifically includes the following steps: Initialization state and action: At the beginning of each time slot, the state of the edge server includes: the number of all vehicle terminals in the current coverage area, the number of possible service migrations, and the number of computing resources that can be provided; the actions taken by the edge server include: the number of bandwidth resource blocks and computing resource blocks allocated to each vehicle terminal, and the number of service migrations that need to be initiated; State representation: Use a vector to represent the current state of the edge server; Select action: Based on the current state of the edge server, use the policy network in the deep deterministic policy gradient algorithm to select an action; Execute actions and observe rewards: According to the selected actions, actually perform resource allocation and service migration; according to the results of executing the actions, calculate rewards; rewards are calculated based on service delays. If the service delay is lower than or equal to a threshold, the reward is positive; if the service delay exceeds the threshold, the reward gradually decreases until it becomes zero; Update state, policy, and value function: Update the state of the edge server at the next moment based on the executed actions and results, and use the value network in the deep deterministic policy gradient algorithm to evaluate the value of the selected action and update the policy network accordingly. At the same time, use experience replay to update the value network based on the actual actions executed and the observed rewards; Loop iteration: Repeat the above steps until the predetermined training cycle is reached or the termination condition is met.

Images