Unknown resource sensing and task unloading method based on AUCB algorithm

Abstract

The invention discloses an unknown resource sensing and task unloading method based on an AUCB algorithm, and belongs to the field of calculation task unloading research. According to the method, driving vehicles on a road are divided into task vehicles for generating calculation tasks and service vehicles for providing calculation services, firstly, a priority model of the calculation tasks is established through an entropy evaluation method, and it is ensured that the calculation tasks generated by all the task vehicles at the same moment are processed according to priority ranking; then a multi-arm bandit model is used for modeling a calculation task unloading problem, idle calculation and communication resources of service vehicles are sensed and explored, an adaptive confidence interval upper bound algorithm is used for modeling the calculation task unloading problem, and task confidence is recorded and updated; and finally, evaluating each rocker reward value represented by the vehicle, and selecting an optimal service vehicle to execute calculation task unloading, thereby realizing unloading strategy solving, and further improving system effectiveness and service quality of vehicle-mounted edge calculation task unloading.

Description

Technical Field

[0001] The present invention belongs to the research field of computing task offloading, and specifically relates to an unknown resource perception and task offloading method based on AUCB algorithm. Background Art

[0002] With the development of the Internet of Vehicles and intelligent connected vehicles, vehicles have been endowed with more computing, storage, sensing and communication capabilities. Under the dual effects of artificial intelligence and 5G technology, computing-intensive applications such as intelligent driving, image navigation, augmented reality (AR), and virtual reality (VR) are constantly emerging in the Internet of Vehicles. However, the computing resources currently carried by vehicles are difficult to meet the computing power requirements brought about by the development of the above technologies, which not only affects the driving and riding experience, but also brings new uncertain risks to driving safety. Therefore, the current lack of computing resources for intelligent connected vehicles is considered to be a bottleneck factor restricting the development of the Internet of Vehicles. In order to break through the constraints brought by the shortage of computing resources to the development of the Internet of Vehicles, in addition to increasing the computing resource configuration of the vehicle itself, computing task offloading is considered to be a very effective solution. Task offloading technology can offload the vehicle's own computing tasks to nodes with rich computing resources, greatly reducing the original amount of computing and ensuring that applications that require a large amount of computing resources can run efficiently.

[0003] In the current research on offloading computing tasks in the Internet of Vehicles, the architecture based on the center and edge cloud has made great progress and breakthroughs. However, these studies rarely consider the new challenges brought by new intelligent connected vehicle tasks such as computationally intensive and delay-sensitive. Such tasks are generally unloaded to the center cloud or edge cloud with a high time delay, which is unacceptable for security service tasks. Secondly, in the Internet of Vehicles scenario, vehicles usually frequently switch communications with servers such as base stations or road side units (RSUs), which is particularly obvious when the vehicle is traveling at high speed, which will inevitably have a serious impact on the performance of computing task offloading. Most importantly, computing task offloading under high-speed movement is highly dependent on traffic infrastructure, and such tasks are difficult to achieve in the scenario of decentralized center cloud and decentralized edge cloud. The vehicle cloud-based method has many advantages such as high cost performance, high utilization rate and stable communication, which are not available in the task offloading method based on the center and edge cloud. First, the on-board resources are cost-effective and can effectively improve the utilization efficiency of computing resources. Secondly, the vehicles are moving towards each other, and the communication environment between them is relatively stable, which is caused by the smaller relative speed between the vehicles moving towards each other, which makes task offloading more reliable. In addition, vehicle cloud-based offloading methods are usually associated with smaller vehicles and do not generate large-scale privacy leakage and security issues.

[0004] Therefore, VEC came into being. VEC deploys computing resources closer to the terminal vehicle, and uses task offloading technology to transfer computing-intensive applications from the vehicle to VEC servers on various adjacent infrastructures, or uses collaboration between vehicles to call on the idle computing resources of other vehicles to jointly complete computing tasks to obtain lower latency. However, the current VEC computing task offloading technology still faces huge challenges, mainly because the Internet of Vehicles scenarios are complex, diverse and heterogeneous. These challenges not only affect the allocation of vehicle computing resources and the formulation of actual task offloading strategies, but also the subjectivity of vehicle participation in task offloading. With the continuous improvement of people's awareness of safety and privacy, many car owners believe that their data and computing resources are privacy-sensitive due to their personal subjective consciousness, which makes their vehicle resources in a non-shared state. The current mainstream task offloading methods do not have strong environmental perception and cannot complete computing task offloading when computing resources are unknown, which greatly reduces the reliability of task offloading technology. At the same time, the limitations of traditional optimization algorithms are becoming increasingly apparent with the increase in the scale of the Internet of Vehicles. Specifically, these traditional algorithms do not have active learning capabilities, which leads to serious deficiencies in adapting to new application environments and expanding new tasks. More importantly, traditional optimization algorithms are relatively simple, while the computational process of computing task offloading is usually more complex, which is difficult for traditional methods to cope with, which brings new challenges to task offloading. In addition, the limited coverage and computing resources of a single offloading node also restrict the time delay and energy consumption performance optimization of computing task offloading. Summary of the invention

[0005] Aiming at the complex, diverse and heterogeneous characteristics of Internet of Vehicles scenarios, the present invention proposes an Internet of Vehicles unknown computing resource perception and task offloading method based on an adaptive upper confidence bound (AUCB) algorithm, which transforms the unknown computing resource perception problem in the Internet of Vehicles into a MAB model and uses the AUCB algorithm to estimate the reward value of each "rocker arm" represented by the vehicle to achieve perception of the state of on-board computing resources.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method for unknown resource perception and task offloading based on the AUCB algorithm. The method divides the vehicles on the road into task vehicles that generate computing tasks and service vehicles that provide computing services. First, the entropy method is used to establish a priority model for computing tasks to ensure that the computing tasks generated by each task vehicle at the same time are processed in order of priority; then the multi-armed bandit model is used to model the computing task offloading problem, the idle computing and communication resources of the service vehicle are perceived and explored, and the adaptive confidence interval upper bound algorithm is used to model the computing task offloading problem, and the task confidence is recorded and updated; finally, each "rocker arm" reward value represented by the vehicle is evaluated, and the optimal service vehicle is selected to perform computing task offloading, thereby realizing the offloading strategy solution, further improving the system utility and service quality of vehicle-mounted edge computing task offloading.

[0008] The beneficial effects of the present invention over the prior art are as follows: the vehicle-mounted edge computing method of the adaptive confidence interval upper bound algorithm of the present invention can better complete task offloading in Internet of Vehicles scenarios with complex, diverse and heterogeneous characteristics, and when computing resources are unknown, and has a smaller average regret value, a shorter average delay and a higher probability of task completion. The present invention can solve the problems of insufficient VEC servers or RSUs, which lead to the difficulty in completing computing offloading tasks of known resources and the failure of computing task offloading, and improve the reliability of computing task offloading; this method is also suitable for realizing computing task offloading in an environment where the resource status is unknown, which can not only improve the perception and decision-making ability of the vehicle group system, but also is an effective supplement to the existing VEC technology, solving the problem of shortage of computing resources in the Internet of Vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is the workflow diagram of the unknown resource task offloading method based on the AUCB algorithm;

[0010] Figure 2 This is a comparison chart of the average regret values ​​of each algorithm;

[0011] Figure 3 This is a comparison chart of the average delay of each algorithm;

[0012] Figure 4 Comparison chart of the completion probability of each algorithm task;

[0013] Figure 5 This is a graph showing the impact of the number of task splits on average latency. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0015] Example 1

[0016] A method for unknown resource perception and task offloading based on AUCB algorithm, characterized in that: the method specifically comprises:

[0017] Step 1: Vehicular Edge Computing (VEC) scenario modeling

[0018] In the VEC task offloading scenario, the driving vehicles are divided into two roles: one is the task vehicle responsible for generating computing tasks, and the other is the service vehicle that provides computing resources to support task offloading. When the task vehicle faces a computing task that cannot be completed by itself, it will offload this task to the surrounding service vehicles. The service vehicle uses its unoccupied computing resources to create a virtual machine so that this part of the resources can be used by external tasks and will not interfere with the application operation of the vehicle itself. Considering that several computing tasks may appear at the same time in a specific scenario, it is particularly important to effectively sort these parallel tasks that need to be offloaded. The present invention proposes a strategy for task priority sorting based on the entropy method. The strategy determines the processing order of tasks by calculating the entropy value of task parameters, and forms a priority queue based on this. The size of the entropy value of the parameter directly reflects its relative importance in the entire task priority queue, that is, the larger the entropy value of the task parameter, the higher its priority, and thus it occupies a higher priority position in the task execution priority queue.

[0019] When several computing tasks occur simultaneously, and these parallel tasks that need to be offloaded are effectively sorted, the task offloading time slot is defined as t = {t 1 ,t 2 ,...,t k}, the interval is denoted as Δt, at t k At this moment, each service vehicle only performs a single computing task, so the computing tasks generated by different task vehicles within Δt time are represented as a set T = {T 1 ,T 2 ,...,T N}, the feature of each task is represented by T i ={d i ,ω i ,τ i ,ρ i}, where d i ,ω i ,τ i ,ρ i They represent the data size (unit: bit), computational intensity (unit: cycle / bit), maximum delay limit (unit: s) and importance of each offload task, and are defined in t k The set of service vehicles available at any time is represented by S = {S1 ,S 2 ,...,S M};

[0020] Expand the task set described above into the following matrix:

[0021]

[0022] Then use α r , r = {1,2,3,4} (equivalent to α i,r ) represents each column vector of the matrix, and the probability of occurrence P of each element is solved i,r ,Right now:

[0023]

[0024] Therefore, the entropy value of each parameter is expressed as

[0025]

[0026] The weight corresponding to each parameter is obtained from the entropy value of each parameter:

[0027]

[0028] Define the weight vector as W = {w 1 ,w 2 ,w 3 ,w 4}, then the priority vector of each task is expressed as

[0029]

[0030] After the calculation is completed, the task offloading requests are sorted according to the obtained priority vector and added to the priority queue in sequence; in this way, the computing tasks generated at the same time point will be processed according to the specified priority order, and those tasks generated at different time points will be executed according to the time order in which they were offloaded;

[0031] Step 2: Unload optimization modeling

[0032] In the offloading optimization model, when a task vehicle needs to offload a task, it will send complete task data to all available service vehicles, and can subdivide a large computing task into several subtasks with similar computing intensity for offloading; after the service vehicles complete the subtasks assigned to them, they can immediately feedback the calculation results to the task vehicle without waiting for the completion of other subtasks in the same large task, and prepare to process subsequent computing tasks in the task priority queue;

[0033] Each time a computing task is offloaded, all task data needs to be offloaded to the service vehicle collection. There are m service vehicles in the network, and the maximum communication bandwidth of service vehicle j is Then the time required to unload the task data to service vehicle j is

[0034]

[0035] In formula (6), is the transmission rate between the service vehicle and the task vehicle (unit: MB / s), d i is the data size of each unloading task; after all task data are unloaded, the task calculation is performed. Since the transmission rates between different vehicles in the actual environment are different, the actual transmission time should be expressed as

[0036]

[0037] For any given j, Indicates that there is an S j ;

[0038] The maximum idle computing capacity that service vehicle j can provide is expressed as The computing power provided to the accepted tasks is Therefore, the time required for the task is calculated by service vehicle j as

[0039]

[0040] m represents the number of vehicles for the computation task;

[0041] Since the actual computational delay of a task is determined by the last completed subtask, the actual time consumed by task i can be obtained only when all subtasks are completed:

[0042]

[0043] The total time delay of the task is

[0044]

[0045] Since the goal is to minimize the average delay of the system, the optimization problem can be expressed as

[0046]

[0047] t k is the time to start unloading the kth task, Indicates the total time delay of the task;

[0048] Step 3: Uninstall unknown resource tasks

[0049] Computational task offloading requires the selection of the optimal service vehicle to minimize task delay. In the initial stage, the task vehicle does not know in advance which service vehicles can provide the best computing performance and minimum delay. This decision-making process can be formalized as a multi-armed bandit (MAB) problem, in which the task vehicle is analogous to a casino player, and the service vehicle set is equivalent to the multiple arms of the bandit. In this analogy, the task vehicle performs computing task offloading by selecting different service vehicles. The probability distribution of the benefits brought by each offloading to a specific service vehicle is unknown, so a strategy combining exploration and utilization is required. This strategy involves finding a balance between exploring the performance of different service vehicles and using known information to select the best service vehicle, aiming to find the best service vehicle combination that can minimize delay. After multiple offloading tasks, by recording the delay results of each operation, the task vehicle can continuously update its estimate of the performance of each service vehicle, thereby obtaining its probability distribution of benefits. In this way, after multiple rounds of training, the task vehicle can tend to select those service vehicle combinations that can minimize delay, so as to achieve the purpose of optimizing the decision of computing task offloading.

[0050] In the AUCB algorithm, the basis for action selection is

[0051]

[0052] In formula (12), Q t (a) represents the average actual benefit of executing action a before time t. The larger its value, the greater the probability of selection. The square root term represents the upper limit of the confidence interval of the average benefit of executing the action, that is, the maximum expected return in theory. The larger its value, the more credible the average benefit. N t (a) is the number of times action a is executed before time t, and n is the number of times all actions are executed before time t. This shows that the algorithm focuses on exploring the benefits of various actions in the initial stage, and uses the action with the highest estimated benefit when the exploration results tend to be stable, thus achieving a dynamic balance between exploration and utilization.

[0053] In the process of task offloading, time delay and benefit show a negative correlation, that is, the lower the time delay, the higher the benefit, so the transmission and calculation time delay d of the available task offloading j The reciprocal of represents the benefit obtained by executing the unloading action of the task, so the cumulative average benefit before executing the tth task is

[0054]

[0055] During the multiple rounds of exploration in the initial stage, sufficient exploration of a specific service vehicle will lead to its performance benefits gradually becoming stable, which is manifested as a continuous decrease in the variance of benefits. Based on this, it is reasonable and necessary to consider the impact of the variance of benefits when the service vehicle handles subtasks on its confidence. This can improve the confidence of service vehicles that have been fully explored and show stable performance. In addition, during the utilization process, service vehicles with a smaller vehicle-to-vehicle distance D are given priority for task unloading to ensure unloading efficiency and communication timeliness. Assuming that the computing task is divided into M subtasks, which means that the number of service vehicles selected to accept task unloading is also M, then the confidence before executing the tth task becomes

[0056]

[0057] V t (i) is expressed as

[0058]

[0059] is the variance, which is used to describe the degree of dispersion of data;

[0060] The selection basis of the unloading vehicle is

[0061]

[0062] When M is 1, the computing task is not split, and the single optimal service vehicle is selected in formula (16). If M is greater than 1, it indicates that the computing task needs to be split into multiple subtasks for unloading. The strategy at this time is to unload these subtasks independently to different service vehicles to avoid the same unloading set of each subtask, which can be regarded as M independent computing task unloading. By considering the variance of the service vehicle's revenue and the distance between the task vehicle and the service vehicle, this method provides an adaptive mechanism for computing task unloading decisions. This mechanism can not only improve computing efficiency to a certain extent, but also reduce the negative impact of long-distance communication on computing delay, thereby finding the best computing task unloading strategy in the dynamically changing VEC environment.

[0063] In order to test the performance of the unknown resource task offloading method based on the AUCB algorithm, a comparative experiment was set up to compare the performance with the random offloading Random algorithm, the ε-Greedy algorithm, and the original UCB algorithm. The relevant simulation parameters required for the experiment refer to the requirements for the verification of typical business service capabilities of the Internet of Vehicles in the "Internet of Vehicles White Paper (2023 Edition)" released by the China Academy of Information and Communications Technology. The parameters for information interaction and collaborative perception services are shown in Table 1.

[0064] Table 1 Experimental parameter settings

[0065] Parameter name Value Number of service vehicles 20 Maximum delay limit 50-150ms Task Computation Intensity 10~200cycle / bit Vehicle transmission rate 5~20MB / s Task data volume 100~200kb Communication bandwidth 20MHz Iterations 3000

[0066] (1) Comparison of average regret values

[0067] The average regret value refers to the difference between the actual average return generated by the algorithm and the average return generated under ideal conditions. The smaller the regret value, the better the algorithm performance. The size of this regret is an important indicator for measuring the performance of the MAB algorithm. The average regret value of each algorithm changes with the number of iterations. Figure 2 As shown in the figure, the Random algorithm randomly selects service vehicles to offload computing tasks and does not use any rocker information, so the average regret value is significantly larger than other algorithms; the ε-Greedy algorithm limits the probability of exploration and utilization by setting the parameter ε. The algorithm performance is easily affected by the parameter value, and the performance is relatively unstable. It performs slightly better in the average regret value indicator; the UCB algorithm does not require additional parameter settings, and each execution strategy selection will refer to information such as average revenue and rocker selection times, so it performs better in the average regret value indicator; and the AUCB algorithm takes the revenue variance of the service vehicle processing tasks into confidence considerations, improves the confidence of service vehicles that have been fully explored and have stable revenue, and tries to select service vehicles that are closer to the target for task offloading, so the average regret value performance is the best.

[0068] (2) Average delay comparison

[0069] The average latency refers to the time delay to complete the whole process of computing task unloading, computing, and returning. It is an important performance indicator in mobile edge computing and the main optimization direction of this invention. The average latency of each algorithm changes with the number of iterations. Figure 3 As shown in the figure, the average delay of the Random algorithm, ε-Greedy algorithm and UCB algorithm after convergence decreases successively and remains in the range of 80-85ms. The performance of the ε-Greedy algorithm and the UCB algorithm is similar, but because they ignore the interference of the distance between the task vehicle and the service vehicle on vehicle communication, they relatively evenly offload the computing tasks to the service vehicles within the range during the exploration phase, which leads to the extension of the average delay and increases the possibility of task offloading timeout. The average delay of the AUCB algorithm after convergence remains in the range of 70-75ms, which is the best performance and has a significant effect in reducing the average delay.

[0070] (3) Comparison of task completion probability

[0071] The task completion probability is the ratio of the number of computing tasks that complete the computing task offloading within the maximum delay limit to the total number of computing tasks that have been executed. This indicator can effectively measure the adaptability of the algorithm to the computing task offloading. The task completion probability of the algorithm changes with the number of iterations. Figure 4As shown in the figure, in the converged state, the task completion probability from high to low is AUCB algorithm, UCB algorithm, Random algorithm, and ε-Greedy algorithm. Figure 4 It can be seen that all algorithms have the situation where the computing task unloading exceeds the maximum delay limit, resulting in the failure to complete the task. The main reason is that the matching degree between the task vehicle and the service vehicle is insufficient, and the task vehicle has not found a more suitable service vehicle within the limited range to unload the computing task. Therefore, the AUCB algorithm optimizes the algorithm confidence to improve the algorithm's utility in finding the optimal unloading service vehicle set and executing the task unloading action to obtain the highest benefit, so that the probability of task completion after convergence is significantly improved and maintained above 82%, the highest among the four algorithms.

[0072] (4) Comparison of the impact of the number of computational task splits

[0073] Given that computing tasks in the VEC environment have strict requirements on time delay, it is necessary to ensure that the task completes the calculation and returns the result within the set maximum delay limit. Therefore, this paper simulates the scenario of splitting the computing task into 1 to 3 subtasks and offloading them separately to explore the impact of different numbers of splits on the average delay of task completion. Figure 5 As shown in the figure, the increase in the number of task divisions makes the average delay of the computing tasks of the AUCB algorithm lower and the convergence speed faster. This is because after the computing task is divided into multiple subtasks, the task offloading strategy selects multiple service vehicles at the same time when executing this computing task, which is equivalent to casino players operating multiple arms of MAB at the same time. The exploration efficiency is increased by M times, so that a single offloading will update the average revenue of multiple service vehicles, greatly improving the confidence interval accuracy of the service vehicle set, resulting in better algorithm performance.

Claims

1. A method for unknown resource perception and task offloading based on AUCB algorithm, characterized in that: The method is as follows: firstly, the entropy method is used to establish a priority model of computing tasks to ensure that the computing tasks generated by each task vehicle at the same time are processed in order of priority; then, the multi-armed bandit model is used to model the computing task offloading problem, the idle computing and communication resources of the service vehicle are sensed and explored, and the adaptive confidence interval upper bound algorithm is used to model the computing task offloading problem, and the task confidence is recorded and updated; finally, each "rocker" reward value represented by the vehicle is evaluated, and the optimal service vehicle is selected to perform computing task offloading, thereby realizing the solution of the offloading strategy.

2. According to claim 1, a method for unknown resource perception and task offloading based on AUCB algorithm is characterized in that: The method is specifically as follows: Step 1: Vehicle edge computing scenario modeling When several computing tasks occur simultaneously, the task offloading time slot is defined as t = {t1, t2, ..., t k }, the interval is denoted as Δt, at t k At this moment, each service vehicle only performs a single computing task, so the computing tasks generated by different task vehicles within Δt time are represented as a set T = {T1, T2, ..., T N }, the feature of each task is represented by T i ={d i ,ω i ,τ i ,ρ i }, where d i ,ω i ,τ i ,ρ i They represent the data size (unit: bit), computational intensity (unit: cycle / bit), maximum delay limit (unit: s) and importance of each offload task, and are defined in t k The set of service vehicles available at any time is represented by S = {S1, S2, ..., S M }; Expand the task set described above into the following matrix: Then use α r , r = {1,2,3,4} represents each column vector of the matrix, and the probability of occurrence P of each element is solved i,r ,Right now: Therefore, the entropy value of each parameter is expressed as The weight corresponding to each parameter is obtained from the entropy value of each parameter: Define the weight vector as W = {w1,w2,w3,w4}, then the priority vector of each task is expressed as After the calculation is completed, the task offloading requests are sorted according to the obtained priority vector and added to the priority queue in sequence; in this way, the computing tasks generated at the same time point will be processed according to the specified priority order, and those tasks generated at different time points will be executed according to the time order in which they were offloaded; Step 2: Unload optimization modeling Each time a computing task is offloaded, all task data needs to be offloaded to the service vehicle collection. There are m service vehicles in the network, and the maximum communication bandwidth of service vehicle j is Then the time required to unload the task data to service vehicle j is In formula (6), is the transmission rate between the service vehicle and the task vehicle (unit: MB / s), d i The data size of each offloading task; After all the task data are unloaded, the task calculation is performed. Since the transmission rates between different vehicles in the actual environment are different, the actual transmission time should be expressed as For any given j, Indicates that there is an S j ; The maximum idle computing capacity that service vehicle j can provide is expressed as The computing power provided to the accepted tasks is Therefore, the time required for the task is calculated by service vehicle j as m represents the number of vehicles for the computation task; Since the actual computational delay of a task is determined by the last completed subtask, the actual time consumed by task i can be obtained only when all subtasks are completed: The total time delay of the task is Since the goal is to minimize the average delay of the system, the optimization problem can be expressed as t k is the time to start unloading the kth task, Indicates the total time delay of the task; Step 3: Uninstall unknown resource tasks In the AUCB algorithm, the basis for action selection is In formula (12), Q t (a) represents the average actual benefit of executing action a before time t; the square root term represents the upper limit of the confidence interval of the average benefit of executing the action, that is, the maximum expected return in theory; N t (a) is the number of times action a is executed before time t, and n is the number of times all actions are executed before time t; In the task offloading process, the transmission and calculation time delay d of the task offloading is used j The reciprocal of represents the benefit obtained by executing the unloading action of the task, so the cumulative average benefit before executing the tth task is In the process of utilization, the service vehicle with the smaller inter-vehicle distance D is preferentially selected for task unloading. Assuming that the calculation task is divided into M subtasks, this means that the number of service vehicles selected to accept task unloading is also M. Then the confidence before executing the tth task becomes V t (i) is expressed as is the variance, which is used to describe the degree of dispersion of data; The selection basis of the unloading vehicle is When M is 1, the computing task is not split, and the single optimal service vehicle is selected by formula (16). If M is greater than 1, it indicates that the computing task needs to be split into multiple subtasks for unloading. The strategy at this time is to unload these subtasks independently to different service vehicles to avoid the same unloading set of each subtask. It can be regarded as M independent computing task unloading.

Images