Vehicle-connected MEC computing task scheduling method based on dynamic pricing and terminal equipment mobility

Abstract

The invention relates to an Internet-of-Vehicles MEC computing task scheduling method based on dynamic pricing and terminal equipment mobility, which comprises the following steps of: establishing an Internet-of-Vehicles mobile edge computing system, dividing the Internet-of-Vehicles mobile edge computing system into a user side and a task side, establishing a user side task unloading model, and in binary task unloading scheduling, performing task unloading on the user side. Establishing a mathematical model for maximizing the long-term utility of the mobile user vehicle under the condition of satisfying budget and time delay limitation; establishing an edge side task processing model, establishing a mathematical model for maximizing the overall utility of the roadside unit, introducing a simplified variable and a relaxation integer constraint condition, solving the mathematical model for maximizing the overall utility of the roadside unit through a KKT condition, and obtaining a service price for enabling the overall utility of the roadside unit to be optimal; and based on the optimal service pricing, a multi-agent reinforcement learning algorithm is adopted to solve the maximized mobile user vehicle long-term utility mathematical model, and a final edge computing server for mobile user vehicle computing task unloading is determined.

Description

Technical Field

[0001] The present invention belongs to the field of mobile edge computing and task offloading, and relates to a computing task scheduling method based on dynamic pricing and terminal device mobility. Background Art

[0002] Multi-access Edge Computing (MEC) is an emerging computing method that reduces the burden on the cloud by completing part or all of the computing work that originally needed to be processed in the cloud on edge devices. MEC has a three-layer architecture, the first layer is the cloud, the second layer is MEC, and the third layer is the terminal mobile device. In the MEC environment, the terminal device can offload computing tasks to nearby edge servers for execution, thereby improving computing efficiency and saving device energy consumption. Task offloading is one of the key technologies of MEC, and its types are mainly binary task offloading and partial task offloading.

[0003] In the MEC environment, reasonable resource management and optimization are crucial due to the limited and uneven distribution of edge server resources. By designing a reasonable service price, the edge server can motivate users to offload tasks to appropriate nodes, thereby achieving effective use of resources. Mobile User Vehicle (MUV) makes unloading decisions based on price, demand and its own benefits, forming a dynamic resource competition and allocation process. Roadside Unit (RSU) is equipped with an edge computing server. Mobile user vehicles move along a given trajectory and plan to offload computing tasks to RSU when resources are insufficient. In order to optimize within each time slot, each roadside unit determines the rental price of the unit computing resource, and MUV makes unloading decisions based on these prices. The present invention adopts a dynamic pricing method to adjust the pricing of task offloading under different circumstances, which can better balance the load, improve resource utilization, and optimize resource allocation. Summary of the invention

[0004] In order to solve the technical problem in the prior art that uneven group task unloading causes congestion of some edge server services, the technical solution adopted in this application is: a vehicle-connected MEC computing task scheduling method based on dynamic pricing and terminal device mobility, characterized in that it includes the following steps:

[0005] Establishing a mobile edge computing system for an Internet of Vehicles, and dividing the mobile edge computing system for an Internet of Vehicles into a user side and a task side, wherein the user side includes a plurality of mobile user vehicles, and the task side includes a plurality of roadside units, each of which is provided with an edge computing server; the user side and the task side perform task offloading through a macro base station;

[0006] Establish a user-side task offloading model. In binary task offloading scheduling, establish a mathematical model to maximize the long-term utility of mobile user vehicles under the conditions of meeting budget and delay constraints.

[0007] Establish an edge-side task processing model, control the number of task backlogs in the processing queue through a dynamic price setting mechanism, and establish a mathematical model to maximize the overall utility of roadside units under the conditions of not causing service congestion and energy consumption constraints.

[0008] Simplified variables are introduced, integer constraints are relaxed, and the mathematical model of maximizing the overall utility of roadside units is solved through the Carlo-Kuhn-Tucker condition to obtain the optimal service pricing for the overall utility of roadside units.

[0009] Based on the optimal service pricing, a multi-agent reinforcement learning algorithm is adopted to solve the mathematical model of maximizing the long-term utility of mobile user vehicles and determine the final edge computing server for offloading the computing tasks of mobile user vehicles.

[0010] Further: the process of establishing the user-side task offloading model is as follows:

[0011] The IoV mobile edge computing system is divided into time slots, and the following process is performed in each time slot t:

[0012] The generation task of mobile user vehicles within the coverage area of ​​the th roadside unit follows a Poisson process with an average rate of λ i , roadside unit index represents the set of mobile user vehicles within the coverage area of ​​the jth roadside unit in the tth time slot,

[0013] Tuple for task n To describe, is the generation time of task n, T n is the latency constraint of task n, X represents the average offloaded data size of each task, and Y represents the average computational load required to complete the task;

[0014] Assuming that the channel coherence time is greater than the length of a time slot τ0, the transmission rate R i，j (t) is:

[0015]

[0016] Where B is the bandwidth, d i，j is the distance between mobile user vehicle i and the jth roadside unit in the tth time slot, α is the path loss exponent, represents the transmission power of mobile user vehicle i, σ 2 represents the background noise power, e iFor inter-cell interference, obtain the unloaded transmission delay for:

[0017]

[0018] Among them, a i，j，n = 0, 1 is the binary unloading decision of the mobile user vehicle. In each time slot, the mobile user vehicle generates n i (t) computing tasks, the number of tasks to be unloaded by mobile user vehicle i in this time slot is N i (t) = n i (t)+N i (t-1)-∑ j a i，j，n (t-1);

[0019] Among them, at the beginning there is N i (1) = n i (1), so the unloading decision a of mobile user vehicle i in the tth time slot is i (t) is an N i (t)×J i (t) is a binary matrix; where J i (t) represents the number of edge nodes connected to the mobile user vehicle i through the vehicle-to-infrastructure (V2I) link.

[0020] Further: In the binary task offloading scheduling, the process of establishing a mathematical model for maximizing the long-term utility of mobile user vehicles under the conditions of satisfying budget and delay constraints is as follows:

[0021] Each mobile user vehicle evaluates the value budget E of each task service based on the unloading duration and service pricing. i，j，n :

[0022] The estimated time required for the entire task offloading is DT i，j，n for:

[0023]

[0024] in is the processing delay of task n at the roadside unit j; thus, the personal value function E of mobile user vehicle i i，j，n (t) is:

[0025]

[0026] Among them, β i is a trade-off parameter used to balance the unit computing service price p of roadside unit node j j and unloading performance 1 / DTi，j，n , mobile user vehicle i needs to pay resource rental cost a to rent computing resources of roadside unit i，j，n (t)p j (t);

[0027] Estimate the personal utility of mobile users' vehicles based on value budget, service pricing and opportunity cost:

[0028] Mobile user vehicles use opportunity cost O to balance immediate benefits and potential opportunities and risks. i (t) describes the bias in the value estimate caused by changes in the price of the alternative roadside unit, where:

[0029] And has an initial value O i (t=0)=0, where |·| is the operator of the number of set elements, and the long-term utility U of mobile user vehicle i i (t) is:

[0030] U i (t) = η∑ j (E i，j，n (t)-a i，j，n (t)p j (t))+O i (t) (4)

[0031] Among them, η is the weight coefficient;

[0032] Maximize the long-term utility U of mobile users’ vehicles i (t), the specific formula is as follows:

[0033]

[0034] stC1:a i，j，n (t)DT i，j，n ≤T n

[0035] C2: a i,j，n (t)P j (t)≤E i，j，n (t)

[0036]

[0037] C4:a i，j，n (t)∈0,1

[0038] Among them, τ0 is the length of a time slot, p j (t) is the calculated service price of roadside unit j in time slot t, T nis the delay constraint of task n. In order to maximize the user utility within the task delay limit, C1 gives the constraint of task service delay, C2 is the task budget limit, C3 gives the computing task scheduling constraint, and C4 is the task scheduling binary variable.

[0039] Further: the establishment of the edge-side task processing model includes the process of estimating the unit task processing time, updating the number of backlog tasks and judging service congestion, which is specifically as follows:

[0040] S31: The unit task processing time estimation is as follows:

[0041] The task of reaching the roadside unit comes from multiple mobile user vehicles, still following the Poisson process, and its arrival rate λ j (t) is:

[0042] λ j (t) = ∑ i λ i (t)(∑ n a i,j，n (t) / n i (t)) (6)

[0043] Where: f j is the computing capacity of roadside unit j, the arriving task flow forms an M / M / 1 queue, and the service rate is μ j =f j / Y, calculate the estimated processing time of task n for:

[0044]

[0045] S32: The roadside edge-side task processing model includes updating the number of backlog tasks and determining service congestion. The specific process is as follows:

[0046] At the beginning of each time slot, the jth roadside unit updates the number of its unprocessed backlog tasks and determines whether service congestion occurs. At the beginning of time slot t+1, the number of backlog tasks of roadside unit j is Q j (t+1) is:

[0047]

[0048] The jth roadside unit adjusts the price p of its unit computing resources j , to influence the unloading decision of mobile users’ vehicles, Represents the queue size, then the number of backlogged tasks needs to satisfy the constraint When the number of tasks arriving is greater than the upper limit of the edge node service capacity, that is, When , it is considered that service congestion occurs, μ jτ0 is the amount of tasks that a roadside unit can serve in one time slot.

[0049] Further: the process of controlling the number of task backlogs in the processing queue through the dynamic price setting mechanism and establishing a mathematical model for maximizing the overall utility of the roadside unit under the conditions of not generating service congestion and energy consumption constraints is as follows:

[0050] S31: Set the roadside unit service price p j ;

[0051] A linear function is used to characterize the relationship between price and computing resource supply:

[0052]

[0053] Among them, γ j is the demand curve factor, which is a negative number, and p0 is the initial price, which is a natural number;

[0054] S32: Calculate roadside unit revenue;

[0055] Roadside units provide computing services for tasks in a first-come, first-served and first-paid-before-service manner. The revenue of the jth roadside unit is:

[0056]

[0057] in, is the normalized energy consumption;

[0058] S33: The roadside unit sets an appropriate price and establishes a mathematical model to maximize the overall utility of the roadside unit under the constraint of mission capacity. The details are as follows:

[0059]

[0060]

[0061] Further: the simplified variables are: roadside unit task acceptance quantity q(t);

[0062] The process of introducing simplified variables, relaxing integer constraints, solving the mathematical model of maximizing the overall utility of roadside units through Karush-Kuhn-Tucker Conditions (KKT) and obtaining the optimal service pricing for the overall utility of roadside units is as follows:

[0063] S41: Estimate the task acceptance of roadside units;

[0064] Let q(t) = ∑ i，n a i，j，n(t), then the number of unloading tasks to be performed by the roadside unit in formula (8) is:

[0065]

[0066] Among them, the condition Q is satisfied j (x)≤u j τ;

[0067] S42: Relax integer constraints;

[0068] Relax the integer constraint on the variable q(t) and obtain the optimal task acceptance q*(t) through the KKT condition:

[0069]

[0070] in, is the floor operator, β j is the Lagrange multiplier;

[0071] S43: Calculate optimal pricing for roadside units;

[0072] The optimal pricing is obtained based on the relationship between the optimal task acceptance volume and price and the computing resource supply.

[0073] Further: Based on the optimal service pricing, the multi-agent reinforcement learning algorithm is used to solve the mathematical model of maximizing the long-term utility of the mobile user vehicle, and the process of determining the final edge computing server for offloading the computing task of the mobile user vehicle is as follows:

[0074] S51. The surrounding roadside unit prices and the mobile user vehicle location are taken as the state s: s = {s1, s2, ..., s s}; Each different price and position combination corresponds to a different state s s , the state is a discrete space;

[0075] The unloading decision of the mobile user's vehicle is taken as action A i : is the roadside unit index, |A i |The number of roadside units available for mobile user vehicle i;

[0076] Use the utility value as the reward r: r = U i (t);

[0077] S52: Initialize Q value Q(s, a)←0, probability strategy π(s, a)←1 / |A i | and the number of iterations C(s)←0; after initialization, the mobile user vehicle implements the following process in each iteration:

[0078] Select action a: Select action a in state s according to the probability strategy π(s, a), and observe the corresponding reward r and the next state s′ corresponding to the mobile user’s environment after selecting the action;

[0079] Update the value in the Q table: According to the reward r and state s′, update the Q value: Q(s, a)←(1-a)Q(s, a)+a(r+γmax a′ Q(s′, a′));

[0080] Update the average probability strategy Update C(s)←C(s)+1; for any a′∈A i ,have

[0081] Update the probability strategy π: constrain the strategy π to a reasonable probability distribution;

[0082] If a=argmax a′ Q(s′,a′), then π(s,a)←π(s,a)+δ; otherwise, when when , δ=δω; otherwise, δ=δ l , δ ω >δ l ;

[0083] Until the absolute value of the difference between the Chebyshev norms of two adjacent iterations of the Q table is less than the iteration stop condition 10 -4 , get the final probability strategy π, and obtain action a according to the probability strategy under the corresponding environment state s as the final edge computing server.

[0084] Through the implementation of the above technical solution, the beneficial effects of the present invention are:

[0085] The present application discloses a computing task scheduling method based on dynamic pricing and terminal device mobility, which uses the terminal device mobility to offload tasks to relatively idle edge nodes as much as possible, thereby achieving balanced single-hop group task offloading; the edge server designs appropriate service prices to guide the task offloading of user groups, and mobile user vehicles compete for resources based on prices and their own needs to make offloading decisions to obtain maximum benefits.

[0086] The purpose of the present invention is to provide a vehicle-connected MEC computing task scheduling method based on dynamic pricing and terminal device mobility, which can effectively maximize the long-term utility of mobile user vehicles while meeting budget and delay constraints. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A network diagram of the vehicle-connected MEC system provided by the present invention;

[0088] Figure 2 A graph showing the relationship between the long-term utility of a mobile user's vehicle and the number of mobile user's vehicles in the present invention;

[0089] Figure 3 This is a relationship diagram between the average vehicle utility of a single mobile user and the computing power of a roadside unit in the present invention. DETAILED DESCRIPTION

[0090] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments:

[0091] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0092] Figure 1 A network diagram of the vehicle-connected MEC system provided by the present invention;

[0093] A method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility includes the following steps:

[0094] S1: Establish a mobile edge computing system for an Internet of Vehicles, and divide the mobile edge computing system for an Internet of Vehicles into a user side and a task side, wherein the user side includes multiple mobile user vehicles, and the task side includes multiple roadside units, each of which is provided with an edge computing server; the user side and the task side perform task offloading through a macro base station;

[0095] S2: Establish a user-side task offloading model. In binary task offloading scheduling, establish a mathematical model that maximizes the long-term utility of mobile user vehicles under the conditions of meeting budget and delay constraints;

[0096] S3: Establish an edge-side task processing model, control the number of task backlogs in the processing queue through a dynamic price setting mechanism, and establish a mathematical model to maximize the overall utility of roadside units under the conditions of not generating service congestion and energy consumption constraints.

[0097] S4: Introduce simplified variables, relax integer constraints, and solve the mathematical model of maximizing the overall utility of roadside units through KKT conditions to obtain the service pricing that optimizes the overall utility of roadside units;

[0098] S5: Based on the optimal service pricing, a multi-agent reinforcement learning algorithm is used to solve the mathematical model of maximizing the long-term utility of mobile user vehicles and determine the final edge computing server for offloading the computing tasks of mobile user vehicles.

[0099] After step S1 is executed, S2 and S3 are executed in parallel, and S3, S4, and S5 are executed sequentially;

[0100] The vehicle-connected mobile edge computing system includes I roadside units and, mobile user vehicles; an edge computing server is installed on the roadside unit side, which serves as an edge node to provide computing services for mobile users. It is assumed that the mobile user vehicle moves along a given trajectory. When the mobile user vehicle does not have sufficient computing resources to run computing-intensive and delay-constrained tasks, these computing tasks are offloaded to the edge node for processing; represents the set of mobile user vehicles within the coverage area of ​​the t-th roadside unit in the t-th time slot, Represents the set of edge nodes that mobile user vehicle i is connected to through the V2I link in the tth time slot. Considering the mobility of mobile user vehicles in the coverage area of ​​the macro base station, the time slots are divided in the vehicle-connected mobile edge computing system and scheduling decisions are made in each time slot. In a time slot t, each roadside unit will determine the rental price of the unit computing resource, and each mobile user vehicle makes a binary unloading decision based on the given price, that is, decides to unload or not unload to the roadside unit within the current V2I communication range.

[0101] The process of establishing the user-side task offloading model is as follows:

[0102] The vehicle-connected mobile edge computing system is divided into time slots, and the following process is performed in each time slot t:

[0103] The mobile user vehicle generation task within the coverage area of ​​the jth (j∈J) roadside unit follows a Poisson process with an average rate of λi. Tuple for task n To describe, is the generation time of task n, T n is the delay constraint of task n, X represents the average offload data size of each task, and Y represents the average computational load required to complete the task.

[0104] Assuming that the channel coherence time is greater than the length of a time slot τ, the transmission rate R i，j (t) is:

[0105]

[0106] Where B is the bandwidth, d i，j is the distance between the mobile user vehicle i and the jth roadside unit in the th time slot, α is the path loss exponent, represents the transmission power of mobile user vehicle i, σ 2 represents the background noise power, e i For inter-cell interference, obtain the unloaded transmission delay for:

[0107]

[0108] Among them, a i，j，n = 0, 1 is the binary unloading decision of the mobile user vehicle. In each time slot, the mobile user vehicle generates n i (t) computing tasks, the number of tasks to be unloaded by mobile user vehicle i in this time slot is N i (t) = n i (t)+N i (t-1)-∑ j a i，j，n (t-1). Among them, at the beginning there are N i (1) = n i (1). Therefore, the unloading decision a of mobile user vehicle i in the tth time slot is i (t) is a binary matrix of Ni(t)×Ji(t), where Ji(t) represents the number of edge nodes that can be connected to mobile user vehicle i through V2I.

[0109] In the binary task offloading scheduling, the process of establishing a mathematical model to maximize the long-term utility of mobile user vehicles under the conditions of meeting budget and delay constraints is as follows:

[0110] Each mobile user vehicle evaluates the value budget E of each task service based on the unloading duration and service pricing. i，j，n :

[0111] The estimated time required for the entire task offloading is in is the processing delay of task n at roadside unit j, thus, the personal value function E of mobile user vehicle i i，j，n (t) is:

[0112]

[0113] Among them, β i is a trade-off parameter used to balance the unit computing service price p of roadside unit node j j and unloading performance 1 / DT i，j，n Mobile user vehicle i needs to pay resource rental cost a to rent computing resources of roadside unit i，j，n (t)p j (t).

[0114] Estimate the personal utility of mobile users' vehicles based on value budget, service pricing and opportunity cost:

[0115] Mobile users use opportunity cost to balance immediate benefits and potential opportunities and risks. Describes the value estimation bias caused by the price change of the optional roadside unit, and has an initial value O i (t=0)=0. Among them, |·| is an operator for the number of elements in a set. The long-term utility U of mobile user vehicle i i (t) is:

[0116] U i (t) = η∑ j (E i，j，n (t)-a i，j，n (t)p j (t))+O i (t) (4)

[0117] Among them, η is the weight coefficient used to normalize and distinguish user types, which depends on the user type. For example, risk-sensitive users prefer smaller η.

[0118] Maximize the long-term utility of mobile user vehicles:

[0119]

[0120] stC1:a i，j，n (t)DT i，j，n ≤T n

[0121] C2: a i，j，n (t)P j (t)≤E i，j，n (t)

[0122]

[0123] C4:a i，j，n (t)∈0,1

[0124] Among them, τ0 is the length of a time slot, p j (t) is the calculated service price of roadside unit j in time slot t, T n is the delay constraint of task n. In order to maximize the user utility within the task delay limit, C1 gives the constraint of task service delay, C2 is the task budget limit, C3 gives the computing task scheduling constraint, and C4 is the task scheduling binary variable.

[0125] The establishment of the edge-side task processing model includes the process of estimating the unit task processing time, updating the number of backlog tasks, and judging service congestion, which is specifically as follows:

[0126] Roadside units will control the number of task backlogs in the processing queue through a dynamic price setting mechanism, preventing service congestion while also increasing service profits.

[0127] S31. The process of estimating the time required for processing a roadside unit task is as follows:

[0128] The tasks arriving at the roadside unit come from multiple users and still follow the Poisson process, with an arrival rate of

[0129] λ j (t) = ∑ i λ i (t)(∑ n a i,j，n (t) / n i (t)) (6)

[0130] f j is the computing capacity of roadside unit j, the arriving task flow forms an M / M / 1 queue, and the service rate is μ j =f j / Y (the number of tasks that the roadside unit can process per second). Calculate the estimated processing time for task n for:

[0131]

[0132] S32. Update the number of backlog tasks and determine service congestion. The specific process is as follows:

[0133] At the beginning of each time slot, the jth roadside unit updates the number of its unprocessed backlog tasks and determines whether service congestion occurs. At the beginning of time slot t+1, the number of backlog tasks of roadside unit j is Q j (t+1) is:

[0134]

[0135] The jth roadside unit adjusts the price p of its unit computing resources j , to influence the unloading decision of mobile users’ vehicles. Represents the queue size, then the number of backlogged tasks needs to satisfy the constraint When the number of tasks arriving is greater than the upper limit of the edge node service capacity, that is, When μ j τ0 is the amount of tasks that a roadside unit can serve in one time slot.

[0136] The process of establishing a mathematical model to maximize the overall utility of roadside units by controlling the number of tasks in the processing queue through a dynamic price setting mechanism and satisfying the conditions of not generating service congestion and energy consumption constraints is as follows:

[0137] S31: Set the roadside unit service price p j ;

[0138] The demand curve is a downward sloping curve related to price and demand, using a linear function to characterize the relationship between price and the supply of computing resources:

[0139]

[0140] Among them, γ j A negative number.

[0141] S32: Calculate the roadside unit revenue. The specific process is as follows:

[0142] Roadside units provide computing services for tasks on a first-come, first-served basis and first-paid-before-served basis. The revenue V of the jth roadside unit is j (t) is:

[0143]

[0144] in, is the normalized energy consumption.

[0145] S33: The roadside unit sets an appropriate price and establishes a mathematical model to maximize the overall utility of the roadside unit under the constraint of mission capacity. The details are as follows:

[0146]

[0147]

[0148] The simplified variable is the task acceptance quantity q(t) of the roadside unit;

[0149] The process of introducing simplified variables, relaxing integer constraints, solving the mathematical model of maximizing the overall utility of roadside units through KKT conditions, and obtaining the optimal service pricing for the overall utility of roadside units is as follows:

[0150] S41: Estimate the task acceptance of roadside units;

[0151] Let q(t) = ∑ i，n a i，j，n (t), then the number of unloading tasks to be performed by the roadside unit in formula (8) is:

[0152]

[0153] Among them, the condition Q is satisfied j (x)≤u j τ.

[0154] S42: Relax the integer constraints as follows:

[0155] Relax the integer constraint on the variable q(t) and obtain the optimal task acceptance through the KKT condition:

[0156]

[0157] in, is the floor operator, β j is the Lagrange multiplier.

[0158] S43: Calculate optimal pricing for roadside units;

[0159] The optimal pricing is obtained based on the relationship between the optimal task acceptance volume and price and the computing resource supply.

[0160] The optimal service pricing method uses a multi-agent reinforcement learning algorithm (Win or Learn Fast-Policy Hill Climbing, WoLF-PHC algorithm) to solve the mathematical model of maximizing the long-term utility of mobile user vehicles, and the process of determining the final edge computing server for offloading the computing tasks of mobile user vehicles is as follows:

[0161] On the user side, based on the optimal service pricing, the multi-agent reinforcement learning algorithm WoLF-PHC is used to solve problem P1.

[0162] S51: Mobile user vehicles to implement multi-agent strong chemistry algorithm WoLF-PHC:

[0163] The surrounding roadside unit prices and the mobile user's vehicle location are taken as the state s: s = {s1, s2, ..., s s}; Each different price and position combination corresponds to a different state s s , the state is a discrete space.

[0164] The unloading decision of the mobile user's vehicle is taken as action A i : is the roadside unit index, |A i |The number of roadside units available for mobile user vehicle i.

[0165] Use the utility value as the reward r: r = U i (t);

[0166] S52: Initialize Q value Q(s, a)←0, probability strategy π(s, a)←1 / |A i | and the number of iterations C(s)←0. After initialization, the mobile user vehicle implements the following process in each iteration:

[0167] Select action a: Select action a in state s according to the probability strategy π(s, a), and observe the corresponding reward r and the next state s′ corresponding to the mobile user’s environment after selecting the action;

[0168] Update the value in the Q table: According to the reward r and state s′, update the Q value: Q(s, a)←(1-a)Q(s, a)+a(r+γmax a′ Q(s′, a′));

[0169] Update the average probability strategy Update C(s)←C(s)+1; for any a′∈A i ,have

[0170] Update the probability strategy π: constrain the strategy π to a reasonable probability distribution;

[0171] If a=argmax a′ Q(s′,a′), then π(s,a)←π(s,a)+δ; otherwise, when When δ=δ ω ; otherwise, δ=δ l , δ ω >δ l .

[0172] Until the absolute value of the difference between the Chebyshev norms of two adjacent iterations of the Q table is less than the iteration stop condition 10 -4 , and the final probability strategy is Figure 2 , 3 As shown in Figure 2, the present invention gives the relationship between the long-term utility of a mobile user vehicle and the number of mobile user vehicles, as well as the average utility of a single mobile user vehicle and the computing power of the roadside unit. It can be seen from the two figures that the performance of the proposed algorithm is better than the baseline solution (when selecting action a in each time slot, mobile user vehicle j moves from A to B). j This is because in each time slot iteration, the action is selected according to the strategy, and each mobile user vehicle under the proposed algorithm is updated based on the price information, considering the current optimal decision and future opportunity utility from the perspective of long-term utility; therefore, the proposed algorithm can effectively use price information to coordinate the supply and demand distribution of computing and communication resources.

[0173] Figure 2The figure is a graph showing the relationship between the long-term utility and the number of mobile user vehicles when the computing power of the roadside unit is poor (f∈[1,5]GHz) and strong (f∈[5,25]GHz). Due to the limited resources available to the roadside unit in the network and penalties for unfinished tasks, the stronger the computing power of the roadside unit, the higher the corresponding long-term utility, and this advantage increases with the number of mobile user vehicles. It can be seen that when f is small, the baseline utility decreases with the increase in the number of mobile user vehicles, and the proposed algorithm increases slightly until the number of mobile user vehicles increases to 10, at which point the utility decreases due to severe resource shortages. When f is large, as the number of mobile user vehicles increases, the baseline utility is basically stable, and the utility of the proposed algorithm increases.

[0174] Figure 3 The relationship between the long-term utility of an average single mobile user vehicle and the computing power of the roadside unit when the number of mobile user vehicles is small (I = 10) and large (I = 20). As the computing power of the roadside unit increases, the four schemes can dispatch more computing resources to meet the demand. The proposed algorithm performs better in the sparse mobile user vehicle area than in the dense mobile user vehicle area, but the gap between the two decreases as the computing power of the roadside unit increases. The baseline performs consistently in the two mobile user vehicle density areas because they are randomly selected and not affected by environmental factors.

[0175] The vehicle-connected MEC computing task scheduling method based on dynamic pricing and terminal device mobility proposed in the present invention takes into account the limited resources of edge servers and the uneven load distribution that may lead to service congestion problems. It combines the dynamic service price design with the mobility of mobile users' vehicles to design a task scheduling scheme, which is more realistic. In addition, the task scheduling scheme based on the multi-agent strong chemistry algorithm WoLF-PHC proposed in the present invention can significantly outperform the representative baselines considered under various network settings.

[0176] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention. The techniques, shapes, and structural parts not described in detail in the present invention are all known technologies.

Claims

1. A method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility, characterized by: The following steps are involved: Establishing a mobile edge computing system for an Internet of Vehicles, and dividing the mobile edge computing system for an Internet of Vehicles into a user side and a task side, wherein the user side includes a plurality of mobile user vehicles, and the task side includes a plurality of roadside units, each of which is provided with an edge computing server; the user side and the task side perform task offloading through a macro base station; Establish a user-side task offloading model. In binary task offloading scheduling, establish a mathematical model to maximize the long-term utility of mobile user vehicles under the conditions of meeting budget and delay constraints. Establish an edge-side task processing model, control the number of task backlogs in the processing queue through a dynamic price setting mechanism, and establish a mathematical model to maximize the overall utility of roadside units under the conditions of not causing service congestion and energy consumption constraints. Simplified variables are introduced, integer constraints are relaxed, and the mathematical model of maximizing the overall utility of roadside units is solved through the Carlo-Kuhn-Tucker condition to obtain the optimal service pricing for the overall utility of roadside units. Based on the optimal service pricing, a multi-agent reinforcement learning algorithm is adopted to solve the mathematical model of maximizing the long-term utility of mobile user vehicles and determine the final edge computing server for offloading the computing tasks of mobile user vehicles.

2. The method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility according to claim 1 is characterized in that: The process of establishing the user-side task offloading model is as follows: The IoV mobile edge computing system is divided into time slots, and the following process is performed in each time slot t: The mobile user vehicle generation task within the coverage area of ​​the jth roadside unit follows a Poisson process with an average rate of λ i , roadside unit index represents the set of mobile user vehicles within the coverage area of ​​the jth roadside unit in the tth time slot, Tuple for task n To describe, is the generation time of task n, T n is the latency constraint of task n, X represents the average offloaded data size of each task, and Y represents the average computational load required to complete the task; Assuming that the channel coherence time is greater than the length of a time slot τ0, the transmission rate R i,j (t) is: Where B is the bandwidth, d i,j is the distance between mobile user vehicle i and the jth roadside unit in the tth time slot, α is the path loss exponent, represents the transmission power of mobile user vehicle i, σ 2 represents the background noise power, e i For inter-cell interference, obtain the unloaded transmission delay for: Among them, a i,j,n =0,1 is the binary unloading decision of the mobile user vehicle. In each time slot, the mobile user vehicle generates n i (t) computing tasks, the number of tasks to be unloaded by mobile user vehicle i in this time slot is N i (t) = n i (t)+N i (t-1)-∑ j a i,j,n (t-1); Among them, at the beginning there is N i (1) = n i (1), so the unloading decision a of mobile user vehicle i in the tth time slot is i (t) is an N i (t)×J i (t) is a binary matrix; where J i (t) represents the number of edge nodes connected to mobile user vehicle i through the V2I vehicle-infrastructure link.

3. The method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility according to claim 1 is characterized in that: In the binary task offloading scheduling, the process of establishing a mathematical model to maximize the long-term utility of mobile user vehicles under the conditions of meeting budget and delay constraints is as follows: Each mobile user vehicle evaluates the value budget E of each task service based on the unloading duration and service pricing. i,j,n : The estimated time required for the entire task offloading is DT i,j,n for: in is the processing delay of task n at roadside unit j; thus, the personal value function E of mobile user vehicle i i,j,n (t) is: Among them, β i is a trade-off parameter used to balance the unit computing service price p of roadside unit node j j and unloading performance 1 / DT i,j,n , mobile user vehicle i needs to pay resource rental cost a to rent computing resources of roadside unit i,j,n (t)p j (t); Estimate the personal utility of mobile users' vehicles based on value budget, service pricing and opportunity cost: Mobile user vehicles use opportunity cost O to balance immediate benefits and potential opportunities and risks. i (t) describes the bias in the value estimate caused by changes in the price of the alternative roadside unit, where: And has an initial value O i (t=0)=0, where |·| is the operator of the number of set elements, and the long-term utility U of mobile user vehicle i i (t) is: U i (t)=η∑ j (E i,j,n (t)-a i,j,n (t)p j (t))+O i (t) (4) Among them, η is the weight coefficient; Maximize the long-term utility U of mobile users’ vehicles i (t), the specific formula is as follows: s.t.C1:a i,j,n (t)DT i,j,n ≤T n C2:a i,j,n (t)P j (t)≤E i,j,n (t) C4:a i,j,n (t)∈0,1 Among them, τ0 is the length of a time slot, p j (t) is the calculated service price of roadside unit j in time slot t, T n is the delay constraint of task n. In order to maximize the user utility within the task delay limit, C1 gives the constraint of task service delay, C2 is the task budget limit, C3 gives the computing task scheduling constraint, and C4 is the task scheduling binary variable.

4. The method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility according to claim 1, characterized in that: The establishment of the edge-side task processing model includes the process of estimating the unit task processing time, updating the number of backlog tasks, and judging service congestion, which is specifically as follows: S31: The unit task processing time estimation is as follows: The task of reaching the roadside unit comes from multiple mobile user vehicles, still following the Poisson process, and its arrival rate λ j (t) is: λ j (t)=∑ i λ i (t)(∑ n a i,j,n (t) / n i (t)) (6) Where: f j is the computing capacity of roadside unit j, the arriving task flow forms an M / M / 1 queue, and the service rate is μ j =f j / Y, calculate the estimated processing time of task n for: S32: The roadside edge-side task processing model includes updating the number of backlog tasks and judging service congestion. The specific process is as follows: At the beginning of each time slot, the jth roadside unit updates the number of its unprocessed backlog tasks and determines whether service congestion occurs. At the beginning of time slot t+1, the number of backlog tasks of roadside unit j is Q j (t+1) is: The jth roadside unit adjusts the price p of its unit computing resources j , to influence the unloading decision of mobile users’ vehicles, Represents the queue size, then the number of backlogged tasks needs to satisfy the constraint When the number of tasks arriving is greater than the upper limit of the edge node service capacity, that is, When , it is considered that service congestion occurs, μ j τ0 is the amount of tasks that a roadside unit can serve in one time slot.

5. The method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility according to claim 1, characterized in that: The process of establishing a mathematical model to maximize the overall utility of roadside units by controlling the number of tasks in the processing queue through a dynamic price setting mechanism and satisfying the conditions of not generating service congestion and energy consumption constraints is as follows: S31: Set the roadside unit service price p j ; A linear function is used to characterize the relationship between price and computing resource supply: Among them, γ j is the demand curve factor, which is a negative number, and p0 is the initial price, which is a natural number; S32: Calculate roadside unit revenue; Roadside units provide computing services for tasks in a first-come, first-served and first-paid-before-service manner. The revenue of the jth roadside unit is: in, is the normalized energy consumption; S33: The roadside unit sets an appropriate price and establishes a mathematical model to maximize the overall utility of the roadside unit under the constraint of mission capacity, as follows:

6. The method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility according to claim 1, characterized in that: The simplified variables are: roadside unit task acceptance quantity q(t); The process of introducing simplified variables, relaxing integer constraints, solving the mathematical model of maximizing the overall utility of roadside units through the Carlo-Kuhn-Tucker condition, and obtaining the optimal service pricing for the overall utility of roadside units is as follows: S41: estimating the task acceptance of roadside units; Let q(t) = ∑ i,n a i,j,n (t), then the number of unloading tasks to be performed by the roadside unit in formula (8) is: Among them, the condition Q is satisfied j (x)≤u j τ; S42: Relaxing integer constraints; Relax the integer constraint on the variable q(t) and obtain the optimal task acceptance q through the KKT condition * (t) is: in, is the floor operator, β j is the Lagrange multiplier; S43: Calculate the optimal pricing of roadside units; The optimal pricing is obtained based on the relationship between the optimal task acceptance and price and the supply of computing resources.

7. The method for scheduling vehicle-connected MEC computing tasks based on dynamic pricing and terminal device mobility according to claim 1, characterized in that: The process of solving the mathematical model of maximizing the long-term utility of mobile user vehicles based on optimal service pricing and using a multi-agent reinforcement learning algorithm to determine the final edge computing server for offloading the computing tasks of mobile user vehicles is as follows: S51. The surrounding roadside unit prices and the mobile user vehicle location are taken as the state s: s = {s1, s2, ..., s s }; Each different price and position combination corresponds to a different state s s , the state is a discrete space; The unloading decision of the mobile user's vehicle is taken as action A i : is the roadside unit index, |A i |The number of roadside units available for mobile user vehicle i; Use the utility value as the reward r: r = U i (t); S52: Initialize Q value Q(s,a)←0, probability strategy π(s,a)←1 / |A i | and the number of iterations C(s)←0; after initialization, the mobile user vehicle implements the following process in each iteration: Select action a: Select action a in state s according to the probability strategy π(s,a), and observe the corresponding reward r and the next state s′ corresponding to the mobile user’s environment after selecting the action; Update the value in the Q table: According to the reward r and state s′, update the Q value: Q(s,a)←(1-a)Q(s,a)+a(r+γmax a′ Q(s′,a′)); Update the average probability strategy Update C(s)←C(s)+1; for any a′∈A i ,have Update the probability strategy π: constrain the strategy π to a reasonable probability distribution; If a = argmax a′ Q(s′, a′), then π(s, d) ← π(s, a) + δ; otherwise, When , δ = δ ω ; otherwise, δ = δ l , δ ω > δ l ; Until the absolute value of the difference between the Chebyshev norms of two adjacent iterations of the Q table is less than the iteration stop condition 10 -4 , get the final probability strategy π, and obtain action a according to the probability strategy under the corresponding environment state s as the final edge computing server.

Images