MEC resource scheduling method and system based on deep learning

Abstract

The invention discloses an MEC resource scheduling method and system based on deep learning, relates to the field of mobile communication resource scheduling, and provides a resource scheduling strategy which solves the problem of mobility generated when tasks are allocated to direct connection service vehicles. And the method is formulated by using the advantages of the non-direct-connected vehicles in task calculation. Considering that the connection limitation of direct-connection and non-direct-connection vehicles is crucial for determining a feasible solution, an optimization function and a constraint condition which take client vehicle utility minimization as a target are determined according to the time required for task execution and the relationship between the connectable time of a client vehicle and a service vehicle; and solving by using a routing forwarding strategy optimization algorithm so as to obtain an optimal resource scheduling strategy. Meanwhile, the corresponding communication model is selected by considering the distance between the vehicles, so that the average time required by resource scheduling is reduced, and the system is more efficient.

Description

Technical Field

[0001] The present invention relates to the field of mobile communication resource scheduling, and in particular to a MEC resource scheduling method and system based on deep learning. Background Art

[0002] Vehicular IoT has gained tremendous attention in the research community in recent years and has experienced significant growth with the emergence of emerging devices and technologies. Applications such as autonomous driving, which have strict latency constraints and high processing requirements, place significant demands on computing and storage resources. In the prior art, resource scheduling has received much attention, especially in studying server-based resource scheduling strategies. These strategies involve using roadside units (RSUs) as edge servers with communication, computing, and storage capabilities, and the scheduling of resources is done on a single edge server or task offloading based on mobility across multiple edge servers. However, the penetration rate of edge servers is very low and the implementation of edge servers is quite expensive, and it is impractical to fully deploy edge servers along road sections. Mobile edge computing (MEC) combined with vehicular networks is a potential strategy to meet this demand. However, the complex channel environment and high mobility of vehicles lead to constantly changing network topologies, making communication links fragile, which poses a major challenge to making optimal decisions. An efficient resource scheduling process should also ensure that the tasks sent to the service vehicles are completed before the connection between them is terminated, as the contact time between vehicles is short. Similarly, since vehicle users are selfish, they may not be allowed to use computing and storage resources unless they are well compensated. Therefore, how to design an effective method to schedule MEC resources is a major challenge. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings existing in the prior art and to provide a MEC resource scheduling method and system based on deep learning. The resource scheduling method is formulated by solving the mobility problem caused by assigning tasks to directly connected service vehicles and taking advantage of the advantages of non-directly connected vehicles in task calculation. For directly connected vehicles, any given task must be completed before exiting the communication range; in the case of non-directly connected vehicles, they can only participate in task calculation after the connection between the client and the service vehicle is established. Considering the connection limitations of directly connected and non-directly connected vehicles is crucial to determining a feasible solution, and a routing forwarding strategy optimization algorithm is proposed. At the same time, considering the selection of the corresponding communication model based on the distance between vehicles, the average time required for resource scheduling is reduced, making the system more efficient.

[0004] The present invention first discloses a MEC resource scheduling method based on deep learning, and the specific steps are as follows:

[0005] Step 1: According to the vehicle request resource task, establish an MEC network architecture based on the client vehicle and multiple service vehicles;

[0006] Step 2: Under the MEC network architecture, a mobility model is constructed. Different communication models are determined by the distance between the client vehicle and the service vehicle. The communication models include a cellular communication model and a millimeter wave communication model. Different task models are constructed by the resource processing mode, and the corresponding task execution time is obtained according to different task models.

[0007] Step 3, determining the optimization function and constraint conditions with the goal of minimizing the utility of the client vehicle based on the relationship between the time required for task execution and the connectable time of the client vehicle and the service vehicle;

[0008] Step 4: Model the optimization objective as a Markov decision process and use the routing forwarding strategy optimization algorithm to obtain the optimal resource scheduling strategy.

[0009] Furthermore, the mobility model constructed in step 2 is specifically:

[0010] Assume that each vehicle moves at a random speed v, the speed is independent and follows a Gaussian distribution; since the vehicle speed varies with time, it is selected from a truncated Gaussian distribution to prevent negative speeds. The truncated Gaussian probability density function is expressed as:

[0011]

[0012] in, is the Gaussian probability density function; V max =μ+3θ is the maximum speed; V min =μ-3θ is the minimum speed; erf(·) is the error function; μ is the average speed; θ is the standard deviation of the speed;

[0013] The vehicle speed is V min <v<V max The expressions in scope are:

[0014]

[0015] Furthermore, the cellular communication model constructed in step 2 is specifically:

[0016] Each vehicle includes multiple antennas for 5G connectivity; vehicle v n and vehicle v i The transmission rate between can be calculated as follows:

[0017]

[0018] Among them, P trepresents the transmission power of the vehicle; h represents the channel fading coefficient; N0 is the Gaussian noise power; W cc and represent bandwidth and path loss respectively;

[0019] The path loss is calculated as follows:

[0020]

[0021] Among them, d n,i v n and v i The distance between

[0022] Calculate the vehicle v before time t i The remaining distance to leave its coverage area is calculated as follows:

[0023]

[0024] Among them, vehicle v n With vehicle v i The positions at time t are represented by x n (t), y n (t) and x i (t), y i (t) indicates; r i Indicates vehicle v i The coverage area radius;

[0025] Vehicle n In vehicle v i The time spent in the coverage area is calculated as:

[0026]

[0027] in, Indicates vehicle v n and v i the difference in speed with respect to their relative directions;

[0028] The transmission rate varies with time and is expressed as Vehicle n and vehicle v i The average transfer rate between is:

[0029]

[0030] The millimeter wave communication model constructed is as follows:

[0031] Each vehicle is equipped with a directional antenna, which is aligned before transmission; the antenna gain on vehicle i is expressed as a function of the steering angle φ, and the millimeter wave antenna gain for:

[0032]

[0033] Among them, the mathematical gains of the main lobe and side lobe are and φ represents the deviation angle from the axis direction; the beam width of the main lobe is φ b express;

[0034] Indicates vehicle v n and vehicle v i Transmission rate:

[0035]

[0036] in, Defined as:

[0037]

[0038] The estimated value of the link signal-to-noise ratio of the millimeter wave is obtained by the following formula:

[0039]

[0040] Among them, the transmission power of the vehicle in the millimeter wave link is P i Indicated by; Noise power is represented by N o Indicates; millimeter wave bandwidth is expressed in W mm denoted by ; path loss is denoted by φ; vehicle v n and vehicle v i The distance between the vehicles under millimeter wave coverage is expressed as d n,i The main lobe array gain is expressed as Indicated by; shadow fading is represented by ρ α express.

[0041] Further, the specific steps of determining the communication model according to the distance between the client vehicle and the service vehicle are:

[0042] When the transmission time T of task i k,i Less than or equal to vehicle v n In vehicle v i Duration of stay in the coverage area Then determine whether it is within the millimeter wave range; when the vehicle v n In vehicle v i If the distance between them is within the millimeter wave communication range, the transmission is carried out in millimeter wave mode, and the current transmission rate is set to If not, then when the vehicle v i and the next vehicle v j If the distance between them is within the millimeter wave communication range, the millimeter wave is used for relay transmission, and the current transmission rate is set to When the vehicle v i and the next vehicle v j If the distance between them is not within the millimeter wave communication range, the cellular mode is used for relay transmission, and the current transmission rate is set to r hcc .

[0043] Furthermore, in step 2, different task models are constructed according to the resource processing mode:

[0044] A client vehicle that performs N tasks is represented by the task set N = {1, ..., N}; M = {1, ..., M} defines the set of available service vehicles; each task i∈N is represented by D = s i ,ω i Indicates; where s i Indicates the size of the input data; ω i represents the number of CPU cycles required to complete the task; the client vehicle is represented as o; task i and The uninstall variable is represented by x k,i ∈{0,1}; where x oi =1 indicates that task i is a local calculation; x k,i =1 indicates that task i is unloaded to the corresponding service vehicle k.

[0045] Furthermore, the time required for task execution of different task models is as follows:

[0046] For local computation, use f n represents the local computing power of client vehicle o, that is, the number of CPU cycles per second; for client vehicle n, the time required to perform task i in the nearby area is:

[0047]

[0048] Then the processing time of all tasks executed locally is calculated as:

[0049]

[0050] For vehicle unloading calculation, the time T required to complete task i for a given service vehicle k is k,i It consists of three parts: transmission time Execution time and output time

[0051]

[0052] The time required for processing a task from client vehicle n to service vehicle k is:

[0053]

[0054] Furthermore, the time required for vehicle unloading calculation is specifically:

[0055] Direct transmission time: The time required to upload task i from the client vehicle to its directly connected vehicle k is:

[0056]

[0057] Among them, S i Indicates the size of the input data; The speed selection can be determined by the distance between vehicles;

[0058] The time for client vehicle n to connect to its directly connected vehicle k is given by Indicates that if the following equation is satisfied, the directly connected vehicle k can complete all assigned tasks:

[0059]

[0060] Indirect transmission time: Assuming that each indirect path contains u direct paths connecting client vehicle n to its service vehicle k, the connectivity of the path is expressed as:

[0061]

[0062] Select the path with the shortest connection time to upload the task;

[0063] Task calculation time: The time required for service vehicle k to complete task i is:

[0064]

[0065] Feedback time: Since the size of the output data is significantly smaller than the input data, the feedback time Ignore.

[0066] Furthermore, in step 3, the optimization function and constraints for minimizing the utility of the client vehicle are specifically as follows:

[0067] The non-directly connected vehicle task offloading problem is defined as the client vehicle utility minimization problem; the problem is expressed as:

[0068]

[0069]

[0070] Among them, C1 means that each task can only be calculated on a single vehicle; C2 means that each task can be processed locally or by offloading; C3 means that in order to successfully offload the task, the processing time of the directly connected service vehicle must be less than or equal to the connection time between them; C4 means that the sum of the execution time of all tasks assigned to the service vehicle must be less than or equal to the connection time between the vehicle and the client vehicle.

[0071] Furthermore, in step 4, the optimization objective is modeled as a Markov decision process:

[0072] State space: The state of the system in period τ is expressed as S(τ) = {D' τ ,M' τ ,r' h ,W τ ,pτ}, where D' τ is the set of distances between vehicles in the cycle; M' τ is the set of all scheduled data in the period; the total communication bandwidth of the period τ is given by W τ The transmission power of each vehicle is denoted by pτ; the availability of the communication link between the client and the serving vehicle is denoted by r' h express;

[0073] Action space: The ability of the agent to decide whether each task should be unloaded and to which vehicle constitutes the action space of the system. For the unloading decision of a single task k, x k is a decision vector, each x i Indicates whether task k is unloaded to a specific vehicle i:

[0074] x k =(x1,x2,…,x n )

[0075] Where x is limited to 0 or 1; i ranges from 1 to n, where n is the total number of available service vehicles;

[0076] The decision set for all tasks unloading within the period τ is denoted by x τ express:

[0077] x τ =(x1,x2,…,x K )

[0078] Among them, x τ is a decision vector, each x i Indicates whether a task is unloaded to a specific vehicle, x i The limit is 0 or 1; the range of i is 1 to K, where K is the total number of tasks that need to be completed in the current cycle;

[0079] Reward function: within time τ, the discounted cumulative reward obtained by the agent from the initial state to the final state of the system is determined as follows:

[0080]

[0081] where γ is the discount factor of the reward; τ end represents the time step of the system termination state; r(τ) is the immediate reward obtained by the agent at time step τ.

[0082] Specifically, the optimal resource scheduling strategy is obtained by using the routing forwarding strategy optimization algorithm as follows:

[0083] Step 41: Input the initial policy network parameters πθ and the critic network parameters ω;

[0084] Step 42: For each iteration i < N, where N is the number of iterations; perform the following steps;

[0085] 1) Interact with the environment using the policy network parameters for a continuous T time steps, update the speed and relative position between vehicles according to the mobility model; collect state, action, and reward data for subsequent network training;

[0086] 2) Calculate where represents estimating the true value of state s τ using the cumulative discounted reward collected by n-step sampling after state s τ ; V πθ (s τ , ω) is the estimated value of state s τ provided by the critic network; γ represents the discount factor used for rewards; update the policy network parameters πθ using the gradient policy method;

[0087] 3) Minimize to reduce the error in state value evaluation, where E τ [·] represents the expected value; and update the critic network parameters ω using the gradient descent method;

[0088] Step 43: Repeat processes 1)-3), and when the maximum number of iterations is reached, output the optimized policy network parameters and critic network parameters, thereby obtaining the optimal resource scheduling strategy.

[0089] On the other hand, the present invention also provides a deep learning-based MEC resource scheduling system, which includes:

[0090] at least one processor; and a memory communicatively connected to the at least one processor; where

[0091] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute any one of the above-mentioned deep learning-based MEC resource scheduling methods.

[0092] The present invention relates to a MEC resource scheduling method and system based on deep learning. The resource scheduling method is formulated by solving the mobility problem caused by assigning tasks to directly connected service vehicles and taking advantage of the advantages of non-directly connected vehicles in task calculation. For directly connected vehicles, any given task must be completed before exiting the communication range; in the case of non-directly connected vehicles, they can only participate in task calculation after the connection between the client and the service vehicle is established. Considering the connection restrictions of directly connected and non-directly connected vehicles is crucial to determining a feasible solution. The optimization function and constraints with the goal of minimizing the utility of the client vehicle are determined by the relationship between the time required for task execution and the connectable time of the client vehicle and the service vehicle, and the routing forwarding strategy optimization algorithm is used to solve it, thereby obtaining the optimal resource scheduling strategy. At the same time, considering the selection of the corresponding communication model based on the distance between vehicles, the average time required for resource scheduling is reduced, making the system more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 This is a schematic diagram of vehicle types and connection methods under the MEC architecture;

[0094] Figure 2 A schematic diagram of the structure of a MEC resource scheduling system based on deep learning is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0095] The detailed description set forth below in conjunction with the accompanying drawings is intended to be a description of various exemplary embodiments of the present invention, and is not intended to represent the only embodiment that can practice the present invention. For the purpose of providing a thorough understanding of the present invention, the detailed description includes specific details. However, it is apparent to those skilled in the art that the present invention can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid blurring the concept of the present invention.

[0096] like Figure 1As shown, there are vehicles in two directions on a straight road (such as a highway). In this case, there are two types of vehicles: vehicles that perform client tasks and vehicles that provide services. Service vehicles can also act as relay vehicles to achieve indirect communication. Service vehicles pool their unused computing resources to complete tasks generated by client vehicles. It is worth mentioning that the functions of each vehicle may vary depending on its tasks or computing resources. When implementing resource scheduling, there are two methods that can be used to complete the tasks of client vehicles. When the client vehicle handles the processing of all its tasks, the calculations are performed locally. In the second method, when resources are unloaded, the client vehicle sends its work to nearby vehicles with additional computing power.

[0097] In order to determine where and how to compute its tasks to meet demand, the client vehicle first has to gather information about available service vehicles. Only then can it create an effective task allocation algorithm. In this optimization problem, the best offloading decision is determined by considering both processing delay and incentive payment. This includes wireless transmission delays for receiving and uploading data as well as vehicle computation delays. However, unlike conventional MEC, the distance between client and service vehicles may affect the total delay. If the use of vehicle resources is to be maximized and the experience of vehicle users is to be enhanced, it is crucial to incentivize vehicles with surplus processing resources to participate in task offloading through some kind of incentive system. Client vehicles need to consider the cost of using service vehicles to process tasks. Link connectivity is crucial to ensure that tasks are offloaded. Links in vehicle networks often fail because nodes are constantly moving. Therefore, connectivity between vehicles will significantly affect the allocation of tasks. Better link connectivity makes it more likely that client vehicles will hand over their tasks to service vehicles.

[0098] The availability of the link is a key factor in designing non-directly connected service vehicles, as they are more susceptible to outages than directly connected vehicles. Non-directly connected service vehicles can receive offloaded tasks from client vehicles. If the connection between the two vehicles remains stable during the offloading process, the results can be sent back to the client vehicle. Client vehicles can achieve optimal offloading decisions by addressing this challenge. When a client vehicle must rely on one or more service vehicles for task processing, it makes this decision through offloading decisions. It then sends the relevant tasks to the involved service vehicles. Once these tasks are received, each service vehicle will contribute to the task processing. The service vehicles complete the tasks and report the results. Once the client receives all the relevant data, the application can run.

[0099] Example 1

[0100] 1. Mobility Model

[0101] Assume that each vehicle moves at a random speed, which is independent and follows a Gaussian distribution. Since the vehicle speed varies over time, each vehicle is assigned a random speed v, selected from a truncated Gaussian distribution to prevent negative speeds. The truncated Gaussian probability density function is expressed as:

[0102]

[0103] in, is the Gaussian probability density function; V max =μ+3θ is the maximum speed; V min =μ-3θ is the minimum speed; erf(·) is the error function; μ is the average speed; θ is the standard deviation of the speed;

[0104] Based on formula (1), the vehicle speed is V min <v<V max The expressions in scope are:

[0105]

[0106] 2. Communication model

[0107] Vehicle-to-vehicle (V2V) communications use 5G NR-V2X Mode-2 for cellular and mmWave connectivity. Each vehicle includes multiple antennas for mmWave and 5G connectivity. The distance between vehicles determines the communication mode. Details of the cellular and mmWave modes are as follows:

[0108] 1) Cellular mode

[0109] The NR-V2X Mode-2 protocol handles the cellular link in vehicle-to-vehicle communications. n and vehicle v i The transmission rate between can be calculated as follows:

[0110]

[0111] Among them, P t represents the transmission power of the vehicle; h represents the channel fading coefficient; N0 is the Gaussian noise power; W cc and represent bandwidth and path loss respectively;

[0112] The path loss is calculated as follows:

[0113]

[0114] Among them, d n,i v n and v i The distance between

[0115] In addition, the vehicle v n In vehicle v i The time spent in the coverage area is a key metric used in analysis to prevent scheduling failures; using this information, the time that a vehicle v spends before time t can be calculated. i The remaining distance to leave its coverage area is calculated as follows:

[0116]

[0117] Among them, vehicle v n In vehicle v i The positions at time t are represented by x n (t), y n (t) and x i (t), y i (t) indicates; r i Indicates vehicle v i The coverage area radius;

[0118] Vehicle n In vehicle v i The time spent in the coverage area is calculated as:

[0119]

[0120] in, Indicates vehicle v n and v i the difference in speed with respect to their relative directions;

[0121] The transmission rate varies with time and can be expressed as

[0122] Therefore, the vehicle v n and vehicle v i The average transfer rate between is:

[0123]

[0124] 2) Millimeter wave mode

[0125] In order to take advantage of mmWave in vehicle-to-vehicle (V2V) communication mode, it is assumed that each vehicle is equipped with a directional antenna. Therefore, before starting to use mmWave communication for task transmission, the vehicle's antenna must be aligned. Therefore, the antenna gain on vehicle i is expressed as a function of the steering angle φ, and the general mmWave antenna gain is for:

[0126]

[0127] Among them, the mathematical gains of the main lobe and side lobe are and φ represents the deviation angle from the axis direction; the beam width of the main lobe is φ b express;

[0128] Indicates vehicle v n and vehicle v i Transmission rate:

[0129]

[0130] in, Defined as:

[0131]

[0132] The estimated value of the link signal-to-noise ratio of the millimeter wave is obtained by the following formula:

[0133]

[0134] Among them, the transmission power of the vehicle in the millimeter wave link is P i Indicated by; Noise power is represented by N o Indicates; millimeter wave bandwidth is expressed in W mm denoted by ; path loss is denoted by φ; vehicle v n and vehicle v i The distance between the vehicles under millimeter wave coverage is expressed as d n,i The main lobe array gain is expressed as Represents shadow fading with ρ α express.

[0135] 3. Task Model

[0136] Consider a client vehicle that performs N tasks, represented by the task set N = {1, ..., N}; M = {1, ..., M} defines the set of available service vehicles. Each task i∈N is represented by a tuple D = s i ,ω i Indicates; where s i Indicates the size of the input data; ω i represents the number of CPU cycles required to complete a task; a client vehicle, denoted as o, can perform its task locally or remotely; tasks i and The uninstall variable is represented by x k,i ∈{0,1}; where x oi =1 indicates that task i is a local calculation; x k,i =1 means that task i is unloaded to the corresponding service vehicle k;

[0137] 1) Local computing

[0138] Use f nrepresents the local computing power of client vehicle o, that is, the number of CPU cycles per second; for client vehicle n, the time required to perform task i in the nearby area is:

[0139]

[0140] Then the total processing time of all tasks executed locally can be calculated as:

[0141]

[0142] 2) Vehicle unloading

[0143] For a client vehicle, the total time required to complete task i for each given service vehicle k is the time interval T k,i , consists of three parts, transmission time Execution time and output time

[0144]

[0145] The time required for processing a task from client vehicle n to service vehicle k is:

[0146]

[0147] (1) Direct transmission time

[0148] The time required to upload task i from the client vehicle to its directly connected vehicle k is:

[0149]

[0150] Among them, S i Indicates the size of the input data; The speed can be selected according to the vehicle distance;

[0151] The specific selection method is:

[0152] When the transmission time T of task i k,i Less than or equal to vehicle v n In vehicle v i Duration of stay in the coverage area Then determine whether it is within the millimeter wave range; when the vehicle v n In vehicle v i If the distance between them is within the millimeter wave communication range, the transmission is carried out in millimeter wave mode, and the current transmission rate is set to If not, then when the vehicle v i and the next vehicle v jIf the distance between them is within the millimeter wave communication range, the millimeter wave is used for relay transmission, and the current transmission rate is set to When the vehicle v i and the next vehicle v j If the distance between them is not within the millimeter wave communication range, the cellular mode is used for relay transmission, and the current transmission rate is set to

[0153] Due to vehicle mobility constraints, the task transferred from client vehicle n to directly connected service vehicle k must be completed before the connection between the client vehicle and the service vehicle is disconnected. Therefore, if the computation offloading is completed before the connection between the two vehicles is lost, then the offloading is successful; the time when the client vehicle n is connected to its directly connected vehicle k is given by Indicates that if the following equation is satisfied, the directly connected vehicle k can complete all assigned tasks:

[0154]

[0155] (2) Indirect transmission time

[0156] The relay node can pass the data received by the client vehicle n to any service vehicle k; assuming that each indirect path contains u direct vehicle paths connecting the client vehicle n to its service vehicle k, the connectivity of path j is expressed as:

[0157]

[0158] The connection time of the indirect path is the minimum of the connection times of all the directly connected vehicle paths in the path; this is because the stability of the entire path is determined by the least stable path; the path with the smallest connection time is selected to upload the task.

[0159] (3) Task calculation

[0160] The time required for service vehicle k to complete task i is:

[0161]

[0162] The present invention does not consider the time required for result feedback, because the size of the output data is significantly smaller than the input data, and the feedback time is negligible.

[0163] 4. Optimization Issues

[0164] The non-directly connected vehicle task offloading problem is defined as the client vehicle utility minimization problem to improve the client vehicle's QoE; the problem is expressed as:

[0165]

[0166] Among them, the constraints of P1 are as follows: C1 means that each task can only be calculated on a single vehicle; C2 means that each task can be processed locally or by offloading; C3 means that in order to successfully offload the task, the processing time of the directly connected vehicle service vehicle must be less than or equal to the connection time between them; C4 means that the sum of the execution time of all tasks assigned to the service vehicle must be less than or equal to the connection time between the vehicle and the client vehicle.

[0167] 5. Offloading of non-directly connected vehicle tasks

[0168] In the context of connected vehicles, vehicles facilitate information exchange by periodically emitting beacons that convey details of their speed and current location. In this network, candidate service vehicles are defined as those that are within the direct vehicle distance of the client vehicle's current location. This proximity is established through the client vehicle's continuous communication with nearby directly connected vehicles, enabling efficient task transfer between them. It is important to emphasize that vehicles located outside the direct vehicle distance of the client location can also be used for service. A relay transmission mechanism is adopted to enable communication between the client vehicle and these more distant service vehicles. This mechanism relies on intermediate vehicles as relay nodes, effectively bridging the gap between the client and the remote service vehicles.

[0169] 1) Markov decision process

[0170] (1) State space

[0171] In order for the system to function properly, the agent must make decisions based on information about the tasks that need to be completed in the current scheduling period and the available vehicle communication resources. When delegating tasks to service vehicles, it is critical to maintain the maximum range of communication; the distance between vehicles as described in Equation (5) and the millimeter wave or cellular communication method defined in Equations (7) and (10), respectively, determine whether vehicles can communicate with each other.

[0172] The state of the system in period τ is expressed as S(τ) = {D' τ ,M' τ ,r' h ,W τ ,pτ}, where D' τ is the set of all unloading requirements in this cycle; M' τ is the set of all available vehicle servers in this period; the total communication bandwidth of period τ is given by W τ The transmission power of each vehicle is denoted by pτ; the availability of the communication link between the client and the serving vehicle is denoted by r' h Indicator matrix expression;

[0173] (2) Action Space

[0174] The ability of the agent to decide whether each task should be unloaded and to which vehicle constitutes the action space of the system. For the unloading decision of a single task k, x k is a decision vector, each x i Indicates whether task k is unloaded to a specific vehicle i:

[0175] x k =(x1,x2,…,x n ) (twenty one)

[0176] Where x is limited to 0 or 1; i ranges from 1 to n, where n is the total number of available service vehicles;

[0177] The decision set for all tasks unloading within the period τ is denoted by x τ express:

[0178] x τ =(x1,x2,…,x K ) (twenty two)

[0179] Among them, x τ is a decision vector, each x i Indicates whether a task is unloaded to a specific vehicle, x i The limit is 0 or 1; the range of i is 1 to K, where K is the total number of tasks that need to be completed in the current cycle;

[0180] If the service vehicle assigned a computation task is out of communication range with the client vehicle that originally owned the task, the task is considered to have not been successfully offloaded;

[0181] (3) Reward Function

[0182] At time τ, the reward is determined by the ratio that takes into account the total amount obtained by the service vehicle and the sum of all tasks that the service vehicle attempts to unload at that time; the discounted cumulative reward obtained by the agent from the initial state of the system to the final state is determined as follows:

[0183]

[0184] Among them, γ is the discount factor of the reward; τ end represents the time step of the system's terminal state; r(τ) is the immediate reward obtained by the agent at time step τ;

[0185] 2) Routing and forwarding strategy optimization algorithm

[0186] The routing forwarding strategy optimization algorithm is implemented by using the critic network and the policy network; the specific process is as follows:

[0187] Step 41: Input the initial policy network parameters πθ and the critic network parameters ω;

[0188] Step 42: For each iteration i < N, where N is the number of iterations; perform the following steps;

[0189] 1) Interact with the environment using the policy network parameters for a continuous T time steps, update the speeds and relative positions between vehicles according to the mobility model; collect state, action, and reward data for subsequent network training;

[0190] 2) Calculate where represents the true value of state s estimated using the cumulative discounted reward collected by n-step sampling after state s τ ; V τ (s πθ , ω) is the estimated value of state s provided by the critic network τ ; γ represents the discount factor for rewards; update the policy network parameters πθ using the gradient policy method; τ

[0191] 3) Minimize to reduce the error in state value evaluation, where E τ [·] represents the expected value; and update the critic network parameters ω using the gradient descent method;

[0192] Step 43: Repeat Steps 1 - 3; when the maximum number of iterations is reached, output the optimized policy network parameters and critic network parameters to obtain the optimal resource scheduling policy.

[0193] Example 2

[0194] As can be seen from Figure 2 , this example provides a deep learning-based MEC resource scheduling system, which includes:

[0195] At least one processor; and a memory communicatively connected to the at least one processor; where

[0196] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to implement the deep learning-based MEC resource scheduling method described in Example 1 above.

[0197] ​As an embodiment of the present invention, the communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0198] As an embodiment of the present invention, the memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0199] As an embodiment of the present invention, the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0200] In summary, the present invention proposes a resource scheduling strategy, which is formulated by solving the mobility problem caused by assigning tasks to directly connected vehicles and service vehicles, and taking advantage of the advantages of non-directly connected vehicles in task calculation. For directly connected vehicles, any given task must be completed before exiting the communication range; in the case of non-directly connected vehicles, they can only participate in task calculation after the connection between the client and the service vehicle is established. Considering the connection restrictions of directly connected vehicles and non-directly connected vehicles is crucial to determining a feasible solution. The optimization function and constraints with the goal of minimizing the utility of the client vehicle are determined by the relationship between the time required for task execution and the connectable time of the client vehicle and the service vehicle. The routing forwarding strategy optimization algorithm is used to solve it, thereby obtaining the optimal resource scheduling strategy. At the same time, considering the selection of the corresponding communication model based on the distance between vehicles, the average time required for resource scheduling is reduced, making the system more efficient.

[0201] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A MEC resource scheduling method based on deep learning, the specific steps are as follows: Step 1: According to the vehicle request resource task, establish an MEC network architecture based on the client vehicle and multiple service vehicles; Step 2: Under the MEC network architecture, a mobility model is constructed. Different communication models are determined by the distance between the client vehicle and the service vehicle. The communication models include a cellular communication model and a millimeter wave communication model. Different task models are constructed by the resource processing mode, and the corresponding task execution time is obtained according to different task models. Step 3, determining the optimization function and constraint conditions with the goal of minimizing the utility of the client vehicle based on the relationship between the time required for task execution and the connectable time of the client vehicle and the service vehicle; Step 4: Model the optimization objective as a Markov decision process and use the routing forwarding strategy optimization algorithm to obtain the optimal resource scheduling strategy.

2. According to a deep learning-based MEC resource scheduling method according to claim 1, it is characterized in that: The mobility model constructed in step 2 is specifically: Assume that each vehicle moves at a random speed v, the speed is independent and follows a Gaussian distribution; since the vehicle speed varies with time, it is selected from a truncated Gaussian distribution to prevent negative speeds. The truncated Gaussian probability density function is expressed as: in, is the Gaussian probability density function; V max =μ+3θ is the maximum speed; V min =μ-3θ is the minimum speed; erf(·) is the error function; μ is the average speed; θ is the standard deviation of the speed; The vehicle speed is V min <v<V max The expressions in scope are:

3. According to the deep learning-based MEC resource scheduling method of claim 1, it is characterized in that: The cellular communication model constructed in step 2 is specifically: Each vehicle includes multiple antennas for 5G connectivity; vehicle v n and vehicle v i The transmission rate between can be calculated as follows: Among them, P t represents the transmission power of the vehicle; h represents the channel fading coefficient; N0 is the Gaussian noise power; W cc and represent bandwidth and path loss respectively; The path loss is calculated as follows: Among them, d n,i v n and v i The distance between Calculate the vehicle v before time t i The remaining distance to leave its coverage area is calculated as follows: Among them, vehicle v n With vehicle v i The positions at time t are represented by x n (t), y n (t) and x i (t), y i (t) indicates; r i Indicates vehicle v i The coverage area radius; Vehicle n In vehicle v i The time spent in the coverage area is calculated as: in, Indicates vehicle v n and v i the difference in speed with respect to their relative directions; The transmission rate varies with time and is expressed as Vehicle n and vehicle v i The average transfer rate between is: The millimeter wave communication model is specifically as follows: Each vehicle is equipped with a directional antenna, which is aligned before transmission; the antenna gain on vehicle i is expressed as a function of the steering angle φ, and the millimeter wave antenna gain for: The gains of the main lobe and side lobe are and φ represents the deviation angle from the axis direction; the beam width of the main lobe is φ b express; Indicates vehicle v n and vehicle v i Transmission rate: in, Defined as: The estimated value of the link signal-to-noise ratio of the millimeter wave is obtained by the following formula: Among them, the transmission power of the vehicle in the millimeter wave link is P i Indicated by; Noise power is represented by N o Indicates; millimeter wave bandwidth is expressed in W mm denoted by ; path loss is denoted by φ; vehicle v n and vehicle v i The distance between the vehicles under millimeter wave coverage is expressed as d n,i The main lobe array gain is expressed as Indicated by; shadow fading is represented by ρ α express.

4. According to the deep learning-based MEC resource scheduling method of claim 3, it is characterized in that: The specific steps to determine the communication model based on the distance between the client vehicle and the service vehicle are: When the transmission time T of task i k,i Less than or equal to vehicle v n In vehicle v i Duration of stay in the coverage area Then determine whether it is within the millimeter wave range; when the vehicle v n In vehicle v i If the distance between them is within the millimeter wave communication range, the transmission is carried out in millimeter wave mode, and the current transmission rate is set to If not, then when the vehicle v i and the next vehicle v j If the distance between them is within the millimeter wave communication range, the millimeter wave is used for relay transmission, and the current transmission rate is set to When the vehicle v i and the next vehicle v j If the distance between them is not within the millimeter wave communication range, the cellular mode is used for relay transmission, and the current transmission rate is set to 5. The MEC resource scheduling method based on deep learning according to claim 1, characterized in that: In step 2, different task models are constructed according to the resource processing mode: A client vehicle that performs N tasks is represented by the task set N = {1, ..., N}; M = {1, ..., M} defines the set of available service vehicles; each task i∈N is represented by D = s i ,ω i Indicates; where s i Indicates the size of the input data; ω i represents the number of CPU cycles required to complete the task; the client vehicle is represented as o; task i and The uninstall variable is represented by x k,i ∈{0,1}; where x oi =1 indicates that task i is a local calculation; x k,i =1 indicates that task i is unloaded to the corresponding service vehicle k.

6. A MEC resource scheduling method based on deep learning according to claim 5, characterized in that The time required for task execution of different task models is as follows: For local computation, use f n represents the local computing power of client vehicle o, that is, the number of CPU cycles per second; for client vehicle n, the time required to perform task i in the nearby area is: Then the processing time of all tasks executed locally is calculated as: For vehicle unloading calculation, the time T required to complete task i for a given service vehicle k is k,i It consists of three parts: transmission time Execution time and output time The time required for processing a task from client vehicle n to service vehicle k is:

7. A MEC resource scheduling method based on deep learning according to claim 6, characterized in that The time required for vehicle unloading calculation is as follows: Direct transmission time: The time required to upload task i from the client vehicle to its directly connected vehicle k is: Among them, S i Indicates the size of the input data; The speed selection can be determined by the distance between vehicles; The time for client vehicle n to connect to its directly connected vehicle k is given by Indicates that if the following equation is satisfied, the directly connected vehicle k can complete all assigned tasks: Indirect transmission time: Assuming that each indirect path contains u direct paths connecting client vehicle n to its service vehicle k, the connectivity of the path is expressed as: Select the path with the shortest connection time to upload the task; Task calculation time: The time required for service vehicle k to complete task i is: Feedback time: Since the size of the output data is significantly smaller than the input data, the feedback time Ignore.

8. The MEC resource scheduling method based on deep learning according to claim 1, characterized in that: The optimization function and constraints for minimizing the client vehicle utility in step 3 are as follows: The non-directly connected vehicle task offloading problem is defined as the client vehicle utility minimization problem; the problem is expressed as: Among them, C1 means that each task can only be calculated on a single vehicle; C2 means that each task can be processed locally or by offloading; C3 means that in order to successfully offload the task, the processing time of the directly connected service vehicle must be less than or equal to the connection time between them; C4 means that the sum of the execution time of all tasks assigned to the service vehicle must be less than or equal to the connection time between the vehicle and the client vehicle.

9. According to the deep learning-based MEC resource scheduling method of claim 1, in step 4, the optimization target is modeled as a Markov decision process, and the optimal resource scheduling strategy is obtained by using the routing forwarding strategy optimization algorithm: Modeling as a Markov decision process is as follows: State space: The state of the system in period τ is expressed as S(τ) = {D' τ ,M' τ ,r' h ,W τ ,pτ}, where D' τ is the set of distances between vehicles in the cycle; M' τ is the set of all scheduled data in the period; the total communication bandwidth of the period τ is given by W τ The transmission power of each vehicle is denoted by pτ; the availability of the communication link between the client and the serving vehicle is denoted by r' h express; Action space: The ability of the agent to decide whether each task should be unloaded and to which vehicle constitutes the action space of the system. For the unloading decision of a single task k, x k is a decision vector, each x i Indicates whether task k is unloaded to a specific vehicle i: x k =(x1,x2,…,x n ) where x is restricted to 0 or 1; i ranges from 1 to n, and n is the total number of available service vehicles; The decision set for all tasks unloading within the period τ is denoted by x τ express: x τ =(x1,x2,…,x K ) Among them, x τ is a decision vector, each x i Indicates whether a task is unloaded to a specific vehicle, x i The limit is 0 or 1; the range of i is 1 to K, where K is the total number of tasks that need to be completed in the current cycle; Reward function: within time τ, the discounted cumulative reward obtained by the agent from the initial state to the final state of the system is as follows: Among them, γ is the discount factor of the reward; τ end represents the time step of the system's terminal state; r(τ) is the immediate reward obtained by the agent at time step τ; The routing forwarding policy optimization algorithm is specifically as follows: Step 41: Input the initial policy network parameters πθ and the critic network parameters ω; Step 42: For each iteration i < N, where N is the number of iterations; perform the following steps; 1) Interact with the environment using the policy network parameters for a continuous T time steps, update the speeds and relative positions between vehicles according to the mobility model; collect state, action, and reward data for subsequent network training; 2) Calculation in, Indicates that in state s τ The state s is then estimated using the cumulative discounted rewards collected over n steps of sampling τ The true value of V πθ (s τ ,ω) is the state s provided by the critic network τ The estimated value of ; γ represents the discount factor used for rewards; the policy network parameters πθ are updated using the gradient policy method; 3) Minimize This reduces the error in evaluating the state value, where E τ [·] represents the expected value; and the critic network parameters ω are updated using gradient descent; Step 43: Repeat processes 1)-3), and when the maximum number of iterations is reached, output the optimized policy network parameters and critic network parameters, so as to obtain the optimal resource scheduling policy.

10. A MEC resource scheduling system based on deep learning, characterized in that: The system includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a deep learning-based MEC resource scheduling method according to any one of claims 1-9.

Images