A computing offloading method and system in many-to-many edge computing scenarios
By using Markov decision-making process modeling and strategy iteration method in many-to-many edge computing scenarios, the resource allocation of edge servers and mobile terminals is optimized, and the problem of minimizing the energy consumption of heterogeneous tasks in many-to-many scenarios is solved, and the problem of minimizing heterogeneous tasks in many-to-many scenarios is realized. The optimal unloading strategy for minimizing energy consumption.
Patent Information
- Application Number
- CN202111254759.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-27
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-10-27
AI Technical Summary
The existing technology lacks computational offloading methods for many-to-many edge computing scenarios, especially when there is a priority relationship between heterogeneous tasks. How to reasonably utilize the limited computing power of edge servers to minimize energy consumption has not been effective. solve.
Using Markov decision-making process modeling, an offload model is constructed through the strategy iteration method, combining the position constraints, computing power constraints and energy consumption optimization of edge servers and mobile terminals, and formulate the optimal offload strategy to achieve tasks in many-to-many scenarios. Minimize energy consumption.
In many-to-many edge computing scenarios, the optimal unloading strategy can be accurately formulated to ensure the minimization of energy consumption during task unloading, and has strong practicality and efficiency.
Smart Images

Figure CN113900739B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of heterogeneous task offloading, and in particular to a computing offloading method and system in a many-to-many edge computing scenario. Background Art
[0002] With the rapid development of science and technology, due to their limited resources and computing performance, smart terminals often face insufficient computing power when processing computing-intensive and time-sensitive tasks. For this reason, the edge computing model that uses edge servers to process and analyze data has emerged and complements the traditional cloud computing model. However, edge servers are often lightweight, and how to reasonably utilize the limited computing power of edge servers has become an important issue that edge computing needs to solve. Computation offloading, as a key technology of edge computing, can provide computing resources for resource-constrained devices to run computing-intensive applications under limited computing power, speed up computing speed, and save energy. It can also be said that computation offloading in edge computing is to offload the computing tasks of the terminal to the edge cloud, thereby solving the deficiencies of the terminal in terms of resource storage, computing performance, and energy.
[0003] In the existing technologies, only the different requirements for terminal computing power due to the data volume of heterogeneous tasks are considered, but the impact when there is a priority relationship between multiple tasks is ignored. For example, some tasks are preemptive, that is, while other tasks are being processed, such preemptive tasks can preempt other task resources and be processed first; and most of the existing research focuses on "many-to-one" or "one-to-many" scenarios. In addition to the above situations, it also includes "many-to-many" scenarios. The so-called "many-to-many" is defined as follows: information is transmitted between multiple mobile terminals and multiple edge servers, and there is no fixed information transmission correspondence between each edge server and the mobile terminal. Instead, the task offloading strategy is adjusted in real time according to the actual situation, and is not limited to the number of mobile terminals and edge servers. However, there are few technical solutions in the existing technology that study the "many-to-many" offloading scenario. Summary of the invention
[0004] In order to solve the problems existing in the above-mentioned prior art such as how to model the offloading scenarios in the "many-to-many" scenarios, the present invention provides a computing offloading method in a many-to-many edge computing scenario, which can perform task offloading modeling for heterogeneous tasks generated by terminals of different natures with the goal of minimizing energy consumption.
[0005] To achieve the above technical objectives, the present invention provides a method for offloading computing in a many-to-many edge computing scenario, comprising:
[0006] Obtain edge computing scenario data in a multi-edge server and multi-mobile terminal scenario, and build an edge computing scenario based on the edge computing scenario data;
[0007] Based on the edge computing scenario, an offloading model is constructed, where the offloading model is modeled based on the Markov decision process;
[0008] The unloading model is solved by the strategy iteration method to obtain the optimal unloading strategy.
[0009] Optionally, the edge computing scenario data includes an edge server set, a vehicle set, a vehicle task set and remaining computing capacity.
[0010] Optionally, the process of building an edge computing scenario includes:
[0011] Place location constraints on edge server sets and vehicle sets;
[0012] Performing remaining computing capacity constraints on the remaining computing capacity;
[0013] Calculate local processing energy consumption and offloading processing energy consumption based on the vehicle task set, and minimize energy consumption constraints based on local processing energy consumption and offloading processing energy consumption;
[0014] The location constraint results, computing power constraint results and minimum energy consumption constraint results are integrated to obtain the edge computing scenario.
[0015] Optionally, the offloading model includes a state space, an action space, a reward function, a strategy and a return;
[0016] The state space includes a number of different states, including the position of each vehicle, the remaining computing capacity of each vehicle, and the remaining computing capacity of each edge server;
[0017] The action space includes a number of different actions, including local processing and edge server offload processing actions;
[0018] The reward function is constructed based on local processing energy consumption and offload processing energy consumption;
[0019] The strategy is the probability of different actions under different states;
[0020] The reward is the total reward obtained by calculating the reward function.
[0021] Optionally, the process of solving the unloading model by the policy iteration method includes:
[0022] Based on the unloading model, a state value function is constructed, where the state value function is the mathematical expectation sum of the reward function values of each vehicle when different strategies are adopted under different states, where the reward function value is calculated by the reward function;
[0023] Initialize the state and strategy, calculate the state value function based on the initialized state and strategy, obtain the initial state value, and then iteratively update the state and strategy. During the update process, update the initial state value based on the iteratively updated state and strategy until the updated state value converges, and obtain the strategy under the converged state value, that is, the optimal unloading strategy.
[0024] In order to better achieve the above technical objectives, the present invention also provides a computing offloading system in a many-to-many edge computing scenario, including:
[0025] The first construction module is used to obtain edge computing scenario data in a multi-edge server and multi-mobile terminal scenario, and construct an edge computing scenario based on the edge computing scenario data;
[0026] The second building module is used to build an offloading model based on the edge computing scenario, where the offloading model is modeled based on a Markov decision process;
[0027] The solution module is used to solve the unloading model through the strategy iteration method to obtain the optimal unloading strategy.
[0028] Optionally, the edge computing scenario data in the first construction module includes an edge server set, a vehicle set, a vehicle task set and remaining computing capacity.
[0029] Optionally, the first building block includes:
[0030] The first constraint module is used to perform location constraints on the edge server set and the vehicle set;
[0031] The second constraint module is used to constrain the remaining computing capacity;
[0032] The third constraint module calculates the local processing energy consumption and the unloading processing energy consumption based on the vehicle task set, and performs a minimization energy consumption constraint based on the local processing energy consumption and the unloading processing energy consumption;
[0033] The edge computing scenario construction module is used to integrate the location constraint results, computing power constraint results and minimum energy consumption constraint results to obtain the edge computing scenario.
[0034] Optionally, the offloading model in the second building module includes a state space, an action space, a reward function, a strategy and a return;
[0035] The state space includes a number of different states, including the position of each vehicle, the remaining computing capacity of each vehicle, and the remaining computing capacity of each edge server;
[0036] The action space includes a number of different actions, including local processing and edge server offload processing actions;
[0037] The reward function is constructed based on local processing energy consumption and offload processing energy consumption;
[0038] The strategy is the probability of different actions under different states;
[0039] The reward is the total reward obtained by calculating the reward function.
[0040] Optionally, solve module package block:
[0041] The first processing module is used to construct a state value function based on the unloading model, wherein the state value function is the mathematical expectation sum of the reward function values of each vehicle when different strategies are adopted under different states, wherein the reward function value is calculated by the reward function;
[0042] The second processing module is used to initialize the state and strategy, calculate the state value function based on the initialized state and strategy, obtain the initial state value, and then iteratively update the state and strategy. During the updating process, the initial state value is updated based on the iteratively updated state and strategy until the updated state value converges, and the strategy under the converged state value is obtained, that is, the optimal unloading strategy.
[0043] The present invention has the following technical effects:
[0044] The method and system proposed in the present invention take into account the size of the task data volume and the priority relationship between tasks, take minimizing energy consumption as the goal, and offload heterogeneous tasks in a "many-to-many" scenario. It can accurately formulate relevant optimal offloading strategies and minimize energy consumption during task offloading, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0046] Figure 1 A flowchart of a problem-solving approach used when solving the optimal allocation strategy provided in the first embodiment of the present invention;
[0047] Figure 2 A schematic diagram of the distribution of edge servers and terminals in a multi-edge server and multi-terminal scenario provided in the first embodiment of the present invention;
[0048] Figure 3 This is a schematic diagram of a problem-solving system used to solve the optimal allocation strategy provided in the second embodiment of the present invention. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] In order to solve the problems existing in the prior art such as how to model the uninstallation scenario in the "many-to-many" scenario, the present invention provides the following solutions:
[0051] Embodiment 1
[0052] like Figure 1-2 The present invention provides a method for offloading computing tasks in edge computing scenarios with multiple mobile terminals and multiple edge servers. During the operation of multiple terminals, for the generated heterogeneous tasks, from the two perspectives of task priority and the size of the task itself, with the goal of minimizing terminal energy consumption, offloading modeling is performed, and finally, the established offloading model is solved using a greedy algorithm using a policy iteration method to obtain the optimal offloading strategy under each state.
[0053] Since the present invention is mainly aimed at unloading heterogeneous tasks generated by various AGV vehicles in industrial parks, the mobile terminals involved below are represented by various types of AGV vehicles. The specific contents are as follows:
[0054] Step 1: Obtain edge computing scenario data for multiple edge servers and multiple mobile terminals, and build edge computing scenarios based on the edge computing scenario data, that is, build edge computing scenarios for multiple edge servers and multiple mobile terminals.
[0055] In an industrial park covered by multiple edge servers, multiple AGV vehicles perform transportation tasks in the park. Assume that there are m edge servers forming a set EC = {EC1,EC2,EC3KEC m}, and evenly distributed in the industrial park; the number of AGV vehicles is n, forming a set AGV = {AGV1, AGV2, AGV3KAGV n};AGV i There may be multiple tasks at a time, AGVi The generated task set is T i ={T i1 ,T i2 ,T i3 K}, and T i ∈T={T1,T2KT n} represents the total set of tasks generated by all AGV vehicles.
[0056] There are two ways to process all tasks: local processing and offloading to an edge server for processing. Where ω represents the urgency of the task. According to the urgency, the tasks are divided into: urgent tasks and non-urgent tasks, which are marked as {0, 1} by the AGV vehicles respectively. For urgent tasks, considering that such tasks have high requirements for delay time, they are directly processed locally and can preempt the computing resources of non-urgent tasks; for non-urgent tasks, the modeling method of the Markov decision process is used to make decisions with the goal of minimizing the energy consumption of the AGV vehicle to determine the processing method of each task. Indicates the data size of the task. θ indicates the priority value of the task after quantification. The priority value is determined according to the urgency of the task, its sensitivity to delay, and its impact on the overall operation of the industrial park. During the scheduling process, high-priority tasks are processed first in the task cache queue.
[0057] During the operation of any AGV vehicle, according to the AGV i The location and remaining computing power of EC j The remaining computing capacity and the energy consumption of each edge server determine which edge server the task needs to be offloaded to. That is, the tasks generated by the AGV vehicle can be offloaded to any nearby edge server as needed, but the following conditions must be met:
[0058]
[0059] l represents the distance between the vehicle and the edge server where the task is finally unloaded; l0 represents the maximum signal coverage radius of all edge servers; Indicates the remaining computing capacity of each edge server; Indicates the remaining computing capacity of each AGV vehicle.
[0060] Since this patent is aimed at offloading computing tasks of AGV vehicles in industrial parks, AGV vehicles need to be charged every time they work for a period of time. In order to make full use of the battery energy, it is necessary to prioritize the energy of AGV vehicles during the calculation process, that is, the energy consumption of AGV vehicles is minimized; and since the edge server has a continuous energy source, the energy consumption of the edge server is not considered in the calculation. As for the energy consumption during the transmission process, there is no energy consumption during the transmission process during the local processing process. When the calculation results are returned to the AGV vehicle after being unloaded to the edge server, the energy consumption can be basically ignored due to the small amount of data of the calculation results. In summary, this article only considers the energy consumption of local calculations of AGV vehicles and the energy consumption of the unloading process to the edge server.
[0061] That is, energy consumption in the case of local processing
[0062]
[0063] C per P represents the number of CPU cycles required for local calculation of each unit of data by the AGV vehicle. per Represents the energy consumption of each cycle of the AGV vehicle's local calculation
[0064] ② Energy consumption when tasks are offloaded to edge servers
[0065]
[0066] P cs represents the transmission power, and v represents the data transmission rate
[0067] Step 2: Build an offloading model based on the edge computing scenario, that is, establish a "many-to-many" edge computing task offloading model
[0068] At present, the commonly used task offloading modeling methods in edge computing include convex optimization, game theory, Markov decision, Lyapunov optimization and other methods. Considering that this scenario is mainly aimed at the computing task offloading scenarios of various AGV vehicles in the industrial park, the scenario will continuously generate new tasks, the state is constantly changing, and the state at the next moment is independent of the historical state, but only related to the state at the previous moment. In step 2, the Markov decision process is used for modeling, and the advantages and disadvantages of each strategy are converted into the data size of the final return. Under the condition of sufficient computing resources, the optimal solution is given; conversely, an approximate solution to the optimal solution that is acceptable to people can also be given.
[0069] The unloading model is mainly composed of five tuples {S, A, R, π, r}, where S is the state space and the state S in this time period is t ={x,y,f AGV ,f EC}, including the location of each AGV vehicle, the remaining computing capacity of each AGV vehicle, and the remaining computing capacity of each edge server; A is the action space, which consists of the actions of all vehicles in this time period, A={a1,a2,a3...a n}, A represents the set of processing methods for all tasks generated by each AGV, including: local processing and offloading to an edge server for processing; R is the reward function, which is mainly related to the total energy consumption of each AGV vehicle in processing tasks; π is the strategy, which represents the probability of the AGV vehicle taking various actions in each state; r is the return, which represents the total reward obtained in the end. The reward function R is defined as follows:
[0070] ① For tasks processed locally (divided into two cases depending on whether they exceed the upper limit of the AGV vehicle's computing power)
[0071]
[0072] ②Tasks offloaded to edge cloud servers (divided into two cases depending on whether they exceed the computing capacity limit of the edge cloud server to which they are offloaded)
[0073]
[0074] Since the ultimate goal of the Markov decision process used in the present invention is to determine whether all tasks generated by each AGV vehicle are offloaded to the edge server for processing or processed locally, the final reward can be obtained to be equal to the sum of the reward functions of each AGV vehicle:
[0075]
[0076] In the process of task offloading, the remaining computing capacity of each AGV and each edge server needs to be considered. That is to say, the tasks accepted by each edge server or processed by the AGV itself cannot exceed its processing capacity limit.
[0077] Step 3: Solve the unloading model through the strategy iteration method to obtain the optimal unloading strategy, that is, determine the optimal unloading strategy
[0078] For the unloading model obtained above, the policy iteration method is used to solve the optimal strategy. In the Markov decision process method, each state corresponds to a state value function:
[0079] V π (s i )=E π [∑R i |s i ]
[0080] This function represents the state of s iTherefore, when the strategy adopted is π, the sum of the mathematical expectations of the reward function values R after all tasks generated by each AGV vehicle make decisions.
[0081] This method gives an initial state s0 and an initial strategy π0, obtains the V(s) value of s0 under the strategy, improves the strategy based on the V(s) value, and then continues to perform a new round of strategy improvement based on the new V(s) value until it converges. The greedy algorithm can efficiently obtain an approximate solution to the optimal strategy.
[0082] Under random initial state s0 and initial strategy π0:
[0083] Step 3-1: Randomly select an AGV vehicle and adjust only the task allocation strategy of the AGV vehicle. At the same time, it is necessary to ensure the computing power upper limit of each server and the computing power upper limit of the AGV vehicle itself, so as to maximize V(s) without changing the allocation strategy of other AGV vehicles;
[0084] Step 3-2: As in step 3-1, only the task allocation strategy of one AGV vehicle is randomly changed, but the allocation strategy of the AGV vehicle in step 1 remains unchanged;
[0085] Step 3-3: Same as step 3-2, only the task allocation strategy of one AGV vehicle is randomly changed, but the allocation strategy of the AGV vehicles selected in step 1 and step 2 remains unchanged;
[0086] And so on, after one cycle, we finally get a strategy that is better than the initial strategy π0.
[0087] Start the next cycle: By changing the AGV vehicle selected in step 1, we can get the best strategy in various situations, and compare and select the strategy with the largest V(s) value. At this time, the total energy consumption of the terminal is the smallest under this strategy.
[0088] For ease of understanding, the present invention provides the following example: In an industrial park covered by multiple edge servers, multiple AGV vehicles perform transportation tasks in the park. Assume that there are 3 edge servers forming a set EC = {EC1, EC2, EC3}, and they are evenly distributed in the industrial park; the number of AGV vehicles is 5, forming a set AGV = {AGV1, AGV2, AGV3, AGV4, AGV5}; AGV i There may be multiple tasks at a time, AGV i The generated task set is T i ={T i1 ,T i2 ,T i3 K}, and T i ∈T={T1,T2KT n} represents the total set of tasks generated by all AGV vehicles.
[0089] According to the above scenario, the Markov decision process is used to establish the unloading model, which is mainly composed of five tuples {S, A, R, π, r}, where S is the state space, A is the action space, R is the reward function, π is the strategy and r is the return.
[0090] After the task is generated, the state is s0, and the strategy π0 is randomly generated:
[0091] Step 3-1: Randomly select an AGV (for example, AGV1), and only adjust the sub-strategy of AGV1. At the same time, it is necessary to ensure that the tasks assigned to each server and vehicle will not reach the computing power limit of each server and the computing power limit of the AGV itself, and maximize V(s) without changing the sub-assignment strategies of all other AGVs (AGV2, AGV3, AGV4, AGV5);
[0092] Step 3-2: As in step 3-1, only the sub-strategy of one AGV vehicle (for example, AGV2) is randomly changed to maximize V(s), but the sub-strategy of AGV1 in step 1 remains consistent with the result of step 1 and the sub-strategies of AGV3, AGV4, and AGV5 remain unchanged;
[0093] Step 3-3: As in step 3-2, only the task allocation strategy of one AGV vehicle (for example, AGV3) is randomly changed to maximize V(s), but the sub-strategies of AGV1 and AGV2 selected in steps 1 and 2 remain consistent with those in step 2;
[0094] And so on, after one cycle, we finally get a strategy that is better than the initial strategy π0.
[0095] Start the next cycle: change the AGV vehicle selected in step 1 (for example, the initially selected vehicle is any one of AGV2, AGV3, AGV4, and AGV5), and you can get the best strategy in various situations. Compare the results and select the strategy with the largest V(s) value. At this time, the total energy consumption of the terminal is the smallest under this strategy.
[0096] Embodiment 2
[0097] In order to better achieve the above technical objectives, Figure 3 As shown, the present invention also provides a computing offloading system in a many-to-many edge computing scenario, including:
[0098] The first construction module is used to obtain edge computing scenario data in a multi-edge server and multi-mobile terminal scenario, and construct an edge computing scenario based on the edge computing scenario data;
[0099] The second building module is used to build an offloading model based on the edge computing scenario, where the offloading model is modeled based on a Markov decision process;
[0100] The solution module is used to solve the unloading model through the strategy iteration method to obtain the optimal unloading strategy.
[0101] Optionally, the edge computing scenario data in the first construction module includes an edge server set, a vehicle set, a vehicle task set and remaining computing capacity.
[0102] Optionally, the first building block includes:
[0103] The first constraint module is used to perform location constraints on the edge server set and the vehicle set;
[0104] The second constraint module is used to constrain the remaining computing capacity;
[0105] The third constraint module calculates the local processing energy consumption and the unloading processing energy consumption based on the vehicle task set, and performs a minimization energy consumption constraint based on the local processing energy consumption and the unloading processing energy consumption;
[0106] The edge computing scenario construction module is used to integrate the location constraint results, computing power constraint results and minimum energy consumption constraint results to obtain the edge computing scenario.
[0107] Optionally, the offloading model in the second building module includes a state space, an action space, a reward function, a strategy and a return;
[0108] The state space includes a number of different states, including the position of each vehicle, the remaining computing capacity of each vehicle, and the remaining computing capacity of each edge server;
[0109] The action space includes a number of different actions, including local processing and edge server offload processing actions;
[0110] The reward function is constructed based on local processing energy consumption and offload processing energy consumption;
[0111] The strategy is the probability of different actions under different states;
[0112] The reward is the total reward obtained by calculating the reward function.
[0113] Optionally, solve module package block:
[0114] The first processing module is used to construct a state value function based on the unloading model, wherein the state value function is the mathematical expectation sum of the reward function values of each vehicle when different strategies are adopted under different states, wherein the reward function value is calculated by the reward function;
[0115] The second processing module is used to initialize the state and strategy, calculate the state value function based on the initialized state and strategy, obtain the initial state value, and then iteratively update the state and strategy. During the update process, the initial state value is updated based on the iteratively updated state and strategy until the updated state value converges, and the strategy under the converged state value is obtained, that is, the optimal unloading strategy. In addition, the system described in the present invention corresponds to the method, and the specific explanation of the method will not be repeated here.
[0116] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for offloading computation in a many-to-many edge computing scenario, characterized in that: include: Obtain edge computing scenario data in a multi-edge server and multi-mobile terminal scenario, and build an edge computing scenario based on the edge computing scenario data; Based on the edge computing scenario, an offloading model is constructed, where the offloading model is modeled based on the Markov decision process; The unloading model is solved by the strategy iteration method to obtain the optimal unloading strategy; The offloading model includes state space, action space, reward function, strategy and return; The state space includes a number of different states, including the position of each vehicle, the remaining computing capacity of each vehicle, and the remaining computing capacity of each edge server; The action space includes a number of different actions, including local processing and edge server offload processing actions; The reward function is constructed based on local processing energy consumption and offload processing energy consumption; The strategy is the probability of different actions under different states; The reward is the total reward obtained by calculating the reward function; The process of solving the unloading model through the policy iteration method includes: Based on the unloading model, a state value function is constructed, where the state value function is the mathematical expectation sum of the reward function values of each vehicle when different strategies are adopted under different states, where the reward function value is calculated by the reward function; Initialize the state and strategy, calculate the state value function based on the initialized state and strategy, obtain the initial state value, and then iteratively update the state and strategy. During the update process, update the initial state value based on the iteratively updated state and strategy until the updated state value converges, and obtain the strategy under the converged state value, that is, the optimal unloading strategy; The process of obtaining edge computing scenario data in a multi-edge server and multi-mobile terminal scenario and constructing an edge computing scenario based on the edge computing scenario data includes: In an industrial park covered by several edge servers, several types of AGV vehicles perform transportation tasks in the park; suppose there are m edge servers forming a set EC = {EC1,EC2,EC3...EC m }, and evenly distributed in the industrial park; the number of AGV vehicles is n, forming a set AGV = {AGV1, AGV2, AGV3...AGV n };AGV i There are multiple tasks at a time, AGV i The generated task set is T i ={T i1 ,T i2 ,T i3 ...}, and T i ∈T={T1,T2...T n } represents the total set of tasks generated by all AGV vehicles; After all tasks are generated, there are two ways to process them: local processing and offloading to an edge server for processing; Where ω represents the urgency of the task. According to the urgency, the tasks are divided into urgent tasks and non-urgent tasks, which are marked as {0, 1} by the AGV vehicles respectively. Urgent tasks are directly processed locally and can preempt the computing resources of non-urgent tasks. For non-urgent tasks, the modeling method of the Markov decision process is used to make decisions with the goal of minimizing the energy consumption of the AGV vehicle to determine the processing method of each task. Indicates the data size of the task; θ indicates the priority value of the task after quantification. The priority value is determined according to the urgency of the task, the sensitivity to delay, and the impact on the overall operation of the industrial park. During the scheduling process, tasks with high priority are processed first in the task cache queue; Based on the edge computing scenario, an offloading model is constructed, that is, a "many-to-many" edge computing task offloading model is established. Specifically: The unloading model is mainly composed of five tuples {S, A, R, π, r}, where S is the state space and the state S in this time period is t ={x,y,f AGV ,f EC }, including the location of each AGV vehicle, the remaining computing capacity of each AGV vehicle, and the remaining computing capacity of each edge server; A is the action space, which consists of the actions of all vehicles in this time period, A={a1,a2,a3...a n }, A represents the set of processing methods of each AGV for all tasks generated by itself, including: local processing and offloading to an edge server for processing; R is the reward function, which is mainly related to the total energy consumption of each AGV vehicle in processing tasks; π is the strategy, which represents the probability of the AGV vehicle taking various actions in each state; r is the return, which represents the total reward obtained in the end; the reward function R is defined as follows: ① For tasks processed locally, there are two situations depending on whether they exceed the upper limit of the AGV vehicle's computing capacity. ②Tasks offloaded to edge cloud servers: Since the ultimate goal of using the Markov decision process is to determine whether all tasks generated by each AGV vehicle are offloaded to the edge server for processing or processed locally, the final reward is equal to the sum of the reward functions of each AGV vehicle: During the task offloading process, the remaining computing capacity of each AGV and each edge server should be considered. That is, the tasks accepted by each edge server or processed by the AGV itself cannot exceed its processing capacity limit. The unloading model is solved by the strategy iteration method to obtain the optimal unloading strategy, that is, to determine the optimal unloading strategy, specifically: For the unloading model obtained above, the strategy iteration method is used to solve the optimal strategy; Each state in the Markov decision process method corresponds to a state value function: V π (s i )=E π [ΣR i |s i ] This function represents the state of s i , so when the adopted strategy is π, the sum of the mathematical expectations of the reward function values R after all tasks generated by each AGV vehicle make decisions; Given an initial state s0 and an initial strategy π0, find the V(s) value of s0 under the strategy, improve the strategy according to the V(s) value, and then continue to make a new round of strategy improvements according to the new V(s) value until convergence, and use the greedy algorithm to get the approximate solution of the optimal strategy; Under the random initial state s0 and the initial strategy π0, the method specifically includes the following steps: Step 1: Randomly select an AGV vehicle and adjust only the task allocation strategy of the AGV vehicle, while ensuring the computing power upper limit of each server and the computing power upper limit of the AGV vehicle itself, so as to maximize V(s) without changing the allocation strategy of other AGV vehicles; Step 2: As in step 1, only the task allocation strategy of one AGV vehicle is randomly changed, but the allocation strategy of the AGV vehicle in step 1 remains unchanged; Step 3: As in step 2, only the task allocation strategy of one AGV vehicle is randomly changed, but the allocation strategy of the AGV vehicles selected in steps 1 and 2 remains unchanged; By analogy, after one cycle, we finally get a strategy that is better than the initial strategy π0; Start the next cycle: change the AGV vehicle selected in step 1, and then get the strategies for various situations. Compare and select the strategy with the largest V(s) value. At this time, the total energy consumption of the terminal is the smallest under this strategy.
2. A computing offloading system based on the computing offloading method in the many-to-many edge computing scenario according to claim 1, characterized in that: include: The first construction module is used to obtain edge computing scenario data in a multi-edge server and multi-mobile terminal scenario, and construct an edge computing scenario based on the edge computing scenario data; The second building module is used to build an offloading model based on the edge computing scenario, where the offloading model is modeled based on a Markov decision process; The solution module is used to solve the unloading model through the strategy iteration method to obtain the optimal unloading strategy; The offloading model in the second building block includes a state space, an action space, a reward function, a strategy and a return; The state space includes a number of different states, including the position of each vehicle, the remaining computing capacity of each vehicle, and the remaining computing capacity of each edge server; The action space includes a number of different actions, including local processing and edge server offload processing actions; The reward function is constructed based on local processing energy consumption and offload processing energy consumption; The strategy is the probability of different actions under different states; The reward is the total reward obtained by calculating the reward function; The solution modules include: The first processing module is used to construct a state value function based on the unloading model, wherein the state value function is the mathematical expectation sum of the reward function values of each vehicle when different strategies are adopted under different states, wherein the reward function value is calculated by the reward function; The second processing module is used to initialize the state and strategy, calculate the state value function based on the initialized state and strategy, obtain the initial state value, and then iteratively update the state and strategy. During the updating process, the initial state value is updated based on the iteratively updated state and strategy until the updated state value converges, and the strategy under the converged state value is obtained, that is, the optimal unloading strategy.
