A method and system for edge offloading resource allocation based on Q-learning

Abstract

The application discloses an edge unloading resource allocation method based on Q learning, and the method comprises the following steps: acquiring parameters of a current environment of a MEC system and user parameters in the system; adopting a preset allocation method to calculate an optimal resource allocation strategy which makes the overall energy consumption of user mobile devices minimum, as an optimal resource allocation scheme; the preset allocation method comprises the following steps: initializing an unloading strategy set Π, a state space S and an action space A ; ensuring the completion time of user computing tasks as a condition, and taking the minimum overall energy consumption of user mobile devices as a target, iteratively optimizing a pre-constructed Q function to obtain a state space and corresponding optimal actions which are synchronous with the completion time of user computing tasks; and calculating the optimal resource allocation strategy which makes the overall energy consumption of user mobile devices minimum according to the state space and the corresponding optimal actions. The application can minimize the overall energy consumption of user mobile devices on the basis of ensuring the completion time of user computing tasks.

Description

TECHNICAL FIELD

[0001] The application relates to an edge offloading resource allocation method and system based on Q learning, and belongs to the technical field of communication. BACKGROUND

[0002] In recent years, with the rapid development of smart mobile devices (SMD), numerous new applications such as face recognition, augmented reality and video streaming have emerged, and especially with the advent of 5G and the diffusion of smart mobile devices, the demand for mobile traffic will increase explosively. In fact, Cisco estimates that the number of global mobile devices will reach 11.5 billion in 2021, however, due to the limited processing capacity of existing base stations and mobile devices, the growth of mobile traffic will be bottlenecked, in addition, the energy consumption problem of mobile devices is one of the reasons that limits the speed of data transmission and processing. Research shows that the battery capacity of smart mobile devices has only increased by 29% in recent years, which is far slower than the demand for energy consumption of daily growing computing tasks.

[0003] In order to solve this problem, edge computing technology has attracted widespread attention, and the biggest difference between edge computing and remote cloud (MCC, Mobile Cloud Computing) is that edge cloud is closer to users, so users will not consume more energy on transmitting computing tasks. This move allows users to offload a large number of computing tasks to edge cloud servers for processing with less energy consumption, thereby reducing local energy consumption. In recent years, the academic and industrial communities have carried out research on MEC offloading problems. However, how to allocate edge offloading resources, there is no unified method to achieve it, which is a difficult problem faced by edge cloud. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art, and provide an edge offloading resource allocation method and system based on Q learning, which can minimize the overall energy consumption of user mobile devices while ensuring the completion time of user computing tasks.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] In a first aspect, the present application provides an edge offloading resource allocation method based on Q learning, which comprises the following steps:

[0007] Obtaining parameters of the current environment of the MEC system and user parameters in the system;

[0008] Based on the obtained parameters, an optimal resource allocation strategy that minimizes the overall energy consumption of user mobile devices is calculated by using a preset allocation method, and the optimal resource allocation strategy is taken as the optimal resource allocation scheme;

[0009] The preset allocation method is used to calculate a resource allocation strategy that minimizes the overall energy consumption of the user mobile device, and the resource allocation strategy comprises:

[0010] According to the obtained parameters, a task offloading and resource allocation joint optimization task is initialized, and a state space S and an action space A of the task are obtained.

[0011] The Q function is iteratively optimized to obtain a state space and an optimal action corresponding to the state space, which are synchronized with the time for the user to complete the computing task, under the condition of ensuring the completion time of the user computing task and the target of minimizing the overall energy consumption of the user mobile device.

[0012] The optimal resource allocation strategy is calculated according to the obtained state space and the optimal action corresponding to the state space.

[0013] In combination with the first aspect, further, the state space S of the task is initialized according to the obtained parameters, and the initialization comprises:

[0014] The state of each CPU is s=(i, e, ch, m), wherein i represents a current to-be-offloaded subtask, i.e., i∈V, V represents a set of to-be-offloaded subtasks; e represents a computing amount of the to-be-offloaded subtask, i.e., e∈ε, ε represents a set of computing amounts of offloaded subtasks; ch represents a channel selected by the user mobile device, i.e., ch∈Ch, Ch represents a set of channels of the mobile device; and m represents a current task processed by a local CPU0 or a MEC system CPU m , i.e., m∈{0,..., M}, M represents a total number of tasks; and the state space S is a set of states of all CPUs, i.e., S={s=(i, e, ch, m)}.

[0015] In combination with the first aspect, further, the action space A of the task is initialized according to the obtained parameters, and the initialization comprises:

[0016] An action a i,s ∈{0,..., M} is taken by each current to-be-offloaded subtask i∈V in a state s∈S, i.e., a m ∈{0,..., M} is taken by each current to-be-offloaded subtask i∈V in a state s∈S, i.e., a i,s ∈{0,..., M}}.

[0017] In combination with the first aspect, further, the Q function is pre-constructed, and the Q function is represented by the following formula:

[0018]

[0019] In formula (1), Rw(s,a) represents a reward function under each state action, and δ represents a decay factor of each iteration.

[0020] According to the first aspect, further, the iterative optimization of the pre-constructed Q function is represented by the following formula:

[0021] Q(i,a) = (1-p k )Q(i,a) + p k [C(i,a,j)-C k t(i,a,j)+ηmin b∈A(j) Q(j,b)] (2)

[0022] In formula (2), i represents a current subtask to be offloaded; j represents a next subtask to be offloaded after the subtask i is completed; t represents a transmission time from the current subtask i to be offloaded to the next subtask j to be offloaded; a and b represent actions taken in different states of the subtask; p k is a learning rate of Q learning, if the value of p k is equal to 0, the Q learning algorithm degenerates into a greedy strategy;

[0023] In formula (2), C k represents an average cost, and is represented by the following formula:

[0024]

[0025] In formula (3), r k represents a weighting factor, total_cost represents a total energy consumption required by all tasks, and total_time represents an overall task completion time, and is represented by the following formula:

[0026] total_cost = total_cost + C(i,a,j) (4)

[0027] total_time = total_time + t(i,a,j) (5)

[0028] The overall energy consumption of the user mobile device includes an offloading cost and a local cost, and is represented by the following formula:

[0029]

[0030] In formula (2), C0(S,a i,s ) represents the offloading cost, and C l (S,a i,s ) represents the local execution cost, wherein S represents a system state, a i,san action taken in the current state.

[0031] In combination with the first aspect, preferably, the objective of minimizing the overall energy consumption of the user mobile device is to minimize the energy consumption of the user computing device on the basis of ensuring the completion time of the user computing task, and a pre-constructed Q function can be obtained, including:

[0032] The initial value of the Q learning is set to an arbitrary value, the initial value of the total iteration number k is set to 1, and the average overhead in the kth iteration is C k , the initial value of C 1 is set to 0, the initial state is i, a series of optional actions in the state i is A(i), the learning rate p k and τ k are represented by the following formula:

[0033]

[0034]

[0035] The learning rate p k and τ k in the Q learning are functions of the iteration number k and are less than 1;

[0036] In the kth iteration, the optimal action is selected to minimize the overall energy consumption of the user mobile device, so that each state can select the state with the minimum energy consumption in the current state. The selection is a greedy selection, and the power for each greedy selection is set to 1-p(k). p(k) is represented by the following formula:

[0037]

[0038] In formula (9), G1 and G2 are variable parameters of the experience selection probability, and G2≥G1.

[0039] The non-greedy selection action is an experience selection, and the probability of the experience selection decays with the increase of the iteration number k. Let a be the action selection each time. If the greedy strategy is selected, let Otherwise Let j represent the subsequent state after the action selection of the state i, C(i, a, j) represent the data transmission overhead from the state i to the state j, and t(i, a, j) represent the data transmission time from the state i to the state j. A pre-constructed Q function is obtained.

[0040] In combination with the first aspect, further, the optimal resource allocation strategy is calculated when the overall energy consumption of the user mobile device is minimized according to the obtained state space and the corresponding optimal action, and is calculated by the following formula:

[0041]

[0042] In formula (7), Rw(s, a) represents a reward function under each state action, and δ represents a decay factor of each iteration.

[0043] In a second aspect, the present application provides a Q-learning-based edge offloading resource allocation system, comprising:

[0044] An acquisition module is configured to acquire parameters of a current environment of a MEC system and user parameters in the system.

[0045] An optimization calculation module is configured to calculate an optimal resource allocation strategy that minimizes the overall energy consumption of user mobile devices based on the acquired parameters and a preset allocation method, and take the optimal resource allocation strategy as an optimal resource allocation scheme.

[0046] The optimization calculation module comprises:

[0047] An initialization module is configured to initialize a set of offloading strategies Π, a state space S and an action space A of a joint optimization task of task offloading and resource allocation of a Q-learning algorithm according to the acquired parameters.

[0048] A first calculation module is configured to perform iterative optimization on a pre-constructed Q function to obtain a state space and an optimal action corresponding thereto that are synchronized with the time for a user to complete a computing task, with the condition that the time for the user to complete the computing task is ensured and the target that the overall energy consumption of the user mobile devices is minimized.

[0049] A second calculation module is configured to calculate an optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile devices based on the obtained state space and the optimal action corresponding thereto.

[0050] In a third aspect, the present application provides a Q-learning-based edge offloading resource allocation device, comprising a processor and a storage medium.

[0051] The storage medium is configured to store instructions.

[0052] The processor is configured to operate according to the instructions to perform the steps of the method of the first aspect.

[0053] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the method of the first aspect.

[0054] Compared with the prior art, the Q-learning-based edge offloading resource allocation method and system provided by the embodiments of the present application have the following beneficial effects:

[0055] This invention employs a pre-defined allocation method to calculate a resource allocation strategy that minimizes the overall energy consumption of the user's mobile device. This includes: initializing the task offloading and resource allocation joint optimization task offloading strategy set Π, state space S, and action space A of the Q-learning algorithm based on the acquired parameters; iteratively optimizing the pre-constructed Q-function with the condition of ensuring the completion time of the user's computing task and the objective of minimizing the overall energy consumption of the user's mobile device, obtaining a state space synchronized with the user's computing task completion time and its corresponding optimal action; and calculating the optimal resource allocation strategy that minimizes the overall energy consumption of the user's mobile device based on the obtained state space and its corresponding optimal action. The allocation method provided by this invention has lower complexity and faster convergence speed.

[0056] This invention acquires the parameters of the current environment of the MEC system and the user parameters in the system; based on the acquired parameters, it calculates the optimal resource allocation strategy that minimizes the overall energy consumption of the user's mobile device using a preset allocation method, and uses this as the optimal resource allocation scheme; this invention can reduce latency and improve user experience; this invention can reduce energy consumption and improve resource utilization efficiency; this invention can minimize the overall energy consumption of the user's mobile device while ensuring the completion time of the user's computing tasks. Attached Figure Description

[0057] Figure 1 This is a flowchart of an edge offloading resource allocation method based on Q-learning provided in Embodiment 1 of the present invention. Detailed Implementation

[0058] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0059] Example 1:

[0060] like Figure 1 As shown, this embodiment of the invention provides an edge offloading resource allocation method based on Q-learning, including:

[0061] Obtain the parameters of the current environment of the MEC system and the user parameters in the system;

[0062] Based on the obtained parameters, the optimal resource allocation strategy that minimizes the overall energy consumption of the user's mobile device is calculated using a preset allocation method, and this strategy is taken as the optimal resource allocation scheme.

[0063] Among them, the resource allocation strategies that minimize the overall energy consumption of user mobile devices, calculated using a preset allocation method, include:

[0064] Initialize the task offloading and resource allocation joint optimization task of the Q-learning algorithm with the obtained parameters, a set of offloading strategies Π, a state space S and an action space A.

[0065] With the condition of ensuring the user computing task completion time and the target of minimizing the overall energy consumption of the user mobile device, the pre-constructed Q function is iteratively optimized to obtain a state space synchronized with the user computing task completion time and the corresponding optimal action.

[0066] According to the obtained state space and the corresponding optimal action, the optimal resource allocation strategy is calculated when the overall energy consumption of the user mobile device is minimized.

[0067] The preset allocation method is used to calculate the resource allocation strategy that minimizes the overall energy consumption of the user mobile device, and the specific steps are as follows:

[0068] Step 1: Initialize the task offloading and resource allocation joint optimization task of the Q-learning algorithm with the obtained parameters, a set of offloading strategies Π, a state space S and an action space A.

[0069] Step 1.1: Initialize the state space S.

[0070] The state of each CPU is s=(i, e, ch, m), where i represents the current to-be-offloaded subtask, satisfying i∈V, V represents the set of offloaded subtasks; e represents the to-be-offloaded subtask computation size, satisfying e∈ε, ε represents the set of offloaded subtask computation sizes; ch represents the channel selected by the user mobile device, satisfying ch∈Ch, Ch represents the set of channels of the mobile device; m represents the current task handled by the local CPU0 or the MEC system handled by the CPU m , satisfying m∈{0,...M}, M represents the total number of tasks; the state space S is the set of states of all CPUs, represented as S={s=(i, e, ch, m)}.

[0071] Step 1.2: Initialize the action space A.

[0072] The action a i,s ∈{0,...,M} of each current to-be-offloaded subtask i∈V in the state s∈S is selected as a m , selecting the current task handled by the local CPU0 or the MEC system handled by the CPU i,s , then the action space A is the set of all actions, represented as A={a i,s ∈{0,...,M}}.

[0073] Step 2: Iteratively optimize the pre-constructed Q function to obtain the state space and the corresponding optimal action in synchronization with the user's completion time of the computing task, with the condition of ensuring the user's computing task completion time and the target of minimizing the overall energy consumption of the user's mobile device.

[0074] Step 2.1: Initialize the initial value of Q-learning as an arbitrary value, the initial value of the total number of iterations k as 1, the average overhead at the kth iteration as C k , the initial value of C 1 set as 0, the initial state as i, the series of optional actions at state i as A(i), the learning rate p k and τ k represented by the following formula:

[0075]

[0076]

[0077] The learning rate p k and τ k in Q-learning are functions of the number of iterations k and are less than 1.

[0078] Step 2.2: At the kth iteration, select the optimal action to minimize the overall energy consumption of the user's mobile device, and enable each state to select the state with the minimum energy consumption at the current state. This selection is a greedy selection, and the power for each greedy selection is set as 1-p(k), p(k) is represented by the following formula:

[0079]

[0080] In formula (3), G1 and G2 are variable parameters representing the probability of experience selection, and G2≥G1.

[0081] Step 2.3: The non-greedy selected action is an experience selection, and the probability of experience selection will decay with the increase of the number of iterations k. Let a be the action selection at each time, if the greedy strategy is selected, then Otherwise Let j represent the subsequent state after the action selection at state i, C(i,a,j) represent the data transmission overhead from state i to state j, t(i,a,j) represent the data transmission time from state i to state j, and the pre-constructed Q function is obtained by the following formula:

[0082]

[0083] In formula (4), Rw(s,a) represents the reward function under each state action, and δ represents the decay factor of each iteration.

[0084] Step 2.4: Iterative optimization is performed on the pre-constructed Q function, which is represented by the following equation:

[0085] Q(i, a) = (1 - p k )Q(i, a) + p k [C(i, a, j) - C k t(i, a, j) + ηmin b∈A(j) Q(j, b)] (5)

[0086] In equation (5), i represents the current subtask to be offloaded; j represents the next subtask to be offloaded after completing the subtask i; t represents the transmission time from the current subtask i to be offloaded to the next subtask j to be offloaded; a and b represent the actions taken in different states of the subtask; p k is the learning rate of Q-learning, and if p k is equal to 0, the Q-learning algorithm degenerates into a greedy strategy.

[0087] In equation (5), C k represents the average cost, which is represented by the following equation:

[0088]

[0089] In equation (6), r k represents the weighting factor, total_cost represents the total energy consumption required for all tasks, and total_time represents the overall task completion time, which is represented by the following equation:

[0090] total_cost = total_cost + C(i, a, j) (7)

[0091] total_time = total_time + t(i, a, j) (8)

[0092] The overall energy consumption of the user mobile device includes the offloading cost and the local cost, which is represented by the following equation:

[0093]

[0094] In equation (5), C0(S, a i,s ) represents the offloading cost, and C l (S, a i,s ) represents the local execution cost, where S represents the system state, a i,s represents the action taken in the current state.

[0095] Step 3: According to the obtained state space and its corresponding optimal action, the optimal resource allocation strategy is calculated when the overall energy consumption of the user mobile device is minimized, which is calculated by the following equation:

[0096]

[0097] In formula (10), Rw(s,a) represents a reward function under each state action, and δ represents a decay factor of each iteration.

[0098] In order to solve formula (10), the method first initializes a user offloading strategy set and an action space, because the initial value requirement of the reinforcement learning algorithm is not high, the strategy set can be set to a set of 0s first, then the subtask nodes of all tasks, the current action, the channel state and the subtask data size are obtained, and the CPU remaining computing capacity size of each subtask in the current MEC is obtained, then a state is selected in each action selectable state set to calculate the completion time and the required energy consumption of the computing task, and the Q value is calculated according to formula (10), until learning converges.

[0099] The application can minimize the overall energy consumption of the user mobile device on the basis of ensuring the completion time of the user computing task.

[0100] Embodiment two:

[0101] The embodiment of the application provides an edge offloading resource allocation system based on Q learning, which comprises:

[0102] The acquisition module is configured to acquire parameters of a current environment of the MEC system and user parameters in the system.

[0103] The optimization calculation module is configured to calculate an optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device based on the acquired parameters by using a preset allocation method, and take the optimal resource allocation strategy as an optimal resource allocation scheme.

[0104] The optimization calculation module comprises:

[0105] The initialization module is configured to initialize an offloading strategy set Π, a state space S and an action space A of a task offloading and resource allocation joint optimization task of the Q learning algorithm according to the acquired parameters.

[0106] The first calculation module is configured to perform iterative optimization on the pre-constructed Q function to obtain a state space and an optimal action corresponding to the state space that are synchronous with the completion time of the user computing task, with the condition of ensuring the completion time of the user computing task and the target of minimizing the overall energy consumption of the user mobile device.

[0107] The second calculation module is configured to calculate an optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device based on the obtained state space and the optimal action corresponding to the state space.

[0108] Embodiment three:

[0109] The embodiment of the present application provides an edge unloading resource allocation device based on Q learning, comprising a processor and a storage medium.

[0110] The storage medium is used for storing instructions.

[0111] The processor is used for operating according to the instructions to perform the steps of the method in the embodiment one.

[0112] Embodiment four:

[0113] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by the processor to realize the steps of the method in the embodiment one.

[0114] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0115] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks Figure 1 The function specified in one block or multiple blocks.

[0116] These computer program instructions can also be stored in a computer readable storage medium, which can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which realize the functions specified in the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks Figure 1 The function specified in one block or multiple blocks.

[0117] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the flow Figure 1 one flow or multiple flows and / or the functions specified in the flow

[0118] The above description is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A Q-learning-based edge offloading resource allocation method, characterized in that, The method comprises: obtaining parameters of a current environment of a MEC system and user parameters in the system; based on the obtained parameters, using a preset allocation method to calculate an optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device, and taking the optimal resource allocation strategy as the optimal resource allocation scheme; wherein the preset allocation method comprises: Initializing a task offloading and resource allocation joint optimization task of a Q-learning algorithm according to the obtained parameters , state space S , and action space A ; The state space of the task offloading and resource allocation joint optimization task is initialized according to the obtained parameters S , comprising: The state of each CPU is s =( i , e , ch , m ), wherein i represents a subtask to be unloaded currently, satisfying , V represents a set of subtasks to be unloaded currently; e represents a computation amount of the subtask to be unloaded, satisfying , and ε represents a set of computation amounts of the subtasks to be unloaded; ch represents a channel selected by a user mobile device, satisfying , Ch represents a set of channels in which the mobile device is located; m represents that a current task is processed by a local or a MEC system, the current task is processed by the local- CPU 0, and the current task is processed by the MEC system- CPU m , satisfying , M represents a total number of CPUs in the MEC system; a state space S is a set of states of all CPUs, and is represented as S ={ s=( i , e , ch , m )}. The action space of the task of jointly optimizing task offloading and resource allocation of the Q learning algorithm is initialized according to the obtained parameters A , comprising: each current subtask to be unloaded in the state the action taken is the action space A is the set of all actions, denoted as ; iteratively optimizing a pre-constructed Q function to obtain a state space synchronized with the user's completion time of the computing task and the corresponding optimal action, under the condition of ensuring the user's completion time of the computing task and the target of minimizing the overall energy consumption of the user mobile device; wherein the pre-constructed Q function is represented by the following formula: （1） In formula (1), Rw s a represents a reward function under each state action, and δ represents a decay factor for each iteration.​​ wherein the iterative optimization of the pre-constructed Q function is represented by the following formula: （2） In formula (2), i denotes the current subtask to be unloaded; j denotes the completion of the subtask i the next subtask to be unloaded; t denotes the transmission time from the current subtask to be unloaded i to the next subtask to be unloaded j ; a and b denotes the action taken in different states of the subtask; is the learning rate of Q-learning, and if the value of is equal to 0, the Q-learning algorithm degenerates into a greedy strategy; In formula (2), represents the average overhead, which is expressed by the following formula: （3） In formula (3), denotes a weighting factor, denotes the total energy consumption required for all tasks, denotes the overall task completion time, which is expressed by the following formula: （4） （5） The overall energy consumption of the user mobile device includes offloading overhead and local overhead, which is represented by the following formula: （6） In formula (2), represents the offload overhead, represents the local execution overhead, wherein represents the system state, represents the action taken in the current state; According to the obtained state space and the corresponding optimal action, the optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device is calculated; wherein the optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device is calculated by the following formula: （7） In formula (7), represents the reward function under each state action, and δ represents the decay factor of each iteration.

2. A Q-learning based edge offloading resource allocation system for performing the Q-learning based edge offloading resource allocation method of claim 1, characterized by The method comprises: an acquisition module for obtaining parameters of a current environment of a MEC system and user parameters in the system; an optimization calculation module for calculating an optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device based on the obtained parameters, using a preset allocation method, and taking the optimal resource allocation strategy as the optimal resource allocation scheme; wherein the optimization calculation module comprises: An initialization module is configured to initialize a set of offloading strategies for the joint optimization task of task offloading and resource allocation according to the obtained parameters , state space S , and action space A ; a first calculation module for iteratively optimizing a pre-constructed Q function to obtain a state space synchronized with the user's completion time of the computing task and the corresponding optimal action, under the condition of ensuring the user's completion time of the computing task and the target of minimizing the overall energy consumption of the user mobile device; a second calculation module for calculating an optimal resource allocation strategy that minimizes the overall energy consumption of the user mobile device according to the obtained state space and the corresponding optimal action.

3. An edge offloading resource allocation apparatus based on Q-learning, characterized by, comprises a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to perform the steps of the method of claim 1.

4. A computer readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the steps of the method of claim 1.

Images