A method and system for task offloading and resource allocation based on inter-user dependency in a mobile edge computing system

Abstract

This application discloses a method and system for task offloading and resource allocation based on inter-user dependencies in a mobile edge computing system. The method includes the following steps: S1, initializing state information; S2, in response to the completion of state information initialization, making an offloading decision and outputting the action selection result; S3, performing environmental interaction based on the output action selection result; S4, in response to the completion of environmental interaction, performing neural network parameter training; S5, determining whether the neural network parameter training has reached the preset maximum number of training rounds; if the maximum number of training rounds has been reached, executing S6, outputting all results. This application, through reasonable modeling of task dependencies between users, provides a solution that can fully utilize the computing power of deployed edge servers while satisfying the constraints of cross-terminal task relevance, thereby reducing task completion time and terminal energy consumption.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a method and system for task unloading and resource allocation based on inter-user dependencies in a mobile edge computing system. Background Technology

[0002] The rapid growth and functional diversification of emerging mobile applications such as intelligent transportation systems, autonomous driving, extended reality, and the Industrial Internet of Things have generated massive demands for mobile terminal access and computational data processing. To alleviate core network traffic load and reduce application response latency to improve user experience, Mobile Edge Computing (MEC) technology, which deploys computing and storage devices at the network edge, has been widely adopted. MEC technology breaks down the physical distance between end users and traditional cloud servers, providing ultra-low latency and energy-efficient services with ample resources. In MEC systems, performance-critical business requests from terminals are offloaded to edge servers, addressing the issues of insufficient computing power and limited energy of individual devices. To further improve the utilization of communication and computing resources and meet the performance requirements of computationally intensive and latency-sensitive tasks in terms of computing power, response time, and energy consumption, it is often necessary to jointly optimize computation offloading and resource management in MEC systems. Through efficient configuration of various communication and computing resources, excessive network overhead and traffic load can be avoided while further ensuring a better user experience. Methods related to task offloading and resource allocation in MEC systems primarily consider whether terminal computing tasks need to be offloaded and the amount of data to be offloaded. They also aim to efficiently utilize limited communication and computing resources through MEC node deployment and allocation, and network state awareness, reducing task completion time, system energy consumption, or the trade-offs between the two. However, the random variations in wireless channels between time slots and the mobility of terminals place MEC networks in a highly dynamic state. Traditional optimization algorithms cannot meet the rapid response requirements of MEC system services when terminals only have partial information. Deep reinforcement learning (DRL) addresses these challenges. As an artificial intelligence technique, it trains neural networks to learn the optimal action selection strategy for the current state through continuous interaction between the agent and the current environment. The double-deep Q-network (DDQN) algorithm in DRL can adapt to the discrete offloading action space and provide a solution for optimizing long-term rewards in dynamic scenarios. Some solutions idealistically assume that each computing task is independent, and therefore the amount of computation to be offloaded is also independent. However, in reality, mobile applications are often composed of a set of dependent tasks, ensuring the processing priority of specific tasks to achieve data transfer or functional continuity. In order to preserve the temporal correlation between tasks during the unloading process, the entire task is usually modeled as a directed acyclic graph (DAG), and fine-grained unloading and corresponding resource allocation are performed based on the analysis and processing of this graph.

[0003] Although some methods have been proposed to address the task offloading and resource allocation problems of MEC systems in dynamic scenarios and with different task dependencies, there are still some gaps in the existing technology that need to be filled:

[0004] 1) Currently, there is a lack of universality in problem modeling regarding application scenarios and task structures. In terms of application scenarios, dynamic scenarios need to be considered, such as time-varying wireless channel fading, continuous user movement, and random generation of service requests. In terms of task structures, under the trend of the Internet of Things, the task dependencies between users need to be considered, and the temporal correlation constraints of data across terminals need to be satisfied. Furthermore, in the aforementioned dynamic networks with highly spatiotemporally varying topologies, designing a solution that satisfies the low response latency of MEC systems becomes extremely challenging.

[0005] 2) The structured characteristics of the task and the system potential brought by the collaboration between edge servers and terminals have not been fully explored. Respecting and utilizing the parallelization approach when designing optimization problems and solutions can further improve key indicators such as task completion latency and terminal energy consumption.

[0006] Therefore, how to achieve task offloading and resource allocation in dynamic MEC networks while meeting the delayed response to service requests, in view of the spatiotemporal correlation caused by terminal mobility and task dependencies between users, has become an urgent problem to be solved in this field. Summary of the Invention

[0007] This application provides a task offloading and resource allocation method based on inter-user dependencies in a mobile edge computing system, comprising the following steps: S1, initializing state information; S2, in response to the completion of state information initialization, making an offloading decision and outputting the action selection result; S3, performing environmental interaction based on the output action selection result; S4, in response to the completion of environmental interaction, performing neural network parameter training; S5, determining whether the neural network parameter training has reached the preset maximum number of training rounds; if the maximum number of training rounds has not been reached, incrementing the training round count by 1 and returning to step S2; if the maximum number of training rounds has been reached, executing S6 and outputting all results.

[0008] As mentioned above, the initialization state information includes the design of algorithm elements, the division of the execution window for the initial periodic task request, the random initialization of the agent's neural network parameters, and the definition of each parameter of the system.

[0009] As described above, the unloading decision and the output action selection result include the following sub-steps: input the state information of the subtask observed by the agent in the current environment; select the unloading action based on the ε-greedy policy according to the input subtask state information; and output the action selection result.

[0010] As above, subtask vs The status information includes three parts: its own task information, information about its dependencies, and information about other subtasks within the same execution window.

[0011]

[0012] in, Subtask v s The distance between the corresponding terminal and the edge server in the current time slot.

[0013] This indicates the execution of other subtasks within window i. The influence weight of the current task on the l-th feature dimension.

[0014] As described above, the environmental interaction based on the output action selection result includes the following sub-steps: taking the output action selection result as input; based on the output action selection result, solving for the optimal transmission power for transmitting subtask data under a given channel through bisection-based transmission power control; based on the optimal transmission power, solving for the subchannel allocation result in the form of weighted bipartite graph matching using the KM algorithm; calculating the task processing overhead and returning the corresponding reward signal based on the action selection result, the subchannel allocation result, and the optimal transmission power; calculating the reward signal and observing the task status information in the next execution window.

[0015] A task offloading and resource allocation system based on inter-user dependencies in a mobile edge computing system includes an initialization unit, an offloading decision unit, an environment interaction unit, a neural network parameter training unit, a maximum training round count determination unit, and an output unit. The initialization unit initializes state information; the offloading decision unit makes offloading decisions and outputs the selected action; the environment interaction unit performs environment interaction based on the output action selection result; the neural network parameter training unit trains neural network parameters; the maximum training round count determination unit determines whether the neural network parameter training has reached a preset maximum training round count; and the output unit outputs all results if the maximum training round count has been reached.

[0016] As mentioned above, the initialization unit initializes state information, including the design of algorithm elements, the division of the execution window for the initial periodic task request, the random initialization of the agent's neural network parameters, and the definition of various system parameters.

[0017] As described above, the unloading decision unit makes an unloading decision and outputs the action selection result, which includes the following sub-steps: inputting the state information of the subtask observed by the agent in the current environment; selecting the unloading action based on the ε-greedy policy according to the input subtask state information; and outputting the action selection result.

[0018] As shown above, in the unloading decision unit, subtask v s The status information includes three parts: its own task information, information about its dependencies, and information about other subtasks within the same execution window.

[0019]

[0020] in, Subtask v s The distance between the corresponding terminal and the edge server in the current time slot. This indicates the execution of other subtasks within window i. The influence weight of the current task on the l-th feature dimension.

[0021] As described above, the environment interaction unit performs the following sub-steps based on the output action selection result: taking the output action selection result as input; based on the output action selection result, solving for the optimal transmission power for transmitting subtask data under a given channel through bisection-based transmission power control; based on the optimal transmission power, solving for the subchannel allocation result in the form of weighted bisection graph matching using the KM algorithm; calculating the task processing overhead and returning the corresponding reward signal based on the action selection result, the subchannel allocation result, and the optimal transmission power; and after calculating the reward signal, observing the task status information in the next execution window.

[0022] This application has the following beneficial effects:

[0023] (1) This application targets computationally intensive and latency-sensitive application scenarios involving multiple users, such as intelligent transportation and autonomous driving. By reasonably modeling the task dependency relationship between users, the solution provided can make full use of the computing power of the deployed edge servers while satisfying the constraints of cross-terminal task correlation, thereby reducing task completion time and terminal energy consumption.

[0024] (2) This application uses DDQN in deep reinforcement learning technology for task offloading decision, which can adapt to the needs of MEC system for rapid response in dynamic scenarios without prior knowledge, and integrates sub-channel allocation and transmit power control in the environmental interaction link, thereby further improving system performance through reasonable resource allocation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0026] Figure 1 This is a flowchart of a task unloading and resource allocation method based on inter-user dependencies in a mobile edge computing system provided in an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of the internal structure of a task unloading and resource allocation system oriented towards inter-user dependencies in a mobile edge computing system provided according to an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0029] This application proposes a task offloading and resource allocation method based on inter-user dependencies in a dynamic MEC system. By jointly optimizing task offloading, sub-channel allocation, and power control, the method reduces the long-term task completion latency and terminal energy consumption weighted overhead of the system.

[0030] Example 1

[0031] Assume there are N mobile terminals in the system, denoted as It consists of a fixed-location edge server (ES). Service requests are dynamically generated based on the current network status and user demand. Each service request corresponds to a task cycle, and the task G is published in cycle τ. τ It consists of a set of dependent subtasks. Simultaneously, each task cycle τ is divided into a series of execution windows (time slots) {1,2,…,i,…,I}, where different terminals and edge servers can process the subtasks in parallel within each time slot. The total system bandwidth B is divided into K orthogonal subchannels, denoted as... In addition, using The variable representing the unloading decision for the subtask; Sub-channel allocation variables representing sub-tasks; It refers to the controlled transmission power.

[0032] like Figure 1 As shown in this embodiment, a method for task offloading and resource allocation based on inter-user dependencies in a mobile edge computing system is provided, which specifically includes the following steps:

[0033] Step S110: Initialize state information.

[0034] The design of system initialization parameters includes the design of elements of the DDQN algorithm, the division of the execution window for the initial periodic task request, the random initialization of the agent's neural network parameters, and the definition of various system parameters.

[0035] Consider a system consisting of N mobile terminals. This is a MEC system consisting of a fixed-location edge server (ES). It is assumed that service requests are dynamically generated based on the current network state and user demand, with each service request corresponding to a task cycle, and the tasks G issued during cycle τ. τ It consists of a set of dependent subtasks, and corresponds to a set of task participants. That is, complete task G. τ It needs to be done by the system Multiple terminals participate together. Simultaneously, each task cycle τ is divided into a series of execution windows (slots) {1,2,…,i,…,I}, where different terminals and edge servers can process subtasks in parallel within each slot. Task G τ The structure can be represented as a directed acyclic graph describing G. τ ={V τ U τ}, node set V τ ={v 1 ,v 2 …,v S} represents the S subtasks that make up this task, where subtask v s Represented as v s = s ,c s ,χ s > Represents the amount of communication data, the amount of computation, and the corresponding terminal, respectively; the set of directed edges U τ It records the dependency relationship between the two subtasks. If u qs =1, v is called q For v s The direct preceding task, denoted as Accordingly, v s For v q The direct successor task is denoted as

[0036] The total system bandwidth is defined as B, and it is divided into K orthogonal sub-channels, denoted as... Then the bandwidth of each sub-channel is B0 = B / K. Bandwidth allocation variable. Indicates time slot i subtask v s Whether the k-th subchannel is used for data transmission, and the corresponding uplink rate is:

[0037] ​

[0038] in, It is time slot i that transmits subtask v on subchannel k. s The transmission power of the corresponding data This represents the corresponding terminal n (n = χ). s The current channel gain on sub-channel k, σ 2 This is the noise power. Additionally, the system maintains a fixed bandwidth of B. D Communication resources are used for end-to-end D2D data transmission.

[0039] Therefore, time slot i terminal n occupies subchannel k to transmit subtasks to the edge server. The corresponding data The communication latency and the energy consumed by the corresponding terminal n are expressed as follows:

[0040]

[0041]

[0042] Edge servers dynamically create and delete a corresponding number of virtual machines (VMs) to perform parallel computation on multiple tasks within the same time slot. Considering that I / O interference between concurrently existing VMs can affect task processing speed, in time slot i, subtask v... s The computation time required for processing by ES is expressed as:

[0043]

[0044] in This represents the number of subtasks offloaded to the server for processing in time slot i, d0 = c s / f ES f represents the unit computation time required to create only one virtual machine. ES ζ represents the computing power of the MEC server, and ζ represents the degradation factor caused by I / O interference between VMs. The corresponding energy consumption of the terminal is then expressed as:

[0045]

[0046] in It is the static power of terminal n.

[0047] Due to the computing power f of terminal n n Limited by availability, subtasks can be executed locally or offloaded to edge servers for processing. Subtask v s It is independent and indivisible, unloading indicator variable. Subtask v s Offload the data to the edge server for processing in time slot i; otherwise... The processing of a subtask requires three steps: input data acquisition, processor computation, and result return. The input data consists of two parts: data directly collected or cached by the device and data obtained from the results of previous tasks. Based on practical considerations, this invention assumes that the process of the edge server returning the computation result can be ignored, but the process of the terminal transmitting the computation result to the processor of its subsequent tasks cannot be ignored. Assume subtask v... s The calculation result is If the preceding subtask v q The calculation result is processed locally. Need to be sent to the current subtask v s The corresponding terminal j is either uploaded to ES. Since the calculation result of the preceding subtask is naturally part of the input data of the subsequent subtask, this invention merges the result return process of the preceding task into the processing process of the current task for unified analysis and representation.

[0048] Specifically, the processing overhead of each subtask consists of two parts: basic overhead and dependency overhead. Subtask v in slot i s If processed locally Its basic latency overhead can be expressed as the ratio of computational load to the computational power of terminal n, and the corresponding terminal energy consumption is as follows:

[0049]

[0050]

[0051] in This is the value of the switched capacitor.

[0052] If the current subtask v s Uninstalled The basic latency and energy consumption consist of their own communication and computation processes, and can be expressed in specific terms using -.

[0053]

[0054]

[0055] However, considering the cross-terminal dependencies between tasks, additional dependency overhead will be incurred due to the transmission of computation results. If a certain preceding subtask v q , Executed locally in a previous time slot i′ The corresponding dependency overhead is:

[0056]

[0057]

[0058] in k′ is represented as v q Calculation results Transmission allocated sub-channels, This represents the D2D communication rate between MTn and MTj.

[0059] In summary, based on the different situations described above, subtask v s = s ,c s ,χ s The completion latency and terminal power consumption are expressed as follows:

[0060]

[0061]

[0062] This embodiment provides a unified representation of the processing latency and energy consumption corresponding to the current subtask and its preceding unloading results.

[0063] The first two terms in the above expression (taking task processing latency as an example) This refers to the basic overhead of the current subtask unloading and local computation, assuming all preceding subtasks have been unloaded to Elasticsearch for execution. The latter item... Given a preceding subtask that can be computed locally, this refers to the handling method for the current subtask (whether it should be unloaded). And whether the current subtask and its predecessor belong to the same terminal. XOR operation The dependency overhead resulting from different combinations of cases (0 when n=j; 1 when n≠j).

[0064] Consider the parallel processing of subtasks within an execution window (slot). The processing cost of a task within slot i is represented as the weighted sum of the maximum completion latency of each subtask and the terminal's energy consumption:

[0065]

[0066] Where α D and α E The latency and energy consumption weighting coefficient can be flexibly set according to different application requirements. This represents the set of subtasks within execution window i. Therefore, the task G published during task cycle τ... τ The time-delay energy-weighted overhead can be represented by the sum of the overheads of each execution window:

[0067]

[0068] Therefore, in this embodiment, the system needs to meet certain conditions.​ and Under the constraints, minimize the weighted sum of long-term task completion latency and terminal energy consumption overhead of the dynamic MEC system.

[0069] After defining and determining the above formulas, initialize all state information parameters.

[0070] Step S120: In response to the initialization of the completion status information, make an unloading decision and output the action selection result.

[0071] This embodiment uses the DDQN algorithm to make unloading decisions for subtasks within each execution window, requiring specific design of its space, action space, and reward function. Step S120 specifically includes the following sub-steps:

[0072] Step S1201: Input the state information of the subtasks observed by the agent in the current environment.

[0073] Subtask v s The status information includes three parts: its own task information, information about its dependencies, and information about other subtasks within the same execution window.

[0074]

[0075] in, Subtask v s The distance between the corresponding terminal and the edge server in the current time slot. This indicates the execution of other subtasks within window i. The influence weight of the current task on the l-th feature dimension.

[0076] Action space and the unloading indicator variable defined above Consistency. That is... This indicates that the subtask is computed locally; This indicates that the subtask has been offloaded to the edge server for processing.

[0077] Step S1202: Select the unloading action based on the input subtask status information using an ε-greedy strategy.

[0078] The agent will use the aforementioned state information obtained from the environment during each round of decision-making. i s is used as the input to the neural network. Based on the ε-greedy policy, the network outputs the Q value corresponding to each action in that state, and the unloading action is selected based on the Q value.

[0079] The agent selects unloading actions for subtasks within the current execution window based on an ε-greedy policy and executes the corresponding actions in the current environment. The ε-greedy policy means selecting the unloading action with the largest Q-value output by the current neural network with a probability of 1-ε; otherwise, the action will be selected according to a random policy to achieve a balance between exploration and exploitation.

[0080] Step S1203: Output the action selection result.

[0081] Step S130: Select the result based on the output action and perform environmental interaction.

[0082] During the environmental interaction process, the first step is to allocate sub-channels and control transmit power based on the offloading decision result, which indicates communication needs and sub-tasks. Then, based on environmental information such as the current channel state, terminal location, and sub-task size, as well as the offloading decision and resource allocation results, the task processing overhead of this window is obtained. Finally, a value negatively correlated with the overhead is fed back to the agent in the form of a reward signal, serving as the environment's evaluation of the action selection.

[0083] Step S130 specifically includes the following sub-steps:

[0084] Step S1301: Select the output action result as input.

[0085] Step S1302: Based on the output action selection result, the optimal transmission power for transmitting subtask data under a given channel is solved by using a bisection-based transmission power control.

[0086] The implementation of the transmit power control submodule is based on a derived quasi-convex function of transmit power. By performing a bisection method under the constraint of the transmit power range, the optimal transmit power for transmitting the corresponding data of the subtask under any given channel can be obtained.

[0087] Step S1303: Based on the optimal transmit power, solve the sub-channel allocation result in the form of weighted bipartite graph matching using the KM algorithm.

[0088] The sub-channel allocation result is specifically the channel variable allocated to each task.

[0089] The above-mentioned transmit power control based on the bisection method and the KM algorithm solution are both common methods in the field in the prior art, and will not be elaborated here.

[0090] Step S1304: Calculate the task processing overhead and return the corresponding reward signal based on the action selection result, sub-channel allocation result and optimal transmit power.

[0091] For state action pairs within the same execution window A unified value related to the cost of this window is returned as the immediate reward signal r. i Specifically, it is formalized as the negative of the logarithm of the actual processing cost of all subtasks in this window, base 10, i.e.:

[0092] r i =-log 10 cost i (17)

[0093] To ensure consistency, a virtual entry task and a virtual exit task are added before and after each task graph, both with zero data and computational cost. The execution window is divided based on the maximum number of directed edges from the subtask node to the entry task. Although latency and energy consumption overhead are related to the combination of unloading decisions for dependent subtasks between adjacent time slots, according to the task processing model established in this embodiment, the overhead caused by dependencies within this execution window i can be independently determined based on the given unloading decisions of the corresponding preceding subtasks. Therefore, by sequentially minimizing the task processing overhead within each execution window, the long-term task completion latency and terminal energy consumption weighted overhead of the system can be optimized.

[0094] Step S1305: Calculate the completion reward signal and observe the task status information in the next execution window.

[0095] In response to the calculation of the completion reward signal, observe the task status information within the next execution window. Complete the environmental interaction.

[0096] Step S140: In response to completing the environmental interaction, perform neural network parameter training.

[0097] The neural network parameters will be updated after the training start condition is met. Step S140 specifically includes the following sub-steps:

[0098] Step S1401: [This step is incomplete and requires further context.] The status, actions, updated status, and rewards are stored as a sample in the experience pool.

[0099] Step S1402: Determine whether the conditions for starting training are met.

[0100] Specifically, if the conditions for starting training are met, training will begin if the number of samples in the experience pool is greater than a specified number, and step S1403 will be executed; otherwise, step S1406 will be executed to exit the neural network training.

[0101] Step S1403: Randomly select a batch of samples from the experience pool, perform gradient descent, and train and update the parameters of MainNet.

[0102] The parameter update process and update target of DDQN are as follows:

[0103]

[0104]

[0105] Where ω i and These are the MainNet and TargetNet parameters, commonly used in quasi-static target network techniques in deep reinforcement learning algorithms based on DQN, used to ensure training stability. η is the learning rate for parameter updates, and γ is a discount factor representing the agent's emphasis on future long-term rewards.

[0106] Step S1404: Determine whether the TargetNet parameters need to be updated.

[0107] The specific condition for whether an update is needed is as follows: if the current number of training rounds is an integer multiple of the preset TargetNet update frequency, then an update is needed; otherwise, it is not needed.

[0108] If an update is needed, proceed to step S1405; otherwise, proceed to step S1406 to exit neural network training.

[0109] Step S1405: Replace the TargetNet parameters with the parameters of the current MainNet.

[0110] Step S1406: Exit neural network training.

[0111] Step S150: Determine whether the neural network parameter training has reached the preset maximum number of training rounds.

[0112] If the preset maximum number of training rounds has not been reached, increment the training round count by 1 and return to step 120. If the preset maximum number of training rounds has been reached, proceed to step S160.

[0113] Step 160: Output all results.

[0114] The process involves determining whether the system has reached the termination condition. If the maximum number of training rounds has been met, the training is terminated. Under the guidance of the current DDQN, the system selects the action to unload the output task, outputs the results of task unloading, sub-channel allocation and transmit power control, as well as the corresponding system overhead.

[0115] Example 2

[0116] like Figure 2As shown, this application provides a task unloading and resource allocation system for user-dependent relationships in a mobile edge computing system, wherein the system specifically includes: an initialization unit 210, an unloading decision unit 220, an environment interaction unit 230, a neural network parameter training unit 240, a maximum training round number judgment unit 250, and an output unit 260.

[0117] The initialization unit 210 is used to design the system initialization parameters, including the design of the elements of the DDQN algorithm, the division of the execution window for the initial periodic task request, the random initialization of the agent's neural network parameters, and the definition of various system parameters.

[0118] The total system bandwidth is defined as B, and it is divided into K orthogonal sub-channels, denoted as... Then the bandwidth of each sub-channel is B0 = B / K. Bandwidth allocation variable. Indicates time slot i subtask v s Whether the k-th subchannel is used for data transmission, and the corresponding uplink rate is:

[0119]

[0120] in, It is time slot i that transmits subtask v on subchannel k. s The transmission power of the corresponding data This represents the corresponding terminal n (n = χ). s The current channel gain on sub-channel k, σ 2 This is the noise power. Additionally, the system maintains a fixed bandwidth of B. D Communication resources are used for end-to-end D2D data transmission.

[0121] Therefore, time slot i terminal n occupies subchannel k to transmit subtasks to the edge server. The corresponding data The communication latency and the energy consumed by the corresponding terminal n are expressed as follows:

[0122]

[0123]

[0124] Edge servers dynamically create and delete a corresponding number of virtual machines (VMs) to perform parallel computation on multiple tasks within the same time slot. Considering that I / O interference between concurrently existing VMs can affect task processing speed, in time slot i, subtask v... s The computation time required for processing by ES is expressed as:

[0125]

[0126] in This represents the number of subtasks offloaded to the server for processing in time slot i, d0 = c s / f ES f represents the unit computation time required to create only one virtual machine. ES ζ represents the computing power of the MEC server, and ζ represents the degradation factor caused by I / O interference between VMs. The corresponding energy consumption of the terminal is then expressed as:

[0127]

[0128] in It is the static power of terminal n.

[0129] Due to the computing power f of terminal n n Limited by availability, subtasks can be executed locally or offloaded to edge servers for processing. Subtask v s It is independent and indivisible, unloading indicator variable. Subtask v s Offload the data to the edge server for processing in time slot i; otherwise... The processing of a subtask requires three steps: input data acquisition, processor computation, and result return. The input data consists of two parts: data directly collected or cached by the device and data obtained from the results of previous tasks. Based on practical considerations, this invention assumes that the process of the edge server returning the computation result can be ignored, but the process of the terminal transmitting the computation result to the processor of its subsequent tasks cannot be ignored. Assume subtask v... s The calculation result is If the preceding subtask v q The calculation result is processed locally. Need to be sent to the current subtask v s The corresponding terminal j is either uploaded to ES. Since the calculation result of the preceding subtask is naturally part of the input data of the subsequent subtask, this invention merges the result return process of the preceding task into the processing process of the current task for unified analysis and representation.

[0130] Specifically, the processing overhead of each subtask consists of two parts: basic overhead and dependency overhead. Subtask v in slot i s If processed locally Its basic latency overhead can be expressed as the ratio of computational load to the computational power of terminal n, and the corresponding terminal energy consumption is as follows:

[0131]

[0132]

[0133] in This is the value of the switched capacitor.

[0134] If the current subtask v s Uninstalled The basic latency and energy consumption consist of their own communication and computation processes, and can be expressed in specific terms using -.

[0135]

[0136]

[0137] However, considering the cross-terminal dependencies between tasks, additional dependency overhead will be incurred due to the transmission of computation results. If a certain preceding subtask v q , Executed locally in a previous time slot i′ The corresponding dependency overhead is:

[0138]

[0139]

[0140] in k′ is represented as v q Calculation results Transmission allocated sub-channels, This represents the D2D communication rate between MTn and MTj.

[0141] In summary, based on the different situations described above, subtask v s = s ,c s ,χ s The completion latency and terminal power consumption are expressed as follows:

[0142]

[0143]

[0144] This embodiment provides a unified representation of the processing latency and energy consumption corresponding to the current subtask and its preceding unloading results.

[0145] The first two terms in the above expression (taking task processing latency as an example) This refers to the basic overhead of the current subtask unloading and local computation, assuming all preceding subtasks have been unloaded to Elasticsearch for execution. The latter item... Given a preceding subtask that can be computed locally, this refers to the handling method for the current subtask (whether it should be unloaded). And whether the current subtask and its predecessor belong to the same terminal. ​XOR operation The dependency overhead resulting from different combinations of cases (0 when n=j; 1 when n≠j).

[0146] Consider the parallel processing of subtasks within an execution window (slot). The processing cost of a task within slot i is represented as the weighted sum of the maximum completion latency of each subtask and the terminal's energy consumption:

[0147]

[0148] Where α D and α E The latency and energy consumption weighting coefficient can be flexibly set according to different application requirements. This represents the set of subtasks within execution window i. Therefore, the task G published during task cycle τ... τ The time-delay energy-weighted overhead can be represented by the sum of the overheads of each execution window:

[0149]

[0150] Therefore, in this embodiment, the system needs to meet certain conditions. and Under the constraints, minimize the weighted sum of long-term task completion latency and terminal energy consumption overhead of the dynamic MEC system.

[0151] After defining and determining the above formulas, initialize all state information parameters.

[0152] The unloading decision unit 220 is used to make unloading decisions and output the selection result of the action.

[0153] This embodiment employs the DDQN algorithm to make unloading decisions for subtasks within each execution window, requiring specific design of its space, action space, and reward function. The unloading decision unit 220 specifically includes the following sub-modules: a status information input module, an unloading action selection module, and an action output module.

[0154] The status information input module is used to input the status information of the subtasks observed by the agent in the current environment.

[0155] Subtask v s The status information includes three parts: its own task information, information about its dependencies, and information about other subtasks within the same execution window.

[0156]

[0157] in, Subtask v sThe distance between the corresponding terminal and the edge server in the current time slot. This indicates the execution of other subtasks within window i. The influence weight of the current task on the l-th feature dimension.

[0158] Action space and the unloading indicator variable defined above Consistency. That is... This indicates that the subtask is computed locally; This indicates that the subtask has been offloaded to the edge server for processing.

[0159] The uninstallation action selection module is used to select uninstallation actions based on an ε-greedy strategy.

[0160] The agent will acquire the aforementioned state information from the environment during each round of decision-making. As input to the neural network, based on the ε-greedy strategy, the network outputs the Q value corresponding to each action in that state, and selects the unloading action based on the Q value.

[0161] The agent selects unloading actions for subtasks within the current execution window based on an ε-greedy policy and executes the corresponding actions in the current environment. The ε-greedy policy means selecting the unloading action with the largest Q-value output by the current neural network with a probability of 1-ε; otherwise, the action will be selected according to a random policy to achieve a balance between exploration and exploitation.

[0162] The action output module is used to output the action selection results.

[0163] The environment interaction unit 230 is used to perform environment interaction based on the selected result of the output action.

[0164] During the environmental interaction process, the first step is to allocate sub-channels and control transmit power based on the offloading decision result, which indicates communication needs and sub-tasks. Then, based on environmental information such as the current channel state, terminal location, and sub-task size, as well as the offloading decision and resource allocation results, the task processing overhead of this window is obtained. Finally, a value negatively correlated with the overhead is fed back to the agent in the form of a reward signal, serving as the environment's evaluation of the action selection.

[0165] The environment interaction unit 230 specifically includes the following sub-modules: input module, optimal transmit power solution module, sub-channel allocation result solution module, reward signal acquisition module, and observation module.

[0166] The input module is used to take the output action selection result as input.

[0167] The optimal transmit power solution module is used to solve for the optimal transmit power for transmitting subtask data under a given channel by using transmit power control based on the output action selection result and the bisection method.

[0168] The implementation of the transmit power control submodule is based on a derived quasi-convex function of transmit power. By performing a bisection method under the constraint of the transmit power range, the optimal transmit power for transmitting the corresponding data of the subtask under any given channel can be obtained.

[0169] The sub-channel allocation result solving module is used to solve the sub-channel allocation result in the form of weighted bipartite graph matching based on the optimal transmit power using the KM algorithm.

[0170] The sub-channel allocation result is specifically the channel variable allocated to each task.

[0171] The above-mentioned transmit power control based on the bisection method and the KM algorithm solution are both common methods in the field in the prior art, and will not be elaborated here.

[0172] The reward signal acquisition module is used to calculate the task processing overhead and return the corresponding reward signal based on the action selection result, sub-channel allocation result and optimal transmission power.

[0173] For state action pairs within the same execution window A unified value related to the cost of this window is returned as the immediate reward signal r. i Specifically, it is formalized as the negative of the logarithm of the actual processing cost of all subtasks in this window, base 10, i.e.:

[0174] r i =-log 10 cost i (17)

[0175] To ensure consistency, a virtual entry task and a virtual exit task are added before and after each task graph, both with zero data and computational cost. The execution window is divided based on the maximum number of directed edges from the subtask node to the entry task. Although latency and energy consumption overhead are related to the combination of unloading decisions for dependent subtasks between adjacent time slots, according to the task processing model established in this embodiment, the overhead caused by dependencies within this execution window i can be independently determined based on the given unloading decisions of the corresponding preceding subtasks. Therefore, by sequentially minimizing the task processing overhead within each execution window, the long-term task completion latency and terminal energy consumption weighted overhead of the system can be optimized.

[0176] The observation module is used to calculate the completion reward signal and observe the task status information in the next execution window.

[0177] In response to the calculation of the completion reward signal, observe the task status information within the next execution window. Complete the environmental interaction.

[0178] The neural network parameter training unit 240 is used for training neural network parameters.

[0179] The neural network parameters will be updated after the training start condition is met. The neural network parameter training unit 240 specifically includes the following sub-modules: storage module, training condition judgment module, parameter update module, parameter update judgment module, replacement module, and exit module.

[0180] The storage module is used to store the current round The status, actions, updated status, and rewards are stored as a sample in the experience pool.

[0181] The training condition judgment module is used to determine whether the conditions for starting training are met.

[0182] Specifically, if the number of samples in the experience pool is greater than a specified number, then training begins and the parameter update module is executed; otherwise, the exit module is entered.

[0183] The parameter update module is used to randomly select a batch of samples from the experience pool, perform gradient descent, and train and update the parameters of MainNet.

[0184] The parameter update process and update target of DDQN are as follows:

[0185]

[0186]

[0187] Where ω i and These are the MainNet and TargetNet parameters, commonly used in quasi-static target network techniques in deep reinforcement learning algorithms based on DQN, used to ensure training stability. η is the learning rate for parameter updates, and γ is a discount factor representing the agent's emphasis on future long-term rewards.

[0188] The parameter update judgment module is used to determine whether the TargetNet parameters need to be updated.

[0189] If an update is needed, proceed to the Replace module; otherwise, proceed to the Exit module.

[0190] The replacement module is used to replace the TargetNet parameters with the parameters of the current MainNet.

[0191] The exit module is used to exit neural network training.

[0192] The maximum training rounds determination unit 250 determines whether the neural network parameter training has reached the preset maximum training rounds.

[0193] If the preset maximum number of training rounds has not been reached, increment the training round count by 1 and return to step 120. If the preset maximum number of training rounds has been reached, execute output unit 260.

[0194] Output unit 260 is used to output all results.

[0195] The process involves determining whether the system has reached the termination condition. If the maximum number of training rounds has been met, the training is terminated. Under the guidance of the current DDQN, the system selects the action to unload the output task, outputs the results of task unloading, sub-channel allocation and transmit power control, as well as the corresponding system overhead.

[0196] This application has the following beneficial effects:

[0197] (1) This application targets computationally intensive and latency-sensitive application scenarios involving multiple users, such as intelligent transportation and autonomous driving. By reasonably modeling the task dependency relationship between users, the solution provided can make full use of the computing power of the deployed edge servers while satisfying the constraints of cross-terminal task correlation, thereby reducing task completion time and terminal energy consumption.

[0198] (2) This application uses DDQN in deep reinforcement learning technology for task offloading decision, which can adapt to the needs of MEC system for rapid response in dynamic scenarios without prior knowledge, and integrates sub-channel allocation and transmit power control in the environmental interaction link, thereby further improving system performance through reasonable resource allocation.

[0199] Although the examples referenced in this application are described for illustrative purposes only and not for limiting the scope of this application, changes, additions and / or deletions to the implementation may be made without departing from the scope of this application.

[0200] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for task offloading and resource allocation based on inter-user dependencies in a mobile edge computing system, characterized in that, Mobile edge computing system is composed of mobile terminals It consists of a fixed-location edge server. Service requests are dynamically generated based on the current network status and user needs. Each service request corresponds to a task cycle, and the cycle is... Released tasks It consists of a set of dependent subtasks, and corresponds to a set of task participants. Each task cycle The internal division is divided into a series of execution time slots. In each time slot, different terminals and edge servers process subtasks in parallel. The structure is represented by a directed acyclic graph. Node set This represents the S subtasks that make up this task, where each subtask... Represented as , These represent the amount of communication data, the amount of computation, and their corresponding terminals, respectively; the set of directed edges. The dependency relationship between the two subtasks is recorded. ,say for The direct preceding task, denoted as , for The direct successor task; the total system bandwidth B is divided into An orthogonal subchannel, denoted as bandwidth of each sub-channel ;use This represents the unloading decision variable for the subtask and the bandwidth allocation variable. Indicates the subtask of time slot i Whether the k-th subchannel is used for data transmission, and the corresponding uplink rate is: , It refers to the controlled transmission power; the method includes the following steps: Step S110: Initialize state information, including the design of DDQN algorithm elements, the allocation of execution time slots for initial periodic task requests, the random initialization of agent neural network parameters, the definition of system parameters, and subtasks. The completion latency and terminal power consumption are expressed as follows: ； ； in, This indicates the number of subtasks offloaded to the server for processing in time slot i. This indicates the computing power of the MEC server. This represents the degradation factor caused by I / O interference between VMs. This represents the computing power of terminal n. Indicates the preceding subtask In a previous time slot i′, the unloading decision variable, XOR operation The value is 0 when n = j, and 1 when n ≠ j. Represented as The calculation results It is time slot i that transmits subtasks on subchannel k. The corresponding data transmission power, The value of the switched capacitor. This represents the static power of terminal n; The task processing overhead within time slot i is represented as the weighted sum of the maximum completion latency of each subtask and the terminal power consumption: ， in and The latency and energy consumption weighting coefficient can be flexibly set according to different application requirements. This represents the set of subtasks within execution slot i; Task cycle Released tasks The latency-weighted overhead is represented by the sum of the overheads of each execution slot: ， The system under the conditions and Under the constraints, minimize the long-term task completion latency and terminal energy consumption weighted sum overhead of the dynamic MEC system; Step S120: Make an unloading decision and output the action selection result; the action space is... This indicates that the subtask is computed locally; This indicates that the subtask has been offloaded to the edge server for processing; subtask The status information includes three parts: its own task information, information about its dependencies, and information about other subtasks within the same execution window. ； in Subtasks The distance between the corresponding terminal and the edge server in the current time slot. This indicates the execution of other subtasks within window i. In the The influence weight of feature dimensions on the current task; Step S130: Based on the output action selection result, perform environmental interaction; based on the output action selection result, solve for the optimal transmission power for transmitting subtask data under a given channel using a bisection-based transmission power control method; based on the optimal transmission power, solve for the subchannel allocation result in the form of a weighted bipartite graph matching using the KM algorithm; based on the action selection result, subchannel allocation result, and optimal transmission power, calculate the task processing overhead and return the corresponding reward signal; in response to the completion of the reward signal calculation, observe the task status information in the next execution window to complete the environmental interaction; Step S140: In response to completing the environmental interaction, perform neural network parameter training; Step S150: Determine whether the neural network parameter training has reached the preset maximum number of training rounds; If the maximum number of training rounds has not been reached, increment the number of training rounds by 1 and return to step S120; If the maximum number of training rounds is reached, proceed to step S160; Step S160: Output all results.

2. The task offloading and resource allocation method for user-dependent relationships in a mobile edge computing system as described in claim 1, characterized in that, The unloading decision process and the output action selection result include the following sub-steps: Input the state information of the subtasks observed by the agent in the current environment; Based on the state information of the input subtask, - A greedy strategy is used to select the unloading action; Output the action selection result.

3. A task offloading and resource allocation system based on inter-user dependencies in a mobile edge computing system, characterized in that, Mobile edge computing system is composed of mobile terminals It consists of a fixed-location edge server. Service requests are dynamically generated based on the current network status and user needs. Each service request corresponds to a task cycle, and the cycle is... Released tasks It consists of a set of dependent subtasks, and corresponds to a set of task participants. Each task cycle The internal division is divided into a series of execution time slots. In each time slot, different terminals and edge servers process subtasks in parallel. The structure is represented by a directed acyclic graph. Node set This represents the S subtasks that make up this task, where each subtask... Represented as , These represent the amount of communication data, the amount of computation, and their corresponding terminals, respectively; the set of directed edges. The dependency relationship between the two subtasks is recorded. ,say for The direct preceding task, denoted as , for The direct successor task; the total system bandwidth B is divided into An orthogonal subchannel, denoted as bandwidth of each sub-channel ;use This represents the unloading decision variable for the subtask and the bandwidth allocation variable. Indicates the subtask of time slot i Whether the k-th subchannel is used for data transmission, and the corresponding uplink rate is: , It refers to the controlled transmission power; The system includes an initialization unit, an unloading decision unit, an environment interaction unit, a neural network parameter training unit, a maximum training rounds determination unit, and an output unit. The initialization unit is used to initialize state information; initializing state information includes the design of elements of the DDQN algorithm, the division of execution time slots for initial periodic task requests, the random initialization of agent neural network parameters, the definition of various system parameters, and subtasks. The completion latency and terminal power consumption are expressed as follows: ； ； in, This indicates the number of subtasks offloaded to the server for processing in time slot i. This indicates the computing power of the MEC server. This represents the degradation factor caused by I / O interference between VMs. This represents the computing power of terminal n. Indicates the preceding subtask In a previous time slot i′, the unloading decision variable, XOR operation The value is 0 when n = j, and 1 when n ≠ j. Represented as The calculation results It is time slot i that transmits subtasks on subchannel k. The corresponding data transmission power, The value of the switched capacitor. This represents the static power of terminal n; The task processing overhead within time slot i is represented as the weighted sum of the maximum completion latency of each subtask and the terminal power consumption: ， in and The latency and energy consumption weighting coefficient can be flexibly set according to different application requirements. This represents the set of subtasks within execution slot i; Task cycle Released tasks The latency-weighted overhead is represented by the sum of the overheads of each execution slot: ， The system under the conditions and Under the constraints, minimize the long-term task completion latency and terminal energy consumption weighted sum overhead of the dynamic MEC system; The unloading decision unit is used to make unloading decisions and outputs the selected action; the action space is... This indicates that the subtask is computed locally; This indicates that the subtask has been offloaded to the edge server for processing; subtask The status information includes three parts: its own task information, information about its dependencies, and information about other subtasks within the same execution window. ； in Subtasks The distance between the corresponding terminal and the edge server in the current time slot. This indicates the execution of other subtasks within window i. In the The influence weight of feature dimensions on the current task; The environment interaction unit is used to perform environment interaction based on the output action selection result; based on the output action selection result, it solves for the optimal transmission power for transmitting subtask data under a given channel by using a bisection-based transmission power control method; based on the optimal transmission power, it solves for the subchannel allocation result in the form of a weighted bipartite graph matching by using the KM algorithm; based on the action selection result, the subchannel allocation result, and the optimal transmission power, it calculates the task processing overhead and returns the corresponding reward signal; in response to the completion of the reward signal calculation, it observes the task status information in the next execution window and completes the environment interaction. The neural network parameter training unit is used to train neural network parameters; The maximum training epochs determination unit is used to determine whether the neural network parameters have reached the preset maximum training epochs. The output unit is used to output all results if the maximum number of training epochs is reached.

4. The task offloading and resource allocation system for user dependencies in a mobile edge computing system as described in claim 3, characterized in that, The unloading decision unit makes an unloading decision and outputs the action selection result, including the following sub-steps: Input the state information of the subtasks observed by the agent in the current environment; Based on the state information of the input subtask, - A greedy strategy is used to select the unloading action; Output the action selection result.

Images