A Dynamic Combinable Context-Aware Mobile Edge Computing Method

Abstract

The present invention discloses a dynamically composable context - adaptive mobile edge computing method, which decouples the deployment process of an end - to - end deep model into two stages: pre - segmentation of computing units and atomic combination and task offloading of a deep neural network (DNN). First, the deep model is pre - segmented based on the finest granularity, and a latency benefit function is designed to filter candidate segmentation points, only retaining the segmentation points that can bring positive latency benefits. These pre - segmented deep model computing units are defined as pre - segmented DNN atoms. Then, efficient selection of the optimal pre - segmented DNN atom combination is supported at runtime. Finally, the DNN atoms are offloaded to the corresponding edge devices with the least additional overhead to complete the entire process of the deep model deployment task. The present invention enables users to obtain latency benefits as soon as possible while minimizing additional offloading consumption.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and particularly relates to a context - adaptive mobile edge computing method. Background Art

[0002] With the development of intelligent Internet of Things technology, the era of human - machine - thing fusion crowd - intelligence computing is gradually coming. It focuses on actively establishing a perception - computing mode that integrates with the environment based on mobile Internet of Things devices (such as wearable devices, smart phones, urban cameras, etc.) to provide convenient intelligent services for humans. To bring a good interaction experience, mobile applications usually require a fast and near - real - time response process, such as face recognition, health monitoring, and fall detection. However, mobile devices are usually resource - constrained and cannot guarantee timely response during the operation of computationally intensive algorithms (such as deep learning models). In order to enable resource - constrained mobile devices to support the efficient operation of AI - driven mobile applications to the greatest extent, it is necessary to solve the problem of adaptive deployment of deep models in dynamic contexts, and coordinate multiple mobile edge devices to complete the computing process, so as to meet the dynamic resource requirements of mobile applications.

[0003] In this context, model splitting technology has become an ideal solution for deploying deep models on heterogeneous mobile and edge clusters. Existing model splitting works adopt serial collaborative computing (splitting between deep model layers) or parallel collaborative computing (splitting within deep model layers and designing a parallel computing paradigm) to seek the optimal splitting point of the deep model, and then deploy the split model layers to different edge devices for running. However, once the deployment context changes, it is necessary to execute the algorithm from scratch to find a new optimal splitting point and redeploy it to the edge device to achieve adaptation to the dynamic context, which will result in unacceptable end - to - end inference latency. Summary of the Invention

[0004] To overcome the deficiencies of the prior art, the present invention provides a dynamically composable context - adaptive mobile edge computing method, which decouples the deployment process of the end - to - end deep model into two stages: pre - splitting of computing units and atomic combination and task offloading of deep neural networks (DNNs). First, the deep model is pre - split based on the finest granularity, and a latency benefit function is designed to filter candidate splitting points, only retaining the splitting points that can bring positive latency benefits. These pre - split deep model computing units are defined as pre - split DNN atoms; then, at runtime, efficient selection of the optimal pre - split DNN atom combination is supported; finally, the DNN atoms are offloaded to the corresponding edge devices with the least additional overhead to complete the entire process of the deep model deployment task. The present invention can enable users to obtain latency benefits as soon as possible while minimizing additional offloading consumption.

[0005] The technical solution adopted by the present invention to solve its technical problems includes the following steps:

[0006] Step 1: Offline pre-segmentation of the deep model;

[0007] The latency benefit function R off is defined as follows:

[0008]

[0009] Where p i ∈P, P represents the set of deep neural network DNN offloading schemes determined by all candidate segmentation points; λ1, λ2 represent the weighted coefficients of positive and negative latency benefits; T exe (p i ) represents the distributed execution latency of p i on the edge device and the mobile device; T dev represents the latency of the model executed completely locally on the mobile device; T tran (p i ) represents the transmission latency generated by transmitting intermediate data, i.e., the feature map, between edge devices during the collaborative computing process of p i ; T user represents the user's latency requirement; ε is the unit step function;

[0010] When T exe (p i ) + T tran (p i ) is less than T user , define ε(T exe (p i ) + T tran (p i ) - T user ) = 0; otherwise, let ε = 1, indicating that this offloading scheme will result in a negative benefit;

[0011] Calculate the latency benefits of all candidate segmentation points of the deep model, and filter out the candidate segmentation points with negative benefits; divide the deep model into multiple computing units according to the filtered candidate segmentation points, and there are no segmentation points with positive benefits inside the obtained computing units; define the obtained computing units as pre-segmented DNN atoms, which constitute a minimized effective search space;

[0012] Step 2: Adaptive atomic dynamic combination;

[0013] Under the constraints of the user-specified latency requirement and the resource budget of the mobile edge device, search for the optimal combination s opt of DNN atoms based on the pre-segmented DNN atoms to maximize the latency benefit R off ;

[0014] The problem is defined as shown in the following formula.

[0015]

[0016] In the formula, s i ∈S, where S is the set of all combinations of pre-partitioned DNN atoms; T exe (s i ) + T tran (s i ) represents the total inference latency of the combination s i ; T user (t) represents the time-varying user latency requirement; represents the time-varying computational resource constraint on the j-th edge device in the dynamic deployment scenario; C j (s i ) represents the amount of computation required to run the target DNN atom for the combination s i on device j; represents the time-varying memory resource constraint on the j-th device; M j (s i ) is the memory space required to run the target DNN atom on device j;

[0017] According to all the pre-partitioned DNN atoms of the deep model, all possible atomic combination schemes, i.e., offloading schemes, are obtained, and these combination schemes are organized into a search graph G = <V, L>; each vertex in G represents an optional combination scheme s i , and the performance parameters of G are characterized by three aspects: the inference latency T(s i ) = T exe (s i ) + T tran (s i ), the distributed memory requirement M(s i ), and the amount of computation C(s i ); the connection <s i , s j > ∈ L in the graph means that only the offloading method of one DNN atom is different between the combination schemes s i and s j ;

[0018] Step 3: Asynchronous offloading during runtime;

[0019] Construct a combination offloading decision algorithm based on the shortest path to determine the optimal offloading order of each atom in the target combination scheme;

[0020] The steps of the combination offloading decision algorithm based on the shortest path are as follows:

[0021] (1) Initialize the weight of the search graph connection as the offloading overhead, i.e., the transmission latency, between adjacent combination schemes; from the current combination scheme s cur to the target combination scheme s tarThe uninstallation overhead is the sum of the weights on the connection path;

[0022] (2) Initialize the auxiliary array L, where each element L[i] represents the minimum overhead from s i to s cur that has been updated currently; if there is a connection from s cur to s i , L[i] is the weight on the connection; otherwise L[i] = ∞;

[0023] (3) Add s cur to the set U, where U represents the set of combination schemes that have currently been determined to have the minimum overhead from s cur ; add the combination schemes adjacent to s cur to the set Q, which represents the set of schemes that have not yet determined the minimum overhead from s cur ;

[0024] (4) Select the s cur with the minimum L value in Q, that is, the minimum overhead from s temp , traverse the adjacent schemes and calculate the overhead of s temp ; if the newly obtained overhead is less than the original overhead, update the L value of the adjacent scheme;

[0025] (5) Add s temp to the set U, and add the newly updated scheme to the set Q;

[0026] (6) Repeat steps (4) and (5) until the set U contains all the combination schemes in the search graph, then the path corresponding to L[tar], which is the minimum overhead from s tar to s cur , is the optimal uninstallation plan.

[0027] Preferably, the resource budget includes a computational resource budget and a memory resource budget.

[0028] The beneficial effects of the present invention are as follows:

[0029] The present invention proposes a combined uninstallation decision algorithm to determine the optimal uninstallation order of pre-partitioned DNN atoms, and realizes end-to-end uninstallation deployment in an asynchronous manner, enabling users to obtain latency benefits as soon as possible while minimizing additional uninstallation consumption. Description of the Drawings

[0030] Figure 1 is the functional module framework diagram of the present invention.

[0031] Figure 2 is the search graph of deploying AlexNet with five atoms on a mobile device and two edge devices in an embodiment of the present invention. Detailed Embodiments

[0032] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0033] The present invention is a dynamic composable context - adaptive mobile edge computing method. The specific process is as follows: First, perform offline pre - segmentation on the deep model. According to the designed latency benefit function, pre - segment the deep model at the basic operator level, compile and store the segmented modules of the model as independent computing units, and define them as pre - segmented DNN atoms. Second, quickly search for the best combination of pre - segmented DNN atoms at runtime to meet the resource budget and latency requirements of the dynamic context. The present invention proposes an adaptive DNN atom dynamic combination algorithm to heuristically improve the search efficiency of the optimal combination of pre - segmented DNN atoms to ensure that the optimal combination has real - time performance and adaptability. Finally, unload the pre - segmented DNN atoms to the mobile edge device according to the searched optimal combination scheme to complete the end - to - end deep model deployment and quickly adapt to the deployment context (including device resource availability, network conditions, and latency requirements). The present invention proposes a combined offloading decision algorithm to determine the optimal offloading order of pre - segmented DNN atoms and adopts an asynchronous method to implement end - to - end offloading deployment, enabling users to obtain latency benefits as soon as possible while minimizing additional offloading consumption.

[0034] To achieve the above tasks, the present invention adopts the following technical solutions:

[0035] Step 1: Offline pre - segmentation of the deep model

[0036] To accurately measure the latency benefits of candidate DNN segmentation points and the subsequent combined offloading schemes, the present invention formally defines the latency benefit function. It simultaneously considers the latency benefits brought by reducing the overall inference latency and meeting the user - defined latency requirements. Specifically, the latency benefit function R off is defined as follows:

[0037]

[0038] In the formula, p i ∈P, where P represents the set of DNN offloading schemes determined by all candidate segmentation points; λ1 and λ2 represent the weighted coefficients of positive and negative latency benefits; T exe (p i ) represents the distributed execution latency of p i on the edge device and the mobile device; T dev represents the latency of completely locally executing the model on the mobile device; T tran (p i ) represents the transmission latency generated by transmitting intermediate data (i.e., feature maps) between edge - end devices during the collaborative computing process of p i ; T userrepresents the user's latency requirement; ε is the unit step function.

[0039] Specifically, the latency gain R off consists of the gain brought by reducing the inference latency and the gain brought by meeting the user's requirements, weighted by λ1 and λ2. Among them, the inference latency gain consists of two parts: the latency acceleration gain T exe (p i ) - T dev and the latency negative gain T tran (p i ). The difference between the two represents the latency acceleration advantage brought by offloading the DNN computing unit according to the offloading scheme p i . The logarithmic operation (i.e., log(·) in the formula) is adopted in the calculation process to reduce the calculation complexity.

[0040] In addition, the present invention defines the user's inference latency requirement as T user . When the inference latency T exe (p i ) + T tran (p i ) is less than T user , define ε(T exe (p i ) + T tran (p i ) - T user ) = 0. Conversely, if the model inference latency exceeds the latency budget, then let ε = 1, indicating that this offloading scheme will get a negative gain.

[0041] Based on the above definitions, the present invention first calculates the latency gains of all candidate segmentation points of the deep model and filters out the segmentation points with negative gains. Then, according to the filtered candidate points, the model is divided into multiple computing units, and there cannot be segmentation points with positive gains inside the obtained computing units. Therefore, the present invention defines them as pre-segmented DNN atoms, which constitute a minimized effective search space.

[0042] Step 2: Adaptive atomic dynamic combination

[0043] Considering that a complex DNN model may contain a large number of combinations of pre-segmented DNN atoms with different performance characteristics, it is a challenge to adaptively select the optimal DNN atom combination according to the dynamic situation at runtime. The present invention models the adaptive combination problem of DNN atoms as an optimization problem at runtime. Specifically, the present invention expects to search for its optimal combination s opt based on the pre-segmented DNN atoms under the constraints of the latency requirements specified by the user and the resource budgets of the mobile edge device (i.e., the computational resource budget and the memory resource budget) offThe problem definition is shown as follows.

[0044]

[0045] Wherein, s i ∈S, and S is the set of all combinations of pre - segmented DNN atoms; T exe (s i ) + T tran (s i ) represents the total inference latency of combination s i ; T user (t) represents the time - varying user latency requirement; represents the time - varying computational resource constraint on the j - th edge device in the dynamic deployment scenario; C j (s i ) represents the amount of computation required to run the target DNN atom for combination s i on device j (measured by floating - point operations, i.e., FLOPs); Similarly, represents the time - varying memory resource constraint on the j - th device; M j (s i ) is the memory space required to run the target DNN atom on device j.

[0046] To formally represent the search space of DNN atom combinations, the present invention constructs a search graph containing various configurations and performance characterizations. Specifically, according to all pre - segmented DNN atoms of a specific deep model, all possible atomic combination schemes (i.e., offloading schemes) can be obtained, and these combinations are organized into a search graph G = <V, L>. Each vertex in G represents an optional combination scheme s i , and its performance parameters are characterized by three aspects: the inference latency T(s i ) = T exe (s i ) + T tran (s i ), the distributed memory requirement M(s i ) and the amount of computation C(s i ). The connection <s i , s j > ∈ L in the graph indicates that only the offloading method of one DNN atom is different between combination schemes s i and s j .

[0047] Specifically, the present invention maps the search graph to a three-dimensional solution space composed of computational complexity, memory occupancy, and inference latency. The fast search problem for the optimal combination can be decomposed into two sub-problems: (1) finding the combined subspace that satisfies the hard constraints; (2) finding the optimal combination of pre-partitioned DNN atoms with the maximum latency benefit. Through prior experiments, the present invention finds that two adjacent atomic combination schemes have similar performance characteristics. Then, if the current combination satisfies the resource constraints, it can be considered that its adjacent combinations will also approximately satisfy the resource constraints. These adjacent combination schemes can constitute an efficient search subspace. Based on this mechanism, the present invention adopts an iterative search method and constructs an "artificial gradient" to heuristically guide the search process of the optimal combination scheme of DNN atoms to achieve the purpose of fast convergence.

[0048] Step 3: Asynchronous offloading during runtime

[0049] To ensure that the mobile device can obtain the latency benefit of computational offloading as soon as possible during runtime and avoid additional offloading overhead, the present invention proposes a combined offloading decision algorithm based on the shortest path to determine the optimal offloading order of each atom in the target combination scheme.

[0050] Specifically, the design of this algorithm follows the following two principles: (1) Give priority to executing with the lowest overhead (i.e., the transmission latency caused by offloading the network structure and parameters to the edge device) during each offloading, so that the mobile device can obtain the latency benefit of computational offloading as early as possible; (2) The overall offloading overhead from the current combination scheme to the target combination scheme is the lowest to ensure that there is no unnecessary offloading consumption. The core steps of the algorithm are as follows:

[0051] (1) Initialize the weight of the connection in the search graph as the offloading overhead (i.e., transmission latency) between adjacent combination schemes. The offloading overhead from the current combination scheme s cur to the target combination scheme s tar is the sum of the weights on the connection path;

[0052] (2) Initialize the auxiliary array L, where each element L[i] represents the minimum overhead from s i to s cur that has been updated currently. If there is a connection from s cur to s i , L[i] is the weight on the connection; otherwise, L[i] = ∞;

[0053] (3) Add s cur to the set U, where U represents the set of combination schemes that have currently been determined to have the minimum overhead from s cur . Add the combination schemes adjacent to s cur to the set Q, which represents the set of schemes whose minimum overhead from s cur has not been determined yet;

[0054] (4) Select the s in Q with the smallest L value (i.e., the minimum cost from s cur to s temp , traverse adjacent solutions and calculate their costs. If the newly obtained cost is less than the original cost, update the L value of the adjacent solution;

[0055] (5) Add s temp to set U, and add the newly updated solution to set Q;

[0056] (6) Repeat steps (4) and (5) until set U contains all combined solutions in the search graph. Then the path corresponding to the minimum cost from s tar to s cur is the optimal offloading plan.

[0057] Once the offloading plan is determined, the present invention will asynchronously offload the atomic executable model file (i.e.,.pkl) to the target device according to the plan.

Claims

1. A dynamic composable context - adaptive mobile edge computing method, characterized in that, It includes the following steps: Step 1: Offline pre-segmentation of the deep model; Time-delay revenue function R off is defined as follows: where p i ∈P, P represents the set of DNN offloading schemes determined by all candidate splitting points; λ1 and λ2 represent the weighted coefficients of the positive and negative benefits of latency; T exe (p i ) represents the distributed execution latency of p i on the edge device and the mobile device; T dev represents the latency of executing the model completely locally on the mobile device; T tran (p i ) represents p i the transmission latency generated by transmitting intermediate data, i.e., feature maps, between edge devices during collaborative computing; T user represents the user's latency requirement; ε is the unit step function; When T exe (p i ) + T tran (p i ) is less than T user , define ε(T exe (p i ) + T tran (p i ) - T user ) = 0; Conversely, let ε = 1, indicating that this offloading scheme will result in negative benefits; Calculate the latency benefits of all candidate segmentation points of the deep model and filter out the candidate segmentation points with negative benefits; divide the deep model into multiple computing units according to the filtered candidate segmentation points, and there are no segmentation points with positive benefits inside the obtained computing units; Define the obtained computing units as pre-segmented DNN atoms, which constitute a minimized effective search space; Step 2: Adaptive atomic dynamic combination; Search for the optimal combination s of DNN atoms based on pre - segmented DNN atoms under the constraints of the latency requirements specified by the user and the resource budget of the mobile edge device opt , to maximize the latency benefit R off ; The problem is defined as shown in the following formula: where s i ∈ S, and S is the set of all combinations of pre - segmented DNN atoms; T exe (s i ) + T tran (s i ) represents the total inference latency of the combination s i ; T user (t) represents the time - varying user latency requirement; represents the time - varying computational resource constraint on the j - th edge device in the dynamic deployment scenario; C j (s i ) represents the computational amount required to run the target DNN atoms of the combination s i on device j; represents the time - varying memory resource constraint on the j - th device; M j (s i ) is the memory space required to run the target DNN atoms on device j; Based on all the pre-segmented DNN atoms of the deep model, all possible atomic combination schemes, i.e., offloading schemes, are obtained. These combination schemes are organized into a search graph G = <V, L>. Each vertex in G represents an optional combination scheme s i , and the performance parameters of G are characterized by three aspects: the inference latency T(s i ) = T exe (s i ) + T tran (s i ), the distributed memory requirement M(s i ), and the computational workload C(s i ). The connection <s i , s j > ∈ L in the graph means that only the offloading method of one DNN atom is different between the combination schemes s i and s j . Step 3: Asynchronous offloading at runtime; Construct a combination offloading decision algorithm based on the shortest path to determine the optimal offloading order of each atom in the target combination scheme; The steps of the combination offloading decision algorithm based on the shortest path are as follows: (1) Initialize the weights of the search graph connections as the offloading costs, i.e., the transmission delays, between atoms of adjacent combination schemes; the offloading cost from the current combination scheme s cur to the target combination scheme s tar is the sum of the weights on the connection path; (2) Initialize the auxiliary array L, where each element L[i] represents the minimum cost of the currently updated path from s i to s cur ; if there is a connection from s cur to s i , L[i] is the weight of the connection; Otherwise, L[i] = ∞; (3) Add s cur to set U, where U represents the set of combination schemes that have currently been determined to have the minimum cost from s cur ; add the combination schemes adjacent to s cur to set Q, which represents the set of schemes whose minimum cost from s cur has not yet been determined; (4) Select the s with the smallest cost among the L values in Q, which is the distance from s cur with the smallest cost temp , traverse adjacent solutions and calculate the cost of s temp ; if the newly obtained cost is less than the original cost, update the L value of the adjacent solution; (5) Add s temp to set U, and add the newly updated solution to set Q; (6) Repeat steps (4) and (5) until set U contains all combination schemes in the search graph. Then the path corresponding to the minimum cost of L[tar] from s tar to s cur is the optimal offloading plan.

2. The dynamic composable context - adaptive mobile edge computing method according to claim 1, wherein The resource budget includes the computational resource budget and the memory resource budget.

Images