An adaptive partitioning method for VGG16 models in edge computing scenarios

Abstract

This invention belongs to the field of edge computing and discloses an adaptive partitioning method for VGG16 models in edge computing scenarios. The method includes: loading the static model file of the VGG16 model; generating a candidate set of split points; obtaining the optimal split point; calculating a loss function based on the computation latency of the VGG16 model and the probability of the optimal split point being selected; and updating the parameters of the deep reinforcement learning model based on the loss function. This invention determines the partitioning method for the neural network model required to complete the inference task, thereby minimizing latency.

Description

Technical Field

[0001] This invention belongs to the field of edge computing, specifically relating to an adaptive partitioning method for VGG16 (Visual Geometry Group 16) models in edge computing scenarios. This method is applicable to edge computing scenarios consisting of a user terminal device and an edge server, and determines the partitioning method for the segmentation points of the neural network model required to complete the inference task, so as to minimize latency. Background Technology

[0002] In an era dominated by data-driven solutions, deep neural networks (DNNs) are constantly evolving in both depth and architecture to meet the ever-evolving demands of contemporary applications such as image classification, object detection, and human-computer interaction. As the performance of these network models improves, their architectures become increasingly complex.

[0003] Emerging mobile edge computing (MEC) technology allocates computing resources at access points to provide cloud-like computing services to nearby mobile devices, enabling devices with limited energy and computing power to apply highly complex DNN models. By partitioning the DNN model and deploying it separately on the user device and the edge server, users can execute only a portion of the DNN model locally without uploading the raw information to the server, thus ensuring both efficiency and privacy.

[0004] Split Computation (SC) offers a feasible solution to the problems of high latency, high energy consumption, and insufficient computing power encountered when deploying complex models on user devices. Specifically, the complete DNN model is split into a head model and a tail model. The head model is deployed on the user device, and the tail model is deployed on an edge server. By setting different optimal split points according to different tasks, end-to-end inference latency and energy consumption can be jointly minimized, thereby enabling collaborative task inference. The combination of split computing and edge computing enables the system operation of collaborative task inference, significantly improving the user experience of performing task inference on user devices and reducing the risk of privacy leaks of raw data.

[0005] Currently, in edge computing scenarios with limited computing power, existing DNN partitioning methods typically train prediction models offline to make partitioning decisions and reduce end-to-end inference latency. This requires a large amount of labeled datasets and may result in long preprocessing times, which is unacceptable for user devices with insufficient computing power. Existing technologies, such as the distributed neural network model partitioning method for edge video analysis published on March 2, 2021 (Publication No. CN112434789A), also use neural network models in edge video analysis scenarios. However, when using this method, there are many types of partitioning decisions for models with a large number of layers. Therefore, the solution space of the algorithm is large, leading to a greater demand for computing power from edge devices. Our method's segmentation point candidate set scheme can effectively reduce the size of the solution space, and the proposed loss function for reinforcement learning algorithms can effectively improve the learning efficiency of the algorithm and accelerate the convergence process. Furthermore, our algorithm can be directly deployed on user devices to run. Summary of the Invention

[0006] The purpose of this invention is to provide an adaptive partitioning method for VGG16 models in edge computing scenarios, which determines the partitioning point positions of the neural network model required to complete the inference task, thereby minimizing latency.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] An adaptive partitioning method for VGG16 models in edge computing scenarios, comprising:

[0009] Step 1: Load the static model file of the VGG16 model;

[0010] Step 2: Based on the static model file, traverse each layer of the VGG16 model. If the current layer is a pooling layer, add the current layer to the split point candidate set and proceed to the next layer; if the current layer is an activation function layer and the next layer is not a pooling layer, add the current layer to the split point candidate set and proceed to the next layer; if the current layer is an activation function layer and the next layer is a pooling layer, proceed directly to the next layer; if the current layer is neither a pooling layer nor an activation function layer, proceed directly to the next layer; continue until all layers have been traversed to obtain the final split point candidate set.

[0011] Step 3: Set the output dimension of the deep reinforcement learning model to the length of the segmentation candidate set, obtain the environmental state input to the deep reinforcement learning model, obtain the probability of each segmentation point being selected in the segmentation candidate set output by the deep reinforcement learning model, and sample the optimal segmentation point based on the probability of each segmentation point being selected. In the VGG16 model, the model before the optimal segmentation point is deployed on the user terminal device for computation, and the model after the optimal segmentation point is deployed on the edge server for computation.

[0012] Step 4: Calculate the loss function based on the computation delay of the VGG16 model and the probability of the optimal split point being selected, and update the parameters of the deep reinforcement learning model based on the loss function.

[0013] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0014] Preferably, the environmental conditions include the transmission rate, signal strength, and channel gain between the user terminal device and the edge server.

[0015] Preferably, the step of obtaining the optimal segmentation point based on the probability of each segmentation point being selected includes:

[0016] The probability of selecting all split points is used as the parameter of the function torch.distributions.Categorical() in the PyTorch deep learning library, and the optimal split point is obtained by the function torch.distributions.Categorical().

[0017] Preferably, the loss function is -log2(p)×L, where L is the computation delay of the VGG16 model and p is the probability of the optimal split point being selected.

[0018] The present invention provides an adaptive partitioning method for VGG16 models in edge computing scenarios, which has the following advantages compared with the prior art:

[0019] Compared with previous optimization methods, in an edge computing scenario consisting of a client device and an edge server, the present invention can solve the problem of selecting a model segmentation scheme in a complex task inference process, quickly determine the optimal segmentation point position for offloading, and at the same time minimize the end-to-end task inference latency. On the one hand, by formulating a suitable candidate segmentation point set scheme based on reinforcement learning, the original m + 1 optional segmentation points can be reduced to n segmentation points (n < m + 1), effectively reducing the solution space size of the algorithm. On the other hand, the reward value parameter in the conventional loss function is a value obtained by simulating the environment, which has a deviation from the reward value obtained from the actual environment. The parameter in the loss function proposed by the present invention is the reward value feedback from the actual environment, which is beneficial for the algorithm to obtain different values according to different environments, thereby further improving the use effect when the algorithm is applied. In addition, probability sampling during segmentation point decision-making can avoid the policy degradation caused by long-term training. And for high-dimensional segmentation point decision-making, there is no need to compare the effects by traversing each segmentation point decision. Instead, this solution samples according to the probability obtained from the deep neural network model to obtain a decision result and directly execute it, and feeds back the actual execution latency of the task to the network to improve performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 An application scenario of an adaptive partitioning method for a VGG16 model in an edge computing scenario of the present invention;

[0021] Figure 2 A flowchart of an adaptive partitioning method for a VGG16 model in an edge computing scenario of the present invention;

[0022] Figure 3 An experimental comparison result graph of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

[0025] To overcome the difficulties of the existing technologies and achieve a relatively low end-to-end inference latency, this embodiment proposes an adaptive partitioning method and system for the VGG16 model in an edge computing scenario, which is applied in scenarios such as Figure 1 As described in the scenario, the application scenario includes a client device (such as Jetson Nano or a smartphone, etc.) and an edge server (such as a server equipped with RTX3090). The client device can communicate with the edge server wirelessly, and the computing power of the client device is limited. Some operations of the VGG16 model need to be performed on the client device, and the remaining operations of the VGG16 model are calculated on the edge server.

[0026] Such as Figure 2 As shown, an adaptive partitioning method for the VGG16 model in an edge computing scenario in this embodiment is also regarded as an edge computing method, including the following steps:

[0027] Step 1: Load the static model file of the VGG16 model. For the convenience of subsequent analysis, a complete VGG16 model loaded can be split into a model composed of a convolutional layer, a pooling layer, an activation function layer, a linear layer, etc.

[0028] Step 2: Generate a candidate set of splitting points: Traverse each layer of the VGG16 model (assuming that the VGG16 model has m layers before splitting, corresponding to m + 1 splitting points, including the splitting points between adjacent layers of the VGG16 model, the splitting point before the first layer of the VGG16 model, and the splitting point after the last layer of the VGG16 model). If the current layer is a pooling layer, add the current layer to the candidate set of splitting points and go to the next layer. If the current layer is an activation function layer and the next layer is not a pooling layer, add the current layer to the candidate set of splitting points and go to the next layer. If the current layer is an activation function layer and the next layer is a pooling layer, directly go to the next layer. If the current layer is neither a pooling layer nor an activation function layer, directly go to the next layer; until all m layers of the VGG16 model have been traversed, the number of splitting points obtained at this time is n (n < m + 1), forming the final candidate set of splitting points. In this embodiment, the original m + 1 optional splitting points are reduced to n splitting points (n < m + 1), effectively reducing the solution space size of the algorithm.

[0029] Step 3: Obtain the optimal segmentation point: Using reinforcement learning, the transmission rate, signal strength, and channel gain between the user device and the edge server are input into the deep reinforcement learning model θ. The output of the deep reinforcement learning model θ is the probability of selecting all segmentation points in the candidate segmentation point set. The size of the output dimension is determined based on the candidate segmentation point set constructed in Step 2, and the sum of all output probabilities is 1. Then, the probability distribution is used as the parameter of the function `torch.distributions.Categorical()` in the PyTorch deep learning library to obtain the optimal segmentation point. Finally, the model before this segmentation point is calculated on the user device, and the calculation result is transmitted to the edge server. The model after this segmentation point is calculated on the edge server, and the result is returned to the user device.

[0030] It is easy to understand that a deep reinforcement learning model includes an agent, and the output of the deep reinforcement learning model is the probability distribution π(a|s) of the agent's actions, where a is the agent's action and s is the state of the current environment.

[0031] Step 4: The delay used in the entire VGG16 model calculation process (the delay is the time between when the user device starts calculating and when it receives the result returned by the edge server) is L. In this embodiment, the loss function is proposed to be -log2(p)×L, where p is the probability that the segmentation point output by the neural network is selected. The parameters of the deep neural network θ are updated using the existing Adam algorithm.

[0032] After the deep reinforcement learning model obtains the output result, the agent is usually trained by setting a reward based on the output result. L is defined as the reward value in the conventional loss function. Here, the delay of the actual task feedback is used as the reward value. In addition, this embodiment introduces the probability value of the agent's direct output on the basis of the delay of the actual task feedback. The combination of probability value and delay enables the model training to integrate actions and actual feedback, and improves the model's prediction ability and performance based on comprehensive quantization.

[0033] Experimental Example

[0034] The experiment used a Jetson Nano as the user terminal device and an edge server equipped with an RTX 3090, both connected to a router to ensure they were on the same local area network. The inference task was image classification, using the VGG16 model (39 layers, m=39, resulting in 40 segmentation points) as the inference model. The signal strength between the user terminal device and the edge server was 0.8, the channel gain was -40 dBm, and multiple experiments were conducted with varying transmission rates of 1 Mbps, 2 Mbps, 4 Mbps, 8 Mbps, and 16 Mbps.

[0035] The experiments involved local computation, fully offloaded computation, the Neurosurgeon scheduler, and the method of this invention (DAPart). The experimental process of the method of this invention is consistent with the steps described in the specific implementation. The length of the resulting candidate set of split points is 18, i.e., n = 18. It is important to note that the process requires moving the user's device to change the state of the current environment input to the deep reinforcement learning model θ. The algorithm converges after one to two hours of training.

[0036] Finally, experimental data were obtained, forming comparative experimental results as follows: Figure 3 As shown, it can be seen that the DAPart method of the present invention has the lowest latency at any transmission rate, and the latency reduction effect is obvious.

[0037] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0038] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. An adaptive partitioning method for VGG16 models in edge computing scenarios, characterized in that, The adaptive partitioning method for VGG16 models in the edge computing scenario includes: Step 1: Load the static model file of the VGG16 model; Step 2: Based on the static model file, traverse each layer of the VGG16 model. If the current layer is a pooling layer, add the current layer to the split point candidate set and proceed to the next layer; if the current layer is an activation function layer and the next layer is not a pooling layer, add the current layer to the split point candidate set and proceed to the next layer; if the current layer is an activation function layer and the next layer is a pooling layer, proceed directly to the next layer; if the current layer is neither a pooling layer nor an activation function layer, proceed directly to the next layer; continue until all layers have been traversed to obtain the final split point candidate set. Step 3: Set the output dimension of the deep reinforcement learning model to the length of the segmentation candidate set, input the environmental state into the deep reinforcement learning model, obtain the probability of each segmentation point being selected in the segmentation candidate set output by the deep reinforcement learning model, and sample the optimal segmentation point based on the probability of each segmentation point being selected. In the VGG16 model, the model before the optimal segmentation point is deployed on the user terminal device for computation, and the model after the optimal segmentation point is deployed on the edge server for computation. The environmental state includes the transmission rate, signal strength, and channel gain between the user terminal device and the edge server. Step 4: Calculate the loss function based on the computation latency of the VGG16 model and the probability of the optimal split point being selected, and update the parameters of the deep reinforcement learning model according to the loss function; the loss function is... ,in The computational latency of the VGG16 model, This represents the probability of the optimal split point being selected.

2. The adaptive partitioning method for VGG16 models in edge computing scenarios according to claim 1, characterized in that, The step of obtaining the optimal segmentation point based on the probability of each segmentation point being selected includes: The probability of selecting all split points is used as the parameter of the function torch.distributions.Categorical() in the PyTorch deep learning library, and the optimal split point is obtained by the function torch.distributions.Categorical().

Images