Edge-end cooperative task division and resource allocation method based on model segmentation

Abstract

The invention discloses an edge-end cooperative task division and resource allocation method based on model segmentation, relates to the field of edge computing video analysis, and integrates CNN channel pruning, bilinear sampling and affine quantization multi-dimensional compression of intermediate transmission feature data; a parameter-shared dynamic neural network is trained based on a sandwich rule and a knowledge distillation technology, and real-time switching of characteristic data compression amplitude is realized; modeling user reasoning performance indexes in edge-end cooperative task division, and constructing an optimization problem of maximizing reasoning accuracy meeting user performance demand constraints and edge resource constraints; a solution algorithm based on dynamic programming is adopted, optimal configuration such as model break points and intermediate feature compression parameters is selected for users, and edge communication resources and computing resources are efficiently allocated, so that the reasoning accuracy under the multi-user condition is improved.

Description

Technical Field

[0001] The present invention relates to the field of edge computing video analysis, and more particularly to a method for edge collaborative task division and resource allocation based on model segmentation. Background Art

[0002] In recent years, with the rapid development of 5G technology and industrial Internet of Things, the number of mobile intelligent terminal devices has grown exponentially. As the computing power of terminal devices has gradually increased, deep neural network models have also been widely deployed on mobile terminal devices, and various intelligent applications have emerged in an endless stream. However, due to the increasingly complex network structure of deep neural network (DNNs) models, the demand for computing resources has gradually increased, and it is difficult for mobile terminal devices with limited computing power to provide high-precision and low-latency services. The contradiction between computing resource requirements and real-time interaction has given rise to the edge computing paradigm. The emergence of edge computing has pushed application tasks from the cloud center to the network edge closer to mobile terminal devices. Data can be analyzed and processed at the network edge with certain computing power, which can meet key requirements such as real-time processing and high computing of applications and realize "edge intelligence".

[0003] When using the edge computing paradigm, a common approach is to upload the original input data directly to the edge server. However, on the one hand, due to the low security of the edge server, there is a problem of user privacy data leakage after uploading the original image data directly to the edge server. On the other hand, due to the bandwidth limitation of the edge server, the continuous transmission of a large amount of original image data will inevitably cause network congestion.

[0004] In order to improve the security of user privacy data and make full use of the computing power of terminal devices, researchers have proposed another edge-side collaborative reasoning solution, namely DNN task division based on model segmentation. The basic idea is to divide the DNN model into two parts along a segmentation point: the head model processed by the user device and the tail model processed by the edge server. The intermediate feature data output by the head model is uploaded to the edge server and used as the input of the tail model to perform reasoning on the edge server.

[0005] However, the current edge collaborative task division scheme based on model segmentation still has the following problems:

[0006] (1) The amount of intermediate data transmitted in traditional task division schemes is large. The uncompressed intermediate data is often larger than the original image. When the network condition is poor, it is difficult to achieve satisfactory system performance due to the pressure from both the computing power of the terminal equipment and the network bandwidth.

[0007] (2) The task division scheme of modifying the DNN structure to compress intermediate transmission data by introducing compression technologies such as pruning or quantization often faces a dilemma. That is, if no additional training for compression is performed, the flexibility of the segmentation points and compression parameters is retained, but the accuracy cost caused by compression is high, which makes it difficult to meet the accuracy requirements of users; and if additional training for compression is performed, only a single predefined segmentation point and compression parameter can be used, resulting in insufficient adaptability and difficulty in meeting fluctuating network environments and complex user needs.

[0008] (3) The task division schemes that introduce dynamic compression technology based on dynamic neural networks are still limited to a given single segmentation point, ignoring the significant impact of the segmentation point in model segmentation on system performance; on the other hand, they do not fully compress the dimensions of intermediate features, ignoring the impact of the height and width of the feature map on compression efficiency. Summary of the invention

[0009] In order to solve the above problems, the present invention provides an edge collaborative task division and resource allocation method based on model segmentation, which improves the processing efficiency and accuracy of deep neural network reasoning tasks in edge computing systems through efficient feature data compression technology and resource allocation strategies.

[0010] To achieve the above object, the present invention provides a method for edge collaborative task division and resource allocation based on model segmentation, which specifically includes the following steps:

[0011] Step S1: Establish a comprehensive system model for the end-to-end inference latency, inference throughput, and accuracy in edge collaborative inference tasks, combined with different inference configurations of users;

[0012] Step S2: Intermediate feature data compression for model segmentation: Based on the edge collaborative task division of model segmentation, the neural network model is divided into a head model and a tail model along a certain segmentation point. The head model is deployed on the end user, and the tail model is deployed on the edge server. The feature data compression module is injected between the head model and the tail model.

[0013] Step S3: According to the multi-segmentation point dynamic neural network training algorithm, a parameter-sharing dynamic neural network using multiple compression parameter configurations is obtained, and multi-segmentation point dynamic neural network training is performed; user demand constraints and edge resource constraints are combined to form a discrete optimization problem with the goal of maximizing accuracy;

[0014] Step S4: Based on the end-to-end inference latency model, inference throughput model, and inference accuracy model established in step S1, and with the help of the dynamic compression of feature data proposed in step S2, a discrete integer programming problem for maximizing inference accuracy is constructed to achieve a trade-off between accuracy, latency, and edge computing or communication resources.

[0015] Step S5: converting the discrete integer programming problem from a single-side multi-terminal scenario into a single user sub-problem in a single-side single-terminal scenario;

[0016] Step S6: Based on the optimal substructure properties of the two-dimensional resource allocation problem, a dynamic programming algorithm is used to obtain the optimal solution to perform edge collaborative task division and resource allocation.

[0017] Preferably, in step S1, the inference configuration Ψ of user i is defined i By model split point s i , intermediate feature channel compression parameter α i , the length and width compression parameter of the intermediate feature β i , intermediate feature quantization width q i The number of discrete computing resources allocated to the edge server r i , the number of discrete communication resources allocated to the edge server b i The system consists of three key variables, and models the three key system performance indicators: end-to-end inference latency, inference throughput, and inference accuracy.

[0018] Preferably, the end-to-end reasoning delay model building specifically includes the following steps:

[0019] End-to-end inference latency of user i Includes: End-user inference latency Intermediate characteristic transmission delay Edge server inference latency The three parts are specifically expressed as follows:

[0020]

[0021] End-user inference latency Affected by the split point and the computing power of the terminal device, an offline measurement method is used. In the preparation stage, the user i is measured and recorded at each possible split point s i The actual inference delay under Reference value of

[0022] Intermediate characteristic transmission delay Affected by the size of the transmitted data and the communication resources allocated by the edge server, the specific expression is as follows:

[0023]

[0024] Among them, Ds i is the split point s i The original uncompressed intermediate feature data size is as follows; is the product of the corresponding compression parameters, which means that the size of the transmitted feature data is compressed from the original [C×H×W×32bit] to [αC×βH×βW×qbit]; B is the total number of communication resources on the edge server; Ω B is the total bandwidth size on the edge server, The communication bandwidth size allocated to user i by the edge server;

[0025] Edge server inference latency Affected by the split point and the computing resources allocated to the edge server, the specific expression is as follows:

[0026]

[0027] Among them, R is the total number of computing resources on the edge server, r i is the number of computing resources allocated to the i-th user, and the ratio of the number of threads allocated by the edge server to the i-th user inference process to the total number of active threads is s i is the split point;

[0028] Use the following formula function Fit the measured data points to obtain the split point s i The function parameters below and Establish the relationship between edge server inference latency and computing resources:

[0029]

[0030] For the actual application scenario of edge servers using GPUs for model inference, on Nvidia GPU devices, the MPS technology provided by Nvidia is used to allocate the computing resources used by users; MPS technology allocates the active threads on a single GPU device to the model inference process of a specified user according to a percentage.

[0031] Preferably, in the edge collaborative reasoning system based on model segmentation, the user's reasoning task is divided into three parallel running parts: head network reasoning, intermediate feature transmission and tail network reasoning. The three parallel running parts correspond to the user device reasoning delay, the intermediate feature transmission delay and the edge server reasoning delay respectively. Based on the principle of pipeline parallelism, the reasoning throughput of user i in the current time slice is determined by the largest delay among the user device reasoning delay, the intermediate feature transmission delay and the edge server reasoning delay:

[0032]

[0033] Among them, Δt is the length of the current time slice, Ψ i A collection of variables for user inference configuration.

[0034] Preferably, in the inference accuracy model, the accuracy of all compression parameter settings is comprehensively measured, and the corresponding Pareto set P is constructed based on the scatter plot of the transmission data size and the corresponding inference accuracy. i :

[0035]

[0036] In the formula, is the product of the corresponding compression parameters, A i (s i ,α i ,β i ,q i ) is the split point s i , compression parameter α i ,β i ,q i The corresponding inference accuracy is for the elements in the Pareto set (s i ,α i ,β i ,q i ), there is no other group (s i ,α j ,β j ,q j ), making the compression ratio Less than At the same time, the accuracy A i (s i ,α j ,β j ,q j ) is greater than A i (s i ,α i ,β i ,q i );

[0037] P i It represents the configuration set with the least impact on accuracy among all compression parameter configurations. Each point on the configuration set represents the highest accuracy achieved under a specific transmission data size.

[0038] Preferably, in step S2, the output of the head model is compressed at the end user, and the compressed data is restored on the edge server as the input of the tail model;

[0039] The feature data compression module compresses the intermediate transmission features from [C×H×W×32bit] to [αC×βH×βW×qbit] at the end user, and then restores them to the original size of [C×H×W×32bit] on the edge server, where C is the number of channels of the intermediate transmission features, H and W are the height and width of the intermediate transmission features respectively, and 32bit means that the intermediate transmission features are stored using 32-bit floating point numbers; α is the channel compression parameter, β is the length and width compression parameter, and q is the quantization width;

[0040] The compression part is based on three feature data compression methods: CNN channel pruning, bilinear interpolation downsampling, and learnable affine quantization. The intermediate transmission features output by the head model are compressed from multiple dimensions. The specific methods are as follows:

[0041] CNN channel pruning: responsible for compressing the number of channels C of intermediate transmission features, denoted by the channel compression parameter α, and inputting the intermediate transmission features into the point convolution layer. Assuming that the weight matrix of the point convolution layer is in the shape of [C,C,1,1], the weight matrix is ​​split and the weight matrix of the shape of [αC,C,1,1] is taken for convolution calculation. The number of output channels of the point convolution layer is pruned from the original C to αC, thus compressing the number of channels of intermediate transmission features.

[0042] Bilinear interpolation downsampling: responsible for compressing the length and width (H and W) of the intermediate transmission features. The compression parameters of H and W are β. The bilinear interpolation method that supports arbitrary numerical scaling is used to downsample the intermediate transmission features.

[0043] Learnable affine quantization: compresses the number of bits of data of intermediate transmission features. The quantization width is q. The intermediate transmission features will be compressed from 32-bit floating point numbers x to q-bit discrete integers x. q , the specific formula is as follows:

[0044]

[0045] where x min 、x max Corresponding to the maximum and minimum values ​​of the intermediate transmission characteristics respectively; Q p is the maximum value of a q-bit discrete integer, Q p =2 q -1; the clamp function is a clipping function that clips the function input to 0 to Q p Within the range; s is a learnable parameter of the neural network, which is learned along with the entire neural network and is used to adjust the Δ used in quantization; The function is the Round function, which is responsible for rounding its function input to an integer;

[0046] The recovery part performs the inverse operation of the compression part in sequence to restore the intermediate feature data to its original size. The specific steps are as follows:

[0047] S21: The edge server dequantizes the received feature data according to the formula Rewrite the q-bit discrete integer x q Restore to 32-bit floating point

[0048] S22: Restore the length and width of the feature data through bilinear interpolation upsampling recovery and depth-separable convolution layer;

[0049] S23: restore the number of channels of feature data through the point convolution layer;

[0050] S24: Insert a switchable batch normalization layer before the tail model to correct the data distribution of the tail model input.

[0051] Preferably, in step S3, an EMA network is used as a shadow network to generate soft labels used for distillation. The weights of the EMA network are updated at the end of each epoch according to the weights of the currently trained network, as shown below:

[0052] W′ τ =ηW τ +(1-η)W′ τ-1 ;

[0053] Where η is the smoothing coefficient, W' τ is the weight of the EMA network at the τth epoch, and W τ is the weight of the trained network at the τth epoch;

[0054] The training process of multi-split point dynamic neural network is divided into two stages. In the first stage, due to the soft labels used in distillation Inaccurate, only the largest subnet is trained; in the second stage, in addition to training the largest subnet, the corresponding smallest subnet and the middle subnet are sampled at the split point, and soft labels are generated through the EMA network As the teacher network, the smallest subnet and the intermediate subnet are distilled.

[0055] Preferably, in step S4, the following discrete integer programming problem for maximizing reasoning accuracy is constructed:

[0056]

[0057] Among them, the objective function is to maximize the inference accuracy A of N users i The sum of C1 and C2 is the user demand constraint, which represents the upper limit of end-to-end delay. The lower bound on inference throughput C3 and C4 are the computing or communication resource constraints on the edge server, and the total number of resource shares R and the total number of communication resource shares B are calculated respectively. i With b i are all discrete integer variables; constraint C5 means that the user's segmentation point and compression parameter configuration are selected from the Pareto set.

[0058] Preferably, in step S5, the original question For user i only, given computing resources r i and communication resources i , original question The scenario is transformed from a single-side multi-end scenario to a single-side single-end scenario, broken down into simplified sub-problems

[0059]

[0060] Among them, the sub-problem The objective function in only considers maximizing the accuracy of user i. Constraints C1 and C2 are end-to-end delay constraints and inference throughput constraints, and constraint C3 is a Pareto set constraint. At this time, the subproblem traverses the corresponding Pareto point set to solve. Given a computing resource r i and communication resources i Under this condition, the maximum inference accuracy that user i can achieve is The corresponding inference configuration used is

[0061] Calculate each type (r i ,b i ) The corresponding binary At this time, the original question is transformed into a two-dimensional resource allocation problem As shown below:

[0062]

[0063] The optimization variable only considers (r i ,b i ) tuple, user's (s i ,α i ,β i ,q i ) in advance calculation In the process, constraints C1 and C2 indicate that the total resources allocated to users do not exceed the edge resource limit.

[0064] Preferably, in step S6, for the dimension resource allocation problem Based on the optimal substructure properties, a dynamic programming algorithm is used to obtain the optimal solution for edge-end collaborative task division and resource allocation, including:

[0065] Define the first k users to use R k Computing resources and B k Communication resources, the maximum accuracy that can be achieved is P k (R k ,B k ), there is a state transfer equation as follows:

[0066]

[0067] in P is the maximum inference accuracy that the kth user can achieve under given computing resources and communication resources y, k -1(R k -x,B k -y) Use R for the first k-1 users k -x computing resources and B k -yThe maximum accuracy that can be achieved by the communication resources, defined as the first k users using R k Computing resources and B k When the communication resources are renew Perform optimal edge-end collaborative task division and resource allocation:

[0068]

[0069] in, Use x for the kth user * Computing resources and * The optimal user reasoning configuration of communication resources is calculated The process of recording and obtaining the original problem The optimal solution has a time complexity of O(NR 2 B 2 ).

[0070] Therefore, the present invention adopts the above-mentioned edge collaborative task division and resource allocation method based on model segmentation, which has the following beneficial effects:

[0071] (1) The present invention combines CNN channel pruning, bilinear interpolation sampling and affine quantization to perform multi-dimensional and comprehensive compression on the channel dimension, height and width dimension and data bit number of the intermediate transmission features, so as to significantly reduce the size of the intermediate transmission data.

[0072] (2) The multi-segmentation point dynamic neural network training algorithm designed by the present invention realizes a weight-sharing dynamic neural network that supports multiple compression parameter configurations and can adapt to different network environments without model switching, greatly improving the flexibility and adaptability of the system;

[0073] (3) The feature data dynamic compression method proposed in the present invention significantly reduces the amount of intermediate transmission feature data through multi-dimensional compression technology, effectively alleviating the network pressure in DNN task division.

[0074] (4) The present invention comprehensively considers multiple factors such as segmentation points, feature data compression, edge computing / communication resources, etc., and provides a comprehensive optimization strategy for DNN task division and resource allocation in single-side multi-terminal scenarios, significantly improving the performance and service quality of the entire system.

[0075] (5) The optimization problem of maximizing the sum of multi-user inference accuracy constructed by the present invention minimizes the impact of compression on inference accuracy while meeting the user's end-to-end latency and inference throughput requirements, thereby achieving an effective trade-off between accuracy, latency, and throughput.

[0076] (6) The problem simplification and decomposition method proposed in the present invention transforms the complex multivariable coupling optimization problem into a two-dimensional resource allocation problem, which greatly reduces the difficulty of solving the problem and improves the practicality of the system.

[0077] (7) The task division and resource allocation algorithm based on dynamic programming designed by the present invention provides an optimal reasoning configuration for each user, achieving a balance between computing / communication resources and multiple users.

[0078] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic diagram of a task division and resource allocation scenario based on model segmentation in the present invention;

[0080] Figure 2 It is a flowchart of task division and resource allocation based on model segmentation in the present invention;

[0081] Figure 3 It is the specific structure of the intermediate feature data compression module for model segmentation in the present invention;

[0082] Figure 4 It is a diagram of a multi-segmentation point dynamic neural network training algorithm in the present invention;

[0083] Figure 5 It is an algorithm diagram of edge collaborative task division and resource allocation based on dynamic programming in the present invention. DETAILED DESCRIPTION

[0084] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0085] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0086] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0087] like Figure 1-Figure 3 As shown, a method for edge collaborative task division and resource allocation based on model segmentation specifically includes the following steps:

[0088] Step S1: Based on the end-to-end inference latency, inference throughput, and accuracy in edge collaborative inference tasks, a comprehensive system model is established in combination with different inference configurations of users; the edge collaborative inference task system modeling comprehensively establishes the models of end-to-end inference latency, inference throughput, and inference accuracy in the inference task, providing theoretical model support for the formation of optimization problems and the design of scheduling algorithms;

[0089] In step S1, the inference configuration Ψ of user i is defined i By model split point s i , intermediate feature channel compression parameter α i , the length and width compression parameter of the intermediate feature β i , intermediate feature quantization width q i The number of discrete computing resources allocated to the edge server r i , the number of discrete communication resources allocated to the edge server b i The system consists of three key variables, and models the three key system performance indicators: end-to-end inference latency, inference throughput, and inference accuracy.

[0090] The end-to-end inference latency model is built by the following steps:

[0091] End-to-end inference latency of user i Includes: End-user inference latency Intermediate characteristic transmission delay Edge server inference latency The three parts are specifically expressed as follows:

[0092]

[0093] End-user inference latency Affected by the split point and the computing power of the terminal device, an offline measurement method is used. In the preparation stage, the user i is measured and recorded at each possible split point s i The actual inference delay under Reference value of

[0094] Intermediate characteristic transmission delay Affected by the size of the transmitted data and the communication resources allocated by the edge server, the specific expression is as follows:

[0095]

[0096] in, is the split point s i The original uncompressed intermediate feature data size is as follows; is the product of the corresponding compression parameters, which means that the size of the transmitted feature data is compressed from the original [C×H×W×32bit] to [αC×βH×βW×qbit]; B is the total number of communication resources on the edge server; Ω B is the total bandwidth size on the edge server, The communication bandwidth size allocated to user i by the edge server;

[0097] Edge server inference latency Affected by the split point and the computing resources allocated to the edge server, the specific expression is as follows:

[0098]

[0099] Among them, R is the total number of computing resources on the edge server, r i is the number of computing resources allocated to the i-th user, and the ratio of the number of threads allocated by the edge server to the i-th user inference process to the total number of active threads is s i is the split point;

[0100] Use the following formula function Fit the measured data points to obtain the split point s i The function parameters below and Establish the relationship between edge server inference latency and computing resources:

[0101]

[0102] For the actual application scenario of edge servers using GPUs for model inference, on Nvidia GPU devices, the MPS technology provided by Nvidia is used to allocate the computing resources used by users; MPS technology allocates the active threads on a single GPU device to the model inference process of a specified user according to a percentage.

[0103] In the edge collaborative reasoning system based on model segmentation, the user's reasoning task is divided into three parallel running parts: head network reasoning, intermediate feature transmission and tail network reasoning. The three parallel running parts correspond to the user device reasoning delay, intermediate feature transmission delay and edge server reasoning delay respectively. Based on the principle of pipeline parallelism, the reasoning throughput of user i in the current time slice is determined by the largest delay among the user device reasoning delay, intermediate feature transmission delay and edge server reasoning delay:

[0104]

[0105] Among them, Δt is the length of the current time slice, Ψ i A collection of variables for user inference configuration.

[0106] In the inference accuracy model, the accuracy of all compression parameter settings is comprehensively measured, and the corresponding Pareto set P is constructed based on the scatter plot of the transmission data size and the corresponding inference accuracy. i :

[0107]

[0108] In the formula, is the product of the corresponding compression parameters, A i (s i ,α i ,β i ,q i ) is the split point s i , compression parameter α i ,β i ,q i The corresponding inference accuracy is for the elements in the Pareto set (s i ,α i ,β i ,q i ), there is no other group (si ,α j ,β j ,q j ), making the compression ratio Less than At the same time, the accuracy A i (s i ,α j ,β j ,q j ) is greater than A i (s i ,α i ,β i ,q i );

[0109] P i It represents the configuration set with the least impact on accuracy among all compression parameter configurations. Each point on the configuration set represents the highest accuracy achieved under a specific transmission data size.

[0110] Step S2: Intermediate feature data compression for model segmentation: Based on the edge collaborative task division of model segmentation, the neural network model is divided into a head model and a tail model along a certain segmentation point. The head model is deployed on the end user, and the tail model is deployed on the edge server. The feature data compression module is injected between the head model and the tail model; according to the multi-segmentation point dynamic neural network training algorithm, a parameter-sharing dynamic neural network that can be configured with multiple compression parameters is obtained, realizing real-time switching of the feature data compression amplitude;

[0111] In step S2, the output of the head model is compressed at the end user, and the compressed data is restored on the edge server as the input of the tail model;

[0112] The feature data compression module compresses the intermediate transmission features from [C×H×W×32bit] to [αC×βH×βW×qbit] at the end user, and then restores them to the original size of [C×H×W×32bit] on the edge server, where C is the number of channels of the intermediate transmission features, H and W are the height and width of the intermediate transmission features respectively, and 32bit means that the intermediate transmission features are stored using 32-bit floating point numbers; α is the channel compression parameter, β is the length and width compression parameter, and q is the quantization width;

[0113] The compression part is based on three feature data compression methods: CNN channel pruning, bilinear interpolation downsampling, and learnable affine quantization. It compresses the intermediate transmission features output by the head model from multiple dimensions to reduce the size of the intermediate transmission data. The specific methods are as follows:

[0114] CNN channel pruning: responsible for compressing the number of channels C of intermediate transmission features, denoted by the channel compression parameter α, and inputting the intermediate transmission features into the point convolution layer. Assuming that the weight matrix of the point convolution layer is in the shape of [C,C,1,1], the weight matrix is ​​split and the weight matrix of the shape of [αC,C,1,1] is taken for convolution calculation. The number of output channels of the point convolution layer is pruned from the original C to αC, thus compressing the number of channels of intermediate transmission features.

[0115] Bilinear interpolation downsampling: responsible for compressing the length and width (H and W) of the intermediate transmission features. The compression parameters of H and W are β. The bilinear interpolation method that supports arbitrary numerical scaling is used to downsample the intermediate transmission features.

[0116] Learnable affine quantization: compresses the number of bits of data of intermediate transmission features. The quantization width is q. The intermediate transmission features will be compressed from 32-bit floating point numbers x to q-bit discrete integers x. q , the specific formula is as follows:

[0117]

[0118] where x min 、x max Corresponding to the maximum and minimum values ​​of the intermediate transmission characteristics respectively; Q p is the maximum value of a q-bit discrete integer, Q p =2 q -1; the clamp function is a clipping function that clips the function input to 0 to Q p Within the range; s is a learnable parameter of the neural network, which is learned along with the entire neural network and is used to adjust the Δ used in quantization; The function is the Round function, which is responsible for rounding its function input to an integer;

[0119] The recovery part performs the inverse operation of the compression part in sequence to restore the intermediate feature data to its original size. The specific steps are as follows:

[0120] S21: The edge server dequantizes the received feature data according to the formula Rewrite the q-bit discrete integer x q Restore to 32-bit floating point

[0121] S22: Restore the length and width of the feature data through bilinear interpolation upsampling recovery and depth-separable convolution layer;

[0122] S23: restore the number of channels of feature data through the point convolution layer;

[0123] S24: Insert a switchable batch normalization layer before the tail model to correct the data distribution of the tail model input.

[0124] Step S3: According to the multi-segmentation point dynamic neural network training algorithm, a parameter-sharing dynamic neural network using multiple compression parameter configurations is obtained, and multi-segmentation point dynamic neural network training is performed; user demand constraints and edge resource constraints are combined to form a discrete optimization problem with the goal of maximizing accuracy;

[0125] The sandwich rule proposed by the research on dynamic neural network is followed, that is, the performance of all subnetworks in the dynamic neural network is constrained between the minimum subnetwork and the maximum subnetwork. The present invention also divides all subnetworks in the neural network into three categories: maximum, minimum and intermediate. Specifically, (1) the maximum subnetwork refers to the original reasoning path in the neural network, which does not pass through the compression-recovery module at the split point, and does not perform any compression on the intermediate feature data. It is the subnetwork with the strongest reasoning performance in the entire neural network; (2) the minimum subnetwork refers to the subnetwork that uses the minimum intermediate transmission feature at each split point. It has the highest degree of information compression and is naturally the subnetwork with the worst reasoning performance at each split point; (3) the intermediate subnetwork refers to the subnetwork that uses the random sampling compression configuration parameters for reasoning at each split point. Combining the sandwich rule with the multi-split point scenario of actual application, in each iteration, the minimum subnetwork and the intermediate subnetwork are sampled for each split point, and they are trained together with the maximum subnetwork, so as to ensure the accuracy of the trained neural network under various compression parameter configurations.

[0126] It is found that if the largest subnet is used as the teacher network to distill the remaining subnets, the entire network will often become unstable during the training process, affecting the final reasoning performance of each subnet. This is because the entire weight-shared neural network will continuously update its weights in each iteration, causing the output generated by the largest subnet between different batches in the same epoch to be unstable, and therefore it is not suitable for distilling the remaining subnets as soft labels. Based on this, in step S3, an EMA network is used as a shadow network to generate the soft labels used for distillation. The weights of the EMA network will be updated at the end of each epoch based on the weights of the currently trained network, as shown below:

[0127] W′ τ =ηW τ +(1-η)W′ τ-1 ;

[0128] Where η is the smoothing coefficient, W' τ is the weight of the EMA network at the τth epoch, and W τ is the weight of the trained network at the τth epoch;

[0129] like Figure 4 As shown, the training process of the multi-segmentation point dynamic neural network is divided into two stages. In the first stage, due to the soft labels used in the distillation Inaccurate, only the largest subnet is trained; in the second stage, in addition to training the largest subnet, the corresponding smallest subnet and the middle subnet are sampled at the split point, and soft labels are generated through the EMA network As the teacher network, the smallest subnet and the intermediate subnet are distilled.

[0130] Step S4: Based on the end-to-end inference latency model, inference throughput model and inference accuracy model established in step S1, with the help of the dynamic compression of feature data proposed in step S2, a discrete integer programming problem for maximizing inference accuracy is constructed to achieve a trade-off between accuracy, latency and edge computing or communication resources; in step S4, the following discrete integer programming problem for maximizing inference accuracy is constructed:

[0131]

[0132] Among them, the objective function is to maximize the inference accuracy A of N users i The sum of C1 and C2 is the user demand constraint, which represents the upper limit of end-to-end delay. The lower bound on inference throughput C3 and C4 are the computing or communication resource constraints on the edge server, and the total number of resource shares R and the total number of communication resource shares B are calculated respectively. i With b i are all discrete integer variables; constraint C5 means that the user's segmentation point and compression parameter configuration are selected from the Pareto set.

[0133] Step S5: Convert the discrete integer programming problem from a single-side multi-terminal scenario to a single user sub-problem in a single-side single-terminal scenario; in step S5, the original problem For user i only, given computing resources r i and communication resources i , original question The scenario is transformed from a single-side multi-end scenario to a single-side single-end scenario, broken down into simplified sub-problems

[0134]

[0135] Among them, the sub-problem The objective function in only considers maximizing the accuracy of user i. Constraints C1 and C2 are end-to-end delay constraints and inference throughput constraints, and constraint C3 is a Pareto set constraint. At this time, the subproblem traverses the corresponding Pareto point set to solve. Given a computing resource r i and communication resourcesi Under this condition, the maximum inference accuracy that user i can achieve is The corresponding inference configuration used is

[0136] Calculate each type (r i ,b i ) The corresponding binary At this time, the original question is transformed into a two-dimensional resource allocation problem As shown below:

[0137]

[0138] The optimization variable only considers (r i ,b i ) tuple, user's (s i ,α i ,β i ,q i ) in advance calculation In the process, constraints C1 and C2 indicate that the total resources allocated to users do not exceed the edge resource limit.

[0139] Step S6: Based on the optimal substructure properties of the two-dimensional resource allocation problem, a dynamic programming algorithm is used to obtain the optimal solution to perform edge collaborative task division and resource allocation.

[0140] In step S6, for the dimension resource allocation problem Based on the optimal substructure properties, a dynamic programming algorithm is used to obtain the optimal solution, perform edge-end collaborative task division and resource allocation, and achieve optimal scheduling of system task division and resource allocation. Specifically, it includes:

[0141] Define the first k users to use R k Computing resources and B k Communication resources, the maximum accuracy that can be achieved is P k (R k ,B k ), there is a state transfer equation as follows:

[0142]

[0143] in P is the maximum inference accuracy that the kth user can achieve under given computing resources and communication resources y, k-1 (R k -x,B k -y) Use R for the first k-1 users k -x computing resources and B k-yThe maximum accuracy that can be achieved by the communication resources, defined as the first k users using R k Computing resources and B k When the communication resources are renew Perform optimal edge-end collaborative task division and resource allocation:

[0144]

[0145] in, Use x for the kth user * Computing resources and * The optimal user reasoning configuration of communication resources is calculated The records are obtained in the process. Figure 5 As shown, the optimal solution to the original problem P1 is obtained, and its time complexity is O(NR 2 B 2 ).

[0146] Therefore, the present invention adopts the above-mentioned model segmentation-based edge collaborative task division and resource allocation method, trains a parameter-sharing dynamic neural network based on the sandwich rule and knowledge distillation technology, and realizes real-time switching of the feature data compression amplitude; models the user reasoning performance indicators in the edge collaborative task division, and constructs an optimization problem that maximizes the reasoning accuracy that meets the user performance requirement constraints and edge resource constraints; adopts a dynamic programming-based solution algorithm to select the optimal model segmentation point and intermediate feature compression parameters for the user, and at the same time efficiently allocates edge communication resources and computing resources to improve its reasoning accuracy under multi-user conditions.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for edge collaborative task division and resource allocation based on model segmentation, characterized by: The specific steps include: Step S1: Establish a comprehensive system model for the end-to-end inference latency, inference throughput, and accuracy in edge collaborative inference tasks, combined with different inference configurations of users; Step S2: Intermediate feature data compression for model segmentation: Based on the edge collaborative task division of model segmentation, the neural network model is divided into a head model and a tail model along a certain segmentation point. The head model is deployed on the end user, and the tail model is deployed on the edge server. The feature data compression module is injected between the head model and the tail model. Step S3: According to the multi-segmentation point dynamic neural network training algorithm, a parameter-sharing dynamic neural network configured with multiple compression parameters is obtained, and multi-segmentation point dynamic neural network training is performed; Combining user demand constraints with edge resource constraints, a discrete optimization problem with the goal of maximizing accuracy is formed; Step S4: Based on the end-to-end inference latency model, inference throughput model, and inference accuracy model established in step S1, and with the help of the dynamic compression of feature data proposed in step S2, a discrete integer programming problem for maximizing inference accuracy is constructed to achieve a trade-off between accuracy, latency, and edge computing or communication resources. Step S5: converting the discrete integer programming problem from a single-side multi-terminal scenario into a single user sub-problem in a single-side single-terminal scenario; Step S6: Based on the optimal substructure properties of the two-dimensional resource allocation problem, a dynamic programming algorithm is used to obtain the optimal solution to perform edge collaborative task division and resource allocation.

2. According to the method of edge collaborative task division and resource allocation based on model segmentation as described in claim 1, it is characterized by: In step S1, the inference configuration Ψ of user i is defined i By model split point s i , intermediate feature channel compression parameter α i , the length and width compression parameter of the intermediate feature β i , intermediate feature quantization width q i The number of discrete computing resources allocated to the edge server r i , the number of discrete communication resources allocated to the edge server b i The system consists of three key variables, and models the three key system performance indicators: end-to-end inference latency, inference throughput, and inference accuracy.

3. According to the method of edge collaborative task division and resource allocation based on model segmentation as described in claim 2, it is characterized by: The end-to-end inference latency model is built by the following steps: End-to-end inference latency of user i Includes: End-user inference latency Intermediate characteristic transmission delay Edge server inference latency The three parts are specifically expressed as follows: End-user inference latency Affected by the split point and the computing power of the terminal device, an offline measurement method is used. In the preparation stage, the user i is measured and recorded at each possible split point s i The actual inference delay under Reference value of Intermediate characteristic transmission delay Affected by the size of the transmitted data and the communication resources allocated by the edge server, the specific expression is as follows: in, is the split point s i The original uncompressed intermediate feature data size is as follows; is the product of the corresponding compression parameters, which means that the size of the transmitted feature data is compressed from the original [C×H×W×32bit] to [αC×βH×βW×qbit]; B is the total number of communication resources on the edge server; Ω B is the total bandwidth size on the edge server, The communication bandwidth size allocated to user i by the edge server; Edge server inference latency Affected by the split point and the computing resources allocated to the edge server, the specific expression is as follows: Among them, R is the total number of computing resources on the edge server, r i is the number of computing resources allocated to the i-th user, and the ratio of the number of threads allocated by the edge server to the i-th user inference process to the total number of active threads is s i is the split point; Use the following formula function Fit the measured data points to obtain the split point s i The function parameters below and Establish the relationship between edge server inference latency and computing resources: For the actual application scenario of edge servers using GPUs for model inference, on Nvidia GPU devices, the MPS technology provided by Nvidia is used to allocate the computing resources used by users; MPS technology allocates the active threads on a single GPU device to the model inference process of a specified user according to a percentage.

4. According to the model segmentation-based edge collaborative task division and resource allocation method of claim 2, it is characterized by: In the edge collaborative reasoning system based on model segmentation, the user's reasoning task is divided into three parallel running parts: head network reasoning, intermediate feature transmission and tail network reasoning. The three parallel running parts correspond to the user device reasoning delay, intermediate feature transmission delay and edge server reasoning delay respectively. Based on the principle of pipeline parallelism, the reasoning throughput of user i in the current time slice is determined by the largest delay among the user device reasoning delay, intermediate feature transmission delay and edge server reasoning delay: Among them, Δt is the length of the current time slice, Ψ i A collection of variables for user inference configuration.

5. According to the method of edge collaborative task division and resource allocation based on model segmentation as described in claim 2, it is characterized by: In the inference accuracy model, the accuracy of all compression parameter settings is comprehensively measured, and the corresponding Pareto set P is constructed based on the scatter plot of the transmission data size and the corresponding inference accuracy. i : In the formula, is the product of the corresponding compression parameters, A i (s i ,α i ,β i ,q i ) is the split point s i , compression parameter α i ,β i ,q i The corresponding inference accuracy is for the elements in the Pareto set (s i ,α i ,β i ,q i ), there is no other group (s i ,α j ,β j ,q j ), making the compression ratio Less than At the same time, the accuracy A i (s i ,α j ,β j ,q j ) is greater than A i (s i ,α i ,β i ,q i ); P i It represents the configuration set with the least impact on accuracy among all compression parameter configurations. Each point on the configuration set represents the highest accuracy achieved under a specific transmission data size.

6. According to the method of edge collaborative task division and resource allocation based on model segmentation as claimed in claim 1, it is characterized by: In step S2, the output of the head model is compressed at the end user, and the compressed data is restored on the edge server as the input of the tail model; The feature data compression module compresses the intermediate transmission features from [C×H×W×32bit] to [αC×βH×βW×qbit] at the end user, and then restores them to the original size of [C×H×W×32bit] on the edge server, where C is the number of channels of the intermediate transmission features, H and W are the height and width of the intermediate transmission features respectively, and 32bit means that the intermediate transmission features are stored using 32-bit floating point numbers; α is the channel compression parameter, β is the length and width compression parameter, and q is the quantization width; The compression part is based on three feature data compression methods: CNN channel pruning, bilinear interpolation downsampling, and learnable affine quantization. The intermediate transmission features output by the head model are compressed from multiple dimensions. The specific methods are as follows: CNN channel pruning: responsible for compressing the number of channels C of intermediate transmission features, denoted by the channel compression parameter α, and inputting the intermediate transmission features into the point convolution layer. Assuming that the weight matrix of the point convolution layer is in the shape of [C,C,1,1], the weight matrix is ​​split and the weight matrix of the shape of [αC,C,1,1] is taken for convolution calculation. The number of output channels of the point convolution layer is pruned from the original C to αC, thus compressing the number of channels of intermediate transmission features. Bilinear interpolation downsampling: responsible for compressing the length and width (H and W) of the intermediate transmission features. The compression parameters of H and W are β. The bilinear interpolation method that supports arbitrary numerical scaling is used to downsample the intermediate transmission features. Learnable affine quantization: compresses the number of bits of data of intermediate transmission features. The quantization width is q. The intermediate transmission features will be compressed from 32-bit floating point numbers x to q-bit discrete integers x. q , the specific formula is as follows: where x min 、x max Corresponding to the maximum and minimum values ​​of the intermediate transmission characteristics respectively; Q p is the maximum value of a q-bit discrete integer, Q p =2 q -1; the clamp function is a clipping function that clips the function input to 0 to Q p Within the range; s is a learnable parameter of the neural network, which is learned along with the entire neural network and is used to adjust the Δ used in quantization; The function is the Round function, which is responsible for rounding its function input to an integer; The recovery part performs the inverse operation of the compression part in sequence to restore the intermediate feature data to its original size. The specific steps are as follows: S21: The edge server dequantizes the received feature data according to the formula Rewrite the q-bit discrete integer x q Restore to 32-bit floating point S22: Restore the length and width of the feature data through bilinear interpolation upsampling recovery and depth-separable convolution layer; S23: restore the number of channels of feature data through the point convolution layer; S24: Insert a switchable batch normalization layer before the tail model to correct the data distribution of the tail model input.

7. According to the method of edge collaborative task division and resource allocation based on model segmentation as described in claim 1, it is characterized by: In step S3, an EMA network is used as a shadow network to generate soft labels for distillation. The weights of the EMA network will be updated at the end of each epoch based on the weights of the currently trained network, as shown below: W′ τ =ηW τ +(1-η)W′ τ-1 ; Where η is the smoothing coefficient, W' τ is the weight of the EMA network at the τth epoch, and W τ is the weight of the trained network at the τth epoch; The training process of multi-split point dynamic neural network is divided into two stages. In the first stage, due to the soft labels used in distillation Inaccurate, only the largest subnet is trained; In the second stage, in addition to training the largest subnet, the corresponding smallest subnet and the middle subnet are sampled at the split point, and the soft labels are generated through the EMA network. As the teacher network, the smallest subnet and the intermediate subnet are distilled.

8. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 6, characterized in that: In step S4, the following discrete integer programming problem for maximizing inference accuracy is constructed: Among them, the objective function is to maximize the inference accuracy A of N users i The sum of C1 and C2 is the user demand constraint, which represents the upper limit of end-to-end delay. The lower bound on inference throughput C3 and C4 are the computing or communication resource constraints on the edge server, and the total number of resource shares R and the total number of communication resource shares B are calculated respectively. i With b i are all discrete integer variables; constraint C5 means that the user's segmentation point and compression parameter configuration are selected from the Pareto set.

9. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 8, characterized in that: In step S5, the original question For user i only, given computing resources r i and communication resources i , original question The scenario is transformed from a single-side multi-end scenario to a single-side single-end scenario, broken down into simplified sub-problems Among them, the sub-problem The objective function in only considers maximizing the accuracy of user i. Constraints C1 and C2 are end-to-end delay constraints and inference throughput constraints, and constraint C3 is a Pareto set constraint. At this time, the subproblem traverses the corresponding Pareto point set to solve. Given a computing resource r i and communication resources i Under this condition, the maximum inference accuracy that user i can achieve is The corresponding inference configuration used is Calculate each type (r i ,b i ) The corresponding binary At this time, the original question is transformed into a two-dimensional resource allocation problem As shown below: The optimization variable only considers (r i ,b i ) tuple, user's (s i ,α i ,β i ,q i ) in advance calculation In the process, constraints C1 and C2 indicate that the total resources allocated to users do not exceed the edge resource limit.

10. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 9, characterized in that: In step S6, for the dimension resource allocation problem Based on the optimal substructure properties, a dynamic programming algorithm is used to obtain the optimal solution for edge-end collaborative task division and resource allocation, including: Define the first k users to use R k Computing resources and B k Communication resources, the maximum accuracy that can be achieved is P k (R k ,B k ), there is a state transfer equation as follows: in P is the maximum inference accuracy that the kth user can achieve under given computing resources and communication resources y, k-1 (R k -x,B k -y) Use R for the first k-1 users k -x computing resources and B k -yThe maximum accuracy that can be achieved by the communication resources, defined as the first k users using R k Computing resources and B k When the communication resources are renew Perform optimal edge-end collaborative task division and resource allocation: in, Use x for the kth user * Computing resources and * The optimal user reasoning configuration of communication resources is calculated Record the process and get the original problem The optimal solution has a time complexity of O(NR 2 B 2 ).

Images