A method for edge collaborative task division and resource allocation based on model segmentation

Abstract

The present invention discloses a method for edge collaborative task division and resource allocation based on model segmentation, which relates to the field of edge computing video analysis, and integrates CNN channel pruning, bilinear sampling and affine quantization to multi-dimensionally compress intermediate transmission feature data; a parameter-sharing dynamic neural network is trained based on the sandwich rule and knowledge distillation technology to realize real-time switching of the feature data compression amplitude; user reasoning performance indicators in edge collaborative task division are modeled, and an optimization problem for maximizing reasoning accuracy that meets user performance requirement constraints and edge resource constraints is constructed; a solution algorithm based on dynamic programming is used to select the optimal model segmentation point and intermediate feature compression parameter configuration for the user, and at the same time, edge communication resources and computing resources are efficiently allocated to improve its reasoning accuracy under multi-user conditions.

Description

Technical Field

[0001] The present invention relates to the field of edge computing video analysis, and in particular to a method for edge collaborative task division and resource allocation based on model segmentation. Background Art

[0002] In recent years, with the rapid development of 5G technology and the Industrial Internet of Things (IIoT), the number of mobile smart terminal devices has grown exponentially. As the computing power of terminal devices continues to increase, deep neural network models have been widely deployed on mobile terminal devices, resulting in a proliferation of intelligent applications. However, due to the increasing complexity of the network structure of deep neural network (DNN) models, the demand for computing resources has gradually increased, making it difficult for mobile terminal devices with limited computing power to provide high-precision, low-latency services. This conflict between computing resource requirements and real-time interaction has given rise to the edge computing paradigm. The emergence of edge computing pushes application tasks from cloud centers to the network edge, closer to mobile terminal devices. Data can be analyzed and processed at the edge of the network, where sufficient computing power is available. This can meet key application requirements such as real-time processing and high computational power, thus realizing "edge intelligence."

[0003] When using the edge computing paradigm, a common approach is to upload raw input data directly to edge servers. However, due to the low security of edge servers, uploading raw image data directly to edge servers can lead to user privacy leakage. Furthermore, due to the bandwidth limitations of edge servers, continuously transmitting large amounts of raw image data inevitably causes network congestion.

[0004] To improve the security of user privacy data and fully utilize the computing power of end devices, researchers have proposed an alternative edge-to-edge collaborative inference solution: DNN task partitioning based on model segmentation. The basic idea is to divide the DNN model into two parts along a partition point: a head model processed by the user device and a tail model processed by the edge server. The intermediate feature data output by the head model is uploaded to the edge server and used as input to the tail model for inference execution.

[0005] However, the current model segmentation-based edge collaborative task division solution still has the following challenges:

[0006] (1) The amount of intermediate data transmitted in traditional task division schemes is large, and the uncompressed intermediate data is often larger than the original image. When the network conditions are poor, it is difficult to achieve satisfactory system performance due to the pressure from both the computing power of the terminal equipment and the network bandwidth.

[0007] (2) The task division scheme of modifying the DNN structure to compress intermediate transmission data by introducing compression technologies such as pruning or quantization often faces a dilemma. That is, if no additional training is performed for compression, the flexibility of the segmentation points and compression parameters is retained, but the accuracy cost caused by compression is high, which makes it difficult to meet the accuracy requirements of users; and if additional training is performed for compression, only a single predefined segmentation point and compression parameter can be used, resulting in insufficient adaptability and difficulty in meeting fluctuating network environments and complex user needs.

[0008] (3) The task division schemes that introduce dynamic compression technology based on dynamic neural networks are still limited to a given single segmentation point, ignoring the significant impact of the segmentation point in model segmentation on system performance; on the other hand, they are not sufficient in compressing the dimensions of intermediate features, ignoring the impact of the height and width of the feature map on compression efficiency. Summary of the Invention

[0009] In order to solve the above problems, the present invention provides a model segmentation-based edge collaborative task division and resource allocation method, which improves the processing efficiency and accuracy of deep neural network inference tasks in edge computing systems through efficient feature data compression technology and resource allocation strategies.

[0010] To achieve the above objectives, the present invention provides a method for edge collaborative task division and resource allocation based on model segmentation, which specifically includes the following steps:

[0011] Step S1: Build a comprehensive system model based on the end-to-end inference latency, inference throughput, and accuracy of edge collaborative inference tasks, taking into account different inference configurations of users.

[0012] Step S2: Intermediate feature data compression for model segmentation: Based on the edge collaborative task division of model segmentation, the neural network model is divided into a head model and a tail model along a certain segmentation point. The head model is deployed on the end user, and the tail model is deployed on the edge server. The feature data compression module is injected between the head model and the tail model;

[0013] Step S3: Based on the multi-split point dynamic neural network training algorithm, a parameter-sharing dynamic neural network using multiple compression parameter configurations is obtained, and the multi-split point dynamic neural network training is performed; combining user demand constraints and edge resource constraints to form a discrete optimization problem with the goal of maximizing accuracy;

[0014] Step S4: Based on the end-to-end inference latency model, inference throughput model, and inference accuracy model established in step S1, and with the help of the dynamic compression of feature data proposed in step S2, a discrete integer programming problem is constructed to maximize inference accuracy, achieving a trade-off between accuracy, latency, and edge computing or communication resources.

[0015] Step S5: Convert the discrete integer programming problem from a single-side multi-terminal scenario to a single-user sub-problem in a single-side single-terminal scenario;

[0016] Step S6: Based on the optimal substructure properties of the two-dimensional resource allocation problem, a dynamic programming algorithm is used to obtain the optimal solution and perform edge-end collaborative task division and resource allocation.

[0017] Preferably, in step S1, the inference configuration Ψ of user i is defined i By model split point s i , intermediate feature channel compression parameter α i , intermediate feature length and width compression parameter β i , intermediate feature quantization width q i The number of discrete computing resources allocated to the edge server r i , the number of discrete communication resources allocated to the edge server b i It consists of six key variables and models three key system performance indicators: end-to-end inference latency, inference throughput, and inference accuracy.

[0018] Preferably, the end-to-end inference delay model building specifically includes the following steps:

[0019] End-to-end inference latency of user i Includes: End-user inference latency Intermediate feature transmission delay Edge server inference latency The three parts are specifically expressed as follows:

[0020]

[0021] End-user inference latency Affected by the split point and the computing power of the terminal device, an offline measurement method is used. In the preparation stage, the user i is pre-measured and recorded at each possible split point s. i The actual inference delay under Reference value of

[0022] Intermediate feature transmission delay Affected by the size of the transmitted data and the communication resources allocated by the edge server, it is specifically expressed as follows:

[0023]

[0024] Among them, Ds i is the split point s i The size of the original uncompressed intermediate feature data; is the product of the corresponding compression parameters, which means that the size of the transmitted feature data is compressed from the original [C×H×W×32bit] to [αC×βH×βW×qbit]; B is the total number of communication resources on the edge server; Ω B is the total bandwidth size on the edge server, The communication bandwidth allocated to user i by the edge server;

[0025] Edge server inference latency Affected by the split point and the computing resources allocated to the edge server, it is specifically expressed as follows:

[0026]

[0027] Where R is the total number of computing resources on the edge server, r i is the number of computing resources allocated to the i-th user, and the ratio of the number of threads allocated to the i-th user inference process by the edge server to the total number of active threads is s i is the split point;

[0028] Use the following formula function Fit the measured data points to obtain the split point s i The function parameters below and Establish the relationship between edge server inference latency and computing resources:

[0029]

[0030] For practical application scenarios where edge servers use GPUs for model inference, Nvidia GPU devices use MPS technology provided by Nvidia to allocate computing resources to users. MPS technology allocates active threads on a single GPU device to a specified user's model inference process based on a percentage.

[0031] Preferably, in the edge-to-edge collaborative reasoning system based on model segmentation, the user's reasoning task is divided into three parallel running parts: head network reasoning, intermediate feature transmission, and tail network reasoning. The three parallel running parts correspond to the user device reasoning delay, intermediate feature transmission delay, and edge server reasoning delay, respectively. Based on the principle of pipeline parallelism, the reasoning throughput of user i in the current time slice is determined by the largest delay among the user device reasoning delay, intermediate feature transmission delay, and edge server reasoning delay:

[0032]

[0033] Among them, Δt is the length of the current time slice, Ψ i Configure a collection of variables for user inference.

[0034] Preferably, in the inference accuracy model, the accuracy of all compression parameter settings is comprehensively measured, and the corresponding Pareto set P is constructed based on the scatter plot of the transmitted data size and the corresponding inference accuracy. i :

[0035]

[0036] Where, is the product of the corresponding compression parameters, A i (s i ,α i ,β i ,q i ) is the split point s i , compression parameter α i ,β i ,q i The corresponding inference accuracy is, for the elements in the Pareto set (s i ,α i ,β i ,q i ), there is no other group (s i ,α j ,β j ,q j ), making the compression ratio Less than At the same time, the accuracy A i (s i ,α j ,β j ,q j ) is greater than A i (s i ,α i ,β i ,q i );

[0037] P i It represents the configuration set with the least impact on accuracy among all compression parameter configurations. Each point on the configuration set represents the highest accuracy achieved under a specific transmission data size.

[0038] Preferably, in step S2, the output of the head model is compressed at the end user, and the compressed data is restored on the edge server as the input of the tail model;

[0039] The feature data compression module compresses the intermediate transmission features from [C × H × W × 32 bits] to [αC × βH × βW × q bits] at the end user, and then restores them to the original size of [C × H × W × 32 bits] on the edge server, where C is the number of channels of the intermediate transmission feature, H and W are the height and width of the intermediate transmission feature, respectively, and 32 bits means that the intermediate transmission feature is stored using 32-bit floating point numbers; α is the channel compression parameter, β is the length and width compression parameter, and q is the quantization width;

[0040] The compression part is based on three feature data compression methods: CNN channel pruning, bilinear interpolation downsampling, and learnable affine quantization. It compresses the intermediate transmission features output by the head model from multiple dimensions. The specific methods are as follows:

[0041] CNN channel pruning: It is responsible for compressing the number of channels C of intermediate transmission features. The channel compression parameter is α. The intermediate transmission features are input into the point convolution layer. Assuming that the weight matrix of the point convolution layer is in the shape of [C, C, 1, 1], the weight matrix is ​​split and the weight matrix of the shape of [αC, C, 1, 1] is taken for convolution calculation. The number of output channels of the point convolution layer is pruned from the original C to αC, thus compressing the number of channels of intermediate transmission features.

[0042] Bilinear interpolation downsampling: responsible for compressing the length and width (H and W) of the intermediate transmission features. The compression parameters of H and W are β. Bilinear interpolation method that supports arbitrary numerical scaling is used to downsample the intermediate transmission features.

[0043] Learnable affine quantization: compresses the number of bits of data for intermediate transmission features. The quantization width is q. The intermediate transmission features will be compressed from 32-bit floating point numbers x to q-bit discrete integers x. q , the specific formula is as follows:

[0044]

[0045] where x min 、x max Corresponding to the maximum and minimum values ​​of the intermediate transmission characteristics respectively; Q p is the maximum value of a q-bit discrete integer, Q p =2 q -1; the clamp function is a clipping function that clips the function input to 0 to Q p Within the range; s is a learnable parameter of the neural network, which is learned along with the entire neural network and is used to adjust the Δ used in quantization; The function is the Round function, which is responsible for rounding its function input to an integer;

[0046] The recovery part performs the inverse operation of the compression part in sequence to restore the intermediate feature data to its original size. The specific steps are as follows:

[0047] S21: The edge server dequantizes the received feature data according to the formula Re-transform the q-bit discrete integer x q Restore to 32-bit floating point

[0048] S22: Restore the length and width of the feature data through bilinear interpolation upsampling and depth-separable convolutional layers;

[0049] S23: Recover the number of channels of feature data through the point convolution layer;

[0050] S24: Insert a switchable batch normalization layer before the tail model to correct the data distribution of the tail model input.

[0051] Preferably, in step S3, an EMA network is used as a shadow network to generate soft labels for distillation. The weights of the EMA network are updated at the end of each epoch based on the weights of the currently trained network, as shown below:

[0052] W′ τ =ηW τ +(1-η)W′ τ-1 ;

[0053] Where η is the smoothing coefficient, W' τ is the weight of the EMA network at the τth epoch, and W τ is the weight of the trained network at the τth epoch;

[0054] The training process of multi-split point dynamic neural network is divided into two stages. In the first stage, due to the soft labels used in distillation Inaccurate, only the largest subnet is trained; in the second stage, in addition to training the largest subnet, the corresponding smallest subnet and the intermediate subnet are sampled at the split point, and soft labels are generated through the EMA network. The smallest subnet and the intermediate subnet are distilled as the teacher network.

[0055] Preferably, in step S4, the following discrete integer programming problem for maximizing inference accuracy is constructed:

[0056]

[0057] Among them, the objective function is to maximize the inference accuracy A of N users i The sum of C1 and C2 is the user demand constraint, which represents the upper limit of end-to-end delay. and the lower bound on inference throughput C3 and C4 are the computing or communication resource constraints on the edge server, respectively calculating the total number of resource shares R and the total number of communication resource shares B, r i with b i are all discrete integer variables; constraint C5 indicates that the user's split point and compression parameter configuration are selected from the Pareto set.

[0058] Preferably, in step S5, the original question For user i only, given computing resources r i and communication resources b i , original question Transform the unilateral multi-end scenario into a unilateral single-end scenario, breaking it down into simplified sub-problems.

[0059]

[0060] Among them, the sub-problem The objective function in only considers maximizing the accuracy of user i. Constraints C1 and C2 are end-to-end delay constraints and inference throughput constraints, and constraint C3 is a Pareto set constraint. At this time, the subproblem is solved by traversing the corresponding Pareto point set. Given a computing resource r i and communication resources b i Under this condition, the maximum inference accuracy that user i can achieve is The corresponding inference configuration used is

[0061] Calculate each type (r i ,b i ) The corresponding pair At this time, the original question is transformed into a two-dimensional resource allocation problem As shown below:

[0062]

[0063] The optimization variables only consider (r i ,b i ) tuple, user's (s i ,α i ,β i ,q i ) in advance calculation In the process, constraints C1 and C2 indicate that the total resources allocated to users do not exceed the edge resource limit.

[0064] Preferably, in step S6, for the dimension resource allocation problem Based on the optimal substructure properties, a dynamic programming algorithm is used to find the optimal solution for edge-end collaborative task division and resource allocation, specifically including:

[0065] Define the top k users using R k Computing resources and B k Communication resources, the maximum accuracy that can be achieved is P k (R k ,B k ), the state transition equation is as follows:

[0066]

[0067] in P is the maximum inference accuracy that the kth user can achieve under given computing resources and communication resources y, k -1(R k -x,B k -y) Use R for the first k-1 users k -x computing resources and B k -yThe maximum accuracy that can be achieved by communication resources, defined as the sum of the first k users using R k Computing resources and B k When communication resources are used, the optimal inference configuration for all users is renew Perform optimal edge-to-edge collaborative task division and resource allocation:

[0068]

[0069] in, Use x for the kth user * Computing resources and * The optimal user reasoning configuration of communication resources is calculated Record the process and solve the original problem The optimal solution has a time complexity of O(NR 2 B 2 ).

[0070] Therefore, the present invention adopts the above-mentioned edge collaborative task division and resource allocation method based on model segmentation, which has the following beneficial effects:

[0071] (1) The present invention combines CNN channel pruning, bilinear interpolation sampling and affine quantization to perform multi-dimensional and comprehensive compression on the channel dimension, height and width dimension and data bit number of the intermediate transmission features, so as to significantly reduce the size of the intermediate transmission data.

[0072] (2) The multi-split point dynamic neural network training algorithm designed by the present invention realizes a weight-sharing dynamic neural network that supports multiple compression parameter configurations and can adapt to different network environments without model switching, greatly improving the flexibility and adaptability of the system;

[0073] (3) The dynamic compression method of feature data proposed in this invention significantly reduces the amount of intermediate transmission feature data through multi-dimensional compression technology, effectively alleviating the network pressure in DNN task division.

[0074] (4) The present invention comprehensively considers multiple factors such as segmentation points, feature data compression, edge computing / communication resources, etc., and provides a comprehensive optimization strategy for DNN task division and resource allocation in single-side multi-terminal scenarios, significantly improving the performance and service quality of the entire system.

[0075] (5) The present invention constructs an optimization problem for maximizing the sum of multi-user inference accuracy, which minimizes the impact of compression on inference accuracy while meeting the end-to-end latency and inference throughput requirements of users, thus achieving an effective trade-off between accuracy, latency, and throughput.

[0076] (6) The problem simplification and decomposition method proposed in the present invention transforms the complex multivariable coupling optimization problem into a two-dimensional resource allocation problem, which greatly reduces the difficulty of solving the problem and improves the practicality of the system.

[0077] (7) The task division and resource allocation algorithm based on dynamic programming designed by the present invention provides each user with the optimal reasoning configuration, achieving a balance between computing / communication resources and multiple users.

[0078] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic diagram of the task division and resource allocation scenario based on model segmentation in the present invention;

[0080] Figure 2 This is a flowchart of task division and resource allocation based on model segmentation in the present invention;

[0081] Figure 3 This is the specific structure of the intermediate feature data compression module for model segmentation in the present invention;

[0082] Figure 4 It is a diagram of the multi-segmentation point dynamic neural network training algorithm in the present invention;

[0083] Figure 5 It is the edge collaborative task division and resource allocation algorithm diagram based on dynamic programming in the present invention. DETAILED DESCRIPTION

[0084] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0085] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0086] The words “include” or “comprising” and similar words used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms “inside”, “outside”, “upper”, “lower”, etc. is based on the orientation or position relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly stipulated and limited, the terms such as “attachment” should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral whole; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0087] like Figure 1-Figure 3 As shown in FIG, a method for task division and resource allocation of edge-end collaboration based on model segmentation includes the following steps:

[0088] Step S1: Build a comprehensive system model for the end-to-end inference latency, inference throughput, and accuracy of edge collaborative inference tasks, taking into account different user inference configurations. This system model comprehensively establishes models for the end-to-end inference latency, inference throughput, and inference accuracy of edge collaborative inference tasks, providing theoretical model support for the formulation of optimization problems and the design of scheduling algorithms.

[0089] In step S1, define the inference configuration Ψ of user i i By model split point s i , intermediate feature channel compression parameter α i , intermediate feature length and width compression parameter β i , intermediate feature quantization width q i The number of discrete computing resources allocated to the edge server r i , the number of discrete communication resources allocated to the edge server b i It consists of six key variables and models three key system performance indicators: end-to-end inference latency, inference throughput, and inference accuracy.

[0090] The end-to-end inference latency model is built by the following steps:

[0091] End-to-end inference latency of user i Includes: End-user inference latency Intermediate feature transmission delay Edge server inference latency The three parts are specifically expressed as follows:

[0092]

[0093] End-user inference latency Affected by the split point and the computing power of the terminal device, an offline measurement method is used. In the preparation stage, the user i is pre-measured and recorded at each possible split point s. i The actual inference delay under Reference value of

[0094] Intermediate feature transmission delay Affected by the size of the transmitted data and the communication resources allocated by the edge server, it is specifically expressed as follows:

[0095]

[0096] in, is the split point s i The size of the original uncompressed intermediate feature data; is the product of the corresponding compression parameters, which means that the size of the transmitted feature data is compressed from the original [C×H×W×32bit] to [αC×βH×βW×qbit]; B is the total number of communication resources on the edge server; Ω B is the total bandwidth size on the edge server, The communication bandwidth allocated to user i by the edge server;

[0097] Edge server inference latency Affected by the split point and the computing resources allocated to the edge server, it is specifically expressed as follows:

[0098]

[0099] Where R is the total number of computing resources on the edge server, r i is the number of computing resources allocated to the i-th user, and the ratio of the number of threads allocated to the i-th user inference process by the edge server to the total number of active threads is s i is the split point;

[0100] Use the following formula function Fit the measured data points to obtain the split point s i The function parameters below and Establish the relationship between edge server inference latency and computing resources:

[0101]

[0102] For practical application scenarios where edge servers use GPUs for model inference, Nvidia GPU devices use MPS technology provided by Nvidia to allocate computing resources to users. MPS technology allocates active threads on a single GPU device to a specified user's model inference process based on a percentage.

[0103] In the edge-to-edge collaborative inference system based on model segmentation, the user's inference task is divided into three parallel running parts: head network inference, intermediate feature transmission, and tail network inference. The three parallel running parts correspond to the user device inference latency, intermediate feature transmission latency, and edge server inference latency, respectively. Based on the principle of pipeline parallelism, the inference throughput of user i in the current time slice is determined by the largest delay among the user device inference latency, intermediate feature transmission latency, and edge server inference latency:

[0104]

[0105] Among them, Δt is the length of the current time slice, Ψ i Configure a collection of variables for user inference.

[0106] In the inference accuracy model, the accuracy of all compression parameter settings is comprehensively measured. Based on the scatter plot of the transmitted data size and the corresponding inference accuracy, the corresponding Pareto set P is constructed. i :

[0107]

[0108] Where, is the product of the corresponding compression parameters, A i (s i ,α i ,β i ,q i ) is the split point s i , compression parameter α i ,β i ,q i The corresponding inference accuracy is, for the elements in the Pareto set (s i ,α i ,β i ,q i ), there is no other group (si ,α j ,β j ,q j ), making the compression ratio Less than At the same time, the accuracy A i (s i ,α j ,β j ,q j ) is greater than A i (s i ,α i ,β i ,q i );

[0109] P i It represents the configuration set with the least impact on accuracy among all compression parameter configurations. Each point on the configuration set represents the highest accuracy achieved under a specific transmission data size.

[0110] Step S2: Intermediate feature data compression for model segmentation: Based on the edge collaborative task division of model segmentation, the neural network model is divided into a head model and a tail model along a certain segmentation point. The head model is deployed on the end user, and the tail model is deployed on the edge server. The feature data compression module is injected between the head model and the tail model. According to the multi-segmentation point dynamic neural network training algorithm, a parameter-sharing dynamic neural network capable of using multiple compression parameter configurations is obtained, realizing real-time switching of the feature data compression amplitude.

[0111] In step S2, the output of the head model is compressed at the end user, and the compressed data is restored on the edge server as the input of the tail model;

[0112] The feature data compression module compresses the intermediate transmission features from [C × H × W × 32 bits] to [αC × βH × βW × q bits] at the end user, and then restores them to the original size of [C × H × W × 32 bits] on the edge server, where C is the number of channels of the intermediate transmission feature, H and W are the height and width of the intermediate transmission feature, respectively, and 32 bits means that the intermediate transmission feature is stored using 32-bit floating point numbers; α is the channel compression parameter, β is the length and width compression parameter, and q is the quantization width;

[0113] The compression part is based on three feature data compression methods: CNN channel pruning, bilinear interpolation downsampling, and learnable affine quantization. It compresses the intermediate transmission features output by the head model from multiple dimensions to reduce the size of the intermediate transmission data. The specific methods are as follows:

[0114] CNN channel pruning: It is responsible for compressing the number of channels C of intermediate transmission features. The channel compression parameter is α. The intermediate transmission features are input into the point convolution layer. Assuming that the weight matrix of the point convolution layer is in the shape of [C, C, 1, 1], the weight matrix is ​​split and the convolution calculation is performed using the weight matrix of the shape of [αC, C, 1, 1]. The number of output channels of the point convolution layer is pruned from the original C to αC, thus compressing the number of channels of the intermediate transmission features.

[0115] Bilinear interpolation downsampling: responsible for compressing the length and width (H and W) of the intermediate transmission features. The compression parameters of H and W are β. Bilinear interpolation method that supports arbitrary numerical scaling is used to downsample the intermediate transmission features.

[0116] Learnable affine quantization: compresses the number of bits of data for intermediate transmission features. The quantization width is q. The intermediate transmission features will be compressed from 32-bit floating point numbers x to q-bit discrete integers x. q , the specific formula is as follows:

[0117]

[0118] where x min 、x max Corresponding to the maximum and minimum values ​​of the intermediate transmission characteristics respectively; Q p is the maximum value of a q-bit discrete integer, Q p =2 q -1; the clamp function is a clipping function that clips the function input to 0 to Q p Within the range; s is a learnable parameter of the neural network, which is learned along with the entire neural network and is used to adjust the Δ used in quantization; The function is the Round function, which is responsible for rounding its function input to an integer;

[0119] The recovery part performs the inverse operation of the compression part in sequence to restore the intermediate feature data to its original size. The specific steps are as follows:

[0120] S21: The edge server dequantizes the received feature data according to the formula Re-transform the q-bit discrete integer x q Restore to 32-bit floating point

[0121] S22: Restore the length and width of the feature data through bilinear interpolation upsampling and depth-separable convolutional layers;

[0122] S23: Recover the number of channels of feature data through the point convolution layer;

[0123] S24: Insert a switchable batch normalization layer before the tail model to correct the data distribution of the tail model input.

[0124] Step S3: Based on the multi-split point dynamic neural network training algorithm, a parameter-sharing dynamic neural network using multiple compression parameter configurations is obtained, and the multi-split point dynamic neural network training is performed; combining user demand constraints and edge resource constraints to form a discrete optimization problem with the goal of maximizing accuracy;

[0125] Following the sandwich rule proposed by the research on dynamic neural networks, the performance of all subnetworks in a dynamic neural network is constrained between the minimum subnetwork and the maximum subnetwork. The present invention also divides all subnetworks in the neural network into three categories: maximum, minimum, and intermediate. Specifically, (1) the maximum subnetwork refers to the original reasoning path in the neural network, which does not pass through the compression-recovery module at the split point and does not perform any compression on the intermediate feature data. It is the subnetwork with the strongest reasoning performance in the entire neural network; (2) the minimum subnetwork refers to the subnetwork that uses the minimum intermediate transmission feature at each split point. It has the highest degree of information compression and is naturally the subnetwork with the worst reasoning performance at each split point; (3) the intermediate subnetwork refers to the subnetwork that uses randomly sampled compression configuration parameters for reasoning at each split point. Combining the sandwich rule with the multi-split point scenario of actual application, in each iteration, the minimum subnetwork and intermediate subnetwork are sampled for each split point and trained together with the maximum subnetwork, thereby ensuring the accuracy of the trained neural network under various compression parameter configurations.

[0126] We found that using the largest subnet as the teacher network to distill the remaining subnets often leads to instability in the entire network during training, affecting the final inference performance of each subnet. This is because the weight-sharing neural network continuously updates its weights in each iteration, causing the output of the largest subnet to be unstable between different batches within the same epoch. Therefore, it is not suitable for distilling the remaining subnets as soft labels. Based on this, in step S3, an EMA network is used as a shadow network to generate the soft labels used for distillation. The weights of the EMA network are updated at the end of each epoch based on the weights of the currently trained network, as shown below:

[0127] W′ τ =ηW τ +(1-η)W′ τ-1 ;

[0128] Where η is the smoothing coefficient, W' τ is the weight of the EMA network at the τth epoch, and W τ is the weight of the trained network at the τth epoch;

[0129] like Figure 4 As shown, the training process of the multi-split point dynamic neural network is divided into two stages. In the first stage, due to the soft labels used in the distillation Inaccurate, only the largest subnet is trained; in the second stage, in addition to training the largest subnet, the corresponding smallest subnet and the intermediate subnet are sampled at the split point, and soft labels are generated through the EMA network. The smallest subnet and the intermediate subnet are distilled as the teacher network.

[0130] Step S4: Based on the end-to-end inference latency model, inference throughput model, and inference accuracy model established in step S1, and with the help of the dynamic compression of feature data proposed in step S2, a discrete integer programming problem for maximizing inference accuracy is constructed to achieve a trade-off between accuracy, latency, and edge computing or communication resources. In step S4, the following discrete integer programming problem for maximizing inference accuracy is constructed:

[0131]

[0132] Among them, the objective function is to maximize the inference accuracy A of N users i The sum of C1 and C2 is the user demand constraint, which represents the upper limit of end-to-end delay. and the lower bound on inference throughput C3 and C4 are the computing or communication resource constraints on the edge server, respectively calculating the total number of resource shares R and the total number of communication resource shares B, r i with b i are all discrete integer variables; constraint C5 indicates that the user's split point and compression parameter configuration are selected from the Pareto set.

[0133] Step S5: Convert the discrete integer programming problem from a unilateral multi-terminal scenario to a single user sub-problem in a unilateral single-terminal scenario; in step S5, the original problem For user i only, given computing resources r i and communication resources b i , original question Transform the unilateral multi-end scenario into a unilateral single-end scenario, breaking it down into simplified sub-problems.

[0134]

[0135] Among them, the sub-problem The objective function in only considers maximizing the accuracy of user i. Constraints C1 and C2 are end-to-end delay constraints and inference throughput constraints, and constraint C3 is a Pareto set constraint. At this time, the subproblem is solved by traversing the corresponding Pareto point set. Given a computing resource r i and communication resources bi Under this condition, the maximum inference accuracy that user i can achieve is The corresponding inference configuration used is

[0136] Calculate each type (r i ,b i ) The corresponding pair At this time, the original question is transformed into a two-dimensional resource allocation problem As shown below:

[0137]

[0138] The optimization variables only consider (r i ,b i ) tuple, user's (s i ,α i ,β i ,q i ) in advance calculation In the process, constraints C1 and C2 indicate that the total resources allocated to users do not exceed the edge resource limit.

[0139] Step S6: Based on the optimal substructure properties of the two-dimensional resource allocation problem, a dynamic programming algorithm is used to obtain the optimal solution and perform edge-end collaborative task division and resource allocation.

[0140] In step S6, for the dimension resource allocation problem Based on the optimal substructure properties, a dynamic programming algorithm is used to find the optimal solution, perform edge-end collaborative task division and resource allocation, and achieve optimal scheduling of system task division and resource allocation. Specifically, it includes:

[0141] Define the top k users using R k Computing resources and B k Communication resources, the maximum accuracy that can be achieved is P k (R k ,B k ), the state transition equation is as follows:

[0142]

[0143] in P is the maximum inference accuracy that the kth user can achieve under given computing resources and communication resources y, k-1 (R k -x,B k -y) Use R for the first k-1 users k -x computing resources and B k-yThe maximum accuracy that can be achieved by communication resources, defined as the sum of the first k users using R k Computing resources and B k When communication resources are used, the optimal inference configuration for all users is renew Perform optimal edge-to-edge collaborative task division and resource allocation:

[0144]

[0145] in, Use x for the kth user * Computing resources and * The optimal user reasoning configuration of communication resources is calculated The records are obtained in the process of Figure 5 As shown, the optimal solution of the original problem P1 is obtained, and its time complexity is O(NR 2 B 2 ).

[0146] Therefore, the present invention adopts the above-mentioned model segmentation-based edge collaborative task division and resource allocation method, trains a parameter-sharing dynamic neural network based on the sandwich rule and knowledge distillation technology, and realizes real-time switching of the feature data compression amplitude; models the user reasoning performance indicators in the edge collaborative task division, and constructs an optimization problem that maximizes the reasoning accuracy that meets the user performance requirement constraints and edge resource constraints; adopts a dynamic programming-based solution algorithm to select the optimal model segmentation point and intermediate feature compression parameters for the user, and at the same time efficiently allocates edge communication resources and computing resources to improve its reasoning accuracy under multi-user conditions.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A model-segmentation-based edge collaborative task division and resource allocation method, characterized by: The specific steps include: Step S1: Build a comprehensive system model based on the end-to-end inference latency, inference throughput, and accuracy of edge collaborative inference tasks, taking into account different inference configurations of users. Step S2: Intermediate feature data compression for model segmentation: Based on the edge collaborative task division of model segmentation, the neural network model is divided into a head model and a tail model along a certain segmentation point. The head model is deployed on the end user, and the tail model is deployed on the edge server. The feature data compression module is injected between the head model and the tail model; Step S3: According to the multi-segmentation point dynamic neural network training algorithm, a parameter-sharing dynamic neural network configured with multiple compression parameters is obtained, and multi-segmentation point dynamic neural network training is performed; Combining user demand constraints with edge resource constraints, a discrete optimization problem is formulated with the goal of maximizing accuracy. Step S4: Based on the end-to-end inference latency model, inference throughput model, and inference accuracy model established in step S1, and with the help of the dynamic compression of feature data proposed in step S2, a discrete integer programming problem is constructed to maximize inference accuracy, achieving a trade-off between accuracy, latency, and edge computing or communication resources. Step S5: Convert the discrete integer programming problem from a single-side multi-terminal scenario to a single-user sub-problem in a single-side single-terminal scenario; Step S6: Based on the optimal substructure properties of the two-dimensional resource allocation problem, a dynamic programming algorithm is used to obtain the optimal solution and perform edge-end collaborative task division and resource allocation.

2. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 1, characterized in that: In step S1, define the inference configuration Ψ of user i i By model split point s i , intermediate feature channel compression parameter α i , intermediate feature length and width compression parameter β i , intermediate feature quantization width q i The number of discrete computing resources allocated to the edge server r i , the number of discrete communication resources allocated to the edge server b i It consists of six key variables and models three key system performance indicators: end-to-end inference latency, inference throughput, and inference accuracy.

3. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 2, characterized in that: The end-to-end inference latency model is built by the following steps: End-to-end inference latency of user i Includes: End-user inference latency Intermediate feature transmission delay Edge server inference latency The three parts are specifically expressed as follows: End-user inference latency Affected by the split point and the computing power of the terminal device, an offline measurement method is used. In the preparation stage, the user i is pre-measured and recorded at each possible split point s. i The actual inference delay under Reference value of Intermediate feature transmission delay Affected by the size of the transmitted data and the communication resources allocated by the edge server, it is specifically expressed as follows: in, is the split point s i The original uncompressed intermediate feature data size; is the product of the corresponding compression parameters, representing that the size of the transmitted feature data is compressed from the original [C × H × W × 32 bits] to [αC × βH × βW × q bits]; B is the total number of communication resources on the edge server; Ω B is the total bandwidth size on the edge server, The communication bandwidth allocated to user i by the edge server; Edge server inference latency Affected by the split point and the computing resources allocated to the edge server, it is specifically expressed as follows: Where R is the total number of computing resources on the edge server, r i is the number of computing resources allocated to the i-th user, and the ratio of the number of threads allocated to the i-th user inference process by the edge server to the total number of active threads is s i is the split point; Use the following formula function Fit the measured data points to obtain the split point s i The function parameters below and Establish the relationship between edge server inference latency and computing resources: For practical application scenarios where edge servers use GPUs for model inference, Nvidia GPU devices use MPS technology provided by Nvidia to allocate computing resources to users. MPS technology allocates active threads on a single GPU device to a specified user's model inference process based on a percentage.

4. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 2, characterized in that: In the edge-to-edge collaborative inference system based on model segmentation, the user's inference task is divided into three parallel running parts: head network inference, intermediate feature transmission, and tail network inference. The three parallel running parts correspond to the user device inference latency, intermediate feature transmission latency, and edge server inference latency, respectively. Based on the principle of pipeline parallelism, the inference throughput of user i in the current time slice is determined by the largest delay among the user device inference latency, intermediate feature transmission latency, and edge server inference latency: Among them, Δt is the length of the current time slice, Ψ i Configure a collection of variables for user inference.

5. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 2, characterized in that: In the inference accuracy model, the accuracy of all compression parameter settings is comprehensively measured. Based on the scatter plot of the transmitted data size and the corresponding inference accuracy, the corresponding Pareto set P is constructed. i : Where, is the product of the corresponding compression parameters, A i (s i ,α i ,β i ,q i ) is the split point s i , compression parameter α i ,β i ,q i The corresponding inference accuracy is, for the elements in the Pareto set (s i ,α i ,β i ,q i ), there is no other group (s i ,α j ,β j ,q j ), making the compression ratio Less than At the same time, the accuracy A i (s i ,α j ,β j ,q j ) is greater than A i (s i ,α i ,β i ,q i ); P i It represents the configuration set with the least impact on accuracy among all compression parameter configurations. Each point on the configuration set represents the highest accuracy achieved under a specific transmission data size.

6. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 1, characterized in that: In step S2, the output of the head model is compressed at the end user, and the compressed data is restored on the edge server as the input of the tail model; The feature data compression module compresses the intermediate transmission features from [C × H × W × 32 bits] to [αC × βH × βW × q bits] at the end user, and then restores them to the original size of [C × H × W × 32 bits] on the edge server, where C is the number of channels of the intermediate transmission feature, H and W are the height and width of the intermediate transmission feature, respectively, and 32 bits means that the intermediate transmission feature is stored using 32-bit floating point numbers; α is the channel compression parameter, β is the length and width compression parameter, and q is the quantization width; The compression part is based on three feature data compression methods: CNN channel pruning, bilinear interpolation downsampling, and learnable affine quantization. It compresses the intermediate transmission features output by the head model from multiple dimensions. The specific methods are as follows: CNN channel pruning: It is responsible for compressing the number of channels C of intermediate transmission features. The channel compression parameter is α. The intermediate transmission features are input into the point convolution layer. Assuming that the weight matrix of the point convolution layer is in the shape of [C, C, 1, 1], the weight matrix is ​​split and the weight matrix of the shape of [αC, C, 1, 1] is taken for convolution calculation. The number of output channels of the point convolution layer is pruned from the original C to αC, thus compressing the number of channels of intermediate transmission features. Bilinear interpolation downsampling: responsible for compressing the length and width (H and W) of the intermediate transmission features. The compression parameters of H and W are β. Bilinear interpolation method that supports arbitrary numerical scaling is used to downsample the intermediate transmission features. Learnable affine quantization: compresses the number of bits of data for intermediate transmission features. The quantization width is q. The intermediate transmission features will be compressed from 32-bit floating point numbers x to q-bit discrete integers x. q , the specific formula is as follows: where x min 、x max Corresponding to the maximum and minimum values ​​of the intermediate transmission characteristics respectively; Q p is the maximum value of a q-bit discrete integer, Q p =2 q -1; the clamp function is a clipping function that clips the function input to 0 to Q p Within the range; s is a learnable parameter of the neural network, which is learned along with the entire neural network and is used to adjust the Δ used in quantization; The function is the Round function, which is responsible for rounding its function input to an integer; The recovery part performs the inverse operation of the compression part in sequence to restore the intermediate feature data to its original size. The specific steps are as follows: S21: The edge server dequantizes the received feature data according to the formula Re-transform the q-bit discrete integer x q Restore to 32-bit floating point S22: Restore the length and width of the feature data through bilinear interpolation upsampling and depth-separable convolutional layers; S23: Recover the number of channels of feature data through the point convolution layer; S24: Insert a switchable batch normalization layer before the tail model to correct the data distribution of the tail model input.

7. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 1, characterized in that: In step S3, an EMA network is used as a shadow network to generate soft labels for distillation. The weights of the EMA network are updated at the end of each epoch based on the weights of the currently trained network, as shown below: W′ τ =ηW τ +(1-η)W′ τ-1 ; Where η is the smoothing coefficient, W' τ is the weight of the EMA network at the τth epoch, and W τ is the weight of the trained network at the τth epoch; The training process of multi-split point dynamic neural network is divided into two stages. In the first stage, due to the soft labels used in distillation Inaccurate, only the largest subnet is trained; In the second stage, in addition to training the largest subnet, the corresponding smallest subnet and intermediate subnet are sampled at the split point, and soft labels are generated through the EMA network. The smallest subnet and the intermediate subnet are distilled as the teacher network.

8. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 6, characterized in that: In step S4, the following discrete integer programming problem for maximizing inference accuracy is constructed: Among them, the objective function is to maximize the inference accuracy A of N users i The sum of C1 and C2 is the user demand constraint, which represents the upper limit of end-to-end delay. and the lower bound on inference throughput C3 and C4 are the computing or communication resource constraints on the edge server, and the total number of resource shares R and the total number of communication resource shares B are calculated respectively. i with b i are all discrete integer variables; constraint C5 indicates that the user's split point and compression parameter configuration are selected from the Pareto set.

9. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 8, characterized in that: In step S5, the original question For user i only, given computing resources r i and communication resources b i , original question Transform the unilateral multi-end scenario into a unilateral single-end scenario, breaking it down into simplified sub-problems. Among them, the sub-problem The objective function in only considers maximizing the accuracy of user i. Constraints C1 and C2 are end-to-end delay constraints and inference throughput constraints, and constraint C3 is a Pareto set constraint. At this time, the subproblem is solved by traversing the corresponding Pareto point set. Given a computing resource r i and communication resources b i Under this condition, the maximum inference accuracy that user i can achieve is The corresponding inference configuration used is Calculate each type (r i ,b i ) The corresponding pair At this time, the original question is transformed into a two-dimensional resource allocation problem As shown below: The optimization variables only consider (r i ,b i ) tuple, user's (s i ,α i ,β i ,q i ) in advance calculation In the process, constraints C1 and C2 indicate that the total resources allocated to users do not exceed the edge resource limit.

10. The method for edge collaborative task division and resource allocation based on model segmentation according to claim 9, characterized in that: In step S6, for the dimension resource allocation problem Based on the optimal substructure properties, a dynamic programming algorithm is used to find the optimal solution for edge-end collaborative task division and resource allocation, specifically including: Define the top k users using R k Computing resources and B k Communication resources, the maximum accuracy that can be achieved is P k (R k ,B k ), the state transition equation is as follows: in P is the maximum inference accuracy that the kth user can achieve under given computing resources and communication resources y, k-1 (R k -x,B k -y) Use R for the first k-1 users k -x computing resources and B k -yThe maximum accuracy that can be achieved by communication resources, defined as the sum of the first k users using R k Computing resources and B k When communication resources are used, the optimal inference configuration for all users is renew Perform optimal edge-to-edge collaborative task division and resource allocation: in, Use x for the kth user * Computing resources and * The optimal user reasoning configuration of communication resources is calculated The records are obtained in the process, and the original problem is solved The optimal solution has a time complexity of O(NR 2 B 2 ).

Images