Depth model lightweight deployment method based on algorithm-system cross-layer adaptive optimization

Abstract

The invention discloses a depth model lightweight deployment method based on algorithm-system cross-layer adaptive optimization, and particularly relates to the field of model lightweight methods.The method comprises the steps that multiple compression operators of a structural layer of a depth model and multiple acceleration operators in the depth model deployment process are determined, taking a plurality of compression operators and a plurality of acceleration operators as a cross-layer search space; determining a target function and a constraint condition according to a target performance index of the depth model; and taking the cross-layer search space, the objective function and the constraint condition as input, and combining a compression operator and an acceleration operator by using a double-layer coevolution optimization algorithm to obtain an optimal technical combination. The optimization requirements of the model and the system level are considered at the same time, the overall optimization effect of the model can be improved, and the model precision is not affected.

Description

Technical Field

[0001] The present application relates to the field of model optimization methods, and in particular to a lightweight deployment method of deep models based on algorithm-system cross-layer adaptive optimization. Background Art

[0002] Compared with transmitting data to a remote center in the cloud, local devices can perform deep learning tasks to ensure accuracy, save communication bandwidth and reduce latency, which enables data processing and model reasoning to be completed quickly on local devices without waiting for network transmission delays. Moreover, when the network connection is unstable, such as disconnection or weak network, requests that rely on the server are often prone to failure. Local model processing can be unaffected by the network and perform computing tasks stably, which is particularly important in complex environments such as unmanned vehicles and autonomous driving, tunnels, and mountainous areas, ensuring the continuous and reliable operation of the system. In addition, users are increasingly concerned about sensitive information and privacy issues, so processing data on local devices is the most effective solution, which not only greatly reduces the risk of user privacy leakage, but also enhances users' confidence in data security. However, modern deep learning models usually have a large number of parameters and complex computing structures, and these devices are often limited by storage capacity, limited computing power, and battery life. Therefore, in order to achieve real-time reasoning on these resource-limited devices, a series of optimization strategies need to be adopted to reduce the memory and computing costs of DNNs. In other words, deep neural network compression has become an urgent and promising topic.

[0003] In terms of existing technologies, although there are some deep learning model compression algorithms that can reduce the size and computational complexity of the model to a certain extent, these methods are often accompanied by performance loss. These "one-size-fits-all" solutions ignore the specific requirements for model performance in different application scenarios, and it is difficult to meet the dual requirements of accuracy and computational efficiency at the same time, thereby reducing the optimization effect. In addition, most of the existing optimization methods focus on optimization at the model structure level, while ignoring system-level acceleration operators, such as hardware acceleration and parallel computing, resulting in limited overall optimization effects. Summary of the invention

[0004] The main purpose of this application is to provide a lightweight deployment method for deep models based on algorithm-system cross-layer adaptive optimization, aiming to solve the problem of limited optimization effect of existing optimization methods.

[0005] To achieve the above-mentioned objectives, the present application provides a lightweight deployment method for deep models based on algorithm-system cross-layer adaptive optimization, including: obtaining multiple compression operators of the structural layer of the deep model, and obtaining multiple acceleration operators of the deep model deployment process, and using multiple compression operators and multiple acceleration operators as cross-layer search spaces; determining objective functions and constraints based on the target performance indicators of the deep model; using a two-layer collaborative evolutionary optimization algorithm, taking the cross-layer search space, objective function and constraints as input, combining the compression operators and acceleration operators to obtain the optimal technical combination.

[0006] Optionally, a two-layer co-evolutionary optimization algorithm is used to combine the compression operator and the acceleration operator to obtain the optimal technology combination, including: in the upper layer of the two-layer co-evolutionary optimization algorithm, the compression method and the acceleration method are used as genes, the objective function is used as the fitness value, and a genetic algorithm is used to combine the compression operator and the acceleration operator to obtain multiple method combinations; in the lower layer of the two-layer co-evolutionary optimization algorithm, multiple method combinations are screened according to the objective function and constraints to obtain the optimal method combination, and the gradient descent algorithm is used to determine the corresponding configuration parameters of the optimal technology combination to obtain the optimal technology combination.

[0007] Optionally, the compression operator includes a compression method and corresponding configuration parameters, and the acceleration operator includes an acceleration method and corresponding configuration parameters.

[0008] Optionally, the configuration parameters corresponding to the compression method include compression rate and compression operator type; the configuration parameters corresponding to the acceleration method include operator fusion combination type and number of parallel groups.

[0009] Optionally, the compression method includes pruning, quantization or low-rank decomposition.

[0010] Optionally, the acceleration method includes fusing convolutional layers with batch normalization layers, parallelism between operators, or computational graph replacement.

[0011] Optionally, the performance indicators include accuracy, latency, memory usage, and computational complexity.

[0012] Optionally, the deep model comprises a convolutional neural network.

[0013] Optionally, the objective function is:

[0014]

[0015] In the formula, A, C, and T represent the accuracy, computational complexity, and latency of the convolutional neural network in dynamic operation, respectively. is the compression method, For the acceleration method, For the target delay, and are runtime accuracy and computational effort, respectively. and They are all importance coefficients, and N(x) is a normalization operation.

[0016] Optionally, the constraints are:

[0017] Where S represents the storage budget of the convolutional neural network in dynamic operation, Budget for target storage.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The lightweight deployment method of deep models based on algorithm-system cross-layer adaptive optimization of the present invention uses compression operators to reduce the scale and complexity of the model at the structural layer, and uses speed-up technology to reduce computing and storage overhead at the system layer, thereby simultaneously considering the optimization requirements at the model and system levels, and can improve the overall optimization effect of the model without affecting the model accuracy; optimizing the deep learning model can effectively solve the deployment problems of deep learning models on embedded devices, such as limited computing power and memory that may cause delays and accuracy requirements not being met, promote the widespread application of deep learning technology in the field of intelligent Internet of Things, and improve user experience and data security; for different application scenarios (mobile devices), objective functions and constraints are set according to their hardware resources and performance requirements, and compression methods and speed-up methods and their configuration parameters are flexibly configured, which has strong versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for lightweight deployment of a deep model based on algorithm-system cross-layer adaptive optimization is provided for this application; Figure 2 This is a latency comparison chart of Example 1 of a deep model lightweight deployment method based on algorithm-system cross-layer adaptive optimization of the present application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The first embodiment of the present invention provides a method for lightweight deployment of a deep model based on algorithm-system cross-layer adaptive optimization. The deep model can be a convolutional neural network. The deep model includes a structure layer and a system layer, such as Figure 1 As shown, the specific steps include: Step S1, determining the structural layer to be optimized of the deep model, obtaining multiple compression operators of the structural layer of the deep model, and obtaining multiple acceleration operators in the deep model deployment process, and using the compression operator and the acceleration operator as a cross-layer search space; wherein the compression operator includes a compression method and corresponding configuration parameters, and the acceleration operator includes an acceleration method and corresponding configuration parameters; The present invention considers the optimization requirements at both the model and system levels, that is, compressing the structural layer of the deep model and accelerating the system scheduling layer on the deployment side of the deep model, making full use of the resources of the hardware equipment without affecting the accuracy of the model, and realizing lightweight deployment of the hardware equipment; compared with the single-layer optimization method, it can more significantly reduce the memory usage and computing latency of the model, while hardly affecting the model accuracy. Among them, the structural layer to be optimized can be determined according to the inter-layer dependency of the deep model using the existing determination method, and the determination method will not be repeated here.

[0023] It is worth noting that for the same deep model configuration, different model layers / operators are mapped to different memory units in different orders (i.e. deployed on different devices), which will also lead to different delays and resource overheads. Therefore, in order to achieve the delay and resource overhead set by the user, the system scheduling layer is formed by setting corresponding strategies (acceleration operators) at the computational graph, operator, and memory allocation levels to cater to the structural layer of the deep model, which can enhance the availability of runtime resources and make full use of hardware resources without affecting the accuracy of the model.

[0024] Furthermore, according to different models, the target performance indicators may include accuracy, latency, memory usage, and computational load indicators. Accuracy can be, for example, classification accuracy. Classification accuracy usually uses accuracy as an evaluation indicator, which is defined as the ratio of correctly classified images to the total number of images. Assuming that the total number of test set samples is N, and the number of samples correctly predicted by the model as positive is TP, the calculation formula for classification accuracy is: Latency refers to the total time of the GPU inference process, covering the entire cycle of the inference process including preprocessing, model inference, data transmission and post-processing, that is, the average time required for the model to complete the classification prediction after inputting an image. Inference latency T represents the time required to complete the inference task of a sample i in a given application scenario of batchsize=16. The calculation formula is 0, where N0 is the total number of samples.

[0025] Exemplarily, compression operators may include pruning, quantization, and low-rank decomposition. Acceleration operators may include fusion of convolutional layers and batch normalization layers, parallelism between operators, and computational graph replacement.

[0026] Step S2, determining the objective function and constraints according to the target performance indicators of the deep model; the performance indicators include accuracy, latency, memory usage, and computational complexity, and the target performance indicators may be, for example, an acceptable accuracy loss range, a maximum latency, and a memory usage limit; Step S3, taking the objective function and constraints as input, and using the two-layer co-evolutionary optimization algorithm to combine the compression operator and the acceleration operator in the cross-layer search space to obtain the optimal technology combination. The core operation of the two-layer co-evolutionary optimization algorithm is to dynamically adjust the combination scheme and parameter configuration according to the device resource limitations and task requirements to achieve the best performance and resource utilization. Specifically, the following steps are included: Step S31, inputting the cross-layer search space, objective function and constraint conditions into the two-layer co-evolutionary optimization algorithm; Step S32, in the upper layer of the double-layer co-evolutionary optimization algorithm input, a genetic algorithm (specifically, an elite non-dominated sorting genetic algorithm, referred to as NSGA-II) is used to combine the compression method and the acceleration method, specifically, the compression method and the acceleration method are used as genes, and the gene combination of individuals represents a possible technology combination; the upper layer objective function is used as the fitness value; Select a part of the solutions as parent individuals according to the fitness value to generate the next generation of individuals; generate a new offspring population using selection, crossover and mutation operations, and use the new offspring population to update the population until the termination condition is reached, thus obtaining multiple method combinations; In the lower layer, multiple method combinations are screened according to the objective function and constraints to obtain the optimal method combination, and the gradient descent algorithm is used to determine the corresponding configuration parameters of the optimal technology combination to obtain the optimal technology combination.

[0027] Step S4, deploying the lightweight model optimized by the above method to the target mobile device (such as a mobile phone or an embedded development board) to perform actual operation tests to achieve lightweight deployment.

[0028] Example 1 takes the convolutional neural network (Resnet18 model) as an example and uses the Resnet18 model to perform the Cifar-100 image classification task Step S10, determining that the structure layer to be optimized of the deep model of the Resnet18 model is the second half of all residual layers, and determining the compression operator of the structure layer and the acceleration operator of the system layer, and using the compression operator and the acceleration operator as the initial cross-layer search space; The compression operator of this embodiment specifically includes pruning and low-rank decomposition. The specific method of pruning is: pruning redundant channels based on channel importance measurement (L1 norm and L2 norm of weights), further reducing the model scale while ensuring model performance. For each channel c, calculate its L1 norm:

[0029] in, is the weight of the cth channel, i and j are the dimensions of the weight matrix. The pruning criterion can be set as: ,in, is the preset threshold.

[0030] Low-rank decomposition, which is done by decomposing the standard convolutional layer into two smaller point-wise convolutional layers according to the following formula to reduce the model complexity;

[0031] in, , There are two point-by-point convolution layers (1x1 convolution), is a diagonal matrix that only retains the first k singular values ​​and their corresponding singular vectors.

[0032] The acceleration operator is the fusion of the convolution layer (CONV) and the batch normalization layer (BN). Specifically, the convolution layer is fused with the batch normalization layer to obtain a fusion layer, which can reduce the computation and storage overhead. The input of the convolution layer is x, the weight is W, the bias is b, the scaling parameter of the batch normalization layer is γ, and the offset parameter is β. The original convolution layer output is: y = CONV(x, W, b), and the batch normalization layer output is: ),in and They are the mean and variance of y respectively. The calculation formula of the fusion layer CONV+BN is:

[0033] Represents a small positive number used to avoid division by zero when calculating the square root of the variance. The value is 10^-3.

[0034] Step S20, determine the objective function and constraints according to the target performance indicators of the deep model. Specifically, for example, the target performance indicators include: the upper-level objective function can ensure that the accuracy loss is controlled within 5% after the original model adopts the compression method in the cross-layer search space, and at the same time requires that the amount of calculation is as small as possible after the acceleration method in the selected optimal technology combination is used to ensure efficient execution without sacrificing the prediction accuracy of the model; the lower-level objective function and constraints focus on minimizing latency to meet the needs of real-time applications. Then the objective function and constraints are:

[0035]

[0036] The constraints are:

[0037] In the formula, A, C, T, and S represent the accuracy, computational complexity, latency, and storage budget of the convolutional neural network in dynamic operation, respectively. is the parameter variable of the compression method, is the parameter variable of the acceleration method. The upper problem involves all variables x = ( ), and are the target latency (i.e., the latency specified by the user) and the storage budget, respectively. and are runtime accuracy and computational effort, respectively. and Both are importance coefficients. The two targets on A and C are expressed by importance coefficients. and Combined together, N(x) is a normalization operation.

[0038] Step S30, initialize the upper and lower layers of the two-layer co-evolutionary optimization algorithm; input the cross-layer search space, objective function and constraints. In addition, the algorithm must follow the memory usage constraints during implementation to ensure that the overall memory does not exceed 30MB, and set the maximum number of iterations to 300. In each iteration, the algorithm first initializes the upper population (pruning, low-rank decomposition, and convolution layer and batch normalization layer fusion are all used as a gene). During the optimization process, a part of the method combination (such as pruning and convolution layer and batch normalization layer fusion) is selected as the parent individual according to the fitness value. The algorithm uses selection, crossover and mutation operations to generate a new offspring population. Through these operations, the solution space is explored to find a better solution, and finally the optimal technology combination is obtained, as shown in Table 1. As can be seen from Table 1, the combination of a pruning rate of 0.5 and conv and bn fusion significantly reduces the complexity and latency of the model while maintaining a high classification accuracy, so that the ResNet18 model can achieve efficient real-time image classification on the JetsonOrin NX platform.

[0039] Table 1 Performance of cross-layer solutions

[0040] The latency of the cross-layer optimization technology of this embodiment is compared with that of the single-layer optimization technology. The experimental results are shown in Figure 2 , Figure 2It shows that although pruning optimization and operator fusion have achieved certain optimization effects when used alone, reducing the latency by 48.64% and 35.09% respectively, when the cross-layer search space is used to build and combine these two optimization technologies, the latency reduction reaches 55.12%, which is nearly 2 times higher than the single-layer optimization technology. In other words, the latency curve after cross-layer optimization is significantly lower than that of other optimization methods, proving the effectiveness of compression operators based on cross-layer search space in practical applications.

[0041] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A lightweight deployment method for deep models based on algorithm-system cross-layer adaptive optimization, characterized in that: include: Obtain various compression operators for the structural layer of the deep model, and obtain various acceleration operators for the deep model deployment process; Using multiple compression operators and multiple acceleration operators as cross-layer search space; Determine the objective function and constraints based on the target performance indicators of the deep model; By using a two-layer collaborative evolution optimization algorithm, the cross-layer search space, the objective function and the constraint conditions are taken as inputs, and the compression operator and the acceleration operator are combined to obtain an optimal technology combination.

2. According to claim 1, the method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization is characterized in that: The double-layer co-evolutionary optimization algorithm is used to combine the compression operator and the acceleration operator to obtain the optimal technology combination, including: In the upper layer of the two-layer co-evolutionary optimization algorithm, the compression method and the acceleration method are used as genes, the objective function is used as the fitness value, and the compression operator and the acceleration operator are combined using a genetic algorithm to obtain multiple method combinations; In the lower layer of the two-layer collaborative evolutionary optimization algorithm, multiple method combinations are screened according to the objective function and constraints to obtain the optimal method combination, and the gradient descent algorithm is used to determine the corresponding configuration parameters of the optimal technology combination to obtain the optimal technology combination.

3. According to claim 1, the method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization is characterized in that: The compression operator includes a compression method and corresponding configuration parameters, and the acceleration operator includes an acceleration method and corresponding configuration parameters.

4. The method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization according to claim 3 is characterized in that: The configuration parameters corresponding to the compression method include the compression rate and the compression operator type; the configuration parameters corresponding to the acceleration method include the operator fusion combination type and the number of parallel groups.

5. According to claim 3, the method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization is characterized in that: The compression methods include pruning, quantization or low-rank decomposition.

6. The method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization according to claim 3 is characterized in that: The acceleration method includes fusion of convolutional layer and batch normalization layer, parallelism between operators or replacement of computational graph.

7. The method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization according to claim 1, characterized in that: The performance indicators include accuracy, latency, memory usage, and computational complexity.

8. The method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization according to claim 1, characterized in that: The deep model includes a convolutional neural network.

9. The method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization according to claim 8, characterized in that: The objective function is: In the formula, A, C, and T are the accuracy, computational complexity, and latency of the convolutional neural network in dynamic operation, respectively. is the parameter variable of the compression method, is the parameter variable of the acceleration method, For the target delay, and are runtime accuracy and computational effort, respectively. and They are all importance coefficients, and N(x) is a normalization operation.

10. The method for lightweight deployment of deep models based on algorithm-system cross-layer adaptive optimization according to claim 9, characterized in that: The constraints are: Where S is the storage budget of the convolutional neural network in dynamic operation, Budget for target storage.

Images