Customized deep neural network model compression method and system based on cloud-edge collaboration

Abstract

The present invention discloses a customized deep neural network model compression method and system based on cloud-edge collaboration. The method is based on a high-precision network model that has been trained for a long time in the cloud. Based on the personalized needs of users, the network model is classified into categories of concern and categories of no concern. Then, the categories of concern, the user's data set and the model compression ratio are combined. The model is lightweighted through a neural network model compression method based on knowledge organization to meet the needs of scenarios with limited node resources on the user's edge side, improve the speed of model reasoning, and ensure a certain model accuracy. The present invention is based on a model training method based on user-focused classification needs and knowledge distillation technology, and is verified based on a public data set in the scenario of image classification.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing and deep learning, and in particular to a customized deep neural network model compression method and system based on cloud-edge collaboration. Background Art

[0002] Deep learning is an important branch of artificial intelligence. Through deep neural networks, deep learning has an increasingly important influence in fields such as machine vision, NLP, and speech recognition. With the development of edge computing, deep learning has begun to play a more important role on a wider range of hardware devices - that is, "edge empowerment", which enables many hardware in the Internet of Things to have more powerful recognition, detection, and intelligent processing capabilities. However, the computing power of many hardware devices in edge computing scenarios is relatively weak, and the speed of deep neural network reasoning is also relatively slow. How to optimize and accelerate deep neural network models before deploying them to edge devices and terminal devices is a difficult problem facing the implementation of artificial intelligence.

[0003] There are two main problems facing edge nodes and terminals when running neural network models. First, the memory of edge nodes and terminals is relatively small, while some deep neural network models are relatively large in size. For example, the VGG-16 network used for image classification has 138M parameters, each of which is of type float32. The running overhead of the model itself is at least 552MB. Second, the computing power of edge nodes and terminals is relatively poor. When encountering application scenarios with high real-time requirements, such as autonomous driving, the inference latency will be crucial. Deep neural network models are relatively large, and when resource capabilities are relatively low, the inference latency is relatively high. The inference time of some models is even at the second level.

[0004] At present, there are four main algorithms for model compression: 1. Design a lighter neural network. The redundancy of model parameters is the premise for ensuring that the model is well trained. By designing the model more carefully, efficiently and lightly, the model size can be reduced to meet the needs of edge and terminal resources. For example, SqueezeNet and MobileNet are both finely designed network structures with relatively good accuracy. 2. Pruning. Through certain metrics, the importance of the trained complex network structure is evaluated, and unimportant weights, convolution kernels, channels and convolution layers are trimmed. 3. Knowledge distillation. The distribution information and feature information in the complex model are transferred to the lightweight model structure through knowledge transfer, which can not only speed up the convergence speed, but also enable the lightweight model to learn some features that the sample set of the complex model does not have, so that the model has better generalization ability. 4. Quantization. Quantization does not fundamentally change the model structure of the complex network, but compresses the original network by reducing the number of bits required for each weight, thereby reducing memory overhead and speeding up reasoning calculations.

[0005] However, none of the above methods are fully integrated with the user needs of fragmented edge scenarios, resulting in a mismatch between the network model provided by the cloud and the network model required by users or "empowerment overload", and the reasoning accuracy of the lightweight model is relatively low. Summary of the invention

[0006] The purpose of the present invention is to provide a customized deep neural network model compression method and system based on cloud-edge collaboration in view of the deficiencies of the prior art. The present invention can solve the problem that edge scenarios are limited by hardware constraints, such as insufficient size and computing power of CPU / GPU / NPU / memory, which leads to the inability to deploy on the edge side or perform real-time reasoning; in fragmented edge scenarios, model compression and acceleration are performed based on the user's customized needs.

[0007] The objective of the present invention is achieved through the following technical solutions: A customized deep neural network model compression method based on cloud-edge collaboration, where users on the edge side receive the deep neural network model transmitted from the cloud, and train the deep neural network model according to the user's customized needs and the user's data set. A cloud-edge collaborative method is adopted. A more universal general neural network model is provided by the cloud, and the edge side uses the hardware resource environment of the edge side to perform secondary training and optimization on the model according to user needs, so as to achieve the purpose of model acceleration. The knowledge distillation method is used to migrate the features of the deep neural network model provided by the cloud to a lightweight network model.

[0008] Furthermore, users are allowed to train lightweight models in two different ways:

[0009] a1) During model training, the classification information of the lightweight model is consistent with the model in the cloud. After the model training is completed, post-processing is performed according to the user's personalized needs.

[0010] a2) When training the model, modify the classification information of the lightweight model and the loss function of the model optimization. During the knowledge distillation process, only the feature information that the user is concerned about is transferred without the need for post-processing.

[0011] Furthermore, the knowledge distillation method can be implemented in two different ways:

[0012] b1) The loss function can be designed based on the distribution information of the model classification Logits, and the distribution characteristics of Logits can be migrated to a lightweight network model.

[0013] b2) The loss function can be designed based on the channel information of the model to migrate the feature map of Channels to a lightweight network model.

[0014] A customized deep neural network model compression system based on cloud-edge collaboration, including:

[0015] Large model generation module. A large model refers to a deep neural network model with multiple capabilities and a relatively complex structure. This module outputs a complex model with multiple classification or recognition capabilities based on the specific resource conditions of the cloud.

[0016] Model distribution module. The model distribution module is mainly used to transfer the model to the edge node through cloud-edge collaboration. The distributed data includes but is not limited to model files, configuration files, and several model-related description files. The model distribution mechanism can be completed through the inter-process communication mechanism, or it can be implemented through some traditional file protocols such as Samba, NFS, etc.

[0017] Edge model compression module. This module is the core module for edge model compression and processing. Its input is the user's personalized data set, the large model distributed in the cloud, and the classification information or target detection information that the user is concerned about. The output is a lightweight model that can be deployed on the edge or terminal side.

[0018] Model deployment module. This module completes the access or deployment of lightweight models. The deployment can be in the form of applications or based on container mechanisms.

[0019] Furthermore, the edge side model compression module has two implementation methods, including:

[0020] 3.1. First, all classifications are distilled, and then the output lightweight model is processed.

[0021] 3.2. Directly use subclass distillation training method.

[0022] Furthermore, the subclass distillation training method can be implemented in two ways, including:

[0023] 3.2.1. During the training process, the Logits-based subclass distillation method is adopted.

[0024] 3.2.2 During the training process, a Channels-based subclass distillation method is adopted.

[0025] Furthermore, step 3.1 is as follows: In the classification task, the probability of each classification is actually calculated using the Softmax function, and Logits is actually the input layer of the Softmax layer. i To represent the input of a certain category i, that is, the Logits of a certain class. The probability output of this category is for

[0026]

[0027] Among them, C is the total number of categories, i=1~C, and T represents the temperature, which is used to control the softness of information distillation from the large model. For exp(z i / T) is the sum from 1 to C. When T = 1, it means that the standard Softmax function is used to calculate the probability distribution. In order to simplify the representation method, use To represent the probability distribution of all categories controlled by temperature, p is used to represent the probability distribution of all categories when T = 1. It can be expressed as

[0028]

[0029] Among them, α is a hyperparameter, which is used to indicate the proportion of knowledge transferred from the large model to the lightweight model. Η is the cross entropy loss function, p s is the probability distribution under the standard softmax function of the lightweight model, Represents the true label 0-1 distribution under the lightweight model calibration sample set. Represents the KL divergence loss function, which is used to measure the difference in probability distribution output by the lightweight model and the large model. Represents the probability distribution of the lightweight model under the softened softmax function with temperature parameter T. Represents the probability distribution of the large model under the softened softmax function with temperature parameter T.

[0030] In the process of Logits distillation training, users use the training set on the edge side and the loss function formed by the combination of the large model and the lightweight model, see formula (2), to perform iterative optimization training. The parameters of the large model are immutable, and the parameters of the lightweight model are continuously updated during training. After a certain number of iterations are completed, the training process ends.

[0031] Furthermore, step 3.2 is as follows: In sub-category distillation, the loss function of formula (2) is still used, but the categories that users pay attention to need to be modified. Suppose the set of all categories is U = {1, 2, ..., C}, U attention is the category that the user is interested in and Then modify the output probability of the category for

[0032]

[0033] Where i',j'∈U attention , For exp(z j′ / T) in Uattention Find the sum on top.

[0034] Furthermore, step 3.2.2 is as follows: The method of subclass knowledge distillation based on Channels adopts a two-stage training method. The purpose of the first stage is to match and synchronize the feature matrix. That is, while ensuring that the large model remains unchanged, a feature map with the same dimension as the tail feature tensor of the lightweight model is trained. In the second stage, feature distillation is performed based on feature maps of the same dimension.

[0035] Phase 1: After the last convolution layer of the large model, add an embedded convolution layer to realize the process of synchronizing the feature tensor. The embedded convolution layer mainly consists of two layers, the first layer is a 1×1 convolution, and the second layer is an SPP layer. The training in the first phase only obtains the parameters of the 1×1 convolution layer, so users only need to use their own data sets. After a small number of epochs, the inference results of the large model can be consistent with the results after adding the embedded convolution layer.

[0036] Phase 2: When defining a lightweight model, the SPP layer needs to be embedded into the last convolutional layer before the fully connected layer. If there is a pooling layer after the convolutional layer, use the SPP layer to replace the pooling layer. In this way, it can be ensured that the feature vectors after the SPP layer of the large model and the lightweight model have the same dimension. In the feature distillation process, the most important goal is to reduce the migration loss between the features of the large model and the features of the lightweight model under the same sample input. Therefore, first of all, a method for measuring the difference distance is needed. Commonly used methods for vector distance measurement are L 1 ,L 2 ,MMD, etc. In the second stage of model training, the parameters of the large model do not change during the gradient calculation process, and only the parameters of the lightweight model need to be updated. Depending on the data source and user business scenario, the training set can use the public data set provided by the cloud or the private data set of edge users.

[0037] Furthermore, in the second stage, considering the speed of gradient descent and the complexity of calculation, L 2 Distance measurement method. Let T,S∈R w×h×c represent the feature representation of the large model and the lightweight model respectively, then the feature distillation loss l distill Defined as

[0038]

[0039] The Channels-based subclass distillation loss is defined as

[0040]

[0041] Here, β is a hyperparameter.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. For the first time, we designed a subclass distillation method based on edge computing scenarios, and conducted an in-depth analysis of the situation where the classification capabilities of complex network models are inconsistent with the classification of user needs, solving the problem of compression through knowledge distillation. This opens up new solutions for the deployment of deep neural networks on the edge and mobile terminals;

[0044] 2. For subclass distillation, we propose Logits-based and Channel-based distillation algorithms. Channels-based knowledge distillation solves the problem of feature tensor mismatch when performing feature migration between large models and lightweight models by adding an embedded convolutional layer;

[0045] 3. Allow users to customize models according to their needs. When the number of user requirements is significantly different from that of the large model, the trained lightweight model can significantly improve the reasoning accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall flow chart of edge-cloud collaborative model compression of the present invention;

[0047] Figure 2 is the edge side model compression flow chart of the present invention;

[0048] Figure 3 is a subclass distillation flow chart of the present invention;

[0049] Figure 4 It is the knowledge distillation logic diagram of the present invention;

[0050] Figure 5 It is the subclass distillation logic diagram of the present invention;

[0051] Figure 6 It is a subclass distillation logic diagram based on Channel of the present invention. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] One of the core concepts of this invention is to make full use of the features of the large model that has been optimized for a long time on the cloud side, and adopt the idea of ​​transfer learning to migrate the useful information in the large model to the lightweight model. On the one hand, the capabilities of the large model are inherited. On the other hand, through short-term training, the model is compressed to a size that can meet the hardware resources of the edge and terminal. At the same time, according to the user's needs for certain classifications or target detection categories, directional distillation is performed, and finally the reasoning accuracy of the specified classification is maintained or even improved.

[0054] The deep neural network models used in the present invention include but are not limited to AlexNet, VGG, GoogleNet, RestNet for image recognition and SSD, YOLO, R-CNN, Fast R-CNN for target detection; the model training process can adopt different deep learning frameworks, including but not limited to Pytorch, TensorFlow, MXNet and PaddlePaddlel.

[0055] Reference Figure 1 , showing the entire process of a customized deep neural network model compression method and system based on cloud-edge collaboration of the present invention. The detailed steps are as follows:

[0056] Step S11: The purpose of the large model generation module is to make full use of the hardware resources and massive data sets in the cloud to train a large model with multiple classifications or target detection categories. The training of a large model may take a very long time. For example, the training of some complex large models requires TB-level image data, and the training time is measured in months. Of course, you can also directly pull large models that have been trained for a long time from the Model Zoo warehouse. The cloud model training module does not consider user needs in edge scenarios, and uses the versatility and accuracy of the model as measurement indicators.

[0057] Step S12: The model distribution module sends the generated large model, configuration files, and description files to the edge nodes that can perform model training based on some existing technologies, such as Socket, Samba, NFS, etc. This can be achieved using an event mechanism, that is, after the cloud completes the preparation of the large model, the Model_Ready event is triggered, and the edge node periodically detects whether the event is triggered. If the event is triggered, the data transmission process is carried out.

[0058] Step S13: Figure 2As shown, after receiving the data related to the large model, the edge model compression module starts to compress the model according to the edge data set, the large model file and the configuration of the user's needs. The present invention provides two model compression methods, which can be selected by configuration: full-category distillation and sub-category distillation. Full-category distillation is to distill and migrate all the classification information in the large model, so after the distillation is completed, post-processing is required according to the user's specified classification requirements. For a detailed implementation logic diagram, see Figure 4 Subclass distillation advances the integration with user needs to the model training stage. For a detailed implementation logic diagram, see Figure 5 .

[0059] Step S131: In the classification task, the probability of each classification is actually calculated using the Softmax function, and Logits is actually the input layer of the Softmax layer. i To represent the input of a certain category i, that is, the Logits of a certain class. The probability output of this category is for

[0060]

[0061] Among them, C is the total number of categories, i=1~C, and T represents the temperature, which is used to control the softness of information distillation from the large model. For exp(z i / T) is the sum from 1 to C. When T = 1, it means that the standard Softmax function is used to calculate the probability distribution. In order to simplify the representation method, use To represent the probability distribution of all categories controlled by temperature, p is used to represent the probability distribution of all categories when T = 1. It can be expressed as

[0062]

[0063] Among them, α is a hyperparameter, which is used to indicate the proportion of knowledge transferred from the large model to the lightweight model. Η is the cross entropy loss function, p s is the probability distribution under the standard softmax function of the lightweight model, Represents the true label 0-1 distribution under the lightweight model calibration sample set. Represents the KL divergence loss function, which is used to measure the difference in probability distribution output by the lightweight model and the large model. Represents the probability distribution of the lightweight model under the softened softmax function with temperature parameter T. represents the probability distribution of the large model under the softened softmax function with temperature parameter T. The T2 The purpose of amplifying the factor is to keep the value of the gradient change at a constant level as the temperature T changes, thus avoiding the gradient explosion.

[0064] See the process of Logits distillation training for details. Figure 5 The user uses the training set on the edge side and the loss function formed by the combination of the large model and the lightweight model, see formula (2), to perform iterative optimization training. The parameters of the large model are immutable, and the parameters of the lightweight model are continuously updated during training. After a certain number of iterations are completed, the training process ends.

[0065] Step S133: The post-processing stage needs to be combined with specific models and tasks. If the model completes a classification task, the final Softmax output of the model is processed, and the unfocused classifications are ignored. Only the value with the highest probability is selected as the classification TOP-1 result in the focused classification. If it is a target detection task, the final output tensor needs to be processed. Generally, this processing is implemented differently for different target detection models.

[0066] Step S132: Figure 3 As shown in Figure 2, in sub-category distillation, the loss function of formula (2) is still used, but the categories that users pay attention to need to be modified. Suppose the set of all categories is U = {1, 2, ..., C}, U attention is the category that the user is interested in and Then modify the output probability of the category for

[0067]

[0068] Where i',j'∈U attention , For exp(z j′ / T) in U attention By modifying the probability output and combining it with the loss function of formula (2), users can ignore the categories they don’t care about, thereby improving the accuracy of the categories they care about.

[0069] Step S1321: The process of model distillation training based on Logits. The principle can be found in step S132.

[0070] Step S1322: The Logits-based subclass distillation method makes the lightweight model as similar to the distribution of the large network as possible from the perspective of the final probability distribution of the classification. However, this method is still relatively extensive because it is impossible to feedback the correlation between classifications from the perspective of features. For example, some classifications have many common features, while some classifications do not have any common features. These cannot be simply displayed from the probability distribution, which ultimately limits the effect of the Logits-based distillation method. This step is to extract the feature information of the convolutional layer, that is, subclass distillation based on Channels (also known as feature distillation).

[0071] The Channels-based subclass knowledge distillation method adopts a two-stage training method. The purpose of the first stage is to match and synchronize the feature matrix. That is, while ensuring that the large model remains unchanged, a feature map with the same dimension as the tail feature tensor of the lightweight model is trained. In the second stage, feature distillation is performed based on feature maps of the same dimension. The logic diagram of the subclass distillation is shown in Figure 6 .

[0072] The large model is a well-trained large network. The Channels distillation-based methods introduced above all migrate the feature information in the large model to the lightweight model through dimensionality transformation and distance calculation during the distillation process. However, this method will cause the problem of feature loss during the dimensionality transformation process. Especially when the large model is different from the lightweight model and the dimensionality of the feature tensor of the last layer is quite different, the effect of feature distillation will be very poor. In fact, the process of synchronizing the feature tensor can be extracted separately. After the last convolution layer of the large model, add an embedded convolution layer to realize the process of synchronizing the feature tensor. For example Figure 6 The embedded convolution layer mainly consists of two layers. The first layer is a 1×1 convolution layer, and the second layer is an SPP (Spartial Pyramid Pooling) layer. The functions of the two parts are as follows:

[0073] 1×1 convolution: The 1×1 convolution layer plays a role in dimensionality reduction. Suppose the dimension of the feature tensor of the last layer of the large model is The dimension of the feature tensor of the last layer of the lightweight model is The purpose of this convolutional layer is to reduce the dimension of the Channels layer (if the dimension c of the lightweight model is 1 >c 2 , it can also be used to increase the dimension), so that the Channel dimension of the feature map is consistent with the lightweight model.

[0074] SPP layer: After passing through the 1×1 convolution layer, the number of channels is guaranteed to be consistent. The SPP layer is used to solve the inconsistency of feature tensors. The SPP layer can output feature maps of arbitrary sizes as output vectors of fixed length. Its design idea is to replace the last pooling layer before the fully connected layer with multiple pyramid-like pooling layers. In the design of the present invention, the SPP layer is placed after the 1×1 convolution layer, so that the vector of the feature map of the large model is consistent with the lightweight model.

[0075] It should be noted that the addition of the SPP layer does not introduce new parameters. Therefore, the first stage of training only obtains the parameters of the 1×1 convolution layer, so the number of parameters is relatively small. Therefore, users only need to use their own datasets and after a small number of epochs, the inference results of the large model can be consistent with the results after adding the embedded convolution layer.

[0076] Feature distillation: When defining a lightweight model, the SPP layer needs to be embedded into the last convolutional layer before the fully connected layer. If there is a pooling layer after the convolutional layer, use the SPP layer to replace the pooling layer. In this way, it can be ensured that the feature vectors after the SPP layer of the large model and the lightweight model have the same dimension. In the feature distillation process, the most important goal is to reduce the migration loss between the features of the large model and the features of the lightweight model under the same sample input. Therefore, first of all, a method for measuring the difference distance is needed. Commonly used methods for vector distance measurement are L 1 ,L 2 ,MMD, etc. Taking into account the speed of gradient descent and the complexity of calculation, L 2 Distance measurement method. Let T,S∈R w×h×c represent the feature representation of the large model and the lightweight model respectively, then the feature distillation loss Defined as

[0077]

[0078] The Channels-based subclass distillation loss in this invention is defined as

[0079]

[0080] Here, β is a hyperparameter.

[0081] In the second stage of model training, the parameters of the large model do not change during the gradient calculation process, and only the parameters of the lightweight model need to be updated. Depending on the data source and user business scenario, the training set can use the public data set provided by the cloud or the private data set of edge users.

[0082] The lightweight model can be deployed in the form of an application or based on a container mechanism.

[0083] Example

[0084] The present invention evaluates the performance of the subclass distillation algorithm based on the CIFAR-10 dataset. The CIFAR-10 dataset contains 50K training data of 32×32 size and 10K test data, including a total of 10 categories. Based on the CIFAR-10 dataset, the algorithm is verified on the VGG classification model. The algorithm is implemented using the Pytorch deep learning framework, and the model training is run on an Ubuntu server with 2 NVIDIA GTX 2080Ti GPUs. The learning rate of the gradient descent SGD is 0.01, the momentum is set to 0.9, and the batchsize is set to 128.

[0085] The large model uses the standard VGG-16. The structure of the compressed small model is obtained by trimming the channels of each convolutional layer of the standard VGG-16. That is, the number of channels of the 13 convolutional layers of the standard VGG-16 are [64, 64, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512], and after trimming, the number of channels is [32, 32, 64, 64, 128, 128, 128, 256, 256, 256, 256, 256]

[0086] The hyperparameters are set as follows: α = 0.95 in Formula 2, temperature T = 2. And β = 5 × 10 2 A larger β means that feature distillation will have a greater impact on the model. The 10 categories of CIFAR-10 are divided into 2 groups, Group-1: {plane, bird, deer, dog, horse}, Group-2: {car, cat, frog, ship, truck}. Here, Group 1 is the category that the user does not pay attention to, and Group 2 is the category that the user pays attention to.

[0087] The model lightweight experiment of cloud-edge collaboration is divided into four groups:

[0088] 1. Use the method described in S131 to first distill all categories and then process user needs (Experiment 1).

[0089] 2. Logits-based subclass distillation method using S1321 (Experiment 2).

[0090] 3. Channels-based subclass distillation method using S1322 (Experiment 3).

[0091] 4. Direct lightweight model training method (Experiment 4).

[0092] After epoch=100 iterative training, the final model accuracy of the four groups of experiments is:

[0093] Experiment 1: mAP - 83.5%

[0094] Experiment 2: mAP-94.4% (present invention)

[0095] Experiment 3: mAP-95.1% (present invention)

[0096] Experiment 4: mAP - 93.1%

[0097] The above embodiments can fully verify the effectiveness of the present invention.

Claims

1. A customized deep neural network model compression method based on cloud-edge collaboration, characterized in that: Users on the edge side receive the deep neural network model transmitted from the cloud, and train the deep neural network model according to the user's customized needs and user data set; adopt the cloud-edge collaboration method; the cloud provides a more universal general neural network model, and the edge side uses the hardware resource environment of the edge side to perform secondary training and optimization on the model according to user needs, so as to achieve the purpose of model acceleration; The knowledge distillation method is used to migrate the features of the deep neural network model provided by the cloud to a lightweight network model. There are two implementation methods: 3.

1. First, all classification distillation is performed, and then the output lightweight model is processed; 3.

2. Directly use the subclass distillation training method, including two implementation methods: 3.2.

1. During the training process, the Logits-based subclass distillation method is adopted; 3.2.2 During the training process, a Channels-based subclass distillation method is adopted; Step 3.1 is as follows: In the classification task, the probability of each classification is actually calculated using the Softmax function, and Logits is actually the input layer of the Softmax layer; here we use z i To represent the input of a certain category i, that is, the Logits of a certain class; and the probability output of this class is generated for Where C is the total number of categories, i = 1 ~ C, T represents the temperature, which is used to control the softness of information distillation from the large model; For exp(z i / T) is the sum from 1 to C; when T = 1, it means that the standard Softmax function is used to calculate the probability distribution; in order to simplify the representation method, use To represent the probability distribution of all categories controlled by temperature, p is used to represent the probability distribution of all categories when T = 1; the total distillation loss of all categories can be expressed as Among them, α is a hyperparameter, which is used to indicate the proportion of knowledge transferred from the large model to the lightweight model; Η is the cross entropy loss function, p s is the probability distribution under the standard softmax function of the lightweight model, represents the true label 0-1 distribution under the lightweight model calibration sample set; KL represents the KL divergence loss function, which is used to measure the difference in probability distribution output by the lightweight model and the large model; Represents the probability distribution of the lightweight model under the softened softmax function with temperature parameter T; Represents the probability distribution of the large model under the softened softmax function with temperature parameter T; In the process of Logits distillation training, the user uses the training set on the edge side and the loss function formed by the combination of the large model and the lightweight model, see formula (2), to perform iterative optimization training; the parameters of the large model are immutable, and the parameters of the lightweight model are continuously updated during training. After a certain number of iterations are completed, the training process ends. Step 3.2 is as follows: In sub-category distillation, the loss function of formula (2) is still used, but the categories that users pay attention to need to be modified; let the set of all categories U = {1, 2,, C}, U attention is the category that the user is interested in and Then modify the output probability of the category for Where i',j'∈U attenti on, For exp(z j′ / T) in U attention Find the sum on top; Step 3.2.2 is as follows: The Channels-based subclass knowledge distillation method adopts a two-stage training method. The purpose of the first stage is to match and synchronize the feature matrix; that is, while ensuring that the large model remains unchanged, a feature map with the same dimension as the tail feature tensor of the lightweight model is trained; in the second stage, feature distillation is performed based on the feature map with the same dimension; Phase 1: After the last convolution layer of the large model, add an embedded convolution layer to realize the process of feature tensor synchronization; the embedded convolution layer mainly includes two layers, the first layer is 1×1 convolution, and the second layer is SPP layer; the training in the first phase only obtains the parameters of the 1×1 convolution layer, so users only need to use their own data sets, and after a small number of epochs, the inference results of the large model can be consistent with the results after adding the embedded convolution layer; The second stage: When defining a lightweight model, the SPP layer needs to be embedded into the last convolution layer before the fully connected layer; if there is a pooling layer after the convolution layer, the SPP layer is used to replace the pooling layer; in this way, the feature vectors after the SPP layer of the large model and the lightweight model can be guaranteed to have the same dimension; in the feature distillation process, the most important goal is to reduce the migration loss between the features of the large model and the features of the lightweight model under the same sample input; therefore, first of all, a difference distance measurement method is needed; the commonly used methods for vector distance measurement are L1, L2, and MMD; in the second stage model training process, the parameters of the large model are not changed during the gradient calculation process, and only the parameters of the lightweight model need to be updated; depending on the data source and user business scenario, the training set can use the public data set provided by the cloud or the private data set of edge users.

2. The customized deep neural network model compression method based on cloud-edge collaboration according to claim 1 is characterized in that: In the second stage, considering the speed of gradient descent and the complexity of calculation, the L2 distance metric is adopted; let T,S∈R w×h×c represent the feature representation of the large model and the lightweight model respectively, then the feature distillation loss l distill Defined as The Channels-based subclass distillation loss is defined as Here, β is a hyperparameter.

3. A customized deep neural network model compression system based on cloud-edge collaboration, characterized in that: A customized deep neural network model compression method based on cloud-edge collaboration for implementing any one of claims 1-2, comprising: Large model generation module; a large model refers to a deep neural network model with multiple capabilities and a relatively complex structure; this module outputs a complex model with multiple classification or recognition capabilities based on the specific resource conditions of the cloud; Model distribution module: The model distribution module is mainly used to transmit the model to the edge node through cloud-edge collaboration; the distributed data includes but is not limited to model files, configuration files and several description files related to the model; the model distribution mechanism can be completed through the inter-process communication mechanism or through some traditional file protocols; Edge-side model compression module: This module is the core module for edge-side model compression and processing. Its input is the user's personalized data set, the large model distributed in the cloud, and the classification information or target detection information that the user is concerned about. The output is a lightweight model that can be deployed on the edge or terminal side. Model deployment module: This module completes the access or deployment of lightweight models; the deployment can be in the form of applications or based on container mechanisms.

4. The customized deep neural network model compression system based on cloud-edge collaboration according to claim 3 is characterized in that: The edge side model compression module has two implementation methods, including: 3.

1. First, all classification distillation is performed, and then the output lightweight model is processed; 3.

2. Directly use subclass distillation training method.

5. The customized deep neural network model compression system based on cloud-edge collaboration according to claim 4 is characterized in that: There are two options for implementing the subclass distillation training method, including: 3.2.

1. During the training process, the Logits-based subclass distillation method is adopted; 3.2.2 During the training process, a Channels-based subclass distillation method is adopted.

6. The customized deep neural network model compression system based on cloud-edge collaboration according to claim 4 is characterized in that: Step 3.1 is as follows: In the classification task, the probability of each classification is actually calculated using the Softmax function, and Logits is actually the input layer of the Softmax layer; here we use z i To represent the input of a certain category i, that is, the Logits of a certain class; and the probability output of this class is generated for Where C is the total number of categories, i = 1 ~ C, T represents the temperature, which is used to control the softness of information distillation from the large model; For exp(z i / T) is the sum from 1 to C; when T = 1, it means that the standard Softmax function is used to calculate the probability distribution; in order to simplify the representation method, use To represent the probability distribution of all categories controlled by temperature, p is used to represent the probability distribution of all categories when T = 1; the total distillation loss of all categories can be expressed as Among them, α is a hyperparameter, which is used to indicate the proportion of knowledge transferred from the large model to the lightweight model; Η is the cross entropy loss function, p s is the probability distribution under the standard softmax function of the lightweight model, Represents the true label 0-1 distribution under the lightweight model calibration sample set; KL represents the KL divergence loss function, which is used to measure the difference in probability distribution output by the lightweight model and the large model; Represents the probability distribution of the lightweight model under the softened softmax function with temperature parameter T; Represents the probability distribution of the large model under the softened softmax function with temperature parameter T; In the process of Logits distillation training, the user uses the training set on the edge side and the loss function formed by the combination of the large model and the lightweight model, see formula (2), to perform iterative optimization training; the parameters of the large model are immutable, and the parameters of the lightweight model are continuously updated during training. After a certain number of iterations are completed, the training process ends.

7. The customized deep neural network model compression system based on cloud-edge collaboration according to claim 6 is characterized in that: Step 3.2 is as follows: In sub-category distillation, the loss function of formula (2) is still used, but the categories that users pay attention to need to be modified; let the set of all categories U = {1, 2,, C}, U attention is the category that the user is interested in and Then modify the output probability of the category for Where i',j'∈U attenti on, For exp(z j′ / T) in U attention Find the sum on top.

8. The customized deep neural network model compression system based on cloud-edge collaboration according to claim 7 is characterized in that: Step 3.2.2 is as follows: The Channels-based subclass knowledge distillation method adopts a two-stage training method. The purpose of the first stage is to match and synchronize the feature matrix; that is, while ensuring that the large model remains unchanged, a feature map with the same dimension as the tail feature tensor of the lightweight model is trained; in the second stage, feature distillation is performed based on the feature map with the same dimension; Phase 1: After the last convolution layer of the large model, add an embedded convolution layer to realize the process of feature tensor synchronization; the embedded convolution layer mainly includes two layers, the first layer is 1×1 convolution, and the second layer is SPP layer; the training in the first phase only obtains the parameters of the 1×1 convolution layer, so users only need to use their own data sets, and after a small number of epochs, the inference results of the large model can be consistent with the results after adding the embedded convolution layer; The second stage: When defining a lightweight model, the SPP layer needs to be embedded into the last convolution layer before the fully connected layer; if there is a pooling layer after the convolution layer, the SPP layer is used to replace the pooling layer; in this way, the feature vectors after the SPP layer of the large model and the lightweight model can be guaranteed to have the same dimension; in the feature distillation process, the most important goal is to reduce the migration loss between the features of the large model and the features of the lightweight model under the same sample input; therefore, first of all, a difference distance measurement method is needed; the commonly used methods for vector distance measurement are L1, L2, and MMD; in the second stage model training process, the parameters of the large model are not changed during the gradient calculation process, and only the parameters of the lightweight model need to be updated; depending on the data source and user business scenario, the training set can use the public data set provided by the cloud or the private data set of edge users.

9. The customized deep neural network model compression system based on cloud-edge collaboration according to claim 8 is characterized in that: In the second stage, considering the speed of gradient descent and the complexity of calculation, the L2 distance metric is adopted; let T,S∈R w×h×c represent the feature representation of the large model and the lightweight model respectively, then the feature distillation loss l distill Defined as: The Channels-based subclass distillation loss is defined as: Here, β is a hyperparameter.

Images