A channel pruning method for visual understanding models used in classification tasks and its application.

Abstract

This invention belongs to the field of visual understanding technology and discloses a channel pruning method and application for a visual understanding model used in classification tasks. The method includes: calculating the importance value of each channel in a trained visual understanding model using a divergence-based channel importance function; wherein, the divergence-based channel importance function is the product of the standard deviation and coefficient of variation of the sample distribution corresponding to the output feature map of each channel in the visual understanding model, and the coefficient of variation is equal to the ratio of the standard deviation of the sample distribution to the expected value of the sample distribution; the standard deviation of the sample distribution is used to reflect the dispersion between samples; the importance values ​​of each channel are sorted, and starting from the channel with the lowest importance value, the corresponding channel is pruned until the required number of parameters and computational cost of the visual understanding model is met, resulting in a pruned lightweight visual understanding model. This invention can improve the classification performance of the model with a high pruning rate.

Description

Technical Field

[0001] This invention belongs to the field of visual understanding technology, and more specifically, relates to a channel pruning method and its application for a visual understanding model used in classification tasks. Background Technology

[0002] In recent years, deep learning technology has demonstrated powerful capabilities in the field of visual understanding. However, with the gradual increase in the number of model parameters and the continuous increase in network depth, the computational resource consumption of deep learning networks has also increased rapidly, placing higher demands on hardware performance. In particular, due to the increasing deployment of models on smartphones and embedded devices, deploying deep neural networks, which consume a large amount of computation and storage costs, on resource-constrained devices is currently a challenge for deep learning technology to move towards larger-scale applications. To address this, model pruning methods have been proposed to lightweight various visual understanding models. Among them, channel pruning (also known as filter pruning) directly removes all parameters from the entire redundant channel. Its sparse pattern has structured characteristics and does not require specific hardware or algorithm library support, making it the most widely used method.

[0003] Model pruning aims to remove redundant parts of the model while minimizing impact on prediction accuracy. Therefore, identifying redundant parts becomes a key focus of pruning methods. Current mainstream methods determine channel importance based on their contribution. These methods assume that channels with larger parameter amplitudes, larger feature map values, or higher feature map rank have a greater impact on the output and are therefore more important and less redundant. Other methods also measure channel importance based on inter-channel relationships; for example, the more similar a channel's parameters are to other channels in a convolutional layer, the higher its redundancy and the lower its importance.

[0004] While the aforementioned methods have been proven to achieve high pruning rates and are relatively stable, they all neglect the discriminative power between samples in each channel during the pruning process, leading to poor classification performance of the pruned visual understanding model. Specifically, if the feature map of a certain channel can fully and comprehensively capture and reflect the differences between samples regarding a certain attribute, then the features of that channel are beneficial for the network to distinguish between samples of different categories. However, if different samples have only extremely subtle feature differences in the feature map of a certain channel, then the features of that channel are not conducive to distinguishing samples of different categories, making it difficult to perform different category recognition tasks. In other words, sample discriminative power has a significant impact on classification results. Therefore, ignoring the discriminative power between samples in each channel during pruning will affect the classification performance of the pruned visual understanding model. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a channel pruning method and application for a visual understanding model for classification tasks, the purpose of which is to improve the classification performance of the pruned visual understanding model.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a channel pruning method for a visual understanding model for a classification task is provided, comprising:

[0007] The importance value of each channel in the trained visual understanding model is calculated using a divergence-based channel importance function. The divergence-based channel importance function is the product of the standard deviation and coefficient of variation of the sample distribution corresponding to the output feature map of each channel in the visual understanding model, and the coefficient of variation is equal to the ratio of the standard deviation of the sample distribution to the expected value of the sample distribution. The standard deviation of the sample distribution is used to reflect the degree of dispersion among samples.

[0008] The importance values ​​of each channel are sorted, and starting from the channel with the lowest importance value, the corresponding channel is pruned until the number of parameters and computational cost required by the visual understanding model are met, thus obtaining the pruned lightweight visual understanding model.

[0009] Furthermore, the visual understanding model includes convolutional layers, and / or linear layers, and activation function layers and normalization layers adjacent to the convolutional layers, and / or linear layers;

[0010] The divergence-based channel importance function is:

[0011]

[0012]

[0013] In the formula, For the j-th channel feature map of the activation function layer adjacent to the i-th layer of the convolutional layer and / or linear layer; express The variance of the distribution it follows, where, The distribution it follows is used to characterize the sample distribution corresponding to the feature map of that channel; express The coefficient of variation of the distribution it follows; express The mathematical expectation of the distribution it follows.

[0014] Furthermore, using mathematical expectation and variance The estimated value Corresponding substitution mathematical expectation and variance

[0015] estimated value The calculation method is as follows:

[0016]

[0017] Where f(x) is the activation function, and x represents a random variable. and The corresponding value represents the estimated value when the activation function is f(x). β j With γ j and represent the learnable offset coefficient and scale coefficient in the j-th channel of the normalized layer, respectively.

[0018] Furthermore, the activation function f(x) is a ReLU activation function, a ReLU6 activation function, or a tanh activation function.

[0019] Furthermore, the trained visual understanding model is obtained by training an untrained visual understanding model using a standard dataset;

[0020] After obtaining the pruned lightweight visual understanding model, the following is also included:

[0021] The pruned lightweight visual understanding model is retrained on the standard dataset to obtain the final lightweight visual understanding model.

[0022] According to a second aspect of the present invention, a method for compressing a visual understanding model for a classification task is provided, comprising:

[0023] The trained visual understanding model is pruned using the channel pruning method described in any one of the first aspects to obtain a pruned lightweight visual understanding model.

[0024] The pruned lightweight visual understanding model is used as the compressed visual understanding model.

[0025] According to a third aspect of the present invention, a visual understanding task classification method is provided, comprising: inputting an image to be classified into a trained lightweight visual understanding model to obtain a classification prediction result; wherein the lightweight visual understanding model is obtained by the channel pruning method described in any one of the first aspects.

[0026] According to a fourth aspect of the present invention, an electronic device is provided, including a computer-readable storage medium and a processor;

[0027] The computer-readable storage medium is used to store executable instructions;

[0028] The processor is configured to read executable instructions stored in the computer-readable storage medium to execute the channel pruning method according to any one of the first aspects, or to execute the visual understanding model compression method according to the second aspect, or to execute the visual understanding task classification method according to the third aspect.

[0029] According to a fifth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the channel pruning method as described in any of the first aspects, or the visual understanding model compression method as described in the second aspect, or the visual understanding task classification method as described in the third aspect.

[0030] According to a sixth aspect of the present invention, a computer program product is provided, which, when run on a computer, causes the computer to perform the channel pruning method as described in any one of the first aspects, or to implement the visual understanding model compression method as described in the second aspect, or to implement the visual understanding task classification method as described in the third aspect.

[0031] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0032] (1) In this invention, to distinguish between different categories of samples during the pruning process, a channel importance function based on divergence is designed. The function is calculated as the product of the standard deviation and coefficient of variation of the sample distribution corresponding to the output feature map of each channel in the visual understanding model. The coefficient of variation of the sample distribution controls the information entropy of the entire feature map distribution; a larger coefficient of variation results in lower information entropy. Considering that model classification is essentially a process of gradually decreasing information entropy, maintaining a low information entropy in the model's feature map is beneficial for model classification. Simultaneously, the standard deviation of the sample distribution reflects the degree of dispersion among samples. This channel importance function considers the influence of the standard deviation of the sample distribution of each channel's output feature map, encouraging greater dispersion among samples, meaning better feature discriminative power of the channel, which is more conducive to capturing subtle differences between samples and resulting in stronger classification performance. Therefore, by ranking the importance values ​​of the channels calculated using the channel importance function based on divergence and pruning the channels with lower importance values, the resulting pruned visual understanding model can improve the classification performance of the pruned visual understanding model.

[0033] (2) Preferably, the channel importance function based on divergence in this invention sorts the importance values ​​of each channel and prunes the corresponding channels starting from the channels with low importance values. This can make the features in the intermediate layers (convolutional layers and / or linear layers) of the visual understanding model retain low information entropy, which is beneficial to reduce the reduction of information entropy in subsequent layers of the visual understanding model and facilitates the improvement of classification performance.

[0034] (3) Furthermore, in order to reduce computational costs, the present invention employs mathematical expectation. and variance The estimated value Corresponding substitution mathematical expectation and variance It also provides a corresponding parameter estimation method, in which only the model parameter (β) needs to be used. j and γ j This method mines feature map distribution information, avoiding the extra computational cost of collecting sample distributions in feature maps using a large amount of real data, and accelerates the pruning process.

[0035] (4) Further, after obtaining the pruned lightweight visual understanding model, the pruned lightweight visual understanding model is retrained on the standard dataset to recover the performance loss caused by channel pruning and further improve the classification performance of the model. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of a channel pruning method for a visual understanding model used for classification tasks in an embodiment of the present invention;

[0037] Figure 2 This is a flowchart of a channel pruning method for a visual understanding model used for classification tasks in an embodiment of the present invention.

[0038] Figure 3a This is a test image in one of the embodiments of the present invention;

[0039] Figure 3b for Figure 3a The corresponding truth label;

[0040] Figure 3c for Figure 3a The corresponding unpruned baseline model that has already been initially trained;

[0041] Figure 3d for Figure 3a The corresponding segmentation prediction results of the lightweight visual understanding model after pruning, obtained by using the DCFP channel pruning method;

[0042] Figure 3e for Figure 3a The corresponding segmentation prediction results of the lightweight visual understanding model obtained by the method in the embodiments of the present invention;

[0043] Figure 4a This is another image to be tested in this embodiment of the invention;

[0044] Figure 4b for Figure 4a The corresponding truth label;

[0045] Figure 4c for Figure 4a The corresponding unpruned baseline model that has already been initially trained;

[0046] Figure 4d for Figure 4a The corresponding segmentation prediction results of the lightweight visual understanding model after pruning, obtained by using the DCFP channel pruning method;

[0047] Figure 4e for Figure 4a The corresponding segmentation prediction results of the lightweight visual understanding model obtained by the method in the embodiments of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Example 1

[0050] like Figure 1 and Figure 2 As shown, the channel pruning method for a visual understanding model used for classification tasks in this embodiment of the invention mainly includes:

[0051] The importance value of each channel in the trained visual understanding model is calculated using a divergence-based channel importance function. The channel importance function is calculated as the product of the standard deviation and the coefficient of variation of the sample distribution corresponding to the output feature map of each channel in the visual understanding model. The coefficient of variation is equal to the ratio of the standard deviation of the sample distribution to the expected value of the sample distribution. The standard deviation of the sample distribution is used to reflect the degree of dispersion among the samples.

[0052] The importance values ​​of each channel are sorted, and pruning is performed on the corresponding channels starting from the lowest importance values ​​until the required number of parameters and computational cost of the visual understanding model is met. This results in a pruned, lightweight visual understanding model, reducing the number of parameters and computational cost. In this embodiment, the required number of parameters for the visual understanding model includes the number of channel parameters and sparsity; the computational cost includes the number of floating-point operations per second of the visual understanding model. Pruning the corresponding channels includes deleting channels with low importance values ​​and their corresponding network parameters. Preferably, the importance values ​​of each channel are sorted in descending order. During pruning, channels and their corresponding network parameters are deleted sequentially from the end of the importance value sorting order until the requirements are met.

[0053] It also includes: fine-tuning the pruned lightweight visual understanding model to obtain the final lightweight visual understanding model.

[0054] Specifically, the pre-trained visual understanding model is obtained by training an untrained visual understanding model using a standard dataset. Each sample in the standard training set consists of an image related to the target visual understanding task and its corresponding ground truth label, which indicates the task result.

[0055] After obtaining the pruned visual understanding model, it is fine-tuned, including retraining the pruned model on the aforementioned standard dataset to obtain the final lightweight visual understanding model. Through retraining, the pruned lightweight model can be adjusted to fit its structure, and excellent classification performance can be recovered based on the larger divergence channels and feature maps retained by the lightweight model.

[0056] In this invention, to distinguish between different categories of samples during pruning, a channel importance function based on divergence is designed. This function is calculated as the product of the standard deviation and coefficient of variation of the sample distribution corresponding to the output feature map of each channel in the visual understanding model. The coefficient of variation controls the information entropy of the entire feature map distribution; a higher coefficient of variation results in lower information entropy. Considering that model classification is essentially a process of gradually decreasing information entropy, maintaining a low information entropy in the model's feature map is beneficial for classification. Simultaneously, the standard deviation of the sample distribution reflects the degree of dispersion among samples. This channel importance function considers the influence of the standard deviation of the sample distribution of each channel's output feature map, encouraging greater dispersion among samples, meaning better feature discriminative power for that channel. This is more conducive to capturing subtle differences between samples and resulting in stronger classification performance. Therefore, ranking the importance values ​​of channels calculated using the divergence-based channel importance function and pruning channels with lower importance values ​​improves the classification performance of the pruned visual understanding model.

[0057] The pruned lightweight visual understanding model is fine-tuned and trained on a standard dataset to optimize the learnable parameters of the remaining channels in the lightweight model, thereby restoring the model's performance and recovering the performance loss caused by channel pruning.

[0058] Preferably, the visual understanding model in this embodiment of the invention includes convolutional layers, and / or linear layers, and activation function layers and normalization layers adjacent to the convolutional layers and / or linear layers. Correspondingly, the channel importance function based on divergence in this embodiment of the invention adopts the ideal divergence channel importance function, specifically:

[0059]

[0060] in, For visual understanding models, the random variable is the feature map of the j-th channel of the activation function layer adjacent to the i-th layer of the convolutional and / or linear layers. For random variables The distribution of the j-th channel feature map of the activation function layer adjacent to the i-th layer of the convolutional layer and / or linear layer is used to characterize the sample distribution corresponding to the channel feature map. This represents a channel importance function based on divergence. The channel importance function represents the ideal divergence. express The distribution it follows The true variance express The true coefficient of variation, and the specific formula for calculating the coefficient of variation is:

[0061]

[0062] in, The coefficient of variation is... express The distribution it follows The true mathematical expectation.

[0063] In this embodiment of the invention, the importance values ​​of each channel are sorted based on the channel importance function of divergence. Starting from the channel with the low importance value, the corresponding channel is pruned. This can make the features in the intermediate layers (convolutional layers and / or linear layers) of the visual understanding model retain low information entropy, which is beneficial to reduce the reduction of information entropy in subsequent layers of the visual understanding model and facilitates the improvement of classification performance.

[0064] As a further design of the present invention, considering real distribution data ( and Inferences are derived from data on a standard dataset using a trained, unpruned visual understanding model. This requires statistical analysis of all feature maps corresponding to the samples. When the dataset and the visual understanding model are large, or when a small portion of the data cannot accurately represent the characteristics of the entire dataset, [further steps are needed]. and The computational cost is very high. Therefore, to reduce the computational cost, we can use... and The estimated value is used to replace the corresponding true value. In this embodiment of the invention, a preferred method for estimating the distribution of feature map samples is provided. and Estimation is performed by using parameters stored in the activation function layers and normalization layers following the convolutional and / or linear layers in each channel to estimate parameters such as the standard deviation and expectation of the output feature map distribution at low computational cost. These estimated values ​​are then used to replace the true values. The specific calculation method is as follows:

[0065]

[0066] in, express The estimated value; and Corresponding to and The estimated value; based on and mathematical expectation The estimated value get.

[0067]

[0068] Where f(x) is the activation function, and x represents a random variable. β j With γ j These represent the learnable bias coefficient and scale factor in the j-th channel of the normalized layer, respectively.

[0069] The estimation method constructed in this embodiment of the invention only requires the learnable offset coefficient β in the j-th channel of the normalization layer. j With scaling factor γ j Substituting the corresponding formula will achieve the desired result. and The estimation does not require the random variables in the feature map. Sampling significantly reduces computational costs.

[0070] Preferably, the activation function f(x) can be a commonly used activation function such as ReLU activation function, ReLU6 activation function or tanh activation function.

[0071] When the activation function f(x) is a ReLU activation function, and They are respectively:

[0072]

[0073] in, and Let represent the expected value and the estimated variance of the convolutional layer-batch normalized layer-ReLU activation function layer, respectively. ReLU(·) is the ReLU activation function, and erf(·) and erfc(·) are the Gaussian error function and the complementary Gaussian error function, respectively.

[0074] When the activation function f(x) is a ReLU6 activation function, and They are respectively:

[0075]

[0076] in, and represents the estimated expected value and estimated variance of the convolutional layer-batch normalization layer-ReLU6 activation function layer, respectively, where ReLU6(·) is the ReLU6 activation function.

[0077] When the activation function f(x) is the tanh activation function, and They are respectively:

[0078]

[0079] in, and Let $\mathbf{ ...

[0080] Preferably, the visual understanding model for classification tasks in the embodiments of the present invention can be a visual understanding model for visual understanding classification prediction tasks, such as an image classification and recognition model, an image segmentation model, or an image detection model.

[0081] Example 2

[0082] This invention provides a method for compressing a visual understanding model for classification tasks, comprising: pruning a trained visual understanding model using the channel pruning method described in Embodiment 1 above to obtain a pruned lightweight visual understanding model; and using the pruned lightweight visual understanding model as the compressed visual understanding model.

[0083] The relevant solutions are described in Example 1 and will not be repeated here.

[0084] The following explanation, in conjunction with some specific application scenarios, further illustrates the beneficial effects that this invention can achieve:

[0085] Taking semantic segmentation as an example of target visual understanding tasks, this paper uses ResNet50 as the backbone network and Deeplabv3 as the decoder, utilizing the ReLU activation function and its corresponding... and The channel importance calculated by the estimation method is used to perform channel pruning flow on the ResNet50-Deeplabv3 model, achieving the goal of reducing the number of floating-point operations per second (FLOPs) by 60%, in order to lightweight the visual understanding model, improve the model's running efficiency and computational resource consumption, and at the same time, try to obtain semantic segmentation performance similar to the original unpruned model.

[0086] The initial visual understanding model was trained using the ADE20K dataset. The ADE20K dataset is an image segmentation dataset for complex scenes, containing 20,000 images as the training set, 2,000 images as the validation set, and 3,000 images as the test set. Each pixel is labeled with a predefined category, and the dataset has 150 predefined semantic labels. The training set was randomly divided into training subsets of the same size, specifically, each training subset was 16 pixels in size. Data augmentation was performed on the data of each training subset to improve accuracy. The data augmentation methods included the following operations: (1) Calculating the mean of each channel of the images in the training set. (2) Subtracting the mean of each image in the training subset. (3) Randomly flipping horizontally and randomly scaling the images, with scaling factors of any one of {0.5, 0.75, 1.0, 1.5, 1.7}. During training, one training subset was trained at a time, and the end of training for all training subsets constituted the end of one iteration. Training was repeated until the number of iterations reached the upper limit, resulting in a trained visual understanding model. In actual training, the upper limit of the number of iterations was preferably 100,000. In the iterative training described above, the training process in one iteration is as follows: the network parameters of the visual understanding model are trained using the forward propagation algorithm and the backpropagation algorithm. Forward propagation calculates the loss function corresponding to each training subset, and backpropagation obtains the gradient corresponding to this training subset. The learning rate is set to 0.0001. Cross-entropy loss is used for loss calculation.

[0087] The proposed divergence-based channel pruning method for visual understanding models is used to prune the ResNet50-Deeplabv3 model. The maximum channel sparsity of each layer is set to 90%, meaning that up to 90% of the channels in the same layer are removed. Channel pruning employs a global importance setting, where each channel is ranked according to its importance against all channels in all layers to be pruned throughout the network, determining the order of subsequent channel pruning. Layers that reach the maximum channel sparsity during pruning are immediately removed from the pruning list, and all remaining channels in that layer are retained; this layer is no longer subject to channel pruning.

[0088] Similarly, the pruned visual understanding model was fine-tuned using the ADE20K dataset, with an optimal maximum of 50,000 iterations. The training subset size, data augmentation, learning rate, and other settings were kept consistent with the initial model training settings.

[0089] A lightweight visual understanding model, optimized by pruning and compression, is used to perform visual understanding tasks in different scenarios. Taking semantic segmentation, a task where prediction quality is relatively easy to demonstrate, as an example, the segmentation prediction results of the lightweight model, with a 60% reduction in floating-point computation per second in a geographic information system, are shown below. Figures 3a-3e , Figures 4a-4e As shown. Among them, Figure 3a and Figure 4a Given two different images to be tested as input, Figure 3b and Figure 4b These are the corresponding truth value labels, with different shades of color representing different categories of truth values. Figures 3c to 3e and Figures 4c to 4e In the image, different shades of color represent different category predictions. Figure 3c and Figure 4c Predicting results for the unpruned, initially trained baseline model (the unpruned, trained visual understanding model). Figure 3d and Figure 4d The image shows the segmentation prediction results of a lightweight visual understanding model after pruning, obtained using the mainstream DCFP channel pruning method. Figure 3e and Figure 4e This is the segmentation prediction result of the lightweight visual understanding model obtained in the embodiments of the present invention.

[0090] It can be seen that, Figures 3a-3e and Figures 4a-4e In comparison Figure 3c and Figure 3e ,as well as Figure 4c and Figure 4e In this embodiment of the invention, the prediction results of the lightweight visual understanding model using the divergence-based channel pruning method for trees and building walls are basically consistent with the prediction results of the baseline model. Furthermore, it is evident that, compared to… Figure 3d and 3e ,as well as Figure 4d and 4e Compared to the prediction results of DCFP, the model in this embodiment of the invention has a significant performance advantage.

[0091] Example 3

[0092] This invention provides a visual understanding task classification method, including:

[0093] The image to be classified in the visual understanding task is input into the trained lightweight visual understanding model to obtain the classification prediction result; wherein, the lightweight visual understanding model is obtained by the channel pruning method in Example 1 above.

[0094] The relevant solutions are described in Example 1 and will not be repeated here.

[0095] Example 4

[0096] This invention provides an electronic device, including a computer-readable storage medium and a processor;

[0097] Computer-readable storage media are used to store executable instructions;

[0098] The processor is used to read executable instructions stored in a computer-readable storage medium to execute the channel pruning method in Embodiment 1, or the visual understanding model compression method in Embodiment 2, or the visual understanding task classification method in Embodiment 3.

[0099] The relevant solutions are described in Examples 1-3, and will not be repeated here.

[0100] Example 5

[0101] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the channel pruning method as in Embodiment 1, the visual understanding model compression method as in Embodiment 2, or the visual understanding task classification method as in Embodiment 3.

[0102] The relevant solutions are described in Examples 1-3, and will not be repeated here.

[0103] Example 6

[0104] This invention provides a computer program product that, when run on a computer, causes the computer to execute the channel pruning method in Embodiment 1, or implement the visual understanding model compression method in Embodiment 2, or implement the visual understanding task classification method in Embodiment 3.

[0105] The relevant solutions are described in Examples 1-3, and will not be repeated here.

[0106] The channel pruning method for a divergence-based visual understanding model provided by this invention overcomes the shortcomings of previous methods that neglected the important characteristic of sample discriminability and thus lacked discriminative features by using the degree of sample distribution dispersion as the pruning standard when performing channel pruning. It can retain more discriminative channels and their corresponding feature maps in the lightweight visual understanding model after pruning, thereby achieving stronger classification performance compared to the current mainstream pruning methods.

[0107] The channel importance function based on divergence provided by this invention offers an effective quantitative indicator of sample discriminability, while also opening up a new approach to measuring channel importance. It adds a new perspective to existing methods for measuring channel importance and exhibits better classification performance than existing methods under a higher pruning ratio.

[0108] The feature map sample distribution estimation method provided by this invention can mine feature map distribution information using only model parameters, avoiding the additional large amount of computational cost caused by collecting a large amount of real data to collect sample distribution in feature maps. It can achieve the estimation of the distribution information of intermediate layer features of visual understanding model with low computational cost, and at the same time achieve the estimation of sample discrimination with high accuracy, thus accelerating the pruning process.

[0109] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A channel pruning method for a visual understanding model used in classification tasks, characterized in that, include: A divergence-based channel importance function is used to calculate the importance value of each channel in a trained visual understanding model. The trained visual understanding model is obtained by training an untrained visual understanding model using a standard dataset. Each sample in the standard training set consists of an image related to the target visual understanding task and its corresponding ground truth label, with the ground truth label indicating the task result. The divergence-based channel importance function is the product of the standard deviation and coefficient of variation of the sample distribution corresponding to the output feature map of each channel in the visual understanding model. The coefficient of variation is equal to the ratio of the standard deviation of the sample distribution to the expected value of the sample distribution. The standard deviation of the sample distribution reflects the degree of dispersion among the samples. The importance values ​​of each channel are sorted, and starting from the channel with the lowest importance value, the corresponding channel is pruned until the number of parameters and computational cost required by the visual understanding model are met, thus obtaining the pruned lightweight visual understanding model. The visual understanding model includes convolutional layers, and / or linear layers, and activation function layers and normalization layers adjacent to the convolutional layers and / or linear layers; The divergence-based channel importance function is: In the formula, For the j-th channel feature map of the activation function layer adjacent to the i-th layer of the convolutional layer and / or linear layer; express The variance of the distribution it follows, where, The distribution it follows is used to characterize the sample distribution corresponding to the feature map of that channel; express The coefficient of variation of the distribution it follows; express The expected value of the distribution it follows; Using mathematical expectation and variance The estimated value , Corresponding substitution mathematical expectation and variance ; estimated value , The calculation method is as follows: in, For activation function, Represents random variables ; and The corresponding activation function is estimated value at time , , and and represent the learnable offset coefficient and scale coefficient in the j-th channel of the normalized layer, respectively.

2. The channel pruning method according to claim 1, characterized in that, The activation function This can be a ReLU activation function, a ReLU6 activation function, or a tanh activation function.

3. The channel pruning method according to claim 1 or 2, characterized in that, The trained visual understanding model is obtained by training an untrained visual understanding model using a standard dataset; After obtaining the pruned lightweight visual understanding model, the following is also included: The pruned lightweight visual understanding model is retrained on the standard dataset to obtain the final lightweight visual understanding model.

4. A method for compressing visual understanding models for classification tasks, characterized in that, include: The trained visual understanding model is pruned using the channel pruning method described in any one of claims 1-3 to obtain a pruned lightweight visual understanding model. The pruned lightweight visual understanding model is used as the compressed visual understanding model.

5. A visual understanding task classification method, characterized in that, include: The image to be classified is input into a trained lightweight visual understanding model to obtain a classification prediction result; wherein the lightweight visual understanding model is obtained by the channel pruning method described in any one of claims 1-3.

6. An electronic device, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium to execute the channel pruning method according to any one of claims 1-3, or to execute the visual understanding model compression method according to claim 4, or to execute the visual understanding task classification method according to claim 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the channel pruning method as described in any one of claims 1-3, or the visual understanding model compression method as described in claim 4, or the visual understanding task classification method as described in claim 5.

8. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to execute the channel pruning method according to any one of claims 1-3, or implement the visual understanding model compression method according to claim 4, or implement the visual understanding task classification method according to claim 5.

Images