Image classification method and device based on pruned neural network, equipment and medium

Abstract

The application discloses a kind of based on pruning neural network's image classification method, device, equipment and medium, comprising: traversing the convolution layer of original image classification model, based on the weight data of the convolution layer Gaussian mixture model is constructed;Calculate the zero activation rate of convolution kernel in each convolution layer;Based on the Gaussian mixture model and the zero activation rate of convolution kernel, calculate the complex convolution correlation index of each convolution kernel, select the convolution layer with the highest mean value of the complex convolution correlation index as the target convolution layer, remove the convolution kernel in the target convolution layer greater than the complex convolution correlation index of preset threshold, obtain the pruned image classification model;The image data to be processed is input into the pruned image classification model, and image classification data is obtained.The method of the application can compress the image classification model while ensuring the accuracy of the model, thereby improving the feasibility of deploying the image classification model on edge nodes and other lightweight devices.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image classification method, apparatus, device, and medium based on a pruned neural network. Background Technology

[0002] Currently, convolutional neural networks (CNNs) are commonly used for image classification. CNN models offer high accuracy and ideal results in visual tasks, but they also have drawbacks such as complex structure, high storage overhead, and high computational resource requirements. However, due to the limited storage and computational resources of edge node devices, existing image classification models are difficult to implement accurately and efficiently on edge hardware platforms.

[0003] Existing methods compress the model by pruning it. However, the norm of the convolutional kernel weights is typically used to determine whether pruning should be performed, which yields poor results. Research shows that clustering methods are more suitable for structured model pruning. Currently, K-means clustering is widely used in model pruning research. This method assumes that the target data is symmetrically distributed about the cluster center circle. However, it cannot accurately characterize the data distribution when dealing with the complex convolutional layer weights. Furthermore, K-means clustering can only assign each data point to one cluster, failing to handle ambiguous boundaries or overlaps in the data effectively. Summary of the Invention

[0004] This application provides an image classification method, apparatus, device, and storage medium based on a pruned neural network, to at least solve the technical problem of difficulty in efficiently pruning convolutional models for image classification in related technologies.

[0005] According to one aspect of the embodiments of this application, an image classification method based on a pruned neural network is provided, comprising:

[0006] The convolutional layers of the original image classification model are traversed, and a Gaussian mixture model is constructed based on the weight data of the convolutional layers. The zero activation rate of the convolutional kernels in each convolutional layer is calculated. Based on the Gaussian mixture model and the zero activation rate of the convolutional kernels, the composite convolutional correlation index of each convolutional kernel is calculated. The composite convolutional correlation index is obtained from the zero activation rate, similarity, and overall distribution influence. The mean of the composite convolutional correlation index of each convolutional layer is calculated. The convolutional layer with the highest mean composite convolutional correlation index is selected as the target convolutional layer. Convolutional kernels in the target convolutional layer whose composite convolutional correlation index is greater than a preset threshold are removed to obtain the pruned image classification model. The image data to be processed is input into the pruned image classification model to obtain image classification data.

[0007] In one embodiment, constructing a Gaussian mixture model based on the weight data of the convolutional layer includes:

[0008] Extract the weight data of the convolutional layer;

[0009] Construct a Gaussian mixture model based on the weight data;

[0010] The parameters of the Gaussian mixture model are solved using the expectation-maximization algorithm to obtain the mean, variance, and probability that the weights of each Gaussian distribution follow each Gaussian distribution.

[0011] In one embodiment, the composite convolution correlation index of each convolution kernel is calculated based on the Gaussian mixture model and the zero activation rate of the convolution kernel, including:

[0012] Based on the Gaussian mixture model, the similarity of each convolutional kernel is calculated;

[0013] Based on the Gaussian mixture model, the overall distribution influence of each convolutional kernel is calculated;

[0014] The composite convolution correlation index of each convolution kernel is obtained by summing the zero activation rate, similarity, and overall distribution influence of each convolution kernel.

[0015] In one embodiment, based on the Gaussian mixture model, the similarity of each convolutional kernel is calculated, including:

[0016] Based on the Gaussian mixture model, the probability that the weight data of the convolution kernel follows each Gaussian distribution is obtained;

[0017] The membership degree of the convolution kernel to the Gaussian distribution is obtained by the ratio of the sum of the probabilities of all weights in the convolution kernel following a certain Gaussian distribution to the number of weights in the convolution kernel.

[0018] The membership set is obtained based on the membership degree of the convolution kernel to all Gaussian distributions;

[0019] A vector is constructed based on the membership set of the convolution kernels, and the cosine similarity between the convolution kernels is calculated using the vector.

[0020] The similarity of each convolutional kernel is determined based on the cosine similarity between the convolutional kernels.

[0021] In one embodiment, based on the Gaussian mixture model, the overall distributional influence of each convolutional kernel is calculated, including:

[0022] The variance of the convolution kernel is calculated based on the membership set of the convolution kernel.

[0023] The degree of influence of the overall distribution of the convolution kernel is determined based on the variance.

[0024] In one embodiment, after removing convolutional kernels whose composite convolution correlation index is greater than a preset threshold, the method further includes:

[0025] Calculate the mean of the compound convolutional correlation index for each convolutional layer in the network;

[0026] Pruning is performed on the convolutional kernels of the convolutional layers with the highest average convolutional correlation index until the average convolutional correlation index of each convolutional layer is lower than a preset threshold, or until the pruning rate is greater than a preset threshold.

[0027] In one embodiment, after pruning is stopped, the method further includes:

[0028] The pruned image classification model was retrained and its parameters were fine-tuned.

[0029] According to one aspect of the embodiments of this application, an image classification apparatus based on a pruned neural network is also provided, comprising:

[0030] The construction module is used to traverse the convolutional layers of the original image classification model and construct a Gaussian mixture model based on the weight data of the convolutional layers;

[0031] The zero activation rate calculation module is used to calculate the zero activation rate of the convolution kernel in each convolutional layer; the correlation index calculation module is used to calculate the composite convolution correlation index of each convolution kernel based on the Gaussian mixture model and the zero activation rate of the convolution kernel. The composite convolution correlation index is obtained from the zero activation rate, similarity and the influence of the overall distribution; and the mean of the composite convolution correlation index of each convolutional layer is calculated.

[0032] The pruning module is used to select the convolutional layer with the highest average composite convolutional correlation index as the target convolutional layer, and remove the convolutional kernels in the target convolutional layer whose composite convolutional correlation index is greater than a preset threshold, so as to obtain the pruned image classification model.

[0033] The image classification module is used to input the image data to be processed into the pruned image classification model to obtain image classification data.

[0034] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described image classification method based on a pruned neural network through the computer program.

[0035] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which is configured to execute the above-described image classification method based on a pruned neural network at runtime.

[0036] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0037] The solution provided in this application uses zero activation rate to evaluate the importance of convolutional kernels and employs a Gaussian mixture model to analyze the distribution of convolutional layer weights in a convolutional neural network. It considers the weight distribution to be a superposition of several Gaussian distributions, assigning a probability to each data point belonging to each cluster, rather than simply assigning it to a single cluster. This improves the applicability of the lightweight algorithm when there are ambiguous boundaries or overlaps in the dataset. Based on the Gaussian distribution, the similarity of convolutional kernels and the overall distribution influence are evaluated. A composite convolutional association index is obtained by combining zero activation rate, similarity, and the degree of overall distribution influence. Convolutional kernels with a composite convolutional association index greater than a preset threshold are removed, resulting in a pruned image classification model. This comprehensively considers the importance, similarity, and overall distribution influence of convolutional kernels. The image processing model pruned using this algorithm, when deployed on an edge computing platform, exhibits faster inference speed, higher model accuracy, and less accuracy loss, demonstrating significant engineering application value. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0039] Figure 1 This is a flowchart of an image classification method based on a pruned neural network according to an embodiment of this application;

[0040] Figure 2 This is a flowchart of another image classification method based on a pruned neural network according to an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of another image classification method based on a pruned neural network according to an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of a Gaussian mixture model for constructing weighted data according to an embodiment of this application;

[0043] Figure 5 This is a schematic diagram illustrating the solution of parameters using the EM algorithm according to an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of an algorithm iteration and parameter fine-tuning according to an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of an image classification device based on a pruned neural network according to an embodiment of this application;

[0046] Figure 8 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0047] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] This application uses a Gaussian mixture model to analyze the distribution of weights in convolutional layers of a convolutional neural network. It assumes that the weight distribution is a superposition of several Gaussian distributions and gives the probability that each weight data follows a certain Gaussian distribution. Based on the Gaussian distribution measurement results of the weight data in the convolutional kernel, the similarity between convolutional kernels and their impact on the overall distribution are evaluated. The APoZ (zero activation rate) value of the convolutional kernel is calculated to evaluate its importance. Then, pruning is performed by combining the similarity, importance and impact on the overall distribution of the convolutional kernels.

[0050] The following is in conjunction with the appendix Figure 1 This application provides a detailed description of the image classification method based on pruned neural networks, as exemplified by the embodiments of this application. Figure 1 As shown, the method mainly includes the following steps:

[0051] S101 iterates through the convolutional layers of the original image classification model and constructs a Gaussian mixture model based on the weight data of the convolutional layers.

[0052] The core task of image classification models is to output the corresponding category of the input image; the accuracy requirement for the specific output value is not high. However, the accuracy of the output category of a trained image classification model is relatively low after pruning. Therefore, this paper designs a convolutional kernel pruning algorithm based on a Gaussian mixture model and zero activation rate for a pre-trained image classification model. ResNet18 is a typical network model used for image classification.

[0053] In one embodiment, the dataset used is the CIFAR-10 dataset, taking the ResNet18 convolutional neural network image classification model as an example.

[0054] Specifically, the convolutional layers in the model are traversed, and a Gaussian mixture model is constructed based on the weight data of the convolutional layers. A Gaussian mixture model refers to the situation where the data follows the superposition of multiple Gaussian distributions, usually meaning that a random variable or a set of data points is a mixture of several different normal distributions. In statistics, this is called a mixture normal distribution or Gaussian mixture model.

[0055] Specifically, the weight data of the convolutional layers is extracted; a Gaussian mixture model is constructed based on the weight data; the parameters of the Gaussian mixture model are solved using the expectation-maximization algorithm to obtain the mean, variance, and probability that the weights follow each Gaussian distribution.

[0056] like Figure 4 As shown, for the pruning target convolutional layer weights X={x1,x2,...,x n} represents the observed data, where n is the number of weights in the convolutional layer. Assuming the weight data follows a superposition of K Gaussian distributions, the GMM (Gaussian Mixture Model) modeling diagram is shown below. Figure 4 As shown.

[0057] The probability density function of the weighted data is:

[0058]

[0059] Where, p i N(x|μ) represents the probability that the weight x follows the i-th Gaussian distribution. i ,∑ i Let μ represent the probability density function of the i-th Gaussian distribution. i and ∑ i Let represent the mean and variance of the Gaussian distribution. After completing the GMM model of the weighted data, it is necessary to solve for the unknown parameter θ in the equation:

[0060] θ={p1,p2,...,p K ,μ1,μ2,...,μ K ,∑1,∑2,...∑ K};

[0061] Furthermore, the unknown parameter θ is determined using maximum likelihood estimation, and the maximum likelihood function L(θ|X) is:

[0062]

[0063] Convolutional neural networks have a large number of weights. If the derivative of the above equation is taken directly, it is difficult to obtain the analytical solution of the parameters. This application uses the EM (Expectation Maximization) algorithm to solve the parameters of the Gaussian mixture model of weight data.

[0064] The EM algorithm is used to solve for the parameters after GMM modeling. The process is as follows: Figure 5 As shown, the process includes acquiring target data, randomly selecting initial parameter values, performing the E step to estimate the probability that a data point belongs to each component based on the current parameters, and performing the M step to re-estimate the parameters of the components based on the probabilities obtained in the E step. By iterating through these two steps, the converged model parameters can eventually be obtained.

[0065] The unknown parameters are solved by the EM algorithm to obtain the mean, variance, and probability that the weights of each Gaussian distribution follow each Gaussian distribution.

[0066] S102 calculates the zero activation rate of the convolutional kernel in each convolutional layer.

[0067] Specifically, the APoZ (Average Percentage of Zeros) value of the network convolutional kernel is calculated. The formula for calculating the APoZ of the convolutional kernel is as follows:

[0068]

[0069] in, Let APoZ represent the APoZ of the i-th layer and c-th channel. This represents the value of the j-th element in the output of the c-th channel of the i-th layer. Then f(·) is set to 1, otherwise it is set to 0. N represents the number of samples for measuring APoZ, and M represents The dimension of the output feature map. The larger the APoZ of the convolution kernel, the greater the proportion of zero outputs, and the less important the convolution kernel is to the neural network.

[0070] S103 calculates the composite convolutional correlation index for each convolutional kernel based on the Gaussian mixture model and the zero activation rate of the convolutional kernel. The composite convolutional correlation index is obtained from the zero activation rate, similarity, and the influence of the overall distribution. The mean of the composite convolutional correlation index for each convolutional layer is also calculated.

[0071] Specifically, the zero activation rate of each convolutional kernel is first obtained to get the APoZ value of each convolutional kernel.

[0072] Furthermore, based on the Gaussian mixture model, the similarity of each convolutional kernel is calculated.

[0073] In one implementation, based on a Gaussian mixture model, the probability that the weight data of the convolution kernel follows each Gaussian distribution is obtained; based on the ratio of the sum of the probabilities that all weight data in the convolution kernel follow a certain Gaussian distribution to the number of weights in the convolution kernel, the membership degree of the convolution kernel to that Gaussian distribution is obtained; based on the membership degree of the convolution kernel to all Gaussian distributions, a membership degree set is obtained.

[0074] Assume the convolutional layer consists of {k1,k2,...,k m It consists of m convolutional kernels, and the j-th convolutional kernel contains... Total M j There are t-th weight values. After being modeled using a Gaussian Mixture Model (GMM), the weight data satisfies the superposition of K Gaussian distributions. The probability that the i-th Gaussian distribution follows is Then there is

[0075] Furthermore, based on the ratio of the sum of probabilities that all weights in the convolution kernel follow a certain Gaussian distribution to the number of weights in the convolution kernel, we obtain the membership degree of the convolution kernel to that Gaussian distribution, which is the degree to which the convolution kernel follows that Gaussian distribution.

[0076]

[0077] in, This indicates that the j-th convolution kernel is associated with the i-th Gaussian distribution c. i The membership degree of the j-th convolution kernel with respect to all Gaussian distributions can be obtained as follows:

[0078] Furthermore, a vector is constructed based on the membership set of the convolution kernels. Each convolution kernel has a membership vector. If two convolution kernels have similar membership to the Gaussian distribution, then the vectors of the two convolution kernels are also similar.

[0079] The cosine similarity between convolutional kernels is calculated using vectors. Based on this cosine similarity, the similarity of each convolutional kernel is determined. For example, with 10 convolutional kernels, calculating the cosine similarity pairwise yields nine similarity values ​​for each kernel compared to the other nine. These nine similarity values ​​are summed to obtain the kernel's overall similarity. Since membership is always positive, there will never be a case where the cosine similarity between two convolutional kernels is negative. In such cases, the cosine similarity value ranges from 0 to 1, where 1 indicates complete overlap and 0 indicates independence. A higher cosine similarity indicates greater similarity between the convolutional kernels and the other kernels.

[0080] Furthermore, based on the Gaussian mixture model, the overall distribution influence of each convolutional kernel is calculated. This includes calculating the variance of the convolutional kernel based on its membership set, and determining the overall distribution influence of the convolutional kernel based on the variance.

[0081] In one embodiment, the membership set of the j-th convolutional kernel to the Gaussian distribution can be obtained as follows: And calculate the variance s(Q) of the membership degree. j When the membership variance of a convolutional kernel is large, it indicates that the kernel only has a high degree of conformity to the superposition of a few Gaussian distributions. Such a convolutional kernel focuses more on extracting local features. The larger the variance, the more the convolutional kernel conforms to a specific Gaussian distribution, and the greater its influence on the overall distribution. When the membership variance of a convolutional kernel is small, it indicates that the convolutional kernel conforms more to the overall distribution of the data. During pruning, such convolutional kernels should be retained first, while convolutional kernels with large variance should be removed.

[0082] Finally, based on the sum of the zero activation rate, similarity, and overall distribution influence of each convolutional kernel, the composite convolutional correlation index of each convolutional kernel is obtained.

[0083] In another alternative implementation, a first APoZ-S value can be obtained based on the sum of the zero activation rate and similarity of each convolutional kernel. Pruning is then performed if the first APoZ-S value is greater than a threshold, in which case only the zero activation rate APoZ and similarity are considered. Alternatively, a second APoZ-S value can be obtained based on the sum of the zero activation rate and the overall distribution influence of each convolutional kernel. Pruning is then performed if the second APoZ-S value is greater than a threshold, in which case only the zero activation rate APoZ and the overall distribution influence (variance s(Q)) are considered. j )).

[0084] According to this step, during pruning, not only is the importance of the convolution kernel considered, but also the similarity between convolution kernels and the overall data conformity are taken into account in combination with the Gaussian distribution, so as to ensure the efficiency, compression rate and accuracy of the pruning model.

[0085] S104 selects the convolutional layer with the highest average composite convolutional correlation index as the target convolutional layer, removes convolutional kernels in the target convolutional layer whose composite convolutional correlation index is greater than a preset threshold, and obtains the pruned image classification model.

[0086] The target convolutional layer is the layer to be pruned, and it is also the convolutional layer with the highest average composite convolutional correlation index. The composite convolutional correlation index of each convolutional kernel in the target convolutional layer is calculated, and the convolutional kernels are sorted based on the composite convolutional correlation index. An appropriate importance threshold T is set according to the required pruning ratio. When the composite convolutional correlation index of a convolutional kernel is greater than T, the convolutional kernel is pruned; otherwise, the convolutional kernel is retained.

[0087] In one embodiment, after removing convolutional kernels with a composite convolutional correlation index greater than a preset threshold, the method further includes:

[0088] Calculate the average composite convolutional correlation index of each convolutional layer in the network; select the convolutional kernel of the convolutional layer with the highest average composite convolutional correlation index for pruning, until the average composite convolutional correlation index of each convolutional layer is lower than a preset threshold, or until the pruning rate is greater than a preset threshold, then stop pruning. The threshold and pruning rate provided in this embodiment can be set according to actual needs, and this application does not impose specific limitations.

[0089] In one embodiment, after stopping pruning, the method further includes: retraining and fine-tuning the parameters of the pruned image classification model. For example... Figure 6 As shown, it is difficult to obtain a network with high compression rate and high accuracy by simply pruning and retraining based on the composite convolutional relevance index value. Therefore, the algorithm is iterated. First, the composite convolutional relevance index (CCRI) value of the convolutional kernel is calculated. The convolutional layer with the highest mean CCRI value is selected, and the kernel of the convolutional layer is pruned until the preset number of iterations is reached. The pruned model is then retrained and the parameters are fine-tuned to output the pruned model.

[0090] S105 inputs the image data to be processed into the pruned image classification model to obtain image classification data.

[0091] Image classification can be performed based on the pruned image classification model, which can be deployed on edge devices with limited computing resources and storage space to achieve image data classification.

[0092] To facilitate understanding of the methods provided in the embodiments of this application, the following description is in conjunction with the appendix. Figure 2 Further description, such as Figure 2 As shown, the method includes:

[0093] Obtain the original model, calculate the APoZ value of the convolutional kernels in the model, and perform GMM modeling on the convolutional layer weight data. Calculate the CCRI value of the convolutional kernels and calculate the mean CCRI of each convolutional layer. Select the convolutional layer with the highest mean CCRI as the target convolutional layer, sort them by importance according to CCRI values, and remove convolutional kernels in the target convolutional layer whose CCRI value is greater than a preset threshold.

[0094] Furthermore, the algorithm is iterated and the parameters are fine-tuned to output the pruned model.

[0095] Figure 3 Another implementation method is shown below, in conjunction with the appendix. Figure 3 Further description, such as Figure 3As shown, the method includes:

[0096] The APoZ values ​​of the convolutional kernels in the model are calculated based on the sample data, and then the mean APoZ value of the convolutional kernels in each convolutional layer is calculated. The convolutional layer with the highest mean value is selected as the target convolutional layer.

[0097] Furthermore, the weight data of the m convolutional kernels of the target convolutional layer are used to model a Gaussian Mixture Model (GMM). The membership variance of the convolutional kernels is calculated, and the APoZ-S value of the convolutional kernels is obtained by summing the membership variance and the zero activation rate. The kernels are then ranked by importance based on their APoZ-S values, and convolutional kernels with APoZ-S values ​​greater than a preset threshold are removed to obtain the pruned convolutional layer.

[0098] According to another aspect of the embodiments of this application, a pruned neural network-based image classification apparatus for implementing the above-described image classification method based on a pruned neural network is also provided. For example... Figure 7 As shown, the device includes:

[0099] Module 701 is used to traverse the convolutional layers of the original image classification model and construct a Gaussian mixture model based on the weight data of the convolutional layers.

[0100] The zero activation rate calculation module 702 is used to calculate the zero activation rate of the convolution kernel in each convolutional layer;

[0101] The correlation index calculation module 703 is used to calculate the composite convolution correlation index of each convolution kernel based on the Gaussian mixture model and the zero activation rate of the convolution kernel. The composite convolution correlation index is obtained from the zero activation rate, similarity and the influence of the overall distribution; and the mean of the composite convolution correlation index of each convolution layer is calculated.

[0102] The pruning module 704 is used to select the convolutional layer with the highest average composite convolutional correlation index as the target convolutional layer, remove convolutional kernels in the target convolutional layer whose composite convolutional correlation index is greater than a preset threshold, and obtain the pruned image classification model.

[0103] The image classification module 705 is used to input the image data to be processed into the pruned image classification model to obtain image classification data.

[0104] It should be noted that the image classification device based on pruned neural networks provided in the above embodiments is only illustrated by the division of the above functional modules when executing the image classification method based on pruned neural networks. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image classification device based on pruned neural networks provided in the above embodiments and the image classification method embodiments based on pruned neural networks belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0105] According to another aspect of the embodiments of this application, an electronic device corresponding to the image classification method based on pruned neural networks provided in the foregoing embodiments is also provided, so as to execute the image classification method based on pruned neural networks described above.

[0106] Please refer to Figure 8 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the image classification method based on a pruned neural network provided in any of the foregoing embodiments of this application.

[0107] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0108] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 801 is used to store programs. After receiving execution instructions, processor 800 executes the programs. The image classification method based on pruned neural networks disclosed in any of the aforementioned embodiments of this application can be applied to processor 800, or implemented by processor 800.

[0109] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0110] The electronic device provided in this application embodiment and the image classification method based on pruned neural networks provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0111] According to another aspect of the embodiments of this application, a computer-readable storage medium corresponding to the image classification method based on pruned neural networks provided in the foregoing embodiments is also provided, wherein a computer program (i.e., a program product) is stored thereon, and when the computer program is run by a processor, it executes the image classification method based on pruned neural networks provided in any of the foregoing embodiments.

[0112] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0113] The computer-readable storage medium provided in the above embodiments of this application and the image classification method based on pruned neural networks provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. An image classification method based on pruned neural networks, characterized in that, include: Traverse the convolutional layers of the original image classification model and construct a Gaussian mixture model based on the weight data of the convolutional layers; Calculate the zero activation rate of the convolutional kernel in each convolutional layer; Based on the Gaussian mixture model and the zero activation rate of the convolution kernel, the composite convolution correlation index of each convolution kernel is calculated. The composite convolution correlation index is obtained from the zero activation rate, similarity, and the degree of influence of the overall distribution. And calculate the mean of the composite convolution correlation index for each convolutional layer; The convolutional layer with the highest average composite convolutional correlation index is selected as the target convolutional layer. Convolutional kernels in the target convolutional layer whose composite convolutional correlation index is greater than a preset threshold are removed to obtain the pruned image classification model. The image data to be processed is input into the pruned image classification model to obtain image classification data.

2. The method according to claim 1, characterized in that, Constructing a Gaussian mixture model based on the weight data of the convolutional layer includes: Extract the weight data of the convolutional layer; Construct a Gaussian mixture model based on the weight data; The parameters of the Gaussian mixture model are solved using the expectation-maximization algorithm to obtain the mean, variance, and probability that the weights of each Gaussian distribution follow each Gaussian distribution.

3. The method according to claim 1, characterized in that, Based on the Gaussian mixture model and the zero activation rate of the convolution kernels, the composite convolution correlation index of each convolution kernel is calculated, including: Based on the Gaussian mixture model, the similarity of each convolutional kernel is calculated; Based on the Gaussian mixture model, the overall distribution influence of each convolutional kernel is calculated; The composite convolution correlation index of each convolution kernel is obtained by summing the zero activation rate, similarity, and overall distribution influence of each convolution kernel.

4. The method according to claim 3, characterized in that, Based on the Gaussian mixture model, the similarity of each convolutional kernel is calculated, including: Based on the Gaussian mixture model, the probability that the weight data of the convolution kernel follows each Gaussian distribution is obtained; The membership degree of the convolution kernel to the Gaussian distribution is obtained by the ratio of the sum of the probabilities of all weights in the convolution kernel following a certain Gaussian distribution to the number of weights in the convolution kernel. The membership set is obtained based on the membership degree of the convolution kernel to all Gaussian distributions; A vector is constructed based on the membership set of the convolution kernels, and the cosine similarity between the convolution kernels is calculated using the vector. The similarity of each convolutional kernel is determined based on the cosine similarity between the convolutional kernels.

5. The method according to claim 4, characterized in that, Based on the Gaussian mixture model, the overall distribution influence of each convolutional kernel is calculated, including: The variance of the convolution kernel is calculated based on the membership set of the convolution kernel. The degree of influence of the overall distribution of the convolution kernel is determined based on the variance.

6. The method according to claim 1, characterized in that, After removing convolution kernels with a correlation index greater than a preset threshold, the method further includes: Calculate the mean of the compound convolutional correlation index for each convolutional layer in the network; Pruning is performed on the convolutional kernels of the convolutional layers with the highest average convolutional correlation index until the average convolutional correlation index of each convolutional layer is lower than a preset threshold, or until the pruning rate is greater than a preset threshold.

7. The method according to claim 6, characterized in that, After pruning stops, the following also applies: The pruned image classification model was retrained and its parameters were fine-tuned.

8. An image classification device based on a pruned neural network, characterized in that, include: The construction module is used to traverse the convolutional layers of the original image classification model and construct a Gaussian mixture model based on the weight data of the convolutional layers; The zero activation rate calculation module is used to calculate the zero activation rate of the convolutional kernel in each convolutional layer; The correlation index calculation module is used to calculate the composite convolution correlation index of each convolution kernel based on the Gaussian mixture model and the zero activation rate of the convolution kernel. The composite convolution correlation index is obtained from the zero activation rate, similarity, and the degree of influence of the overall distribution. And calculate the mean of the composite convolution correlation index for each convolutional layer; The pruning module is used to select the convolutional layer with the highest average composite convolutional correlation index as the target convolutional layer, and remove the convolutional kernels in the target convolutional layer whose composite convolutional correlation index is greater than a preset threshold, so as to obtain the pruned image classification model. The image classification module is used to input the image data to be processed into the pruned image classification model to obtain image classification data.

9. An electronic device, characterized in that, The device includes a processor and a memory storing program instructions, the processor being configured to execute the image classification method based on a pruned neural network as described in any one of claims 1 to 7 when executing the program instructions.

10. A computer-readable medium, characterized in that, It stores computer-readable instructions that are executed by a processor to implement an image classification method based on a pruned neural network as described in any one of claims 1 to 7.

Images