A network compression method and device, and a storage medium

Abstract

The embodiment of the application provides a network compression method and device, and a storage medium, comprising: determining a network layer to be executed weight clustering from a neural network, and acquiring a quantization granularity and a clustering number corresponding to the network layer; determining a parameter sharing granularity corresponding to the network layer according to the quantization granularity; and dividing the network layer into at least one parameter sharing range according to the parameter sharing granularity; respectively assigning a group of quantization parameters to each of the at least one parameter sharing range; and quantizing and clustering the weight values in the at least one parameter sharing range based on at least one group of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a network compression method and device, and a storage medium. BACKGROUND

[0002] Neural networks have been widely used in computer vision, autonomous driving and other fields for classification, identification, noise reduction, super-resolution and other tasks. However, the complex structure of neural networks requires a large amount of power consumption and computing resources even during inference, which severely limits the deployment of such technologies on mobile and embedded platforms with limited power consumption and resources. Therefore, network compression must be performed before deployment. Parameter sharing methods can be used for neural network compression.

[0003] There are currently two parameter sharing methods. The first method is to cluster all individual weights layer by layer on a pre-trained neural network, obtain the center value of each weight cluster to which each weight belongs, and replace the original value of each weight with the center value of the cluster to which each weight belongs. Then, the center values of each cluster in all layers of the pre-trained neural network are trained, and subsequent processing is performed. The second method is to divide the weights of each layer in the pre-trained neural network into several weight groups according to a certain rule, and treat each weight group as a vector in a high-dimensional space. The connected vectors in the high-dimensional space are clustered, and the weight group is also used as the smallest unit for updating in subsequent training. However, the above two methods can result in low accuracy of the compressed neural network. SUMMARY

[0004] The embodiments of the present application provide a network compression method and device, and a storage medium, which can improve the accuracy of the compressed neural network.

[0005] The technical solution of the present application is implemented as follows:

[0006] In a first aspect, the embodiments of the present application provide a network compression method, which comprises:

[0007] determining a network layer to be executed for weight clustering from a neural network, and obtaining a quantization granularity and a cluster number corresponding to the network layer;

[0008] determining a parameter sharing granularity corresponding to the network layer according to the quantization granularity, and dividing the network layer into at least one parameter sharing range according to the parameter sharing granularity;

[0009] allocating a set of quantization parameters to each of the at least one parameter sharing range, and quantizing and clustering the weights in the at least one parameter sharing range based on at least one set of quantization parameters corresponding to the at least one parameter sharing range and the cluster number, to obtain a compressed neural network.

[0010] In a second aspect, the embodiments of the present application provide a network compression device, the device comprising:

[0011] a determination unit configured to determine a network layer to be executed weight clustering from a neural network, and determine a parameter sharing granularity corresponding to the network layer according to a quantization granularity;

[0012] an acquisition unit configured to acquire a quantization granularity and a clustering number corresponding to the network layer;

[0013] a division unit configured to divide the network layer into at least one parameter sharing range according to the parameter sharing granularity;

[0014] an allocation unit configured to allocate a set of quantization parameters to each of the at least one parameter sharing range respectively;

[0015] a compression unit configured to quantize and cluster weights in the at least one parameter sharing range based on at least one set of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network.

[0016] In a third aspect, the embodiments of the present application provide a network compression device, the device comprising: a processor, a memory and a communication bus; the processor executes a running program stored in the memory to implement the network compression method as described above.

[0017] In a fourth aspect, the embodiments of the present application provide a storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the network compression method as described above.

[0018] The embodiments of the present application provide a network compression method and device, and a storage medium, the method comprising: determining a network layer to be executed weight clustering from a neural network, and acquiring a quantization granularity and a clustering number corresponding to the network layer; determining a parameter sharing granularity corresponding to the network layer according to the quantization granularity; dividing the network layer into at least one parameter sharing range according to the parameter sharing granularity; allocating a set of quantization parameters to each of the at least one parameter sharing range respectively; and quantizing and clustering weights in the at least one parameter sharing range based on at least one set of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network. With the implementation scheme described above, the quantization granularity of the network layer is used to determine the parameter sharing range of each network layer, which on the one hand makes the parameter sharing strategy applicable in the quantization scenario, avoids clustering failure after quantization, and further improves the precision of the compressed neural network; and on the other hand, finer-grained parameter sharing is performed on each network layer, which can improve the precision of the compressed neural network. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1A schematic diagram of a parameter sharing method;

[0020] Figure 2 A flow chart of a network compression method provided by an embodiment of the present application;

[0021] Figure 3 A flow chart of an exemplary network compression method provided by an embodiment of the present application;

[0022] Figure 4 A structural schematic of a network compression device provided by an embodiment of the present application Figure 1 ;

[0023] Figure 5 A structural schematic of a network compression device provided by an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0024] In the field of neural network compression, parameter sharing refers to dividing the original neural network into a smaller number of clusters by a clustering algorithm, according to the weights with similar size or the weight groups with small distance. The mean value of all weights in the same cluster is taken as the center value of the cluster, instead of the original value of all weights in the cluster. For the compressed neural network, only the index value of the cluster to which each weight belongs is stored in the storage location of the weight instead of the original weight, and a code table with the original bit width is used to store the center value of each cluster, so as to achieve the purpose of compressing the storage space and the memory read-write bandwidth. For example, the storage location of the weight belonging to the 0th cluster stores 0, and the 0th value in the code table is searched when calculating; the storage location of the weight belonging to the 1st cluster stores 1, and so on. The weight tensor in a network layer has n weights, and the original bit width of each weight is b bits. After performing parameter clustering of k clusters, the compression ratio r is shown in formula (1):

[0025]

[0026] If the training needs to continue thereafter, all weights in the same cluster are updated synchronously, which is equivalent to updating only the center value of each cluster without updating the index value. That is, the weight originally belonging to the ith cluster still belongs to the ith cluster after training, but the ith code table value corresponding to the weight may be updated by training, In this way, the amount of effective parameters in the entire neural network is greatly reduced compared to the original neural network, thereby reducing the degree of freedom of network parameters, achieving the effect of regularization, and helping to avoid overfitting during training. On the other hand, the data stored in the storage location of each weight is converted from a floating-point weight to an integer index value through a clustering algorithm, reducing the number of non-repeated data in the weight tensor and improving the regularity of the data, which is beneficial for subsequent use of existing lossless compression coding techniques to further reduce storage space and transmission bandwidth requirements.

[0027] The specific weight sharing process is shown in Figure 1 The 32-bit floating-point weights are clustered, a 2-bit unsigned integer clustering index is stored in the storage location of each weight, and a code table composed of clustering centers is generated. At the same time, the gradient of the 32-bit floating-point weight is calculated, and the gradients of the same cluster are accumulated to obtain the gradient accumulation value corresponding to each cluster. The clustering center value in the code table is fine-tuned using the gradient accumulation value corresponding to each cluster to implement the process of continuing network training on the compressed neural network.

[0028] Currently, the parameter sharing method of clustering individual weights layer by layer to obtain clustering center values cannot be directly applied to scenarios with quantization requirements. According to the accuracy requirements, we may perform quantization at different granularities, such as layer-by-layer quantization or channel-by-channel quantization. In channel-by-channel quantization, different quantization parameters are used for each channel of each layer of the convolutional neural network. At this time, if all weights in the same layer are clustered together, the weights originally belonging to the same cluster will become different weights in the quantized neural network due to the different quantization parameters applied in different channels, thereby causing the code table to be invalid. Therefore, for scenarios with high accuracy requirements, if channel-by-channel quantization is intended, this technique cannot be directly applied.

[0029] Currently, for the parameter sharing method that each layer of weights is divided into multiple weight groups, and clustering is performed in units of weight groups, the concept of weight group is introduced to try to take advantage of the structured advantage of deep neural networks. However, in high-dimensional space, clustering is less flexible than the previous technology in clustering on a single weight, and may not be able to take care of each weight in the group. And the premise of this technology is to assume that in the same layer of weights, the distribution of each weight group has certain statistical rules, and can form a limited number of clusters, rather than being randomly scattered. From a physical point of view, this implies that the original weight must contain a certain degree of redundancy, so that there are multiple groups of weights that extract similar features at the same time, which is suitable for merging by clustering. But in recent years, models have paid more and more attention to the execution ability of end-to-end reasoning, and have significantly restrained the parameter amount, and instead have focused on the innovation of network structure, such as residual connection, attention mechanism, etc. In this case, the practical significance of this scheme has been reduced, and there is a risk of excessive removal of expression ability, and the accuracy may not be higher than the scheme of clustering a single weight under the same compression rate.

[0030] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are only used for reference and are not intended to limit the embodiments of the present application.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0032] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. It should be noted that the terms "first, second, third" involved in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0033] To solve the above problems, the embodiments of the present application provide a network compression method, as shown in Figure 2 The method can include:

[0034] S101, determining a network layer to be executed weight clustering from a neural network, and obtaining a quantization granularity and a clustering number corresponding to the network layer.

[0035] The network compression method proposed in the embodiments of the present application is applicable to the scene of neural network compression in the fields of image, voice, natural language processing, control, etc. The neural network can be a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), or the like, and the specific type can be selected according to actual conditions, which is not limited in the embodiments of the present application.

[0036] In the embodiments of the present application, the neural network is a trained neural network, and therefore, before S101, the process of training the original neural network to obtain the above neural network is further included. It should be noted that the completion condition of the training is that the neural network converges or the number of training times reaches the maximum step limit. It should be noted that the training here is the training of each weight or each weight group in the original neural network.

[0037] It should be noted that the training type of training the original neural network can be floating-point or integer training, or quantized perception training.

[0038] Further, in order to improve the accuracy of weight clustering, after training the original neural network, the neural network can be further processed to obtain a processed neural network, and the weight clustering accuracy corresponding to the processed neural network is greater than the weight clustering accuracy corresponding to the neural network. The processing process can be retraining based on spectral relaxation K-means regularization, which can adjust the weight distribution of the neural network, so that the weight distribution is more concentrated, and can promote the accuracy of weight clustering; it can also be any processing method that promotes the accuracy of weight clustering.

[0039] It should be noted that for applications with high accuracy requirements, the requirement is to avoid any operation that drastically changes the weight distribution, and at this time, the process of processing the neural network to obtain the processed neural network is not performed.

[0040] In the embodiments of the present application, the original neural network can be a floating-point neural network model, or a fixed-point neural network model with any bit width, such as 16 bits, 12 bits, 11 bits, 10 bits, 8 bits, 6 bits, 4 bits, etc., or an integer neural network model, and the specific type can be selected according to actual conditions, which is not limited in the embodiments of the present application.

[0041] It should be noted that the network layer to be executed weight clustering is part or all of the network layers in the neural network, and the network layer to be executed weight clustering includes but is not limited to convolutional layer, fully connected layer, RNN cycle unit and other network layers suitable for weight clustering.

[0042] In the embodiment of the present application, each network layer is pre-configured with a corresponding quantization granularity, wherein the quantization granularity can be layer-by-layer quantization, channel-by-channel quantization, or quantization according to other standards, which can be selected according to actual conditions, and the embodiment of the present application does not make specific limitations.

[0043] It should be noted that the number of clusters corresponding to different network layers can be the same or different, and the number of clusters corresponding to different network layers can be any reasonable positive integer or an integer power of 2, which can be selected according to actual conditions, and the embodiment of the present application does not make specific limitations.

[0044] In the embodiment of the present application, the number of clusters can be pre-configured or output after processing the neural network as described above. Further, the number of clusters can be fine-tuned using methods such as sensitivity grading analysis.

[0045] S102, determine the parameter sharing granularity corresponding to the network layer according to the quantization granularity; and divide the network layer into at least one parameter sharing range according to the parameter sharing granularity.

[0046] In the embodiment of the present application, the quantization granularity is greater than or equal to the parameter sharing granularity. For example, if the quantization granularity is layer-by-layer quantization, the parameter sharing granularity can be parameter sharing within the layer range, or parameter sharing within the channel range, or parameter sharing within a smaller range; for example, if the quantization granularity is channel-by-channel quantization, the parameter sharing granularity is parameter sharing within the channel range, or parameter sharing within a smaller range.

[0047] For example, for parameter sharing within the layer range, all weights / weight groups in the layer participate in clustering together; for parameter sharing within the channel range, all weights / weight groups in the same channel participate in clustering together, and weights / weight groups in different channels do not participate in clustering together.

[0048] In the embodiment of the present application, if multiple parameter sharing granularities are determined according to the quantization granularity, the parameter sharing granularity of the network layer can be determined based on experience rules or some quantitative or qualitative indicators. The quantitative or qualitative indicators include but are not limited to certain numerical indicators, inference parameters on data sets, network structure characteristics of neural networks, etc.

[0049] S103, assign a set of quantization parameters to each of the at least one parameter sharing range respectively; and quantize and cluster the weights in the at least one parameter sharing range based on the at least one set of quantization parameters corresponding to the at least one parameter sharing range and the number of clusters, to obtain the compressed neural network.

[0050] In the embodiments of the present application, the quantization parameters can be scaling factors and offset values of linear transformation such as scaling and offset.

[0051] In the embodiments of the present application, if the platform deploying the neural network has support of a runtime library and a hardware layer, a compression format in the form of index plus code table is adopted. Specifically, the weights in the at least one parameter sharing range are quantized and clustered based on the at least one set of quantization parameters and the number of clusters, to obtain a set of index values corresponding to each of the at least one parameter sharing range and a code table corresponding to the set of index values, and the code table stores the cluster center values in one parameter sharing range; the set of index values is updated to the storage locations of the weights in the corresponding parameter sharing range, to obtain the compressed neural network. Thus, the corresponding cluster center values can be found from the code table corresponding to the parameter sharing range through the index values in the storage locations of the weights.

[0052] In the embodiments of the present application, if the platform deploying the neural network does not have support of a runtime library and a hardware layer, the corresponding cluster center values are directly stored in the storage locations of the weights. Specifically, the weights in the at least one parameter sharing range are quantized and clustered based on the at least one set of quantization parameters and the number of clusters, to obtain the cluster center values in each of the at least one parameter sharing range; the cluster center values in each of the at least one parameter sharing range are updated to the storage locations of the weights in the corresponding parameter sharing range, to obtain the compressed neural network.

[0053] It should be noted that the clustering algorithm applied to the clustering of the weights in the at least one parameter sharing range can be a K-means algorithm or any other clustering algorithm, which can be selected according to actual conditions, and the embodiments of the present application do not make specific limitations.

[0054] In the embodiments of the present application, the clustering unit for clustering the weights in the at least one parameter sharing range can be a single weight, a weight strip, a filter, a feature map or a set of weights, which can be selected according to actual conditions, and the embodiments of the present application do not make specific limitations.

[0055] Further, after obtaining the compressed neural network, the compressed neural network is further trained to obtain a trained neural network. Here, the training of the compressed neural network is only for the cluster center points.

[0056] In the embodiments of the present application, a loss function including a parameter sharing loss term is first generated, and the compressed neural network is trained by using the loss function to obtain a trained neural network. Other terms of the loss function can be the same as the loss terms used in the process of training the original neural network, or more loss terms can be selected or added in the loss function, or the loss terms in the loss function can be learned during the training process. The specific selection can be made according to the actual situation, and the embodiments of the present application do not make specific limitations.

[0057] In an optional embodiment, in order to reduce the precision loss, the above loss function can use a “demonstration” method, and the error of the compressed neural network and the original neural network and / or other model expression ability can be calculated by the output layer and / or the intermediate layer.

[0058] For example, taking convolution as an example, the parameter sharing loss term is shown in formula (2),

[0059] L ws =‖conv(X,W)-conv(X,W c )‖ n (2)

[0060] Wherein, L ws is the parameter sharing loss term, X is the input feature map tensor, W is the weight tensor before clustering, W c is the weight tensor after the original weight is replaced by the cluster center value, ‖·‖ n denotes the Ln norm, and n can be any reasonable natural number (generally, 2 can be taken).

[0061] It can be understood that the training of the compressed neural network by using the loss function including the parameter sharing loss term can reduce the influence of parameter sharing on the network expression ability, and further improve the precision.

[0062] In the embodiments of the present application, the training of the compressed neural network can be as shown in formula (3), the gradient of each weight value is first calculated, then the gradient of all weights in the same cluster is accumulated to update the cluster center, or under the premise of supporting by a deep learning (DL) framework, the operation graph is modified to redirect each weight value to the center point of the cluster to which it belongs, so that the center point directly participates in the forward inference and the backward propagation update. Figure 1

[0063] ​It should be noted that the data set required for training the original neural network, the data set required for weight clustering preprocessing of the neural network, and the data set required for training the compressed neural network can be the entire training set, a fixed part of the training set in the entire training set, or a part of the training set randomly extracted from the entire training set each time, or another separate data set. The data sets used in multiple cycles of the same step can be different or the same; the data sets used in different steps can be the same or different.

[0064] Further, if there are subsequent processing steps such as further compression, conversion of model format, segment deployment preprocessing, etc., after obtaining the trained neural network, the subsequent processing steps can be entered.

[0065] It can be understood that the quantization granularity of the network layer is used to determine the parameter sharing range of each network layer, which on the one hand enables the parameter sharing strategy to be applicable in the quantization scenario, avoids clustering failure after quantization, and further improves the accuracy of the compressed neural network; on the other hand, finer-grained parameter sharing is performed on each network layer, which can improve the accuracy of the compressed neural network.

[0066] Based on the above embodiment, an embodiment of the present application proposes a network compression method, as shown in Figure 3 The method can include:

[0067] 1. Training the original neural network until the original neural network has converged or the number of training times reaches a maximum step threshold, to obtain a neural network;

[0068] 2. Determining a network layer to be executed for weight clustering from the neural network;

[0069] 3. Iterating through each network layer of the neural network, and selecting a parameter sharing range layer by layer according to the quantization granularity of each network layer;

[0070] 4. Assigning a set of quantization parameters to each parameter sharing range, and clustering and quantizing the weights in each parameter sharing range to obtain a compressed neural network.

[0071] 5. Training the cluster center value in the compressed neural network until the compressed neural network has converged or the number of training times reaches a maximum step threshold, to obtain a trained neural network.

[0072] 6. Post-processing the trained neural network.

[0073] Based on the above embodiment, an embodiment of the present application provides a network compression device 1. As shown in Figure 4 The network compression device 1 includes:

[0074] The determining unit 11 is configured to determine a network layer to be executed weight clustering from the neural network; and determine a parameter sharing granularity corresponding to the network layer according to a quantization granularity.

[0075] The obtaining unit 12 is configured to obtain the quantization granularity and a clustering number corresponding to the network layer.

[0076] The dividing unit 13 is configured to divide the network layer into at least one parameter sharing range according to the parameter sharing granularity.

[0077] The allocating unit 14 is configured to allocate a set of quantization parameters to each of the at least one parameter sharing range respectively.

[0078] The compressing unit 15 is configured to quantize and cluster weights in the at least one parameter sharing range based on at least one set of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network.

[0079] Optionally, the compressing unit 15 is specifically configured to quantize and cluster the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number, to obtain a set of index values corresponding to each of the at least one parameter sharing range and a code table corresponding to the set of index values, the code table storing a clustering center value in a parameter sharing range; and update the set of index values to a storage location of the weights in the corresponding parameter sharing range, to obtain the compressed neural network.

[0080] And / or, the compressing unit 15 is specifically configured to quantize and cluster the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number, to obtain a clustering center value in each of the at least one parameter sharing range; and update the clustering center value in each of the parameter sharing range to a storage location of the weights in the corresponding parameter sharing range respectively, to obtain the compressed neural network.

[0081] Optionally, the apparatus further comprises a generating unit and a training unit.

[0082] The generating unit is configured to generate a loss function comprising a parameter sharing loss term.

[0083] The training unit is configured to train the compressed neural network by using the loss function, to obtain a trained neural network.

[0084] Optionally, the apparatus further comprises a preprocessing unit.

[0085] The preprocessing unit is configured to perform weight cluster preprocessing on the neural network to obtain a processed neural network, and the weight cluster accuracy corresponding to the processed neural network is greater than the weight cluster accuracy corresponding to the neural network.

[0086] Optionally, the cluster number is output after the weight cluster preprocessing on the neural network, and / or is preconfigured.

[0087] Optionally, the quantization granularity is greater than or equal to the parameter sharing granularity.

[0088] The network compression device provided in the embodiment of the present application determines the network layer to be executed for weight cluster from the neural network, and obtains the quantization granularity and the cluster number corresponding to the network layer; determines the parameter sharing granularity corresponding to the network layer according to the quantization granularity; divides the network layer into at least one parameter sharing range according to the parameter sharing granularity; allocates a group of quantization parameters to each of the at least one parameter sharing range; and quantizes and clusters the weight values in the at least one parameter sharing range based on the at least one group of quantization parameters corresponding to the at least one parameter sharing range and the cluster number, to obtain a compressed neural network. As can be seen, the network compression device provided in the embodiment of the present application uses the quantization granularity of the network layer to determine the parameter sharing range of each network layer, which on the one hand makes the parameter sharing strategy applicable in the quantization scenario, avoids cluster failure after quantization, and further improves the precision of the compressed neural network; and on the other hand, finer-grained parameter sharing is performed on each network layer, which can improve the precision of the compressed neural network.

[0089] Figure 5 The component structure of the network compression device 1 provided in the embodiment of the present application is shown in Figure 2 In actual applications, based on the same disclosure concept of the above-mentioned embodiment, as shown in Figure 5 The terminal 1 of the embodiment of the present application includes a processor 16, a memory 17 and a communication bus 18.

[0090] In the process of the specific embodiment, the above-mentioned determination unit 11, acquisition unit 12, division unit 13, allocation unit 14, compression unit 15, generation unit, training unit and preprocessing unit can be implemented by a processor 16 located on the network compression device 1. The processor 16 can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, or a microprocessor. It can be understood that, for different devices, the electronic device for implementing the above-mentioned processor function can also be other, and the present embodiment is not limited specifically.

[0091] In the embodiment of the present application, the above-mentioned communication bus 18 is used to realize the connection communication between the processor 16 and the memory 17; and the processor 16 realizes the network compression method as follows when executing the running program stored in the memory 17.

[0092] The network layer to be executed for weight clustering is determined from the neural network, and the quantization granularity and the clustering number corresponding to the network layer are acquired; the parameter sharing granularity corresponding to the network layer is determined according to the quantization granularity; and the network layer is divided into at least one parameter sharing range according to the parameter sharing granularity; each parameter sharing range in the at least one parameter sharing range is allocated a group of quantization parameters; and the weights in the at least one parameter sharing range are quantized and clustered based on at least one group of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network.

[0093] Further, the processor 16 is further configured to quantize and cluster the weight values in the at least one parameter sharing range based on the at least one set of quantization parameters and the number of clusters, to obtain a set of index values corresponding to each of the at least one parameter sharing range and a code table corresponding to the set of index values, and the code table stores a cluster center value in a parameter sharing range; update the set of index values to the storage locations of the weight values in the corresponding parameter sharing range, to obtain the compressed neural network; and / or quantize and cluster the weight values in the at least one parameter sharing range based on the at least one set of quantization parameters and the number of clusters, to obtain a cluster center value in each of the at least one parameter sharing range; and update the cluster center value in each of the at least one parameter sharing range to the storage locations of the weight values in the corresponding parameter sharing range, to obtain the compressed neural network.

[0094] Further, the processor 16 is further configured to generate a loss function including a parameter sharing loss term; and train the compressed neural network based on the loss function, to obtain a trained neural network.

[0095] Further, the processor 16 is further configured to perform weight clustering preprocessing on the neural network, to obtain a processed neural network, and the weight clustering accuracy corresponding to the processed neural network is greater than the weight clustering accuracy corresponding to the neural network.

[0096] Further, the number of clusters is output after the weight clustering preprocessing on the neural network, and / or is pre-configured.

[0097] Further, the quantization granularity is greater than or equal to the parameter sharing granularity.

[0098] Embodiments of the present application provide a storage medium having a computer program stored thereon, the computer readable storage medium stores one or more programs, the one or more programs are executable by one or more processors, and are applied to a network compression device, and the computer program implements the network compression method as described above.

[0099] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article, or device including the element.

[0100] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing an image display device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present disclosure.

[0101] The above merely describes preferred embodiments of the present application, but is not intended to limit the scope of protection of the present application.

Claims

1. A network compression method, applied to a scenario of neural network compression in the field of computer vision, characterized in that, The method comprises: determining a network layer to be executed weight clustering from a neural network, and acquiring a quantization granularity and a clustering number corresponding to the network layer; the network layer comprises a convolution layer, a full connection layer, and a recurrent unit of a recurrent neural network (RNN); determining a parameter sharing granularity corresponding to the network layer according to the quantization granularity; and dividing the network layer into at least one parameter sharing range according to the parameter sharing granularity; allocating a set of quantization parameters to each of the at least one parameter sharing range, respectively; and quantizing and clustering weights in the at least one parameter sharing range based on at least one set of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network; wherein the quantizing and clustering the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number to obtain the compressed neural network comprises: if a platform deploying the neural network has support of a runtime library and a hardware level, quantizing and clustering the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number to obtain a set of index values corresponding to each of the at least one parameter sharing range and a code table corresponding to the set of index values, the code table storing a clustering center value in a parameter sharing range; and updating the set of index values to a storage location of the weights in the corresponding parameter sharing range, to obtain the compressed neural network; if the platform deploying the neural network does not have support of the runtime library and the hardware level, quantizing and clustering the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number to obtain a clustering center value in each of the at least one parameter sharing range; and updating the clustering center value in each of the at least one parameter sharing range to a storage location of the weights in the corresponding parameter sharing range, to obtain the compressed neural network.

2. The method of claim 1, wherein, After the quantizing and clustering the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number to obtain the compressed neural network, the method further comprises: generating a loss function comprising a parameter sharing loss term; training the compressed neural network using the loss function to obtain a trained neural network.

3. The method of claim 1, wherein, Before the acquiring the quantization granularity and the clustering number corresponding to the network layer, the method further comprises: performing weight clustering preprocessing on the neural network to obtain a processed neural network, the weight clustering accuracy of the processed neural network being greater than that of the neural network.

4. The method of claim 3, wherein, The clustering number is output after the weight clustering preprocessing on the neural network, and / or is preconfigured.

5. The method of claim 1, wherein, The quantization granularity is greater than or equal to the parameter sharing granularity.

6. A network compression device, applied to the scenario of neural network compression in the field of computer vision, characterized in that, The device comprises: The determining unit is configured to determine a network layer to be executed weight clustering from the neural network; and determine a parameter sharing granularity corresponding to the network layer according to a quantization granularity; the network layer comprises a convolution layer, a full connection layer, and a recurrent unit of a recurrent neural network (RNN). The obtaining unit is configured to obtain the quantization granularity and a clustering number corresponding to the network layer. The dividing unit is configured to divide the network layer into at least one parameter sharing range according to the parameter sharing granularity. The allocating unit is configured to allocate a set of quantization parameters to each of the at least one parameter sharing range. The compression unit is configured to quantize and cluster the weights in the at least one parameter sharing range based on the at least one set of quantization parameters corresponding to the at least one parameter sharing range and the clustering number, to obtain a compressed neural network. If the platform for deploying the neural network has a runtime library and hardware level support, the compression unit is specifically configured to quantize and cluster the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number, to obtain a set of index values corresponding to each of the at least one parameter sharing range and a code table corresponding to the set of index values, the code table storing a clustering center value in a parameter sharing range; and update the set of index values to a storage location of the weights in the corresponding parameter sharing range, to obtain the compressed neural network. If the platform for deploying the neural network does not have a runtime library and hardware level support, the compression unit is specifically configured to quantize and cluster the weights in the at least one parameter sharing range based on the at least one set of quantization parameters and the clustering number, to obtain a clustering center value in each of the at least one parameter sharing range; and update the clustering center value in each of the at least one parameter sharing range to a storage location of the weights in the corresponding parameter sharing range, to obtain the compressed neural network.

7. A network compression apparatus, characterized by, The device comprises a processor, a memory, and a communication bus; the processor executes a running program stored in the memory to implement the method of any one of claims 1-5.

8. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the method of any one of claims 1-5.

Images