A model compression method, device, equipment and medium

Abstract

Embodiments of the present application provide a model compression method and device, equipment and medium, to solve the problem of low accuracy of model identification after model compression in the prior art. In the embodiments of the present application, the electronic device determines the sensitive value of each layer in the model to be compressed, and only performs parameter sharing processing on the layer with a high sensitive value, thereby reducing the problem of low accuracy of model compression, and performing parameter sharing processing on the layers in the model can improve the calculation speed of the model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model compression method, apparatus, device and medium. Background Technology

[0002] Deep neural networks have made tremendous progress in various fields recently. In pursuit of better model performance, research institutions have increasingly designed network models with more weights and deeper structures, inevitably leading to redundancy. While model performance has improved significantly, it is difficult to run such large networks on resource-constrained mobile edge devices. Therefore, research on model compression is of great significance in order to reduce the computational cost of models.

[0003] Current model compression methods mainly include the following: 1. Pruning: Pruning the existing network structure by setting parameters below a certain threshold to 0; 2. Model parameter sharing: Multiple parameters share a single value; 3. Model quantization: Reducing the number of bits used to represent floating-point parameters, such as using 8-bit fixed-point representation; 4. Matrix decomposition: Decomposing a large network matrix into multiple smaller network matrices; 5. Knowledge distillation: Using the structural information of a large network to guide the construction and training of smaller networks; 6. Lightweight network design: Designing a more compact network structure with lower computational cost.

[0004] However, model pruning has a problem: while it reduces the number of parameters, the positions of these parameters are randomized, meaning they are accessed randomly. This random access disrupts the CPU's cache structure, leading to low computational performance on the CPU. Furthermore, random access requires encoding and decoding of zero-parameter positions, limiting compression efficiency. Existing model parameter sharing only applies to filters in Convolutional Neural Networks (CNNs), such as sharing a 3x3 or 5x5 convolutional kernel as a single parameter. There are no corresponding methods for other types, such as 1x1 convolutions or fully connected layers (fc), which limits the application of model parameter sharing in model compression. In model quantization, the compression ratio is fixed, and layer-sensitive parameter compression cannot be further utilized. This necessitates multiple design iterations to obtain a suitable business model, such as multiple knowledge distillations followed by model quantization. Matrix factorization, knowledge distillation, and lightweight network design all affect the accuracy of model recognition. In other words, model compression methods in related technologies reduce the accuracy of model recognition. Summary of the Invention

[0005] This application provides a model compression method, apparatus, device, and medium to solve the problem of low model recognition accuracy after model compression in the prior art.

[0006] In a first aspect, embodiments of this application provide a model compression method, the method comprising:

[0007] Add noise to the first preset number of pre-saved samples to generate corresponding noise samples, and input each noise sample into the model to be compressed;

[0008] For each layer of the model to be compressed, obtain each first tensor of that layer; and obtain each second tensor of that layer that is input with the first preset number of samples;

[0009] The sensitivity value of a layer is determined based on the similarity between each first tensor and each second tensor; if the sensitivity value of a layer is greater than a preset threshold, parameter sharing processing is performed on that layer.

[0010] Secondly, embodiments of this application also provide a model compression apparatus, the apparatus comprising:

[0011] The input generation module is used to add noise to a first preset number of pre-saved samples to generate corresponding noise samples, and input each noise sample into the model to be compressed;

[0012] The acquisition module is used to acquire each first tensor of each layer of the model to be compressed; and to acquire each second tensor of the layer of the model into which the first preset number of samples are input.

[0013] The processing module is used to determine the sensitivity value of the layer based on the similarity between each first tensor and each second tensor; if the sensitivity value of a certain layer is greater than a preset threshold, then parameter sharing processing is performed on that layer.

[0014] Thirdly, embodiments of this application also provide an electronic device, which includes at least a processor and a memory, wherein the processor is used to execute the steps of any of the model compression methods described above when executing a computer program stored in the memory.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of any of the above-described model compression methods.

[0016] In this embodiment, the electronic device adds noise to a pre-saved first preset number of samples to generate corresponding noise samples, and inputs each noise sample into the model to be compressed. For each layer of the model to be compressed, each first tensor of that layer is obtained; and each second tensor of that layer is obtained by inputting the preset number of samples into the model. The sensitivity value of that layer is determined based on the similarity between each first tensor and each second tensor. If the sensitivity value of a certain layer is greater than a preset threshold, parameter sharing processing is performed on that layer. Since in this embodiment, the electronic device determines the sensitivity value of each layer in the model to be compressed, and only performs parameter sharing processing on layers with high sensitivity values, the problem of low accuracy in model compression can be reduced, and parameter sharing processing on the layers in the model can improve the computation speed of the model. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of a model compression process provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram illustrating the process of a model compression method in related technologies.

[0020] Figure 3 A detailed schematic diagram illustrating a model compression process provided in an embodiment of this application;

[0021] Figure 4 This application provides a detailed schematic diagram illustrating the process of determining the sensitivity value of a certain layer in an embodiment of the present application.

[0022] Figure 5 A schematic diagram illustrating a model compression process provided in this application embodiment;

[0023] Figure 6 This is a schematic diagram of the structure of a model compression device provided in an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The present application will now be described in further detail with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0026] To improve the accuracy and efficiency of model recognition, embodiments of this application provide a model compression method, apparatus, device, and medium.

[0027] The model compression method includes: adding noise to a pre-saved first preset number of samples to generate corresponding noise samples, and inputting each noise sample into the model to be compressed; for each layer of the model to be compressed, obtaining each first tensor of that layer; and obtaining each second tensor of that layer with the preset number of samples input into the model; determining the sensitivity value of that layer based on the similarity between each first tensor and each second tensor; if the sensitivity value of a certain layer is greater than a preset threshold, then parameter sharing processing is performed on that layer. This improves the accuracy and efficiency of the model's recognition.

[0028] Figure 1 A schematic diagram of a model compression process provided in this application embodiment, the process including the following steps:

[0029] S101: Add noise to the first preset number of samples to generate corresponding noise samples, and input each noise sample into the model to be compressed.

[0030] The model compression method provided in this application is applied to an electronic device, such as a PC or server.

[0031] To compress the model, the electronic device can acquire a pre-saved first preset number of samples, where the samples are original scheduling prompts, original task prompts, or other samples; the first preset number can be a large arbitrary value, such as 100. The electronic device can add noise to these first preset number of samples to generate corresponding noise samples, such as by performing slight tone changes and scaling on the image. Multiple noise samples can be generated for the same sample. After acquiring each noise sample, the electronic device can input each noise sample into the model to be compressed. The first preset number of samples are typical samples.

[0032] S102: For each layer of the model to be compressed, obtain each first tensor of that layer; and obtain each second tensor of that layer in which the first preset number of samples are input.

[0033] To accurately compress the model, the electronic device can acquire each tensor output by each layer of the model to be compressed. For ease of distinction, the electronic device can refer to the tensor output by that layer as the first tensor. The electronic device can also acquire each tensor output by that layer when a first preset number of samples are input into the model to be compressed. For ease of distinction, the electronic device can refer to the tensor output by that layer as the second tensor.

[0034] In this process, after inputting a first preset number of samples into the model to be compressed, each second tensor output by each layer of the model can also be referred to as a typical output. To improve the efficiency of model compression, the electronic device can also quantize the samples and the outputs of each layer of the model.

[0035] S103: Determine the sensitivity value of the layer based on the similarity between each first tensor and each second tensor; if the sensitivity value of a certain layer is greater than a preset threshold, then perform parameter sharing processing on that layer.

[0036] After acquiring each first tensor and each second tensor, the electronic device can determine the similarity between each first tensor and each second tensor. Specifically, for each sample, the electronic device can acquire the second tensor output by the layer when that sample is the input to the model to be compressed, and the first tensor output by the layer when that sample (after adding noise) is the input to the model to be compressed, and determine the similarity between the first tensor and the second tensor. Using this method, the electronic device can determine the similarity corresponding to each sample. Based on the similarity corresponding to each sample, the electronic device can determine the similarity between each first tensor and each tensor. Specifically, the electronic device can determine the average, mode, or median of the similarity corresponding to each sample as the similarity between each first tensor and each second tensor. After determining the similarity between each first tensor and each second tensor, the electronic device can then determine this similarity as the sensitivity value of that layer.

[0037] The higher the similarity between each first tensor and each second tensor of a certain layer, the stronger the layer's ability to handle noise and the higher its sensitivity. Adjusting the parameters of this layer has a smaller impact on the model's recognition accuracy. Therefore, if the sensitivity of a layer is greater than a preset threshold, the layer can be determined as having passed the sensitivity test, and parameter sharing processing can be performed on this layer. The preset threshold can be 90%, meaning a similarity of 90% or higher is considered valid. Specifically, the layer can be controlled to share preset parameters, or the average value of each parameter in the layer can be determined, allowing the layer's parameters to share this average value. This improves the model's processing speed while minimizing the impact on the model's recognition accuracy. In this embodiment, it is equivalent to performing sparsity and noise sensitivity tests layer by layer. Furthermore, electronic devices can also restore model performance by fine-tuning certain parameters within the model. Fine-tuning can significantly improve model performance.

[0038] The electronic device can add noise to the samples in different ways to generate corresponding noise samples. For each layer of the model, the electronic device can obtain the first tensor of each group based on the noise samples generated in different ways. If 10 ways are used to add noise to the samples, 10 groups of first tensors can be obtained. The electronic device can obtain multiple sensitivity values ​​of the layer based on each group of first tensors and each second tensor. Based on the average value of each obtained sensitivity value, it determines whether to perform parameter sharing processing in the future.

[0039] The application scenarios for this model compression include various large model compression and large model lightweighting scenarios.

[0040] Figure 2 This is a schematic diagram of a model compression method in related technologies.

[0041] Depend on Figure 2 It can be seen that the model compression method involves uniformly pruning each layer in each model. For example, a fixed pruning ratio of 50% can be set. Specifically, the parameters of that layer can be sorted by amplitude, and the 50% of data with the lowest amplitude can be set to 0. The parameters that are not set to 0 are marked with a mask. The parameters in the model are restored layer by layer, and the results of the input samples are summarized. Based on the results obtained at this time and the results obtained before the model was adjusted, the accuracy (acc) index of the model recognition is calculated. Each acc index is sorted, and similar pruning parameters are applied to each layer based on experience for similar acc indices. The model performance is restored by fine-tuning the parameters of the mask.

[0042] However, this pruning method is primarily based on amplitude ranking, assuming that larger amplitudes have a greater impact on the model, while smaller amplitudes have a smaller impact. However, due to parameter concentration, this ranking is susceptible to significant noise, making it unstable. Furthermore, experiments show that some parameters with large amplitudes, in specific business scenarios, may actually contribute less to business performance metrics due to model sparsity. Existing processes set different parameters for different layers, but the selection of specific parameters relies heavily on experience. Moreover, the existence of situations where large amplitudes contribute less leads to significant problems in the actual parameter settings for each layer, affecting the final performance of the model.

[0043] In the embodiments of this application, a consistency judgment criterion was established before the layer-by-layer parameter selection, and the validity of the test criterion was determined by adding noise to the samples. When traversing the shared control parameters, the test criterion was verified by experiments. In other words, the sensitivity described in the embodiments of this application is based on experimental judgment results rather than empirical settings.

[0044] Figure 3 A detailed schematic diagram illustrating a model compression process provided in this application embodiment includes the following steps:

[0045] S301: Add noise to the first preset number of samples to generate corresponding noise samples.

[0046] S302: Input each noise sample into the model to be compressed, obtain the first tensor output by each layer of the model, and obtain each second tensor of each layer of the model by inputting a first preset number of samples.

[0047] S303: For each layer, determine the sensitivity value of the layer based on the similarity between each first tensor and each second tensor of the layer.

[0048] S304: For each layer, if the sensitivity value of that layer is greater than a preset threshold, then parameter sharing processing is performed on that layer.

[0049] In this embodiment, the electronic device determines the sensitivity value of each layer in the model to be compressed and performs parameter sharing processing only on layers with higher sensitivity values. This reduces the problem of low accuracy in model compression and improves the computation speed of the model by performing parameter sharing processing on the layers in the model.

[0050] To improve the computational speed of the model, based on the above embodiments, in this embodiment, after adding noise to a pre-saved first preset number of samples to generate corresponding noise samples, and before inputting each noise sample into the model to be compressed, the method further includes:

[0051] Each noise sample is input into the model to be compressed. For each layer of the model to be compressed, each third tensor of that layer is obtained. The sensitivity value of that layer is determined based on the similarity between each third tensor and each second tensor. If the sensitivity value of that layer is greater than a preset threshold, the parameters of that layer are quantized.

[0052] The next step is to perform the parameter quantization process on the model, which involves inputting each noise sample into the model to be compressed.

[0053] In order to improve the computational power of the model, after adding noise to the first preset number of samples to generate corresponding noise samples, the electronic device can input each noise sample into the model to be compressed, and for each layer of the model to be compressed, obtain each third tensor of that layer; and obtain each second tensor of that layer into the model by inputting the first preset number of samples.

[0054] The sensitivity value of a layer is determined based on the similarity between each third tensor and each second tensor. If the sensitivity value of a layer is greater than a preset threshold, the layer passes the sensitivity test, and parameter quantization is performed on that layer. In this way, the electronic device can perform parameter quantization on each layer whose sensitivity is greater than the preset threshold. After parameter quantization, the subsequent step of inputting each noise sample into the model to be compressed is performed on the parameter-quantized model.

[0055] The method described in this application employs both parameter sharing and parameter quantization, which improves the model compression rate, enables better model compression, and is CPU-friendly. This feature can also be used for corresponding operator acceleration optimization in the future.

[0056] To accurately determine the sensitivity value of each layer of the model, based on the above embodiments, in this application embodiment, determining the sensitivity value of the layer based on the similarity between each first tensor and each second tensor includes:

[0057] Filter a second preset number of first important elements from each first tensor; obtain a first vector containing each first important element;

[0058] Filter a second preset number of second important elements from each second tensor; obtain a second vector containing each second important element;

[0059] The sensitivity value of this layer is determined based on the similarity between the first vector and the second vector.

[0060] To accurately determine the sensitivity values ​​of each layer of the model, the electronic device can filter a second preset number of first important elements from each first tensor. This second preset number can be 50% of the number of elements in the first tensor. Specifically, for each first tensor, the electronic device can determine the magnitude of each element, sort the elements according to their magnitudes, filter out the top-ranked second preset number of elements, and determine important elements based on these selected elements. For example, the average value of elements with the same order after sorting the elements of each tensor can be determined as an important element; for instance, the average value of the element with the largest magnitude after sorting the elements of each tensor can be determined as an important element. For ease of distinction, this important element can be called the first important element. After obtaining each first important element, a first vector containing each first important element can be obtained. Specifically, each first important element can be sorted according to its numerical value, and for each first important element, it can be placed at the position corresponding to its order in the vector to obtain the corresponding first vector. This first vector can be called the first importance tensor.

[0061] The electronic device can filter a second preset number of second important elements from each second tensor. Specifically, for each second tensor, the electronic device can determine the magnitude of each element in the second tensor, sort each element in the second tensor according to its magnitude, filter out the second preset number of elements at the top of the sorted order, and determine important elements based on the filtered elements in each second tensor. For example, the average value of elements with the same order after sorting the elements of each tensor can be determined as an important element; for example, the average value of the elements with the largest magnitude after sorting the elements of each tensor can be determined as an important element. For ease of distinction, this important element can be called the second important element. After obtaining each second important element, a second vector containing each second important element can be obtained. Specifically, each second important element can be sorted according to its numerical value, and for each second important element, it can be placed at the position corresponding to its order in the vector to obtain the corresponding second vector. Here, the second vector can be called the second importance tensor.

[0062] Electronic devices measure the model compression effect by measuring the consistency between the first and second vectors. Compared with other measures of model parameter consistency, this is more direct, has fewer restrictions, and is more effective.

[0063] To accurately obtain key elements, based on the above embodiments, in this embodiment, filtering a second preset number of key elements in each tensor includes:

[0064] For each tensor, sort each element in the tensor according to the magnitude of each element in the tensor; wherein, each tensor is either a first tensor or a second tensor; select the first of the sorted elements in a second preset number of target orders;

[0065] For each target order in the sorting, determine the position of the element of the target order in the corresponding tensor of each tensor, and determine the position that appears most frequently as the target position corresponding to the target order; based on the element at the target position in each tensor, determine the important elements of the target order.

[0066] To accurately acquire important elements, the electronic device can determine the amplitude of each element in each tensor. The specific methods for determining the amplitude of each element in a given tensor are existing technologies and will not be elaborated here. After determining the amplitude of each element in the tensor, the elements in the tensor can be sorted according to their amplitude values. In the sorted sequence, the smaller the order, the higher the amplitude, and the greater the importance within the tensor. After sorting each element in the tensor, the electronic device can filter the top few target orders. Each tensor can be either a first tensor or a second tensor.

[0067] For each target order in the sorting process, the electronic device can determine the position of the element corresponding to that target order in the tensor within the tensor itself. This position is the candidate position, and the position that appears most frequently is determined as the target position corresponding to that target order. This target position can be called the tensor importance coordinate. Specifically, the electronic device can generate a histogram hist[i][j] for each target order, where i is the order and j is the corresponding candidate position. Based on the histogram statistics, the position with the highest probability in the histogram is taken as the target position corresponding to that target order. For example, hist[0][0,0,0] = 80000 means that the number of tensors with the element at position [0,0,0] after sorting is 80000. The order 0 refers to the first tensor in the sorting result. Based on the element at the target position in each tensor, the important elements of that target order are determined. Specifically, the maximum or minimum value of the element at the target position in each tensor can be determined as the important elements of that target order. It should be noted that if each tensor is a first tensor, then the important element of the target order is the first important element of the target order; if each tensor is a second tensor, then the important element of the target order is the second important element of the target order. The electronic device solves the problems of unstable amplitude sorting and sparsity by adding noise to the samples and processing the corresponding histograms.

[0068] It should be noted that while ranking elements by amplitude has some merit, the sparsity of large models may lead to some high-amplitude elements contributing less to the business logic, and the amplitude ranking is unstable due to noise. Therefore, this application also designs a new method for determining layer sensitivity. Unlike traditional sensitivity determination methods that obtain model performance metrics through statistics, this application's layer sensitivity is based on important components within the tensor. To this end, noise is added to the samples, the elements in each tensor output by each layer are ranked by amplitude, and histogram statistics are performed based on the coordinates of the importance response. Important elements are extracted based on the statistically important coordinates. The similarity between the first vector composed of important elements extracted from the noisy samples and the vector composed of important elements extracted from the unnoisy original samples is calculated, and it is determined whether the similarity is greater than a preset value. This determines whether parameter sharing processing should be performed on the corresponding layer. This method effectively removes the influence of noise and sparsity.

[0069] In this embodiment, noise is added to the same sample to form different samples, thereby simulating the noise in amplitude and position of the tensor itself. The influence of this noise is removed by statistical target order, and the important elements are obtained by isolating the truly important positions from the tensor, thereby generating the first vector.

[0070] To accurately determine key elements, based on the above embodiments, in this embodiment, determining the key elements of the target order based on the elements at the target position in each tensor includes:

[0071] The average value of the elements at the target position in each tensor is determined as the important element of the target order; or,

[0072] The mode of the element at the target position in each tensor is determined as the important element of the target order; or,

[0073] The median of the elements at the target position in each tensor is determined as the important element of the target order.

[0074] After determining the target position corresponding to each target order in each tensor, the electronic device can determine the important element of each target order by taking the average value of the elements at that target position in each tensor. The electronic device can also determine the important element of each target order by taking the mode of the elements at that target position in each tensor. The electronic device can also determine the important element of each target order by taking the median of the elements at that target position in each tensor. In addition, the electronic device can also use other methods to determine the important element of each target order based on the elements at that target position in each tensor. The specific method of determining the important element of each target order is not limited here.

[0075] To accurately determine the sensitivity value of each layer, based on the above embodiments, in this embodiment, obtaining the vector containing each important element includes:

[0076] For each target order, the important elements of that target order are placed at the corresponding positions in the vector to obtain the corresponding vector.

[0077] After determining each important element, the electronic device can place the important elements of each target order at their corresponding positions for each target order. In this way, the electronic device can place the important elements of each target order into a vector, thus obtaining the corresponding vector.

[0078] Figure 4A detailed process diagram for determining the sensitivity value of a certain layer is provided in the embodiments of this application. The process includes the following steps:

[0079] S401: For each first tensor of this layer, sort each element in the tensor according to the magnitude of each element in the first tensor.

[0080] S402: Filter the second preset number of target orders that are sorted first.

[0081] S403: For each target order in the sorting, determine the position of the element of the target order in the corresponding tensor of each first tensor, and determine the position that appears most frequently as the target position corresponding to the target order.

[0082] S404: Based on the element at the target position in each first tensor, determine the first important element of the target order. And obtain the second important element for each target.

[0083] S405: Get the first vector containing each first important element, and get the second vector containing each second important element.

[0084] S406: Determine the sensitivity value of this layer based on the similarity between the first vector and the second vector.

[0085] To improve the efficiency and accuracy of model recognition, based on the above embodiments, in this embodiment, the parameter sharing process for this layer includes:

[0086] Get the numerical range of the shared parameters stored for this layer;

[0087] Repeat the following steps:

[0088] Obtain the smallest unselected value within the specified range, and divide the parameters of the layer into multiple groups according to the value. For each group, determine the shared parameter value corresponding to the group based on the parameter value of each parameter in that group, and share the shared parameter value with each parameter in the group. Input each noise sample into the model after parameter adjustment, obtain each fourth tensor output by the layer after parameter adjustment, and determine the sensitivity value to be processed for the layer based on the similarity between each fourth tensor and each second tensor. If the sensitivity value to be processed is higher than a preset threshold, then the value is determined to be a selected value.

[0089] The process continues until the determined sensitive value to be processed is lower than the preset threshold, at which point the previously selected value is determined as the target value; then, parameter sharing processing is performed within the corresponding group based on each previously determined shared parameter value.

[0090] When performing parameter sharing processing on a certain layer, if all parameters in that layer share the same parameter, it may affect the accuracy of model recognition. In order to improve the accuracy of model recognition, electronic devices can perform parameter sharing processing on several consecutive adjacent parameters in a certain layer.

[0091] Specifically, the electronic device acquires the numerical range of shared parameters stored for that layer. The values ​​within this range refer to parameters in that layer that are shared; the numerical range can be [2, 16]. The electronic device can repeatedly execute the following steps: acquire the smallest unselected value within the numerical range; divide the parameters of that layer into multiple groups according to the acquired value; specifically, group each acquired value into a group until the number of remaining parameters is less than the acquired value, then group the remaining parameters into a single group. After dividing the parameters of that layer into multiple groups, for each group, determine the corresponding shared parameter value based on the parameter value of each parameter in that group. Specifically, the average, mode, or median of the parameter values ​​of each parameter in that group can be used as the corresponding shared parameter value, and each parameter in that group shares this shared parameter value. Through this method, the electronic device ensures that each parameter in each group shares the same parameter value, thus obtaining the model after parameter adjustment.

[0092] After obtaining the model with adjusted parameters, the electronic device can input each noise sample into the model with adjusted parameters and obtain each fourth tensor output by the layer after adjusting the parameters. The electronic device can determine the processing sensitivity value of the layer based on the similarity between each fourth tensor and each second tensor. Specifically, how to determine the processing sensitivity value of the layer based on the similarity between each fourth tensor and each second tensor has been described in the above embodiments and will not be repeated here. If the processing sensitivity value is higher than the preset threshold, it means that the model can still accurately identify the data, and the value can be determined as the selected value.

[0093] The electronic device repeats the above steps until the determined sensitive value to be processed is lower than a preset threshold. At this point, it indicates that the model cannot accurately identify the value, and the previously selected value can be determined as the target value. Parameter sharing processing is then performed within the corresponding group based on each previously determined shared parameter value. Specifically, how to perform parameter sharing processing within the corresponding group has been described above and will not be repeated here. The target value is the maximum control parameter that meets the conditions. In this embodiment, the parameter sharing processing is not based on prior experience regarding sensitivity, but rather on iterative processes that gradually increase the difficulty, and the consistency of the results is then determined.

[0094] In model compression technology, this application proposes a novel model compression method that simultaneously utilizes parameter sharing and model quantization. During parameter sharing, multiple consecutive adjacent parameters are shared, effectively condensing them into a single parameter, thus breaking away from the traditional method of condensing convolutional kernels into a single parameter. Furthermore, the control parameters for parameter sharing are iteratively applied within a certain range on layers that pass layer sensitivity testing, forcing less sensitive layers to accept more parameter sharing, thereby achieving higher compression efficiency. Moreover, the parameter sharing processing in this application differs from the method based on a fixed convolutional template; by selecting consecutive adjacent target values ​​of parameters, it facilitates control, eliminates the need for positional encoding, further simplifies operator operations, and improves operational efficiency.

[0095] The following describes the model compression method provided in this application embodiment in detail, taking the compression of a large visual model as an example. For instance, the large visual model has 100 layers with parameters. 10,000 typical images of various types are selected as samples. The electronic device can first input the samples into the large model, and then obtain the output of each layer through inference, resulting in a total of 1 million tensors. Ten noisy samples are generated for each image, with the noise addition method involving random chroma transformation, brightness transformation, and resampling. At this point, the parameterized layers of the large model can be quantized first, but parameter sharing is not performed. The noisy image is then inferred through the quantized model, with each layer outputting 100,000 tensors.

[0096] Figure 5 This application provides a schematic diagram of a model compression process, which includes the following steps:

[0097] S501: Add noise to the first preset number of samples to generate corresponding noise samples.

[0098] S502: Input each noise sample into the model to be compressed.

[0099] S503: For each layer of the model to be compressed, obtain each third tensor of that layer, and obtain each second tensor of that layer by inputting a first preset number of samples into the model. Determine the sensitivity value of that layer based on the similarity between each third tensor and each second tensor. If the sensitivity value of that layer is greater than a preset threshold, then perform parameter quantization processing on that layer.

[0100] S504: Input each noise sample into the model after parameter quantization. For each layer of the model after parameter quantization, obtain each first tensor of that layer; and obtain each second tensor of that layer in the model with a first preset number of samples input. Determine the sensitivity value of that layer based on the similarity between each first tensor and each second tensor. If the sensitivity value of a certain layer is greater than a preset threshold, then perform parameter sharing processing on that layer.

[0101] For each of the 100,000 tensors in each layer, each tensor is first sorted by amplitude, and then histograms are calculated according to importance. Assuming it's the first layer, [0,0,0], [5,22,22]... represent the target positions corresponding to the target order. Based on these, important elements are extracted from the 100,000 tensors to generate the first vector. Then, based on the target positions corresponding to the target order, important elements are extracted from the 10,000 original tensors to generate the second vector. For noise samples generated using different noise-adding methods, 10 groups are formed between the important tensors corresponding to the importance tensors and the important tensors extracted from the 100,000 noisy tensors. The similarity between the first and second vectors is determined. Specifically, cosine similarity can be determined. If the similarity is greater than 0.95, the layer can be quantized. If the similarity for layers 55 and 56 is less than 0.95, the layer is not quantized. Starting from the first layer, the initial model of the first layer is quantized, and the shared parameter control parameter is set to 1. The control parameter is iterated in the range [2, 16]. If the sensitivity value determined when iterating to the value 15 is less than 0.95, then the target value for this layer is 14, and parameter sharing processing is performed for this first layer based on the target value of 14. This process continues until parameter sharing processing is performed for each layer whose sensitivity value is higher than the preset threshold. If fine-tuning is required, the modified final model is fine-tuned.

[0102] Figure 6 This is a schematic diagram of a model compression device provided in an embodiment of this application. The device includes:

[0103] The input generation module 601 is used to add noise to a first preset number of samples to generate corresponding noise samples, and input each noise sample into the model to be compressed.

[0104] The acquisition module 602 is used to perform parameter quantization processing on each parameter layer in the model to be compressed; for each layer of the model to be compressed, it acquires each first tensor of that layer; and acquires each second tensor of that layer in the model by inputting the first preset number of samples.

[0105] The processing module 603 is used to determine the sensitivity value of the layer based on the similarity between each first tensor and each second tensor; if the sensitivity value of a certain layer is greater than a preset threshold, then parameter sharing processing is performed on that layer.

[0106] In one possible implementation, the processing module 603 is further configured to, after adding noise to a pre-saved first preset number of samples to generate corresponding noise samples, and before inputting each noise sample into the model to be compressed, further include:

[0107] Each noise sample is input into the model to be compressed. For each layer of the model to be compressed, each third tensor of that layer is obtained. The sensitivity value of that layer is determined based on the similarity between each third tensor and each second tensor. If the sensitivity value of that layer is greater than a preset threshold, the parameters of that layer are quantized.

[0108] The next step is to perform the parameter quantization process on the model, which involves inputting each noise sample into the model to be compressed.

[0109] In one possible implementation, the processing module 603 is specifically configured to: filter a second preset number of first important elements in each first tensor; obtain a first vector containing each first important element; filter a second preset number of second important elements in each second tensor; obtain a second vector containing each second important element; and determine the sensitivity value of the layer based on the similarity between the first vector and the second vector.

[0110] In one possible implementation, the processing module 603 is specifically configured to sort each element in a tensor according to the magnitude of each element in the tensor; wherein each tensor is a first tensor or a second tensor; filter the first preset number of target orders; for each target order in the sorting, determine the position of the element of the target order in the corresponding tensor of each tensor, and determine the position that appears most frequently as the target position corresponding to the target order; and determine the important elements of the target order based on the elements at the target position in each tensor.

[0111] In one possible implementation, the processing module 603 is specifically configured to determine the average value of the elements at the target position in each tensor as an important element of the target order; or, determine the mode of the elements at the target position in each tensor as an important element of the target order; or, determine the median of the elements at the target position in each tensor as an important element of the target order.

[0112] In one possible implementation, the processing module 603 is specifically used to place the important elements of each target order into the corresponding positions in the vector to obtain the corresponding vector.

[0113] In one possible implementation, the processing module 603 is specifically used to obtain the numerical range of shared parameters stored for the layer; repeatedly execute the following steps: obtain the currently unselected and smallest value in the numerical range, and divide the parameters of the layer into multiple groups according to the value; for each group, determine the corresponding shared parameter value according to the parameter value of each parameter in the group, and share the shared parameter value with each parameter in the group; input each noise sample into the model after parameter adjustment, obtain each fourth tensor output by the layer after parameter adjustment, and determine the processing sensitivity value of the layer according to the similarity between each fourth tensor and each second tensor; if the processing sensitivity value is higher than a preset threshold, determine the value as a selected value; until the determined processing sensitivity value is lower than the preset threshold, determine the previously selected value as the target value; perform parameter sharing processing in the corresponding group according to each previously determined shared parameter value.

[0114] Based on the above embodiments, Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 7 As shown, it includes: processor 701, communication interface 702, memory 703 and communication bus 704, wherein processor 701, communication interface 702 and memory 703 communicate with each other through communication bus 704.

[0115] The memory 703 stores a computer program, which, when executed by the processor 701, causes the processor 701 to perform the following steps:

[0116] Add noise to the first preset number of pre-saved samples to generate corresponding noise samples, and input each noise sample into the model to be compressed;

[0117] Perform parameter quantization processing on each parameter layer in the model to be compressed; for each layer of the model to be compressed, obtain each first tensor of that layer; and obtain each second tensor of that layer in the model by inputting the first preset number of samples.

[0118] The sensitivity value of a layer is determined based on the similarity between each first tensor and each second tensor; if the sensitivity value of a layer is greater than a preset threshold, parameter sharing processing is performed on that layer.

[0119] Furthermore, the processor 701 is also configured to input each noise sample into the model to be compressed, and for each layer of the model to be compressed, obtain each third tensor of that layer; determine the sensitivity value of that layer based on the similarity between each third tensor and each second tensor; and if the sensitivity value of that layer is greater than a preset threshold, then perform parameter quantization processing on that layer.

[0120] The next step is to perform the parameter quantization process on the model, which involves inputting each noise sample into the model to be compressed.

[0121] Furthermore, the processor 701 is specifically configured to filter a second preset number of first important elements in each first tensor; and obtain a first vector containing each first important element;

[0122] Filter a second preset number of second important elements from each second tensor; obtain a second vector containing each second important element;

[0123] The sensitivity value of this layer is determined based on the similarity between the first vector and the second vector.

[0124] Furthermore, the processor 701 is specifically configured to sort each element in a tensor according to the magnitude of each element in the tensor; wherein each tensor is a first tensor or a second tensor; and to filter the first of a second preset number of target orders.

[0125] For each target order in the sorting, determine the position of the element of the target order in the corresponding tensor of each tensor, and determine the position that appears most frequently as the target position corresponding to the target order; based on the element at the target position in each tensor, determine the important elements of the target order.

[0126] Further, the processor 701 is specifically configured to determine the average value of the elements at the target position in each tensor as the important element of the target order; or,

[0127] The mode of the element at the target position in each tensor is determined as the important element of the target order; or,

[0128] The median of the elements at the target position in each tensor is determined as the important element of the target order.

[0129] Furthermore, the processor 701 is specifically used to place the important elements of each target order into the corresponding positions in the vector to obtain the corresponding vector.

[0130] Furthermore, the processor 701 is specifically used to obtain the numerical range of the shared parameters stored for this layer;

[0131] Repeat the following steps:

[0132] Obtain the smallest unselected value within the specified range, and divide the parameters of the layer into multiple groups according to the value. For each group, determine the shared parameter value corresponding to the group based on the parameter value of each parameter in that group, and share the shared parameter value with each parameter in the group. Input each noise sample into the model after parameter adjustment, obtain each fourth tensor output by the layer after parameter adjustment, and determine the sensitivity value to be processed for the layer based on the similarity between each fourth tensor and each second tensor. If the sensitivity value to be processed is higher than a preset threshold, then the value is determined to be a selected value.

[0133] The process continues until the determined sensitive value to be processed is lower than the preset threshold, at which point the previously selected value is determined as the target value; then, parameter sharing processing is performed within the corresponding group based on each previously determined shared parameter value.

[0134] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0135] The communication interface 702 is used for communication between the above-mentioned electronic device and other devices.

[0136] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0137] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0138] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, the electronic device performs the following steps:

[0139] The memory stores a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0140] Add noise to the first preset number of pre-saved samples to generate corresponding noise samples, and input each noise sample into the model to be compressed;

[0141] For each layer of the model to be compressed, obtain each first tensor of that layer; and obtain each second tensor of that layer that is input with the first preset number of samples;

[0142] The sensitivity value of a layer is determined based on the similarity between each first tensor and each second tensor; if the sensitivity value of a layer is greater than a preset threshold, parameter sharing processing is performed on that layer.

[0143] In one possible implementation, after adding noise to a pre-saved first preset number of samples to generate corresponding noise samples, and before inputting each noise sample into the model to be compressed, the method further includes:

[0144] Each noise sample is input into the model to be compressed. For each layer of the model to be compressed, each third tensor of that layer is obtained. The sensitivity value of that layer is determined based on the similarity between each third tensor and each second tensor. If the sensitivity value of that layer is greater than a preset threshold, the parameters of that layer are quantized.

[0145] The next step is to perform the parameter quantization process on the model, which involves inputting each noise sample into the model to be compressed.

[0146] In one possible implementation, determining the sensitivity value of the layer based on the similarity between each first tensor and each second tensor includes:

[0147] Filter a second preset number of first important elements from each first tensor; obtain a first vector containing each first important element;

[0148] Filter a second preset number of second important elements from each second tensor; obtain a second vector containing each second important element;

[0149] The sensitivity value of this layer is determined based on the similarity between the first vector and the second vector.

[0150] In one possible implementation, filtering a second preset number of key elements in each tensor includes:

[0151] For each tensor, sort each element in the tensor according to the magnitude of each element in the tensor; wherein, each tensor is either a first tensor or a second tensor; select the first of the sorted elements in a second preset number of target orders;

[0152] For each target order in the sorting, determine the position of the element of the target order in the corresponding tensor of each tensor, and determine the position that appears most frequently as the target position corresponding to the target order; based on the element at the target position in each tensor, determine the important elements of the target order.

[0153] In one possible implementation, the key elements for determining the target order based on the elements at the target position in each tensor include:

[0154] The average value of the elements at the target position in each tensor is determined as the important element of the target order; or,

[0155] The mode of the element at the target position in each tensor is determined as the important element of the target order; or,

[0156] The median of the elements at the target position in each tensor is determined as the important element of the target order.

[0157] In one possible implementation, obtaining the vector containing each important element includes:

[0158] For each target order, the important elements of that target order are placed at the corresponding positions in the vector to obtain the corresponding vector.

[0159] In one possible implementation, the parameter sharing process for this layer includes:

[0160] Get the numerical range of the shared parameters stored for this layer;

[0161] Repeat the following steps:

[0162] Obtain the smallest unselected value within the specified range, and divide the parameters of the layer into multiple groups according to the value. For each group, determine the shared parameter value corresponding to the group based on the parameter value of each parameter in that group, and share the shared parameter value with each parameter in the group. Input each noise sample into the model after parameter adjustment, obtain each fourth tensor output by the layer after parameter adjustment, and determine the sensitivity value to be processed for the layer based on the similarity between each fourth tensor and each second tensor. If the sensitivity value to be processed is higher than a preset threshold, then the value is determined to be a selected value.

[0163] The process continues until the determined sensitive value to be processed is lower than the preset threshold, at which point the previously selected value is determined as the target value; then, parameter sharing processing is performed within the corresponding group based on each previously determined shared parameter value.

[0164] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0166] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A model compression method, characterized in that, The method includes: Noise is added to a first preset number of pre-saved samples to generate corresponding noise samples, and each noise sample is input into the model to be compressed; wherein, adding noise to the samples includes performing tone transformation or scaling on the image; For each layer of the model to be compressed, obtain each first tensor output by that layer; and obtain each second tensor output by that layer in the model by inputting the first preset number of samples. The sensitivity value of a layer is determined based on the similarity between each first tensor obtained from each noise sample and each second tensor obtained from each sample; if the sensitivity value of a layer is greater than a preset threshold, the layer is determined to be a layer that passes the sensitivity test, and parameter sharing processing is performed on the layer. The parameter sharing process for this layer includes: Get the numerical range of the shared parameters stored for this layer; Repeat the following steps: Obtain the smallest unselected value within the specified range, and divide the parameters of the layer into multiple groups according to the value. For each group, determine the shared parameter value corresponding to the group based on the parameter value of each parameter in that group, and share the shared parameter value with each parameter in the group. Input each noise sample into the model after parameter adjustment, obtain each fourth tensor output by the layer after parameter adjustment, and determine the sensitivity value to be processed for the layer based on the similarity between each fourth tensor and each second tensor. If the sensitivity value to be processed is higher than a preset threshold, then the value is determined to be a selected value. The process continues until the determined sensitive value to be processed is lower than the preset threshold, at which point the previously selected value is determined as the target value; then, parameter sharing processing is performed within the corresponding group based on each previously determined shared parameter value.

2. The method according to claim 1, characterized in that, After adding noise to a pre-saved first preset number of samples to generate corresponding noise samples, and before inputting each noise sample into the model to be compressed, the method further includes: Each noise sample is input into the model to be compressed. For each layer of the model to be compressed, each third tensor of that layer is obtained. The sensitivity value of that layer is determined based on the similarity between each third tensor and each second tensor. If the sensitivity value of that layer is greater than a preset threshold, the parameters of that layer are quantized. The next step is to perform the parameter quantization process on the model, which involves inputting each noise sample into the model to be compressed.

3. The method according to claim 1, characterized in that, The step of determining the sensitivity value of the layer based on the similarity between each first tensor obtained based on each noise sample and each second tensor obtained based on each sample includes: Filter a second preset number of first important elements from each first tensor; obtain a first vector containing each first important element; Filter a second preset number of second important elements from each second tensor; obtain a second vector containing each second important element; The sensitivity value of this layer is determined based on the similarity between the first vector and the second vector.

4. The method according to claim 3, characterized in that, The second preset number of important elements to be selected in each tensor includes: For each tensor, sort each element in the tensor according to the magnitude of each element in the tensor; wherein, each tensor is either a first tensor or a second tensor; select the first of the sorted elements in a second preset number of target orders; For each target order in the sorting, determine the position of the element of the target order in the corresponding tensor of each tensor, and determine the position that appears most frequently as the target position corresponding to the target order; based on the element at the target position in each tensor, determine the important elements of the target order.

5. The method according to claim 4, characterized in that, The key elements for determining the order of targets based on the elements at the target position in each tensor include: The average value of the elements at the target position in each tensor is determined as the important element of the target order; or, The mode of the element at the target position in each tensor is determined as the important element of the target order; or, The median of the elements at the target position in each tensor is determined as the important element of the target order.

6. The method according to claim 4, characterized in that, Obtaining a vector containing each important element includes: For each target order, the important elements of that target order are placed at the corresponding positions in the vector to obtain the corresponding vector.

7. A model compression device, characterized in that, The device includes: The input generation module is used to add noise to a first preset number of pre-saved samples to generate corresponding noise samples, and input each noise sample into the model to be compressed; wherein, adding noise to the samples includes performing tone transformation or scaling on the image; The acquisition module is used to acquire each first tensor output by each layer of the model to be compressed; and to acquire each second tensor output by the layer of the model after inputting the first preset number of samples. The processing module is used to determine the sensitivity value of the layer based on the similarity between each first tensor obtained based on each noise sample and each second tensor obtained based on each sample; if the sensitivity value of a certain layer is greater than a preset threshold, the layer is determined to be a layer that passes the sensitivity test, and parameter sharing processing is performed on the layer. Specifically, the processing module is used to obtain the numerical range of shared parameters stored for the layer; repeatedly execute the following steps: obtain the smallest unselected value in the numerical range, and divide the parameters of the layer into multiple groups according to the value; for each group, determine the corresponding shared parameter value based on the parameter value of each parameter in the group, and share the shared parameter value with each parameter in the group; input each noise sample into the model after parameter adjustment, obtain each fourth tensor output by the layer after parameter adjustment, and determine the processing sensitivity value of the layer based on the similarity between each fourth tensor and each second tensor; if the processing sensitivity value is higher than a preset threshold, determine the value as a selected value; until the determined processing sensitivity value is lower than the preset threshold, determine the previously selected value as the target value; perform parameter sharing processing within the corresponding group based on each previously determined shared parameter value.

8. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, wherein the processor is configured to execute the steps of any of the model compression methods described in claims 1-6 when executing a computer program stored in the memory.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, performs the steps of the model compression method according to any one of claims 1-6.

Images