A method and apparatus for compressing and decompressing a neural network model

Abstract

This application discloses a method and apparatus for compressing and decompressing neural network models, used to reduce the storage resource consumption of neural network models. The method includes: acquiring M sets of data to be compressed, each set containing the actual value of at least one parameter of the neural network model, where M is an integer greater than 1; compressing the actual values ​​of each parameter in the M sets of data to be compressed according to M different target compression parameters to obtain M compressed data sets, each compressed data set containing the compressed value of at least one parameter of the neural network model, wherein the M different target compression parameters correspond one-to-one with the M sets of data to be compressed.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for compressing and decompressing neural network models. Background Technology

[0002] With the advancement of science and technology, the impact of artificial intelligence is growing. In recent years, along with the progress of related technologies, artificial intelligence technology has also entered a stage of rapid development. Compared with other artificial intelligence technologies, neural network methods have higher accuracy, but they also have shortcomings. For example, the structure of neural network models is more complex, and the number of parameters in neural network models is larger, resulting in neural network models requiring a large amount of storage resources.

[0003] Therefore, a method for compressing neural network models is needed to reduce the storage resource consumption of neural network models. Summary of the Invention

[0004] This application provides a method and apparatus for compressing and decompressing neural network models, which reduces the storage resource consumption of neural network models.

[0005] The first aspect of this application provides a method for compressing a neural network model, which can be applied to a server or terminal device for training neural network models, including:

[0006] Obtain M sets of data to be compressed. Each set of data contains the actual value of at least one parameter of a neural network model, where M is an integer greater than 1. The parameter of the neural network model can be the weight of a neuron, and the value of M can be adjusted according to actual needs. Compress the actual values ​​of each parameter in the M sets of data to be compressed according to M different target compression parameters to obtain M compressed sets of data. Each compressed set of data contains the compressed value of at least one parameter of the neural network model. The M different target compression parameters correspond one-to-one with the M sets of data to be compressed. The target compression parameters refer to the parameters used in the compression process. For example, the target compression parameter can be a relative error or an absolute error.

[0007] By compressing the actual values ​​of each parameter in M ​​sets of data to be compressed using M different target compression parameters, the compression ratio can be increased without significantly degrading the performance of the neural network model, thereby improving the compression effect and reducing the storage resource consumption of the neural network model.

[0008] In one implementation, the neural network model consists of N layers of neurons, where N is an integer greater than 1; each dataset to be compressed contains the actual values ​​of the parameters of one or more neurons in the N layers.

[0009] In this implementation, the target compression parameters used to compress the actual values ​​of the parameters of neurons in the same layer are the same.

[0010] In one implementation, the M sets of data to be compressed include a first set of data to be compressed; the first set of data to be compressed contains the actual values ​​of the parameters of the multilayer neurons, and the positions of the multilayer neurons contained in the first set of data to be compressed are continuous in the neural network model.

[0011] Since changes in parameters located close to each other have a similar impact on the neural network model, the actual values ​​of the parameters of multi-layer neurons in consecutive locations are grouped into a set of data to be compressed, thereby enabling the compression of the actual values ​​of the parameters of multi-layer neurons in consecutive locations using the same target compression parameter.

[0012] In one implementation, the M sets of data to be compressed include a second set of data to be compressed, which corresponds to a first target compression parameter among the M different target compression parameters. The actual values ​​of each parameter in the M sets of data to be compressed are compressed according to the M different target compression parameters to obtain M compressed data sets. This includes: compressing the actual values ​​of the parameters of X layers of neurons contained in the second set of data to be compressed layer by layer according to the first target compression parameter to obtain X sets of compressed parameter values. The compressed values ​​of the X sets of parameters correspond one-to-one with X layers of neurons and constitute a compressed data set, where X is a positive integer less than N.

[0013] In this implementation, the actual values ​​of the parameters of each layer of neurons are compressed to obtain a set of compressed parameter values.

[0014] In one implementation, a first target compression parameter is used to represent the relative error. The first target compression parameter can take multiple values, such as 1%, 3%, 5%, or 7%. The compression of the actual values ​​of the parameters of X layers of neurons in the second dataset to be compressed, based on the first target compression parameter, to obtain compressed values ​​for X sets of parameters includes: calculating the logarithm of the absolute error of each parameter of the target layer neuron based on the first target compression parameter, where the target layer neuron is any layer in the X layers; calculating the predicted value of each parameter of the target layer neuron based on the actual value of each parameter; obtaining the quantized value of each parameter of the target layer neuron, where the quantized value of the parameter is equal to the ratio of the target difference to the logarithm of the absolute error of the parameter, and the target difference is equal to the difference between the logarithm of the actual value of the parameter and the logarithm of the predicted value of the parameter; and encoding the integer part of the quantized value of each parameter of the target layer neuron to obtain the compressed values ​​of a set of parameters corresponding to the target layer neuron. For example, a dictionary encoding algorithm can be used for encoding. In this algorithm, a dictionary encoding algorithm replaces a string that appears in a dictionary with an index value to achieve compression. In the embodiments of this application, the dictionary encoding algorithm can be understood as using an index value to replace the integer part of the quantization value of a parameter.

[0015] In this implementation, the actual value of the floating-point parameter is converted into an integer value and encoded to achieve lossy compression of the actual value of the parameter, thereby reducing the storage space occupied by the actual value of the parameter. In addition, the embodiments of this application also perform logarithmic transformation on the absolute error, the actual value of the parameter and the predicted value of the parameter, reducing the order of magnitude of the integer value, thereby further reducing the storage space occupied by the compressed value of the parameter.

[0016] In one implementation, after obtaining M sets of data to be compressed, and before compressing the actual values ​​of each parameter in the M sets of data to be compressed according to M different target compression parameters to obtain M compressed data sets, the method further includes: determining M different target compression parameters.

[0017] In one implementation, determining M different target compression parameters includes: obtaining K sets of compression parameters, each set containing M different compression parameters, with each of the M compression parameters corresponding one-to-one with a set of M data to be compressed, where K is an integer greater than 1; compressing the actual values ​​of each parameter in the M data sets to be compressed using the K sets of compression parameters respectively, to obtain K sets of compressed data, with each of the K sets of compressed data corresponding one-to-one with the K sets of compression parameters, and each set of compressed data containing the compressed value of the parameter corresponding to the actual value of each parameter in the M data sets to be compressed; and selecting one set of compression parameters from the K sets of compression parameters based on the K sets of compressed data to obtain M different target compression parameters.

[0018] First, K sets of compression parameters are used to compress the actual values ​​of each parameter in the M datasets to be compressed. Then, based on the obtained K sets of compressed data, the set of compression parameters with the best compression effect is selected, and the M compression parameters in the set of compression parameters with the best compression effect are used as the M target compression parameters. This ensures that the compression effect of compressing the actual values ​​of each parameter in the M datasets to be compressed based on the M target compression parameters is good, so as to minimize the storage overhead of the neural network model.

[0019] In one implementation, selecting a set of compression parameters from K sets of compressed data to obtain M different target compression parameters includes: calculating the compression ratio corresponding to each set of compression parameters in the K sets of compressed data, where the compression ratio corresponding to a set of compression parameters is equal to the ratio of the storage space required by the actual values ​​of each parameter in the M sets of data to be compressed before compression to the storage space required by the set of compressed data obtained by compression using that set of compression parameters; decompressing the K sets of compressed data to obtain K sets of decompressed data, where there is a one-to-one correspondence between the K sets of decompressed data and the K sets of compressed data, and each set of decompressed data contains M sets of data to be compressed. The actual values ​​of each parameter in the set correspond to the decompressed values ​​of the parameters; based on the K sets of decompressed data, calculate the performance value of the neural network model corresponding to each of the K sets of compressed parameters, where the performance value of the neural network model may include at least one of accuracy, precision, recall, and F1 score; based on the compression ratio corresponding to each of the K sets of compressed parameters and the performance value of the neural network model corresponding to each of the K sets of compressed parameters, calculate the score of each of the K sets of compressed parameters; based on the score of each of the K sets of compressed parameters, select a set of compressed parameters from the K sets of compressed parameters to obtain M different target compressed parameters.

[0020] The score for each set of compression parameters is calculated based on the compression ratio and the performance value of the neural network model. Then, a set of compression parameters is selected based on the score, and the M compression parameters in the selected set are used as the M target compression parameters. In this way, the actual values ​​of each parameter in the M sets of data to be compressed are compressed according to the M target compression parameters. This can ensure both good performance of the neural network model and a high compression ratio, thereby reducing the storage resource occupation of the neural network model.

[0021] In one implementation, the score for each compression parameter in the K groups of compression parameters is calculated based on the compression ratio corresponding to each group of compression parameters and the performance value of the neural network model corresponding to each group of compression parameters in the K groups of compression parameters. This includes: obtaining the first product of the compression ratio and the first weight, obtaining the second product of the performance value and the second weight, and then using the sum of the first product and the second product as the score of the compression parameter, where the sum of the first weight and the second weight can be 1.

[0022] This implementation provides a feasible scheme for calculating the score of each compression parameter in K groups of compression parameters.

[0023] In one implementation, obtaining the K sets of compression parameters includes: compressing the actual values ​​of each parameter in the neural network model using Y compression parameters to obtain compressed values ​​of the Y sets of parameters. The compressed values ​​of the Y sets of parameters correspond one-to-one with the Y compression parameters, and each set of compressed values ​​in the Y sets of parameters contains the compressed values ​​of each parameter in the neural network model; selecting a compression parameter from the Y compression parameters as the optimal compression parameter based on the compressed values ​​of the Y sets of parameters. Specifically, a score can also be calculated for each compression parameter, and the optimal compression parameter can be selected from the Y compression parameters based on the score of each compression parameter; and determining the K sets of compression parameters based on the optimal compression parameter and the Y compression parameters.

[0024] Y compression parameters are used to compress all parameters in the neural network model to obtain Y sets of compressed parameter values. Then, the optimal compression parameter is selected from the Y compression parameters based on the compressed values ​​of the Y sets of parameters. Finally, K sets of compression parameters are determined based on the optimal compression parameters, so that the determined K sets of compression parameters can all have good compression effects, and thus the M target compression parameters determined based on the K sets of compression parameters can have good compression effects.

[0025] In one implementation, the M sets of data to be compressed include a third set of data to be compressed and a fourth set of data to be compressed; the neural network model consists of N layers of neurons, including input layer neurons and output layer neurons, where N is an integer greater than 1; the third set of data to be compressed contains the actual values ​​of the parameters of the input layer neurons; the fourth set of data to be compressed contains the actual values ​​of the parameters of the output layer neurons; in any set of K sets of compression parameters, the compression parameter corresponding to the third set of data to be compressed is either the optimal compression parameter or one of the Y compression parameters that is less than the optimal compression parameter, and the compression parameter corresponding to the fourth set of data to be compressed is one of the Y compression parameters that is greater than the optimal compression parameter.

[0026] The compression parameter of the third set of data to be compressed, which contains the actual values ​​of the parameters of the input layer neurons, is relatively small. The compression parameter of the fourth set of data to be compressed, which contains the actual values ​​of the parameters of the output layer neurons, is relatively large. This can improve the compression ratio as much as possible while ensuring that the neural network model has good performance, thereby reducing the storage space occupied by the neural network model.

[0027] A second aspect of this application provides a method for decompressing a neural network model, applicable to devices running neural network models. The method includes: acquiring M compressed data sets, each containing the compressed value of at least one parameter of the neural network model; and decompressing the compressed values ​​of each parameter in the M compressed data sets according to M different decompression parameters to obtain M decompressed data sets. The M different decompression parameters correspond one-to-one with the M compressed data sets, and each decompressed data set contains the decompressed value of at least one parameter of the neural network model. The M different decompression parameters refer to M different target compression parameters used in the compression process.

[0028] By decompressing the compression values ​​of each parameter in M ​​compressed data sets using M different decompression parameters, the compression ratio can be increased without significantly degrading the performance of the neural network model, thereby improving the compression effect and reducing the storage resource occupation of the neural network model.

[0029] In one implementation, the neural network model consists of N layers of neurons, where N is an integer greater than 1; each compressed dataset contains compressed values ​​of the parameters of one or more neurons in the N layers.

[0030] In this implementation, the actual values ​​of the parameters used to decompress the neurons in the same layer are the same.

[0031] In one implementation, the M compressed data sets include a first compressed data set; the first compressed data set contains compressed values ​​of parameters of multi-layer neurons, and the positions of the multi-layer neurons contained in the first compressed data set in the neural network model are continuous.

[0032] Since changes in parameters located close to each other have a similar impact on the neural network model, the positions of multilayer neurons in the compressed dataset are continuous, thus enabling the decompression of the compressed values ​​of parameters of multilayer neurons located in continuous positions using the same decompression parameters.

[0033] In one implementation, the M compressed data sets include a second compressed data set, which corresponds to a first decompression parameter among the M different decompression parameters. The compressed values ​​of each parameter in the M compressed data sets are decompressed according to the M different decompression parameters to obtain M decompressed data sets. This includes: decompressing the compressed values ​​of the parameters of X layers of neurons contained in the second compressed data set layer by layer according to the first decompression parameter to obtain X sets of decompressed parameter values. The decompressed values ​​of the X sets of parameters correspond one-to-one with X layers of neurons and constitute a decompressed data set, where X is a positive integer less than N.

[0034] In this implementation, each time the compressed values ​​of the parameters of a layer of neurons are decompressed, a set of decompressed values ​​of the parameters is obtained.

[0035] In one implementation, decompressing the compressed values ​​of each parameter in M ​​compressed data sets according to M different decompression parameters to obtain M decompressed data sets includes: decompressing the compressed values ​​of each parameter in the first layer of neurons according to a second decompression parameter to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the first layer of neurons. The second decompression parameter is one of the M different decompression parameters. The first layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets. After running the first layer of neurons, decompressing the compressed values ​​of each parameter in the second layer of neurons according to a third decompression parameter to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the second layer of neurons. The third decompression parameter is one of the M different decompression parameters. The second layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets. In the neural network model, the second layer of neurons is adjacent to and located after the first layer of neurons. After running the second layer of neurons, the above steps are repeated until M decompressed data sets are obtained.

[0036] By decompressing the compressed parameter values ​​of only one layer of neurons in memory at a time, the memory resource overhead during the operation of the neural network model can be reduced.

[0037] A third aspect of this application provides an apparatus for compressing a neural network model, comprising:

[0038] The acquisition unit is used to acquire M sets of data to be compressed, each set of data to be compressed containing the actual value of at least one parameter of the neural network model, where M is an integer greater than 1;

[0039] The compression unit is used to compress the actual values ​​of each parameter in the M sets of data to be compressed according to the M different target compression parameters, so as to obtain M compressed data sets. Each compressed data set contains the compressed value of at least one parameter of the neural network model. The M different target compression parameters correspond one-to-one with the M sets of data to be compressed.

[0040] In one implementation, the neural network model consists of N layers of neurons, where N is an integer greater than 1; each dataset to be compressed contains the actual values ​​of the parameters of one or more neurons in the N layers.

[0041] In one implementation, the M sets of data to be compressed include a first set of data to be compressed; the first set of data to be compressed contains the actual values ​​of the parameters of the multilayer neurons, and the positions of the multilayer neurons contained in the first set of data to be compressed are continuous in the neural network model.

[0042] In one implementation, the M sets of data to be compressed include a second set of data to be compressed, and the second set of data to be compressed corresponds to a first target compression parameter among the M different target compression parameters.

[0043] The compression unit is used to compress the actual values ​​of the parameters of the X layers of neurons contained in the second data set to be compressed layer by layer according to the first target compression parameter, so as to obtain the compressed values ​​of X groups of parameters. The compressed values ​​of X groups of parameters correspond one-to-one with the X layers of neurons and constitute a compressed data set, where X is a positive integer less than N.

[0044] In one implementation, the first target compression parameter is used to represent the relative error.

[0045] The compression unit is used to calculate the logarithm of the absolute error of each parameter of the target layer neuron according to the first target compression parameters, wherein the target layer neuron is any layer of X layers of neurons; calculate the predicted value of each parameter of the target layer neuron according to the actual value of each parameter; obtain the quantized value of each parameter of the target layer neuron, wherein the quantized value of the parameter is equal to the ratio of the target difference to the logarithm of the absolute error of the parameter, and the target difference is equal to the difference between the logarithm of the actual value of the parameter and the logarithm of the predicted value of the parameter; and encode the integer part of the quantized value of each parameter of the target layer neuron to obtain the compressed value of a set of parameters corresponding to the target layer neuron.

[0046] In one implementation, the apparatus for compressing the neural network model further includes a determination unit for determining M different target compression parameters.

[0047] In one implementation, a determining unit is used to obtain K sets of compression parameters, each set of compression parameters containing M different compression parameters, with each of the M compression parameters corresponding one-to-one with a set of M data to be compressed, where K is an integer greater than 1; the actual values ​​of each parameter in the M sets of data to be compressed are compressed using the K sets of compression parameters respectively to obtain K sets of compressed data, with each set of compressed data corresponding one-to-one with the K sets of compression parameters, and each set of compressed data containing the compressed value of the parameter corresponding to the actual value of each parameter in the M sets of data to be compressed; based on the K sets of compressed data, a set of compression parameters is selected from the K sets of compression parameters to obtain M different target compression parameters.

[0048] In one implementation, a unit is defined to calculate the compression ratio corresponding to each compression parameter in the K sets of compressed data; decompress the K sets of compressed data to obtain K sets of decompressed data, where the number of decompressed data sets corresponds one-to-one with the number of compressed data sets, and each set of decompressed data sets contains the decompressed value of the parameter corresponding to the actual value of each parameter in the M sets of data to be compressed; calculate the performance value of the neural network model corresponding to each compression parameter in the K sets of compressed data sets based on the K sets of decompressed data; calculate the score of each compression parameter in the K sets of compressed parameters based on the compression ratio and the performance value of the neural network model corresponding to each compression parameter in the K sets of compressed parameters; and select a set of compression parameters from the K sets of compressed parameters based on the scores of each compression parameter in the K sets of compressed parameters to obtain M different target compression parameters.

[0049] In one implementation, a unit is defined to compress the actual values ​​of each parameter in the neural network model using Y compression parameters to obtain Y sets of compressed parameter values. The compressed values ​​of the Y sets of parameters correspond one-to-one with the Y compression parameters, and the compressed value of each set of parameters in the Y sets of parameters contains the compressed values ​​of each parameter in the neural network model. Based on the compressed values ​​of the Y sets of parameters, a compression parameter is selected from the Y compression parameters as the optimal compression parameter. K sets of compression parameters are determined according to the optimal compression parameter and the Y compression parameters.

[0050] In one implementation, the M sets of data to be compressed include a third set of data to be compressed and a fourth set of data to be compressed; the neural network model consists of N layers of neurons, including input layer neurons and output layer neurons, where N is an integer greater than 1; the third set of data to be compressed contains the actual values ​​of the parameters of the input layer neurons; the fourth set of data to be compressed contains the actual values ​​of the parameters of the output layer neurons; in any set of K sets of compression parameters, the compression parameter corresponding to the third set of data to be compressed is either the optimal compression parameter or one of the Y compression parameters that is less than the optimal compression parameter, and the compression parameter corresponding to the fourth set of data to be compressed is one of the Y compression parameters that is greater than the optimal compression parameter.

[0051] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the first aspect of the embodiments of this application.

[0052] A fourth aspect of this application provides an apparatus for decompressing a neural network model, comprising:

[0053] The acquisition unit is used to acquire M compressed data sets, each of which contains the compressed value of at least one parameter of the neural network model;

[0054] The decompression unit is used to decompress the compression values ​​of each parameter in the M compressed data sets according to M different decompression parameters to obtain M decompressed data sets. The M different decompression parameters correspond one-to-one with the M compressed data sets, and each decompressed data set contains the decompression value of at least one parameter of the neural network model.

[0055] In one implementation, the neural network model consists of N layers of neurons, where N is an integer greater than 1; each compressed dataset contains compressed values ​​of the parameters of one or more neurons in the N layers.

[0056] In one implementation, the M compressed data sets include a first compressed data set; the first compressed data set contains compressed values ​​of parameters of multi-layer neurons, and the positions of the multi-layer neurons contained in the first compressed data set in the neural network model are continuous.

[0057] In one implementation, the M compressed data sets include a second compressed data set, which corresponds to the first decompression parameter among the M different decompression parameters.

[0058] The decompression unit is used to decompress the compressed values ​​of the parameters of the X layers of neurons contained in the second compressed data set layer by layer according to the first decompression parameters, so as to obtain the decompressed values ​​of X sets of parameters. The decompressed values ​​of X sets of parameters correspond one-to-one with the X layers of neurons and constitute a decompressed data set, where X is a positive integer less than N.

[0059] In one implementation, a decompression unit is used to decompress the compressed values ​​of each parameter in the first layer of neurons according to a second decompression parameter, to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the first layer of neurons. The second decompression parameter is one of M different decompression parameters. The first layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets. After running the first layer of neurons, the compressed values ​​of each parameter in the second layer of neurons are decompressed according to a third decompression parameter, to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the second layer of neurons. The third decompression parameter is one of M different decompression parameters. The second layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets. In the neural network model, the second layer of neurons is adjacent to the first layer of neurons and located after the first layer of neurons. After running the second layer of neurons, the above steps are repeated until M decompressed data sets are obtained.

[0060] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the second aspect of the embodiments of this application.

[0061] A fifth aspect of this application provides a computer device, including: at least one processor and a memory, the memory storing computer execution instructions executable on the processor, wherein when the computer execution instructions are executed by the processor, the processor executes a method for compressing a neural network model as described in any embodiment of the first aspect above, or a method for decompressing a neural network model as described in any embodiment of the second aspect above.

[0062] The sixth aspect of this application provides a chip or chip system including at least one processor and a communication interface, the communication interface and at least one processor being interconnected via a circuit, the at least one processor being used to run a computer program or instructions to perform a method for compressing a neural network model as described in any of the embodiments of the first aspect above.

[0063] The seventh aspect of this application provides a chip or chip system including at least one processor and a communication interface, the communication interface and at least one processor being interconnected via a circuit, the at least one processor being used to run a computer program or instructions to perform the method for decompressing a neural network model as described in any of the embodiments of the second aspect above.

[0064] The eighth aspect of this application provides a computer-readable storage medium storing one or more computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor executes a method for compressing a neural network model as described in any embodiment of the first aspect above, or a method for decompressing a neural network model as described in any embodiment of the second aspect above.

[0065] A ninth aspect of this application provides a computer program product comprising computer software instructions that can be loaded by a processor to implement the method for compressing a neural network model as described in any of the embodiments of the first aspect above, or the method for decompressing a neural network model as described in any of the embodiments of the second aspect above.

[0066] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0067] M sets of data to be compressed are obtained, each containing the actual value of at least one parameter of a neural network model, where M is an integer greater than 1. The actual values ​​of each parameter in the M sets of data to be compressed are compressed according to M different target compression parameters to obtain M compressed data sets. Each compressed data set contains the compressed value of at least one parameter of the neural network model. The M different target compression parameters correspond one-to-one with the M sets of data to be compressed. Since the same compression parameter has different effects on different parameters in the neural network model, this embodiment compresses the actual values ​​of each parameter in the M sets of data to be compressed according to M different target compression parameters, which can achieve the selection of appropriate compression parameters for each set of data to be compressed, thereby improving the compression effect. Attached Figure Description

[0068] Figure 1(a) is a schematic diagram of the application scenario of the method of this application embodiment;

[0069] Figure 1(b) is a schematic diagram of the architecture of the computer device in an embodiment of this application;

[0070] Figure 2 This is a schematic diagram of the neural network model in the embodiments of this application;

[0071] Figure 3 This is a schematic diagram illustrating an embodiment of the method for compressing a neural network model in this application.

[0072] Figure 4 This is a schematic diagram illustrating the actual values ​​of the parameters of the compressed X-layer neurons in this application.

[0073] Figure 5 This is a schematic diagram illustrating the actual values ​​of the parameters of the target layer neurons in this application embodiment;

[0074] Figure 6 This is a schematic diagram illustrating an embodiment of determining M different target compression parameters in this application.

[0075] Figure 7 This is a schematic diagram illustrating an embodiment of selecting one set of compression parameters from K sets of compression parameters in this application.

[0076] Figure 8 This is a schematic diagram illustrating an embodiment of obtaining K sets of compression parameters in this application.

[0077] Figure 9 This is a schematic diagram illustrating the process of compressing the neural network model in an embodiment of this application;

[0078] Figure 10 This is a schematic diagram illustrating an embodiment of the method for decompressing a neural network model in this application.

[0079] Figure 11 This is a schematic diagram of partial decompression in an embodiment of this application;

[0080] Figure 12 This is a schematic diagram of the structure of the device for compressing the neural network model in the embodiments of this application;

[0081] Figure 13 This is a schematic diagram of the structure of the device for decompressing the neural network model in the embodiments of this application;

[0082] Figure 14 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0083] This application provides a method and apparatus for compressing and decompressing neural network models, which reduces the storage resource consumption of neural network models.

[0084] The embodiments of this application can be applied to computer devices with limited storage space. These computer devices can be in-vehicle terminals, chatbots, and some portable terminal devices.

[0085] Taking an in-vehicle terminal as an example, as shown in Figure 1(a), in autonomous driving or assisted parking scenarios, environmental data such as road conditions around the vehicle are collected by sensors such as cameras. Then, the driving controller (a type of in-vehicle terminal) runs a neural network model to process this environmental data to achieve autonomous driving or assisted parking. Because neural network models are complex and have many parameters, they consume significant storage resources. Since the storage space of in-vehicle terminals is limited, neural network models are usually stored as compressed files within the in-vehicle terminal. Neural network models can be used to process image, audio, and other data collected by in-vehicle sensors.

[0086] For example, as shown in Figure 1(a), the neural network model is trained on a cloud server. After the neural network model is trained, the cloud server compresses the neural network model into a compressed file and then sends the compressed file to the vehicle terminal. The vehicle terminal receives and stores the compressed file. When the vehicle terminal needs to process environmental data, it decompresses the compressed file to obtain the neural network model and runs the neural network model.

[0087] As shown in Figure 1(b), computer devices typically include a processing module, internal memory (RAM), and external memory. External memory can include floppy disk storage, hard disk storage, and optical disk storage, etc.

[0088] In the process of compressing neural network models, computer devices (such as servers) compress the neural network models in memory into compressed files and then store them in external storage.

[0089] During the decompression of the neural network model, the computer device (such as an in-vehicle terminal) reads the compressed file of the neural network model from the external storage into memory, then decompresses the compressed file to obtain the neural network model and runs the neural network model.

[0090] This application does not specifically limit the types of neural network models. For example, neural network models can include feedforward neural network models and feedback neural network models; feedforward neural network models can specifically include convolutional neural network models, adaptive linear neural network models (Adaptive Linear, Adaline), single-layer perceptron models, multi-layer perceptron models, back propagation (BP) neural network models, and radial basis function (RBF) neural network models, etc. Feedback neural network models can include bidirectional associative memory (BAM) network models and Hopfield neural networks, etc.

[0091] Neural network models contain multiple layers of neurons, with each layer containing one or more neurons. Taking a feedforward neural network model as an example, a feedforward neural network model may only include an input layer and an output layer; for example... Figure 2 As shown, a feedforward neural network model can also include an input layer, an output layer, and one or more hidden layers located between the input and output layers. Figure 2 In the feedforward neural network model shown, there are three hidden layers.

[0092] It should be noted that, in the embodiments of this application, compressing the neural network model mainly refers to compressing the parameters of the neural network model. The parameters of the neural network model mainly refer to the weights of each neuron.

[0093] The process of compressing the parameters of a neural network model mainly refers to converting the actual values ​​of the model's parameters into other values ​​(referred to as compressed parameter values) using a certain method, and then storing the compressed parameter values ​​instead of the actual parameter values. Typically, the compressed parameter values ​​occupy less storage space than the actual parameter values, thus saving storage space by compressing the neural network model's parameters. For neural network models containing a large number of parameters, compressing the neural network model's parameters can save a significant amount of storage space.

[0094] To further reduce the storage resource consumption of neural network models, lossy compression methods are commonly used. Lossy compression means that the decompressed values ​​obtained by decompressing the parameters only approximate their actual values, not necessarily their original values. Therefore, running a neural network model based on its decompressed values ​​will result in a performance decrease compared to running it based on its actual values.

[0095] Current lossy compression methods primarily use the same compression parameters to compress all parameters of a neural network model. However, since changes to different parameters in a neural network model have varying degrees of impact on its performance, compressing all parameters with the same parameters does not yield ideal compression results. Therefore, this application employs different compression parameters to compress the parameters of the neural network model separately, thereby improving the compression performance.

[0096] For ease of understanding, the method for compressing neural network models provided in the embodiments of this application is described below. Specifically, please refer to... Figure 3 The illustration shows an embodiment of the method for compressing a neural network model in this application. Figure 3 As shown, this application provides an embodiment of a method for compressing a neural network model. This method can be applied to a server or terminal device that trains the neural network model. After training the neural network model, the server or terminal device uses this method to compress the neural network model. The method specifically includes:

[0097] Step 101: Obtain M sets of data to be compressed. Each set of data to be compressed contains the actual value of at least one parameter of the neural network model, where M is an integer greater than 1.

[0098] The value of M can be adjusted according to actual needs. For example, if the neural network model has a large number of parameters, M can be larger, such as 4 or 5; if the neural network model has a small number of parameters, M can be smaller, such as 2 or 3; usually, M is 3. In addition, the value of M can also be adjusted according to the number of neurons in the neural network model, which will not be detailed here.

[0099] It should be noted that there are multiple methods to obtain the M sets of data to be compressed.

[0100] One feasible approach is to arbitrarily combine the actual values ​​of all parameters of a neural network model to obtain M sets of data to be compressed. In this case, any one of the M sets of data to be compressed can contain the actual values ​​of multiple parameters located in the same neuron layer, or it can contain the actual values ​​of multiple parameters located in different neuron layers. For example, a set of data to be compressed can contain the actual values ​​of multiple parameters located in the input layer, or it can contain the actual values ​​of parameters located in both the input layer and the hidden layer.

[0101] In addition, the actual values ​​of all parameters of the neural network model can be combined according to certain rules to obtain M sets of data to be compressed.

[0102] One feasible approach is to combine the actual values ​​of all parameters of a neural network model by grouping them together within the same neuron layer. For example, the actual values ​​of parameters from neurons in the same layer can be added to a dataset to be compressed.

[0103] At this point, assuming the neural network model consists of N layers of neurons, where N is an integer greater than 1, each set of data to be compressed can contain the actual values ​​of the parameters of one or more neurons in the N layers.

[0104] At this point, the target compression parameters used to compress the actual values ​​of the parameters of neurons in the same layer are the same.

[0105] For any two datasets to be compressed, they can contain the actual values ​​of parameters from the same number of neuron layers, or they can contain the actual values ​​of parameters from different numbers of neuron layers. For example, one dataset may contain the actual values ​​of parameters from one layer of neurons, another may contain the actual values ​​of parameters from two layers of neurons, and yet another may contain the actual values ​​of parameters from three layers of neurons.

[0106] When the dataset to be compressed contains the actual values ​​of the parameters of multiple layers of neurons, the positions of these neurons in the neural network model can vary. This will be discussed in detail below.

[0107] Specifically, the M sets of data to be compressed include a first set of data to be compressed, which contains the actual values ​​of the parameters of the multilayer neurons.

[0108] The first scenario: The multi-layered neurons in the first dataset to be compressed are spaced apart in the neural network model; this can also be understood as any two neurons in the multi-layered neurons of the first dataset to be compressed being non-adjacent. For example, with... Figure 2 Taking the neural network model shown as an example, this neural network model includes 5 layers of neurons. These 5 layers, from the input to the output, are, in order, layer aa, layer bb, layer cc, layer dd, and layer ee. The first dataset to be compressed contains the actual values ​​of the parameters of the neurons in layers aa, cc, and ee. Figure 2 It can be seen that the neurons in layer aa, layer cc, and layer ee are distributed alternately in the neural network model.

[0109] The second scenario: The positions of the multilayer neurons in the neural network model contained in the first dataset to be compressed include both spaced-out and adjacent positions. For example, the first dataset to be compressed could contain the actual values ​​of the parameters of neurons in layer aa, layer bb, and layer ee, based on... Figure 2 It can be seen that the neurons in layer aa and layer bb are adjacent, while the neurons in layer bb and layer ee are spaced apart.

[0110] The third scenario: The positions of the multilayer neurons in the neural network model contained in the first dataset to be compressed are continuous. For example, the first dataset to be compressed contains the actual values ​​of the parameters of neurons in layer aa and layer bb, according to... Figure 2 It can be seen that neurons in layer AA and layer BB are adjacent; the first dataset to be compressed can contain the actual values ​​of the parameters of neurons in layers CC, DD, and EE, according to... Figure 2 It can be seen that the positions of neurons in the cc layer, dd layer, and ee layer are continuous.

[0111] To ensure that the positions of the multi-layered neurons in the first set of data to be compressed are continuous within the neural network model, the neural network model can be divided into M blocks. Each block contains one layer or multiple layers of neurons in continuous positions, and the actual values ​​of the parameters of the neurons in each block constitute a set of data to be compressed. Figure 2 For example, in the embodiments of this application, Figure 2 The neural network model shown is divided into three parts: the first part contains neurons in the aa and bb layers, the second part contains neurons in the cc and dd layers, and the third part contains neurons in the ee layer. According to... Figure 2As can be seen, the neuron layers in both the first and second blocks are continuous.

[0112] It is understandable that changes to parameters in similar locations have a similar impact on neural network models. Therefore, in this embodiment, the actual values ​​of the parameters of multilayer neurons in consecutive locations form a set of data to be compressed, thereby enabling the compression of the actual values ​​of the parameters of multilayer neurons in consecutive locations using the same target compression parameter.

[0113] Step 102: Determine M different target compression parameters.

[0114] Among them, M different target compression parameters are used to compress the actual values ​​of each parameter in the M sets of data to be compressed.

[0115] It should be noted that there are multiple methods for determining M different target compression parameters, and this application does not specifically limit this method. For example, M target compression parameters can be manually set, and determining M different target compression parameters includes: obtaining the user's setting operation, and determining M different target compression parameters based on the setting operation. In addition, the methods for determining M different target compression parameters will be described in detail below.

[0116] In this embodiment of the application, step 102 is optional.

[0117] Step 103: Compress the actual values ​​of each parameter in the M sets of data to be compressed according to the M different target compression parameters to obtain M compressed data sets. Each compressed data set contains the compressed value of at least one parameter of the neural network model. The M different target compression parameters correspond one-to-one with the M sets of data to be compressed.

[0118] The target compression parameter refers to the parameters used in the compression process. For example, the target compression parameter can be a relative error or an absolute error.

[0119] For example, if M is 3, then the actual values ​​of each parameter in the three datasets to be compressed are compressed according to three different target compression parameters. Each target compression parameter is used to compress the actual values ​​of each parameter in one dataset. For each dataset to be compressed, a compressed dataset can be obtained, resulting in three compressed datasets in total.

[0120] It should be noted that, based on M different target compression parameters, various methods can be used to compress the actual values ​​of each parameter in the M datasets to be compressed. The method for compressing the actual values ​​of each parameter in the M datasets to be compressed can be related to the composition of the parameters in the datasets to be compressed.

[0121] In one implementation, the M sets of data to be compressed include a second set of data to be compressed, and the second set of data to be compressed corresponds to a first target compression parameter among the M different target compression parameters.

[0122] The second set of data to be compressed contains the actual values ​​of the parameters of one or more neurons in the N-layer neuron system.

[0123] Based on this, the actual values ​​of each parameter in the M sets of data to be compressed are compressed according to M different target compression parameters to obtain M compressed data sets, including:

[0124] Based on the first target compression parameter, the actual values ​​of the parameters of the X layers of neurons contained in the second data set to be compressed are compressed layer by layer to obtain the compressed values ​​of X groups of parameters. The compressed values ​​of X groups of parameters correspond one-to-one with the X layers of neurons and constitute a compressed data set, where X is a positive integer less than N.

[0125] The process of compressing the actual values ​​of the parameters of neurons in layer X can be understood as compressing the actual values ​​of the parameters of neurons in layer X at the granularity of neuron layers. Specifically, when X is greater than 1, the actual values ​​of the parameters of one layer of neurons are compressed to obtain a set of compressed values ​​of parameters; the actual values ​​of the parameters of another layer of neurons are compressed to obtain another set of compressed values ​​of parameters, and finally the compressed values ​​of X sets of parameters can be obtained.

[0126] It should be noted that there can be multiple methods for determining the actual value of the compression parameters, and this application does not impose any specific limitations on this method. One method for determining the actual value of the compression parameters will be described later.

[0127] The method for compressing a neural network provided in this application embodiment is illustrated below with an example. The parameters of the neural network model include parameter A and parameter B. For example, changes to parameter A have a greater impact on the performance of the neural network model, while changes to parameter B have a smaller impact. In this application embodiment, different compression parameters are used to compress the actual value 'a' of parameter A and the actual value 'b' of parameter B, respectively, to obtain compressed values ​​'aa' and 'bb'. Different compression parameters result in compressed value bb occupying less storage space than compressed value aa, but the value obtained by decompressing compressed value aa differs little from the actual value 'a', while the value obtained by decompressing compressed value bb differs significantly from the actual value 'b'.

[0128] When the actual values ​​'a' and 'b' occupy the same amount of space, compression parameter B achieves a greater compression ratio than compression parameter A, thus further improving the compression effect and reducing the storage space occupied by the neural network model. Although the decompressed value 'bb' differs significantly from the actual value 'b', the impact of changing parameter B on the performance of the neural network model is relatively small. Therefore, compressing parameter B will not lead to a significant decrease in the performance of the neural network model.

[0129] Therefore, in this embodiment of the application, by compressing the actual values ​​of each parameter in the M sets of data to be compressed according to M different target compression parameters, the compression ratio can be improved while ensuring that the performance of the neural network model does not decrease significantly, thereby improving the compression effect and reducing the storage resource occupation of the neural network model.

[0130] As can be seen from the foregoing, various methods can be used to compress the actual values ​​of the parameters of the neurons in layer X contained in the second dataset to be compressed layer by layer. One such method is described below.

[0131] As one possible approach, the first target compression parameter is used to represent the relative error, which can be understood as the user-allowed relative error. Since relative error is usually expressed as a percentage, the first target compression parameter can also be in the form of a percentage. For ease of understanding, a specific example is provided below to illustrate relative and absolute error. For instance, if the actual value of the parameter is 100, and the decompressed value is 95, then using relative error, the relative error is 5%; using absolute error, the absolute error is 5%.

[0132] The first target compression parameter is used to represent the relative error, such as Figure 4 As shown, the actual values ​​of the parameters of the X layers of neurons in the second dataset to be compressed are compressed layer by layer according to the first target compression parameter to obtain the compressed values ​​of the X sets of parameters, including:

[0133] Step 201: Calculate the logarithm of the absolute error of each parameter of the target layer neuron based on the first target compression parameter. The target layer neuron is any layer in the X layer of neurons.

[0134] As explained above, the absolute error of a parameter can be calculated from its relative error. For example, if the compression parameter of the first target is d, the absolute error for a parameter x of a neuron in the target layer ranges from x(1-d) to x(1+d), and the absolute error is 2dx.

[0135] Understandably, in neural network models, the actual values ​​of parameters are usually floating-point numbers, and the order of magnitude of these values ​​is typically high. Therefore, this embodiment performs a logarithmic transformation on the absolute errors of each parameter to obtain the logarithm of the absolute error, thereby reducing the order of magnitude of the absolute error. For example, if the absolute error is 2dx, then the logarithm of the absolute error can be expressed as log2dx.

[0136] The embodiments of this application do not specifically limit the value of the first target compression parameter, for example, it can be 1%, 3%, 5% or 7%.

[0137] Step 202: Calculate the predicted values ​​of each parameter of the target layer neuron based on the actual values ​​of each parameter of the target layer neuron.

[0138] It should be noted that there are various methods for calculating the predicted values ​​of the parameters of neurons in the target layer, and this application does not specifically limit these methods in its embodiments.

[0139] For example, in this embodiment, the parameters of the target layer neurons can be arranged in a certain order, and the predicted value of each parameter in the target layer neurons is equal to the actual value of the previous parameter. For example, if the target layer neurons contain three parameters, and the actual values ​​of these three parameters are x, y, and z respectively, then the predicted value of the parameter corresponding to the actual value y is x, the predicted value of the parameter corresponding to the actual value z is y, and the predicted value of the parameter corresponding to the actual value x can be preset.

[0140] Step 203: Obtain the quantized values ​​of each parameter of the target layer neuron. The quantized value of the parameter is equal to the ratio of the target difference to the logarithm of the absolute error of the parameter. The target difference is equal to the difference between the logarithm of the actual value of the parameter and the logarithm of the predicted value of the parameter.

[0141] For example, the actual value of the parameter is y, and the predicted value of the parameter corresponding to the actual value of y is x, so the target difference is equal to logy-logx; based on the aforementioned explanation of the absolute error, the logarithm of the absolute error of the parameter y can be expressed as log2dy, then the quantized value of the parameter corresponding to the actual value of y is (logy-logx) / log2dy.

[0142] Step 204: Encode the integer part of the quantized values ​​of each parameter of the target layer neuron to obtain a set of compressed values ​​of the parameters corresponding to the target layer neuron.

[0143] It should be noted that there are various methods for encoding the integer part of the quantized value of a parameter, and this application does not specifically limit this method. For example, the encoding method in the current lossless compression method can be used to encode the integer part of the quantized value of the parameter. Specifically, a dictionary encoding algorithm can be used to encode the integer part of the quantized value of the parameter.

[0144] Dictionary encoding algorithm refers to replacing strings that appear in a dictionary with an index value to achieve compression. In this embodiment, dictionary encoding algorithm can be understood as replacing the integer part of the quantization value of a parameter with an index value.

[0145] Since the target layer neurons are any layer of neurons in layer X, steps 201 to 204 are executed to compress other layers of neurons in layer X using the same compression method.

[0146] Based on the above explanation, as Figure 5 As shown, the process of compressing the actual values ​​of the parameters of the target layer neurons in this embodiment can be summarized as follows: First, the actual values ​​of the parameters of the target layer neurons are input; then, the actual values ​​of the parameters of the target layer neurons are logarithmically transformed, and the logarithm of the absolute error of the parameters is calculated; subsequently, the parameters of the target layer neurons are predicted to obtain the predicted values ​​of the parameters; then, the parameters of the target layer neurons are quantized and encoded according to the logarithm of the absolute error of the parameters, that is, the quantized values ​​of each parameter are calculated; finally, the integer part of the quantized value of each parameter is losslessly compressed (for example, by using a dictionary encoding algorithm) to obtain the compressed values ​​of the parameters of the target layer neurons and output them.

[0147] In this embodiment, the actual value of the floating-point parameter is converted into an integer value and encoded to achieve lossy compression of the actual value of the parameter, thereby reducing the storage space occupied by the actual value of the parameter. In addition, this embodiment also performs logarithmic transformation on the absolute error, the actual value of the parameter, and the predicted value of the parameter, reducing the order of magnitude of the integer value, thereby further reducing the storage space occupied by the compressed value of the parameter.

[0148] As explained above, there are multiple methods for determining M different target compression parameters. The following section will detail the methods for determining M different target compression parameters.

[0149] As a feasible approach, such as Figure 6 As shown, the M different target compression parameters are determined as follows:

[0150] Step 301: Obtain K sets of compression parameters. Each set of compression parameters contains M different compression parameters. The M compression parameters correspond one-to-one with the M sets of data to be compressed. K is an integer greater than 1.

[0151] It should be noted that there are multiple methods to obtain the K sets of compression parameters. For example, the K sets of compression parameters can be manually input, which includes obtaining the K sets of compression parameters input by the user.

[0152] In addition, K sets of compression parameters can also be automatically generated using specific methods. This will be discussed later. Figure 7 This paper introduces a method for automatically generating K sets of compression parameters.

[0153] Step 302: Compress the actual values ​​of each parameter in the M sets of data to be compressed using K sets of compression parameters respectively to obtain K sets of compressed data. The K sets of compressed data correspond one-to-one with the K sets of compression parameters, and each set of compressed data contains the compressed value of the parameter corresponding to the actual value of each parameter in the M sets of data to be compressed.

[0154] Each set of compression parameters contains M compression parameters, which are used to compress the actual values ​​of each parameter in the M sets of data to be compressed. Corresponding to each set of compression parameters, a set of compressed data can be obtained.

[0155] It should be noted that the method of compressing the actual values ​​of each parameter in the M sets of data to be compressed using each set of compression parameters can be the same as the aforementioned method of compressing the actual values ​​of each parameter in the M sets of data to be compressed using M different target compression parameters, that is, it can be the lossy compression method provided in the embodiments of this application.

[0156] Specifically, one set of compression parameters in the K sets includes a first compression parameter. When compressing the actual values ​​of each parameter in the second dataset to be compressed using this first compression parameter, the actual values ​​of the parameters of the X-layer neurons in the second dataset to be compressed can be compressed layer by layer according to the first compression parameter. The specific process of layer-by-layer compression can be found in [reference needed]. Figure 4 Please read the relevant explanations carefully.

[0157] Step 303: Select a set of compression parameters from the K sets of compression parameters based on the K sets of compressed data to obtain M different target compression parameters.

[0158] Based on the K sets of compression data, the compression effect of each set of compression parameters in the K sets of compression parameters can be evaluated. Then, based on the compression effect, the set of compression parameters with the best compression effect is selected from the K sets of compression parameters, and the M compression parameters in the set of compression parameters with the best compression effect are taken as the M target compression parameters.

[0159] It should be noted that there are multiple methods for selecting a set of compression parameters from K sets of compressed data, which will be discussed later. Figure 7 This paper provides a detailed introduction to the method of selecting a set of compression parameters from K sets of compression parameters based on K sets of compressed data.

[0160] In this embodiment, K sets of compression parameters are first used to compress the actual values ​​of each parameter in the M sets of data to be compressed. Then, based on the obtained K sets of compressed data, the set of compression parameters with the best compression effect is selected, and the M compression parameters in the set of compression parameters with the best compression effect are used as the M target compression parameters. This ensures that the compression effect of compressing the actual values ​​of each parameter in the M sets of data to be compressed based on the M target compression parameters is good, so as to minimize the storage overhead of the neural network model.

[0161] The following is combined with Figure 7 The process of selecting one set of compression parameters from K sets of compression parameters based on K sets of compressed data to obtain M different target compression parameters is explained in detail.

[0162] like Figure 7 As shown, based on K sets of compressed data, a set of compression parameters is selected from the K sets of compression parameters to obtain M different target compression parameters, including:

[0163] Step 401: Calculate the compression ratio corresponding to each compression parameter in the K groups of compression data.

[0164] For example, the compression ratio corresponding to a set of compression parameters is equal to the ratio of the storage space required by the actual values ​​of each parameter in the M sets of data to be compressed before compression to the storage space required by the set of compressed data obtained by using the set of compression parameters.

[0165] Step 402: Decompress the K sets of compressed data to obtain K sets of decompressed data. The K sets of decompressed data correspond one-to-one with the K sets of compressed data, and each set of decompressed data contains the decompressed value of the parameter corresponding to the actual value of each parameter in the M sets of data to be compressed.

[0166] It is understandable that the process of decompressing K sets of compressed data is the reverse process of compression, so it will not be described in detail here.

[0167] In step 302, if a lossy compression method is used to compress the actual values ​​of each parameter in the M sets of data to be compressed, then in any set of decompressed data in the resulting K sets of decompressed data, the decompressed value of the parameter may not be equal to the actual value of the parameter before compression.

[0168] Step 403: Calculate the performance value of the neural network model corresponding to each of the K sets of compression parameters based on the K sets of decompression data.

[0169] As explained above, the decompressed value of a parameter may differ from its actual value before compression. Therefore, for each set of decompressed data, the performance value of the neural network model can be calculated based on the test data.

[0170] The performance values ​​of a neural network model can include at least one of accuracy, precision, recall, and F1 score.

[0171] Step 404: Based on the compression ratio corresponding to each compression parameter in the K groups of compression parameters, and the performance value of the neural network model corresponding to each compression parameter in the K groups of compression parameters, calculate the score of each compression parameter in the K groups of compression parameters.

[0172] It is understood that there are various methods for calculating the score of each set of compression parameters based on the compression ratio and performance value, and this application embodiment does not specifically limit this method.

[0173] For example, we can obtain the first product of the compression ratio and the first weight, and the second product of the performance value and the second weight. Then, the sum of the first and second products is used as the score of the compression parameters. Specifically, the compression ratio corresponding to each set of compression parameters is represented by g, the performance value of the neural network model corresponding to each set of compression parameters is represented by h, the first weight is represented by J, and the second weight is represented by K. Then, the score of each set of compression parameters = gJ + hK, where the sum of J and K is 1.

[0174] Step 405: Select a set of compression parameters from the K sets of compression parameters based on the score of each set of compression parameters to obtain M different target compression parameters.

[0175] It should be noted that if the sum of the first product and the second product is used as the score for the compression parameters, then the M compression parameters from the set of compression parameters with the highest scores will be selected as the M target compression parameters.

[0176] In this embodiment, a score for each set of compression parameters is calculated based on the compression ratio and the performance value of the neural network model corresponding to each set of compression parameters. Then, a set of compression parameters is selected based on the score, and M compression parameters from the selected set of compression parameters are used as M target compression parameters. In this way, the actual values ​​of each parameter in the M sets of data to be compressed are compressed according to the M target compression parameters, which can ensure both good performance of the neural network model and a high compression ratio, thereby reducing the storage resource occupation of the neural network model.

[0177] The following is combined with Figure 8 The process of obtaining the K sets of compression parameters is explained in detail. For specific details, please refer to [link to documentation]. Figure 8 To obtain K sets of compression parameters, the following are included:

[0178] Step 501: Compress the actual values ​​of each parameter in the neural network model using Y compression parameters to obtain the compressed values ​​of Y groups of parameters. The compressed values ​​of the Y groups of parameters correspond one-to-one with the Y compression parameters, and the compressed values ​​of each group of parameters in the Y groups of parameters contain the compressed values ​​of each parameter in the neural network model.

[0179] It should be noted that in step 501, the actual values ​​of the parameters in the neural network model are compressed using the same compression parameters. With Y compression parameters, Y sets of compressed parameter values ​​can be obtained.

[0180] For example, the Y compression parameters include compression parameter r, compression parameter t, and compression parameter s. Compression parameter r is used to compress the actual values ​​of each parameter in the neural network model to obtain a set of compressed parameter values. Compression parameter t is used to compress the actual values ​​of each parameter in the neural network model to obtain a set of compressed parameter values. Compression parameter s is used to compress the actual values ​​of each parameter in the neural network model to obtain another set of compressed parameter values.

[0181] Step 502: Select one compression parameter from the Y compression parameters as the optimal compression parameter based on the compression value of the Y group of parameters.

[0182] It should be noted that there are multiple methods for selecting the optimal compression parameter from the Y compression parameters based on the compression values ​​of the Y sets of parameters. For example, similar to the method of selecting a set of compression parameters from K sets of compression parameters, one can calculate the compression ratio and the performance value of the neural network model corresponding to each of the Y compression parameters, then calculate the score of each of the Y compression parameters based on the compression ratio and the performance value of the neural network model, and finally select the optimal compression parameter from the Y compression parameters based on the score of each compression parameter.

[0183] The calculation of compression ratio and performance values ​​of neural network model can be understood by referring to the relevant descriptions in the foregoing embodiments.

[0184] Step 503: Determine K sets of compression parameters based on the optimal compression parameters and Y compression parameters.

[0185] It should be noted that there are multiple methods for determining the K sets of compression parameters based on the optimal compression parameters and the Y compression parameters, and this application does not specifically limit these methods.

[0186] For example, M selection ranges of compression parameters can be determined based on the optimal compression parameters and Y compression parameters, with each selection range containing one or more of the Y compression parameters; then, a compression parameter can be randomly selected from each of the M selection ranges to form a set of compression parameters. Repeating this operation will yield K sets of compression parameters.

[0187] For example, given Y compression parameters of 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, and 10%, with an optimal compression parameter of 7% and M of 3, three selection ranges can be determined: one range includes 1%, 2%, 3%, 4%, 5%, 6%, and 7%; another range includes 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, and 10%; and the third range includes 8%, 9%, and 10%. Then, a compression parameter is selected from each of these three ranges to form a set of compression parameters. This process is repeated until K sets of compression parameters are determined.

[0188] In this embodiment, Y compression parameters are used to compress all parameters in the neural network model to obtain the compressed values ​​of Y sets of parameters. Then, the optimal compression parameter is selected from the Y compression parameters based on the compressed values ​​of the Y sets of parameters. Finally, K sets of compression parameters are determined based on the optimal compression parameters, so that the determined K sets of compression parameters can all have a good compression effect, thereby making the M target compression parameters determined based on the K sets of compression parameters have a good compression effect.

[0189] As explained above, changes to different parameters in a neural network model of the same degree have varying effects on the model. Specifically, the inventors discovered that changes to the parameters of neurons near the input of the neural network model have a greater impact on its performance. Therefore, when compressing the parameters of neurons near the input, a smaller target compression parameter (representing relative error) can be selected to appropriately reduce the compression ratio, thus ensuring the performance of the neural network model. Conversely, when compressing the parameters of neurons far from the input, a larger target compression parameter (representing relative error) can be selected to increase the compression ratio with minimal sacrifice to the model's performance. This results in a higher overall compression ratio and better performance of the neural network model.

[0190] Based on this, as an feasible approach, the M sets of data to be compressed include a third set of data to be compressed and a fourth set of data to be compressed.

[0191] A neural network model consists of N layers of neurons, including input layer neurons and output layer neurons, where N is an integer greater than 1.

[0192] The third set of data to be compressed contains the actual values ​​of the parameters of the input layer neurons, and the fourth set of data to be compressed contains the actual values ​​of the parameters of the output layer neurons.

[0193] In any of the K sets of compression parameters, the compression parameter corresponding to the third set of data to be compressed is either the optimal compression parameter or one of the Y compression parameters that is less than the optimal compression parameter. The compression parameter corresponding to the fourth set of data to be compressed is either one of the Y compression parameters that is greater than the optimal compression parameter.

[0194] For example, if the Y compression parameters are 1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, and 10%, and the optimal compression parameter is 7%, then the range of parameters for the third set of data to be compressed can be 1%, 2%, 3%, 4%, 5%, 6%, and 7%, while the range of parameters for the fourth set of data to be compressed can be 8%, 9%, and 10%. This ensures that in any set of the K compression parameters, the compression parameter for the third set of data to be compressed is either the optimal compression parameter or one of the Y compression parameters that is less than the optimal compression parameter, and the compression parameter for the fourth set of data to be compressed is one of the Y compression parameters that is greater than the optimal compression parameter.

[0195] In this embodiment, the compression parameter corresponding to the third data set to be compressed, which contains the actual values ​​of the parameters of the input layer neurons, is relatively small, while the compression parameter corresponding to the fourth data set to be compressed, which contains the actual values ​​of the parameters of the output layer neurons, is relatively large. This can improve the compression ratio as much as possible while ensuring that the neural network model has good performance, thereby reducing the storage space occupied by the neural network model.

[0196] The above section introduced methods for compressing neural network models. Based on the above content, such as Figure 9 As shown, the process of compressing a neural network model can be described as follows:

[0197] The first step is to use Y compression parameters to perform lossy compression on the parameters of the neural network model, and obtain the compressed values ​​of Y sets of parameters. The process of using Y compression parameters for lossy compression can be understood by referring to the relevant explanation of step 501.

[0198] The second step is to score the Y compression parameters based on the compression values ​​of the Y group of parameters to obtain K groups of compression parameters. The process of obtaining K groups of compression parameters through scoring can be understood by referring to the relevant explanations of steps 502 and 503.

[0199] The third step is to divide the neural network model into blocks to obtain M sets of data to be compressed.

[0200] The fourth step is to use K sets of compression parameters to perform lossy compression on the M sets of data to be compressed to obtain K sets of compressed data. The lossy compression process can be understood by referring to the relevant explanation in step 302.

[0201] The fifth step is to score the K sets of compression parameters based on the K sets of compression data to determine M different target compression parameters. The process of determining M different target compression parameters through scoring can be understood by referring to steps 303, 401 to 405.

[0202] The sixth step involves using M different target compression parameters to perform lossy compression on the M sets of data to be compressed, resulting in a compressed file.

[0203] The following is combined with Figure 10 The method for decompressing neural network models is explained. For example... Figure 10 As shown, this application provides an embodiment of a method for decompressing a neural network model. This method can be applied to devices running neural network models, such as in-vehicle terminals and dialogue servers. The method specifically includes:

[0204] Step 601: Obtain M compressed data sets, each containing the compressed value of at least one parameter of the neural network model.

[0205] Step 602: Decompress the compression values ​​of each parameter in the M compressed data sets according to the M different decompression parameters to obtain M decompressed data sets. The M different decompression parameters correspond one-to-one with the M compressed data sets, and each decompressed data set contains the decompression value of at least one parameter of the neural network model.

[0206] Here, M different decompression parameters refer to M different target compression parameters in the compression process.

[0207] It should be noted that since the process of decompressing a neural network model is the reverse process of compressing a neural network model, please refer to the previous explanation of the process of compressing a neural network model for an explanation of the process of decompressing a neural network model.

[0208] As one possible approach, a neural network model consists of N layers of neurons, where N is an integer greater than 1; each compressed dataset contains compressed values ​​of the parameters of one or more neurons in the N layers.

[0209] Since the process of decompressing the neural network model is the reverse process of compressing the neural network model, the embodiments of this application can be understood by referring to the relevant descriptions in the embodiments of the compressed neural network model.

[0210] As one possible approach, the M compressed data sets include a first compressed data set; the first compressed data set contains compressed values ​​of parameters of multilayer neurons, and the positions of the multilayer neurons contained in the first compressed data set in the neural network model are continuous.

[0211] Since the process of decompressing the neural network model is the reverse process of compressing the neural network model, the embodiments of this application can be understood by referring to the relevant descriptions in the embodiments of the compressed neural network model.

[0212] As one possible approach, the M compressed data sets include a second compressed data set, which corresponds to the first decompression parameter among the M different decompression parameters.

[0213] The compressed values ​​of each parameter in M ​​compressed data sets are decompressed according to M different decompression parameters to obtain M decompressed data sets, including:

[0214] Based on the first decompression parameter, the compressed values ​​of the parameters of the X layers of neurons contained in the second compressed data set are decompressed layer by layer to obtain the decompressed values ​​of X sets of parameters. The decompressed values ​​of X sets of parameters correspond one-to-one with the X layers of neurons and constitute a decompressed data set, where X is a positive integer less than N.

[0215] Since the process of decompressing the neural network model is the reverse process of compressing the neural network model, the embodiments of this application can be understood by referring to the relevant descriptions in the embodiments of the compressed neural network model.

[0216] As explained above, in certain scenarios, the external and internal storage space of the device running the neural network model is limited. Furthermore, during operation, the multiple layers of neurons in the neural network model run sequentially, not simultaneously. Therefore, to reduce the memory consumption of the neural network model, the compressed parameter values ​​of each layer of neurons can be decompressed layer by layer according to their respective execution order. Each layer of neurons can be run immediately after its parameters are decompressed, rather than waiting for all layers to be decompressed before running the neural network model.

[0217] Specifically, the compression values ​​of each parameter in the M compressed data sets are decompressed according to M different decompression parameters to obtain M decompressed data sets, including:

[0218] The compressed values ​​of each parameter in the first layer of neurons are decompressed according to the second decompression parameter to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the first layer of neurons. The second decompression parameter is one of M different decompression parameters. The first layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets.

[0219] It should be noted that since the decompression process is the reverse of the compression process, the embodiments of this application will not describe in detail the process of decompressing the compressed values ​​of each parameter in the first layer of neurons. For details, please refer to the relevant descriptions in the embodiments of the compressed neural network model to understand the embodiments of this application.

[0220] After decompression is complete, the first layer of neurons can be run.

[0221] After running the first layer of neurons, the compressed values ​​of each parameter in the second layer of neurons are decompressed according to the third decompression parameter to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the second layer of neurons. The third decompression parameter is one of M different decompression parameters. The second layer of neurons is one of N layers of neurons and belongs to one of M compressed data sets. In the neural network model, the second layer of neurons is adjacent to the first layer of neurons and is located after the first layer of neurons.

[0222] It should be noted that the second layer of neurons is adjacent to and follows the first layer of neurons. This can also be understood as the second layer of neurons being the next layer of neurons after the first layer of neurons. In other words, the second layer of neurons needs to perform calculations based on the results of the first layer of neurons.

[0223] After decompression is complete, run the second layer of neurons.

[0224] Finally, repeat the above steps until the compressed values ​​of the parameters of all neurons in the neural network model are decompressed, resulting in M ​​decompressed data sets.

[0225] like Figure 11 As shown in this embodiment, when a neural network model needs to be run for corresponding calculations, the compressed file of the neural network model can be loaded into memory first, and then the compressed file can be partially decompressed in memory to obtain the decompressed values ​​of the parameters of a specific layer of neurons. The processing module runs the specific layer of neurons according to the decompressed values ​​of the parameters of the specific layer of neurons and the structural data of the specific layer of neurons. After the specific layer of neurons has been run, the decompressed values ​​of the parameters of the specific layer of neurons and the structural data of the specific layer of neurons can be cleared from memory. After that, the processing module will then partially decompress to obtain the decompressed values ​​of the parameters of the next layer of neurons and continue to run the next layer of neurons.

[0226] For example, with Figure 2Taking the neural network model shown as an example, the neurons in layer 'aa' run first; based on the results of the neurons in layer 'aa', the neurons in layer 'bb' run; and so on, with the neurons in layers 'cc', 'dd', and 'ee' running sequentially. Therefore, when running the neural network model, the compressed values ​​of the parameters of the neurons in layer 'aa' can be decompressed first, and then the neurons in layer 'aa' run; then the compressed values ​​of the parameters of the neurons in layer 'bb' are decompressed, and then the neurons in layer 'bb' run, and so on, until the neurons in layer 'ee' are decompressed, at which point the decompression process ends. Therefore, in this embodiment, only the compressed values ​​of the parameters of one layer of neurons are decompressed in memory at a time, thus reducing the memory resource overhead during the operation of the neural network model.

[0227] Please see Figure 12 The diagram below shows the structure of the device for compressing a neural network model in this embodiment.

[0228] like Figure 12 As shown, this application provides an embodiment of an apparatus for compressing a neural network model, comprising:

[0229] The acquisition unit 701 is used to acquire M sets of data to be compressed, each set of data to be compressed containing the actual value of at least one parameter of the neural network model, where M is an integer greater than 1;

[0230] Compression unit 702 is used to compress the actual values ​​of each parameter in M ​​sets of data to be compressed according to M different target compression parameters to obtain M compressed data sets. Each compressed data set contains the compressed value of at least one parameter of the neural network model. The M different target compression parameters correspond one-to-one with the M sets of data to be compressed.

[0231] In one implementation, the neural network model consists of N layers of neurons, where N is an integer greater than 1; each dataset to be compressed contains the actual values ​​of the parameters of one or more neurons in the N layers.

[0232] In one implementation, the M sets of data to be compressed include a first set of data to be compressed; the first set of data to be compressed contains the actual values ​​of the parameters of the multilayer neurons, and the positions of the multilayer neurons contained in the first set of data to be compressed are continuous in the neural network model.

[0233] In one implementation, the M sets of data to be compressed include a second set of data to be compressed, and the second set of data to be compressed corresponds to a first target compression parameter among the M different target compression parameters.

[0234] Compression unit 702 is used to compress the actual values ​​of the parameters of the X-layer neurons contained in the second data set to be compressed layer by layer according to the first target compression parameter, so as to obtain the compressed values ​​of X groups of parameters. The compressed values ​​of X groups of parameters correspond one-to-one with the X-layer neurons and constitute a compressed data set, where X is a positive integer less than N.

[0235] In one implementation, the first target compression parameter is used to represent the relative error.

[0236] Compression unit 702 is used to calculate the logarithm of the absolute error of each parameter of the target layer neuron according to the first target compression parameters, wherein the target layer neuron is any layer of X layers of neurons; calculate the predicted value of each parameter of the target layer neuron according to the actual value of each parameter; obtain the quantized value of each parameter of the target layer neuron, wherein the quantized value of the parameter is equal to the ratio of the target difference to the logarithm of the absolute error of the parameter, and the target difference is equal to the difference between the logarithm of the actual value of the parameter and the logarithm of the predicted value of the parameter; and encode the integer part of the quantized value of each parameter of the target layer neuron to obtain the compressed value of a set of parameters corresponding to the target layer neuron.

[0237] In one implementation, the apparatus for compressing the neural network model further includes a determination unit 703 for determining M different target compression parameters.

[0238] In one implementation, a determining unit 703 is used to obtain K sets of compression parameters, each set of compression parameters containing M different compression parameters, and the M compression parameters correspond one-to-one with M sets of data to be compressed, where K is an integer greater than 1; the actual values ​​of each parameter in the M sets of data to be compressed are compressed using the K sets of compression parameters respectively to obtain K sets of compressed data, the K sets of compressed data correspond one-to-one with the K sets of compression parameters, and each set of compressed data contains the compressed value of the parameter corresponding to the actual value of each parameter in the M sets of data to be compressed; based on the K sets of compressed data, a set of compression parameters is selected from the K sets of compression parameters to obtain M different target compression parameters.

[0239] In one implementation, a determining unit 703 is used to calculate the compression ratio corresponding to each compression parameter in the K sets of compressed data; decompress the K sets of compressed data to obtain K sets of decompressed data, where the number of decompressed data sets corresponds one-to-one with the number of compressed data sets, and each set of decompressed data sets contains the decompression value of the parameter corresponding to the actual value of each parameter in the M sets of data to be compressed; calculate the performance value of the neural network model corresponding to each compression parameter in the K sets of compressed data sets based on the K sets of decompressed data; calculate the score of each compression parameter in the K sets of compressed parameters based on the compression ratio and the performance value of the neural network model corresponding to each compression parameter in the K sets of compressed parameters; and select a set of compression parameters from the K sets of compressed parameters based on the scores of each compression parameter in the K sets of compressed parameters to obtain M different target compression parameters.

[0240] In one implementation, a determining unit 703 is used to compress the actual values ​​of each parameter in the neural network model using Y compression parameters to obtain Y sets of compressed parameter values. The compressed values ​​of the Y sets of parameters correspond one-to-one with the Y compression parameters, and the compressed value of each set of parameters in the Y sets of parameters contains the compressed values ​​of each parameter in the neural network model. Based on the compressed values ​​of the Y sets of parameters, a compression parameter is selected from the Y compression parameters as the optimal compression parameter. K sets of compression parameters are determined according to the optimal compression parameter and the Y compression parameters.

[0241] In one implementation, the M sets of data to be compressed include a third set of data to be compressed and a fourth set of data to be compressed; the neural network model consists of N layers of neurons, including input layer neurons and output layer neurons, where N is an integer greater than 1; the third set of data to be compressed contains the actual values ​​of the parameters of the input layer neurons; the fourth set of data to be compressed contains the actual values ​​of the parameters of the output layer neurons; in any set of K sets of compression parameters, the compression parameter corresponding to the third set of data to be compressed is either the optimal compression parameter or one of the Y compression parameters that is less than the optimal compression parameter, and the compression parameter corresponding to the fourth set of data to be compressed is one of the Y compression parameters that is greater than the optimal compression parameter.

[0242] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the method for compressing the neural network model in the embodiments of this application.

[0243] Please see Figure 13 The present application provides a schematic diagram of the structure of the device for decompressing the neural network model in this embodiment.

[0244] like Figure 13 As shown, this application provides an embodiment of an apparatus for decompressing a neural network model, comprising:

[0245] The acquisition unit 801 is used to acquire M compressed data sets, each of which contains the compressed value of at least one parameter of the neural network model;

[0246] The decompression unit 802 is used to decompress the compression values ​​of each parameter in the M compressed data sets according to the M different decompression parameters to obtain M decompressed data sets. The M different decompression parameters correspond one-to-one with the M compressed data sets, and each decompressed data set contains the decompression value of at least one parameter of the neural network model.

[0247] In one implementation, the neural network model consists of N layers of neurons, where N is an integer greater than 1; each compressed dataset contains compressed values ​​of the parameters of one or more neurons in the N layers.

[0248] In one implementation, the M compressed data sets include a first compressed data set; the first compressed data set contains compressed values ​​of parameters of multi-layer neurons, and the positions of the multi-layer neurons contained in the first compressed data set in the neural network model are continuous.

[0249] In one implementation, the M compressed data sets include a second compressed data set, which corresponds to the first decompression parameter among the M different decompression parameters.

[0250] The decompression unit 802 is used to decompress the compressed values ​​of the parameters of the X layers of neurons contained in the second compressed data set layer by layer according to the first decompression parameters, so as to obtain the decompressed values ​​of X groups of parameters. The decompressed values ​​of X groups of parameters correspond one-to-one with the X layers of neurons and constitute a decompressed data set, where X is a positive integer less than N.

[0251] In one implementation, the decompression unit 802 is used to decompress the compressed values ​​of each parameter in the first layer of neurons according to a second decompression parameter, to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the first layer of neurons. The second decompression parameter is one of M different decompression parameters. The first layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets. After running the first layer of neurons, the compressed values ​​of each parameter in the second layer of neurons are decompressed according to a third decompression parameter, to obtain the decompressed value of the parameter corresponding to the compressed value of each parameter in the second layer of neurons. The third decompression parameter is one of M different decompression parameters. The second layer of neurons is one of N layers of neurons and belongs to one of the M compressed data sets. In the neural network model, the second layer of neurons is adjacent to the first layer of neurons and located after the first layer of neurons. After running the second layer of neurons, the above steps are repeated until M decompressed data sets are obtained.

[0252] For details regarding the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the method for decompressing the neural network model in the embodiments of this application.

[0253] Please see Figure 14 , Figure 14 This is a schematic diagram of the structure of the computer device in the embodiments of this application.

[0254] One embodiment of the computer device in this application may include one or more processors 901, memory 902, and communication interfaces 903.

[0255] The memory 902 can be temporary or persistent storage. Furthermore, the processor 901 can be configured to communicate with the memory 902 and execute a series of instructions stored in the memory 902 on a computer device.

[0256] In this embodiment, the processor 901 can execute the aforementioned... Figure 4 , Figure 6 , Figure 7 , Figure 8 and Figure 10 The steps in the illustrated embodiment will not be described in detail here.

[0257] For example, processor 901 may perform the following steps:

[0258] Obtain M sets of data to be compressed, each set of data to be compressed containing the actual value of at least one parameter of the neural network model, where M is an integer greater than 1;

[0259] The actual values ​​of each parameter in the M sets of data to be compressed are compressed according to the M different target compression parameters to obtain M compressed data sets. Each compressed data set contains the compressed value of at least one parameter of the neural network model. The M different target compression parameters correspond one-to-one with the M sets of data to be compressed.

[0260] At this point, the specific functional module division within processor 901 can be consistent with the aforementioned... Figure 12 The division of functional modules such as the acquisition unit and compression unit described in the previous section is similar, and will not be repeated here.

[0261] For example, processor 901 may perform the following steps:

[0262] Obtain M compressed data sets, each containing the compressed value of at least one parameter of the neural network model;

[0263] The compression values ​​of each parameter in the M compressed data sets are decompressed according to the M different target compression parameters to obtain M decompressed data sets. The M different target compression parameters correspond one-to-one with the M compressed data sets, and each decompressed data set contains the decompressed value of at least one parameter of the neural network model.

[0264] At this point, the specific functional module division within processor 901 can be consistent with the aforementioned... Figure 13 The division of functional modules such as the acquisition unit and decompression unit described in the previous section is similar, and will not be repeated here.

[0265] This application also provides a chip or chip system, which includes at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the aforementioned... Figure 4 , Figure 6 , Figure 7 , Figure 8 and Figure 10 The steps in the illustrated embodiment will not be described in detail here.

[0266] The communication interface in the chip can be an input / output interface, pins, or circuits.

[0267] This application also provides a first implementation of a chip or chip system. The chip or chip system described above in this application further includes at least one memory, which stores instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (such as a read-only memory or random access memory).

[0268] This application also provides a chip or chip system, which includes at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the aforementioned... Figure 4 , Figure 6 , Figure 7 , Figure 8 and Figure 10 The steps in the illustrated embodiment will not be described in detail here.

[0269] The communication interface in the chip can be an input / output interface, pins, or circuits.

[0270] This application also provides a first implementation of a chip or chip system. The chip or chip system described above in this application further includes at least one memory, which stores instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (such as a read-only memory or random access memory).

[0271] This application also provides a computer storage medium for storing computer software instructions for use in a computer device, including instructions for executing programs designed for the computer device.

[0272] The computer device can be as described above. Figure 12 The apparatus for the described compressed neural network model.

[0273] The computer device can be as described above. Figure 13 The apparatus described is for decompressing neural network models.

[0274] This application also provides a computer program product, which includes computer software instructions that can be loaded by a processor to implement the aforementioned features. Figure 4 , Figure 6 , Figure 7 , Figure 8 and Figure 10 The process shown is as follows.

[0275] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0276] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0277] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0278] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0279] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method of compressing a neural network model, the method comprising: The method comprises: obtaining M sets of to-be-compressed data, each set of to-be-compressed data containing actual values of parameters of at least one neural network model, M being an integer greater than 1; compressing the actual values of the parameters in the M sets of to-be-compressed data according to M different target compression parameters to obtain M sets of compressed data, each set of compressed data containing compressed values of at least one parameter of the neural network model, the M different target compression parameters corresponding to the M sets of to-be-compressed data in one-to-one manner, and the M sets of compressed data being stored in a vehicle terminal, the neural network model being used for processing environmental data collected by a vehicle sensor to realize automatic driving or assist parking; the M sets of to-be-compressed data comprise a third set of to-be-compressed data and a fourth set of to-be-compressed data; the neural network model is composed of N layers of neurons, the N layers of neurons comprising input layer neurons and output layer neurons, N being an integer greater than 1; the third set of to-be-compressed data contains actual values of parameters of the input layer neurons; the fourth set of to-be-compressed data contains actual values of parameters of the output layer neurons; the influence degree of a change in the parameters of the input layer neurons on the performance of the neural network model is greater than the influence degree of a change in the parameters of the output layer neurons on the performance of the neural network model, and the target compression parameter corresponding to the third set of to-be-compressed data is smaller than the target compression parameter corresponding to the fourth set of to-be-compressed data.

2. The method of claim 1, wherein, Each set of to-be-compressed data contains actual values of parameters of one or more layers of neurons in the N layers of neurons.

3. The method of claim 2, wherein, the M sets of to-be-compressed data comprise a first set of to-be-compressed data; the first set of to-be-compressed data contains actual values of parameters of multiple layers of neurons, and the multiple layers of neurons contained in the first set of to-be-compressed data are continuous in the neural network model.

4. The method according to claim 2 or 3, characterized in that, the M sets of to-be-compressed data comprise a second set of to-be-compressed data, the second set of to-be-compressed data corresponding to a first target compression parameter in the M different target compression parameters; the compression of the actual values of the parameters in the M sets of to-be-compressed data according to the M different target compression parameters to obtain M sets of compressed data comprises: layer-by-layer compression of the actual values of the parameters of X layers of neurons contained in the second set of to-be-compressed data according to the first target compression parameter to obtain X sets of compressed values of parameters, the X sets of compressed values of parameters corresponding to the X layers of neurons in one-to-one manner and constituting a set of compressed data, X being a positive integer less than N.

5. The method of claim 4, wherein, the first target compression parameter is used to represent a relative error; the layer-by-layer compression of the actual values of the parameters of the X layers of neurons contained in the second set of to-be-compressed data according to the first target compression parameter to obtain X sets of compressed values of parameters comprises: calculating the logarithm of the absolute error of each parameter of a target layer of neurons according to the first target compression parameter, the target layer of neurons being any one of the X layers of neurons; calculating predicted values of the parameters of the target layer of neurons according to the actual values of the parameters of the target layer of neurons; obtaining a quantized value of each parameter of the target layer neuron, wherein the quantized value of the parameter is equal to a ratio of a target difference value and a logarithm of an absolute error of the parameter, and the target difference value is equal to a difference between a logarithm of an actual value of the parameter and a logarithm of a predicted value of the parameter; encoding an integer part of the quantized value of each parameter of the target layer neuron to obtain a compression value of a corresponding set of parameters of the target layer neuron.

6. The method according to any one of claims 1 to 2, characterized in that, After the M sets of to-be-compressed data are obtained, before the actual values of the parameters in the M sets of to-be-compressed data are compressed according to the M different target compression parameters to obtain M sets of compressed data, the method further comprises: determining the M different target compression parameters.

7. The method of claim 6, wherein, The determination of the M different target compression parameters comprises: obtaining K sets of compression parameters, each set of compression parameters comprising M different compression parameters, the M compression parameters corresponding to the M sets of to-be-compressed data in a one-to-one manner, and K being an integer greater than 1; compressing the actual values of the parameters in the M sets of to-be-compressed data by using the K sets of compression parameters respectively to obtain K sets of compressed data, each set of compressed data comprising compression values of parameters corresponding to the actual values of the parameters in the M sets of to-be-compressed data, and the K sets of compressed data corresponding to the K sets of compression parameters in a one-to-one manner; selecting a set of compression parameters from the K sets of compression parameters based on the K sets of compressed data to obtain the M different target compression parameters.

8. The method of claim 7, wherein, The selection of the set of compression parameters from the K sets of compression parameters based on the K sets of compressed data to obtain the M different target compression parameters comprises: calculating a compression ratio corresponding to each set of compression parameters in the K sets of compression parameters based on the K sets of compressed data; decompressing the K sets of compressed data to obtain K sets of decompressed data, each set of decompressed data comprising decompression values of parameters corresponding to the actual values of the parameters in the M sets of to-be-compressed data, and the K sets of decompressed data corresponding to the K sets of compressed data in a one-to-one manner; calculating a performance value of a neural network model corresponding to each set of compression parameters in the K sets of compression parameters based on the K sets of decompressed data; calculating a score of each set of compression parameters in the K sets of compression parameters based on the compression ratio corresponding to each set of compression parameters in the K sets of compression parameters and the performance value of the neural network model corresponding to each set of compression parameters in the K sets of compression parameters; selecting a set of compression parameters from the K sets of compression parameters based on the scores of the K sets of compression parameters to obtain the M different target compression parameters.

9. The method according to claim 7 or 8, characterized in that, The obtaining of the K sets of compression parameters comprises: compressing the actual values of the parameters in the neural network model by using Y compression parameters respectively to obtain Y sets of compression values of parameters, each set of compression values of parameters corresponding to a compression parameter in a one-to-one manner, and each set of compression values of parameters comprising compression values of parameters in the neural network model; selecting a compression parameter from the Y compression parameters as an optimal compression parameter based on the Y sets of compression values of parameters; and selecting a set of compression parameters from the K sets of compression parameters based on the scores of the K sets of compression parameters to obtain the M different target compression parameters. The K groups of compression parameters are determined according to the optimal compression parameter and the Y compression parameters.

10. The method of claim 9, wherein, In any one of the K groups of compression parameters, the compression parameters corresponding to the third to-be-compressed data set are all the optimal compression parameter or one of the Y compression parameters smaller than the optimal compression parameter, and the compression parameters corresponding to the fourth to-be-compressed data set are all one of the Y compression parameters larger than the optimal compression parameter.

11. A method of decompressing a neural network model, the method comprising: Comprise: M compressed data sets are obtained, each of which contains compressed values of at least one parameter of a neural network model, and the M compressed data sets are stored in a vehicle terminal; M decompressed data sets are obtained by decompressing the compressed values of each parameter in the M compressed data sets according to M different decompression parameters, the M different decompression parameters correspond to the M compressed data sets one by one, each decompressed data set contains decompressed values of at least one parameter of a neural network model, and the neural network model is used to process environmental data collected by a vehicle sensor to realize automatic driving or auxiliary parking; The neural network model is composed of N layers of neurons, and N is an integer greater than 1; The M decompressed data sets are obtained by decompressing the compressed values of each parameter in the M compressed data sets according to M different decompression parameters, the M different decompression parameters correspond to the M compressed data sets one by one, each decompressed data set contains decompressed values of at least one parameter of a neural network model, and the neural network model is used to process environmental data collected by a vehicle sensor to realize automatic driving or auxiliary parking; The compressed values of each parameter in the first layer of neurons are decompressed according to a second decompression parameter to obtain decompressed values of the parameters corresponding to the compressed values of each parameter in the first layer of neurons, the second decompression parameter is one of the M different decompression parameters, and the first layer of neurons is one of the N layers of neurons and belongs to one of the M compressed data sets; After the first layer of neurons is run, the compressed values of each parameter in the second layer of neurons are decompressed according to a third decompression parameter to obtain decompressed values of the parameters corresponding to the compressed values of each parameter in the second layer of neurons, the third decompression parameter is one of the M different decompression parameters, the second layer of neurons is one of the N layers of neurons and belongs to one of the M compressed data sets, the second layer of neurons is adjacent to and located after the first layer of neurons in the neural network model, the degree of influence of changes in the parameters of the first layer of neurons on the performance of the neural network model is greater than the degree of influence of changes in the parameters of the second layer of neurons on the performance of the neural network model, and the second decompression parameter is less than or equal to the third decompression parameter; After the second layer of neurons is run, the above steps are repeatedly executed until M decompressed data sets are obtained.

12. The method of claim 11, wherein, Each compressed data set contains compressed values of the parameters of one or more layers of neurons in the N layers of neurons.

13. The method of claim 12, wherein, The M compressed data sets include a first compressed data set; The first compressed data set contains compressed values of the parameters of multiple layers of neurons, and the multiple layers of neurons contained in the first compressed data set are continuous in the neural network model.

14. The method according to claim 12 or 13, characterized in that, The M compressed data sets include a second compressed data set corresponding to a first decompression parameter in the M different decompression parameters; The decompression of the compressed values of the parameters in the M compressed data sets according to the M different decompression parameters to obtain M decompressed data sets includes: The compressed values of the parameters of X-layer neurons included in the second compressed data set are decompressed layer by layer according to the first decompression parameter to obtain X sets of decompressed values of the parameters, the X sets of decompressed values of the parameters correspond to the X-layer neurons one by one and constitute a decompressed data set, and X is a positive integer less than N.

15. An apparatus for compressing a neural network model, the apparatus comprising: The method includes: An acquisition unit is configured to acquire M to-be-compressed data sets, each to-be-compressed data set including actual values of at least one parameter of a neural network model, and M is an integer greater than 1; A compression unit is configured to compress the actual values of the parameters in the M to-be-compressed data sets according to M different target compression parameters to obtain M compressed data sets, each compressed data set including compressed values of at least one parameter of the neural network model, the M different target compression parameters correspond to the M to-be-compressed data sets one by one, the M compressed data sets are stored in a vehicle terminal, and the neural network model is used to process environmental data collected by a vehicle sensor to implement automatic driving or auxiliary parking. The M to-be-compressed data sets include a third to-be-compressed data set and a fourth to-be-compressed data set; The neural network model is composed of N-layer neurons, the N-layer neurons include input-layer neurons and output-layer neurons, and N is an integer greater than 1; The third to-be-compressed data set includes actual values of parameters of the input-layer neurons; The fourth to-be-compressed data set includes actual values of parameters of the output-layer neurons; The influence degree of a change in the parameters of the input-layer neurons on the performance of the neural network model is greater than the influence degree of a change in the parameters of the output-layer neurons on the performance of the neural network model, and a target compression parameter corresponding to the third to-be-compressed data set is smaller than a target compression parameter corresponding to the fourth to-be-compressed data set.

16. An apparatus for decompressing a neural network model, the apparatus comprising: The method includes: An acquisition unit is configured to acquire M compressed data sets, each compressed data set including compressed values of at least one parameter of a neural network model, and the M compressed data sets are stored in a vehicle terminal; A decompression unit is configured to decompress the compressed values of the parameters in the M compressed data sets according to M different decompression parameters to obtain M decompressed data sets, the M different decompression parameters correspond to the M compressed data sets one by one, each decompressed data set includes decompressed values of at least one parameter of the neural network model, and the neural network model is used to process environmental data collected by a vehicle sensor to implement automatic driving or auxiliary parking. The neural network model is composed of N-layer neurons, and N is an integer greater than 1. The decompression unit is specifically configured to decompress the compressed values of the parameters in the first layer of neurons according to a second decompression parameter to obtain decompressed values of the parameters corresponding to the compressed values of the parameters in the first layer of neurons, the second decompression parameter being one of the M different decompression parameters, the first layer of neurons being one of the N layers of neurons and belonging to one of the M compressed data sets; after running the first layer of neurons, decompress the compressed values of the parameters in the second layer of neurons according to a third decompression parameter to obtain decompressed values of the parameters corresponding to the compressed values of the parameters in the second layer of neurons, the third decompression parameter being one of the M different decompression parameters, the second layer of neurons being one of the N layers of neurons and belonging to one of the M compressed data sets, the second layer of neurons being adjacent to and located after the first layer of neurons in the neural network model, the influence degree of the change of the parameters of the first layer of neurons on the performance of the neural network model being greater than the influence degree of the change of the parameters of the second layer of neurons on the performance of the neural network model, and the second decompression parameter being less than or equal to the third decompression parameter; after running the second layer of neurons, the above steps are repeatedly executed until M decompressed data sets are obtained.

17. A computer device, comprising: Comprise: At least one processor and a memory, the memory storing computer execution instructions executable on the processor, when the computer execution instructions are executed by the processor, the processor executes the method as claimed in any one of claims 1-10, or the method as claimed in any one of claims 11-14.

18. A computer-readable storage medium storing one or more computer-executable instructions that, when executed by a processor, cause the processor to perform any of the methods of claims 1-17. When the computer execution instructions are executed by the processor, the processor executes the method as claimed in any one of claims 1-14.

Images