A neural network compression method and related equipment

Abstract

The present application provides a neural network compression method and related equipment, which can enable the compressed neural network to perform image processing of different images with different precisions, thereby ensuring the accuracy of image processing. The method of the present application includes: obtaining a target image; inputting the target image into a first neural network to obtain the number of quantization bits of a second neural network, wherein the second neural network is used to perform image processing on the target image, and the number of quantization bits of the second neural network is positively correlated with the amount of calculation required for image processing; according to the number of quantization bits of the second neural network, the parameters of the second neural network are quantized to obtain the second neural network after quantization processing.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a neural network compression method and related equipment. Background Art

[0002] In recent years, deep neural networks have made great progress in computer vision tasks such as image classification, object detection, and image segmentation. However, deep neural networks often contain a large number of model parameters, which require a large amount of device resources (such as storage space and computing power), making it difficult to run efficiently on terminal devices. Therefore, it is necessary to compress the neural network to reduce the device resources occupied by the neural network.

[0003] Model quantization technology is an effective method to compress neural networks. This technology can convert the parameters of the neural network from high-bit values ​​(for example, 32 bits) to low-bit values ​​(for example, 4 bits) for representation, thereby significantly reducing the resources occupied by the parameters of the neural network.

[0004] When performing model quantization, the number of quantization bits (i.e., the number of bits of the neural network parameters expected by the user) is usually preset and fixed, resulting in the neural network having consistent processing accuracy for any image when performing image processing, and being unable to guarantee the accuracy of image processing (for example, if an image is difficult to process and the neural network has low processing accuracy for it, the processing result will be inaccurate). Summary of the invention

[0005] The embodiments of the present application provide a neural network compression method and related equipment, which can enable the compressed neural network to perform image processing of different images with different precisions, thereby ensuring the accuracy of image processing.

[0006] A first aspect of an embodiment of the present application provides a neural network compression method, the method comprising:

[0007] When it is necessary to process the target image, the target image to be processed can be obtained first. Furthermore, a first neural network and a second neural network can be obtained, wherein the first neural network is used to compress the second neural network, and the second neural network is used to process the target image.

[0008] Then, the target image is input into the first neural network to obtain the number of quantization bits of the second neural network, wherein the number of quantization bits of the second neural network is positively correlated with the amount of calculation required for image processing. For example, if the object in the target image is relatively easy to identify, the amount of calculation required to identify the target image is relatively small, that is, the second neural network can have a lower processing accuracy for the target image, so the number of quantization bits of the second neural network is relatively small. If the object in the target image is relatively difficult to identify, the amount of calculation required to identify the target image is relatively large, that is, the second neural network needs to have a higher processing accuracy for the target image, so the number of quantization bits of the second neural network is relatively large.

[0009] Finally, according to the number of quantization bits of the second neural network, the parameters of the second neural network are quantized to obtain the quantized second neural network, that is, the compressed second neural network. For example, when the number of quantization bits of the second neural network is 4 bits, the parameters of the second neural network are represented by 4-bit values, thereby reducing the device resources occupied by the second neural network.

[0010] It can be seen from the above method that after acquiring the target image, the target image can be input into the first neural network to obtain the number of quantization bits used to compress the second neural network. The number of quantization bits is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the number of quantization bits is large, and if the amount of calculation required for image processing of the target image is small, the number of quantization bits is small. It can be seen that due to the different processing difficulties of the target image, the number of quantization bits output by the first neural network is different, so that the degree of quantization processing of the second neural network is also different. In this way, for target images with lower processing difficulty, the second neural network after quantization processing can perform image processing with lower precision, and for target images with higher processing difficulty, the second neural network after quantization processing can perform image processing with higher precision, thereby ensuring the accuracy of image processing.

[0011] In a possible implementation, the quantization bit number of the second neural network includes the quantization bit number of the M-layer network in the second neural network. According to the quantization bit number of the second neural network, the parameters of the second neural network are quantized to obtain the quantized second neural network, which specifically includes: according to the quantization bit number of the i-th layer network in the second neural network, the parameters of the i-th layer network are quantized to obtain the quantized second neural network, wherein i=1, 2, ..., M, and M is a positive integer. In the aforementioned implementation, the first neural network can output the quantization bit number of the M-layer network in the second neural network. For any one layer of the M-layer network in the second neural network, the parameters of the layer network can be quantized according to the quantization bit number of the layer network, thereby obtaining the quantized second neural network. For example, if the quantization bit number of the first layer network of the second neural network is 4 bits, the parameters of the first layer network are represented by 4-bit values. If the quantization bit number of the second layer network of the second neural network is 3 bits, the parameters of the second layer network are represented by 3-bit values. In this way, the parameters of each layer network of the second neural network can be quantized to obtain the quantized second neural network.

[0012] In one possible implementation, inputting the target image into the first neural network and obtaining the quantization bit number of the second neural network specifically includes: inputting the target image into the first neural network to obtain the probability of the candidate bit number of the i-th layer network in the second neural network; selecting the quantization bit number of the i-th layer network from the candidate bit number of the i-th layer network according to the size of the probability of this part of the candidate bit number. In the aforementioned implementation, after the first neural network performs image processing on the target image, it can obtain the probability of the candidate bit number of any layer network in the M layers of the second neural network. For any layer network in the M layers network, the quantization bit number of the layer network can be accurately selected from the candidate bit number of the layer network based on the size of the probability of the candidate bit number of the layer network.

[0013] In a possible implementation, the method further includes: after obtaining the second neural network after quantization processing, the target image can be input into the neural network, so that the second neural network after quantization processing performs image processing (for example, image classification, target detection and image segmentation, etc.) on the target image to obtain the characteristics of the target image.

[0014] A second aspect of the embodiments of the present application provides a neural network compression method, the method comprising:

[0015] When it is necessary to perform image processing on a target image, the target image to be processed can be obtained first. Furthermore, a third neural network and a second neural network can be obtained, wherein the third neural network is used to compress the input of the second neural network, and the second neural network is used to perform image processing on the target image.

[0016] Then, the target image is input into the third neural network to obtain the target resolution corresponding to the second neural network, wherein the target resolution corresponding to the second neural network is positively correlated with the amount of computation required for image processing. For example, if the object in the target image is relatively easy to identify, the amount of computation required to identify the target image is relatively small, that is, the second neural network can have a lower processing accuracy for the target image, so the target resolution corresponding to the second neural network is relatively small. If the object in the target image is relatively difficult to identify, the amount of computation required to identify the target image is relatively large, that is, the second neural network needs to have a higher processing accuracy for the target image, so the target resolution corresponding to the second neural network is relatively large.

[0017] Finally, the resolution of the target image is adjusted to the target resolution to obtain a target image with the target resolution. For example, when the target resolution corresponding to the second neural network is 168×168 and the original resolution of the target image is 224×224, the resolution of the target image is adjusted from 224×224 to 168×168 to obtain a target image with a resolution of 168×168.

[0018] In the related art, adjusting the resolution of an image is also one of the methods for compressing a neural network. However, the resolution of the adjusted image, as expected by the user, is usually preset and fixed, which results in the neural network processing any image with the same processing accuracy when performing image processing, and the accuracy of image processing cannot be guaranteed. It can be seen from the above method that after acquiring the target image, the target image can be input into the third neural network to obtain the target resolution corresponding to the second neural network, which is used to compress the input of the second neural network, that is, the target image. The target resolution is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the target resolution is large. If the amount of calculation required for image processing of the target image is small, the target resolution is small. It can be seen that due to the different processing difficulties of the target image, the target resolution output by the third neural network is different, so that the target resolution of the adjusted target image is also different. In this way, for a target image with a lower processing difficulty, its resolution can be made smaller, so that the second neural network performs image processing with a lower accuracy. For a target image with a higher processing difficulty, its resolution can be made larger, so that the second neural network performs image processing with a higher accuracy, thereby ensuring the accuracy of image processing.

[0019] In one possible implementation, inputting the target image into the third neural network to obtain the target resolution corresponding to the second neural network specifically includes: inputting the target image into the third neural network to obtain the probability of the candidate resolutions corresponding to the second neural network; and selecting the target resolution corresponding to the second neural network from the candidate resolutions corresponding to the second neural network according to the magnitude of the probability of this part of the candidate resolutions. In the aforementioned implementation, after the third neural network performs image processing on the target image, the probability of the candidate resolutions corresponding to the second neural network can be obtained. Then, according to the magnitude of the probability of each candidate resolution, the target resolution corresponding to the second neural network is accurately selected from this part of the candidate resolutions.

[0020] In one possible implementation, after adjusting the resolution of the target image to the target resolution and obtaining the target image of the target resolution, the method further includes: inputting the target image of the target resolution (i.e., the target image after adjusting the resolution) into a second neural network, so that the second neural network performs image processing (e.g., image classification, target detection, image segmentation, etc.) on the target image of the target resolution to obtain features of the target image of the target resolution.

[0021] A third aspect of an embodiment of the present application provides a model training method, which includes: obtaining an image to be trained; inputting the image to be trained into a first model to be trained to obtain the number of quantization bits of a second model to be trained; quantizing the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained to obtain the second model to be trained after quantization; inputting the image to be trained into the second model to be trained after quantization to obtain features of the image to be trained; updating the parameters of the first model to be trained and the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained and the features of the image to be trained until the model training conditions are met to obtain the first neural network and the second neural network.

[0022] It can be seen from the above method that the first neural network and the second neural network can be obtained by jointly training the first model to be trained and the second model to be trained. The first neural network obtained by this method can accurately obtain the number of quantization bits of the second neural network based on the input target image, and the second neural network obtained by this method can accurately process the target image.

[0023] In a possible implementation, the parameters of the first model to be trained and the parameters of the second model to be trained are updated according to the number of quantization bits of the second model to be trained and the features of the image to be trained until the model training conditions are met to obtain the first neural network and the second neural network. Specifically, the method includes: obtaining the target loss according to the deviation between the number of quantization bits of the second model to be trained and the preset number of bits, and the deviation between the features of the image to be trained and the real features of the image to be trained; and updating the parameters of the first model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met to obtain the first neural network and the second neural network.

[0024] In one possible implementation, the quantization bit number of the second model to be trained includes the quantization bit number of the M-layer network in the second model to be trained, and the parameters of the second model to be trained are quantized according to the quantization bit number of the second model to be trained to obtain the quantized second model to be trained. Specifically, it includes: according to the quantization bit number of the i-th layer network in the second model to be trained, the parameters of the i-th layer network are quantized to obtain the quantized second model to be trained, wherein i=1, 2,…,M, and M is a positive integer.

[0025] In a possible implementation, inputting the image to be trained into the first model to be trained, and obtaining the number of quantization bits of the second model to be trained specifically includes: inputting the image to be trained into the first model to be trained, and obtaining the probability of the candidate number of bits of the i-th layer network in the second model to be trained; and selecting the number of quantization bits of the i-th layer network from the candidate number of bits of the i-th layer network according to the size of the probability.

[0026] In a possible implementation, the deviation between the number of quantization bits of the second model to be trained and the preset number of bits includes the deviation between the number of quantization bits of the i-th layer network in the second model to be trained and the preset number of bits.

[0027] A fourth aspect of an embodiment of the present application provides a model training method, the method comprising: obtaining an image to be trained; inputting the image to be trained into a third model to be trained to obtain a target resolution corresponding to the second model to be trained; adjusting the resolution of the image to be trained to the target resolution to obtain an image to be trained with the target resolution; inputting the image to be trained with the target resolution into the second model to be trained to obtain features of the image to be trained with the target resolution; updating the parameters of the third model to be trained and the parameters of the second model to be trained according to the target resolution and the features of the image to be trained with the target resolution until the model training conditions are met to obtain a third neural network and a second neural network.

[0028] It can be seen from the above method that by jointly training the third model to be trained and the second model to be trained, a third neural network and a second neural network can be obtained. The third neural network obtained by this method can accurately obtain the target resolution corresponding to the second neural network based on the input target image, and the second neural network obtained by this method can accurately process the target image.

[0029] In one possible implementation, according to the target resolution and the features of the image to be trained at the target resolution, the parameters of the third model to be trained and the parameters of the second model to be trained are updated until the model training conditions are met to obtain the third neural network and the second neural network. Specifically, the method includes: obtaining the target loss according to the deviation between the expected value corresponding to the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the actual features of the image to be trained at the target resolution; updating the parameters of the third model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met to obtain the third neural network and the second neural network.

[0030] In one possible implementation, inputting the image to be trained into the third model to be trained to obtain the target resolution corresponding to the second model to be trained specifically includes: inputting the image to be trained into the third model to be trained to obtain the probability of candidate resolutions corresponding to the second model to be trained; and selecting the target resolution corresponding to the second model to be trained from the candidate resolutions corresponding to the second model to be trained according to the size of the probability.

[0031] In a possible implementation, obtaining the target loss according to the deviation between the expected value corresponding to the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution specifically includes: obtaining the target loss according to the deviation between the expected value of the probability of the target resolution and the preset expected value, the deviation between the expected value of the probability of the remaining candidate resolutions except the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution.

[0032] A fifth aspect of an embodiment of the present application provides a neural network compression device, which includes an acquisition module and a processing module; the acquisition module is used to acquire a target image; the processing module is used to input the target image into a first neural network to obtain the quantization bit number of a second neural network, wherein the second neural network is used to perform image processing on the target image, and the quantization bit number of the second neural network is positively correlated with the amount of computation required for image processing; the processing module is also used to quantize the parameters of the second neural network according to the quantization bit number of the second neural network to obtain the second neural network after quantization processing.

[0033] It can be seen from the above device that after acquiring the target image, the target image can be input into the first neural network to obtain the number of quantization bits used to compress the second neural network. The number of quantization bits is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the number of quantization bits is large, and if the amount of calculation required for image processing of the target image is small, the number of quantization bits is small. It can be seen that due to the different processing difficulties of the target image, the number of quantization bits output by the first neural network is different, so that the degree of quantization processing of the second neural network is also different. In this way, for target images with lower processing difficulty, the second neural network after quantization processing can perform image processing with lower precision, and for target images with higher processing difficulty, the second neural network after quantization processing can perform image processing with higher precision, thereby ensuring the accuracy of image processing.

[0034] In one possible implementation, the quantization bit number of the second neural network includes the quantization bit number of the M-layer network in the second neural network, and the processing module is specifically used to quantize the parameters of the i-th layer network according to the quantization bit number of the i-th layer network in the second neural network to obtain the quantized second neural network, where i=1, 2,…, M, and M is a positive integer.

[0035] In one possible implementation, the processing module is specifically used to: input the target image into the first neural network to obtain the probability of the candidate bit number of the i-th layer network in the second neural network; and select the quantization bit number of the i-th layer network from the candidate bit number of the i-th layer network according to the size of the probability.

[0036] In a possible implementation, the processing module is further used to input the target image into the second neural network after quantization processing to obtain the characteristics of the target image.

[0037] A sixth aspect of an embodiment of the present application provides a neural network compression device, which includes an acquisition module and a processing module; the acquisition module is used to acquire a target image; the processing module is used to input the target image into a third neural network to obtain a target resolution corresponding to the second neural network, wherein the second neural network is used to perform image processing on the target image, and the target resolution corresponding to the second neural network is positively correlated with the amount of computation required for image processing; the processing module is also used to adjust the resolution of the target image to the target resolution to obtain a target image with the target resolution.

[0038] It can be seen from the above device that after acquiring the target image, the target image can be input into the third neural network to obtain the target resolution corresponding to the second neural network, which is used to compress the input of the second neural network, that is, the target image. The target resolution is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the target resolution is large. If the amount of calculation required for image processing of the target image is small, the target resolution is small. It can be seen that due to the different processing difficulties of the target image, the target resolution output by the third neural network is different, so that the target resolution of the adjusted target image is also different. In this way, for target images with lower processing difficulty, the resolution can be made smaller, so that the second neural network performs image processing with lower precision. For target images with higher processing difficulty, the resolution can be made larger, so that the second neural network performs image processing with higher precision, thereby ensuring the accuracy of image processing.

[0039] In one possible implementation, the processing module is specifically used to: input the target image into the third neural network to obtain the probability of the candidate resolutions corresponding to the second neural network; and select the target resolution corresponding to the second neural network from the candidate resolutions corresponding to the second neural network according to the size of the probability.

[0040] In a possible implementation, the processing module is further used to input the target image of the target resolution into the second neural network to obtain features of the target image of the target resolution.

[0041] The seventh aspect of the embodiments of the present application provides a model training device, which includes an acquisition module and a training module; the acquisition module is used to acquire the image to be trained; the training module is used to input the image to be trained into the first model to be trained to obtain the quantization bit number of the second model to be trained; the training module is also used to quantize the parameters of the second model to be trained according to the quantization bit number of the second model to be trained to obtain the second model to be trained after quantization; the training module is also used to input the image to be trained into the second model to be trained after quantization to obtain the characteristics of the image to be trained; the training module is also used to update the parameters of the first model to be trained and the parameters of the second model to be trained according to the quantization bit number of the second model to be trained and the characteristics of the image to be trained until the model training conditions are met to obtain the first neural network and the second neural network.

[0042] It can be seen from the above device that the first neural network and the second neural network can be obtained by jointly training the first model to be trained and the second model to be trained. The first neural network obtained by the device can accurately obtain the number of quantization bits of the second neural network based on the input target image, and the second neural network obtained by the device can accurately process the target image.

[0043] In one possible implementation, the training module is specifically used to: obtain a target loss based on a deviation between the number of quantization bits of the second model to be trained and a preset number of bits, and a deviation between features of the image to be trained and real features of the image to be trained; update parameters of the first model to be trained and parameters of the second model to be trained according to the target loss until model training conditions are met, thereby obtaining a first neural network and a second neural network.

[0044] In one possible implementation, the number of quantization bits of the second model to be trained includes the number of quantization bits of the M-layer network in the second model to be trained, and the training module is specifically used to quantize the parameters of the i-th layer network according to the number of quantization bits of the i-th layer network in the second model to be trained to obtain the second model to be trained after quantization, where i=1, 2,…, M, and M is a positive integer.

[0045] In a possible implementation, the training module is specifically used to: input the image to be trained into the first model to be trained to obtain the probability of the candidate bit number of the i-th layer network in the second model to be trained; and select the quantization bit number of the i-th layer network from the candidate bit number of the i-th layer network according to the size of the probability.

[0046] In a possible implementation, the deviation between the number of quantization bits of the second model to be trained and the preset number of bits includes the deviation between the number of quantization bits of the i-th layer network in the second model to be trained and the preset number of bits.

[0047] An eighth aspect of an embodiment of the present application provides a model training device, which includes an acquisition module and a training module; the acquisition module is used to acquire an image to be trained; the training module is used to input the image to be trained into a third model to be trained to obtain a target resolution corresponding to the second model to be trained; the training module is also used to adjust the resolution of the image to be trained to the target resolution to obtain the image to be trained with the target resolution; the training module is also used to input the image to be trained with the target resolution into the second model to be trained to obtain the features of the image to be trained with the target resolution; the training module is also used to update the parameters of the third model to be trained and the parameters of the second model to be trained according to the target resolution and the features of the image to be trained with the target resolution until the model training conditions are met to obtain the third neural network and the second neural network.

[0048] It can be seen from the above device that by jointly training the third model to be trained and the second model to be trained, a third neural network and a second neural network can be obtained. The third neural network obtained by the device can accurately obtain the target resolution corresponding to the second neural network based on the input target image, and the second neural network obtained by the device can accurately process the target image.

[0049] In one possible implementation, the training module is specifically used to: obtain the target loss based on the deviation between the expected value corresponding to the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution; update the parameters of the third model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met, so as to obtain the third neural network and the second neural network.

[0050] In one possible implementation, the training module is specifically used to: input the image to be trained into the third model to be trained to obtain the probability of the candidate resolution corresponding to the second model to be trained; and select the target resolution corresponding to the second model to be trained from the candidate resolutions corresponding to the second model to be trained according to the size of the probability.

[0051] In one possible implementation, the training module is specifically used to obtain the target loss based on the deviation between the expected value of the probability of the target resolution and the preset expected value, the deviation between the expected value of the probability of the remaining candidate resolutions except the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the actual features of the image to be trained at the target resolution.

[0052] A ninth aspect of an embodiment of the present application provides an image processing method, the method comprising: acquiring a target image; inputting the target image into a first neural network to obtain a quantization bit number of a second neural network, wherein the second neural network is used to perform image processing on the target image, and the quantization bit number of the second neural network is positively correlated with the amount of computation required for image processing; quantizing parameters of the second neural network according to the quantization bit number of the second neural network to obtain a quantized second neural network; inputting the target image into the quantized second neural network to obtain features of the target image.

[0053] The tenth aspect of an embodiment of the present application provides an image processing method, the method comprising: acquiring a target image; inputting the target image into a third neural network to obtain a target resolution corresponding to the second neural network, wherein the second neural network is used to perform image processing on the target image, and the target resolution corresponding to the second neural network is positively correlated with the amount of computation required for image processing; adjusting the resolution of the target image to the target resolution to obtain a target image of the target resolution; inputting the target image of the target resolution into the second neural network to obtain features of the target image of the target resolution.

[0054] In an eleventh aspect of an embodiment of the present application, a neural network compression device is provided, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the neural network compression device executes the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0055] The twelfth aspect of the embodiments of the present application provides a model training device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the model training device executes the method described in the third aspect, any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.

[0056] A thirteenth aspect of an embodiment of the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect, the third aspect, or any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.

[0057] The fourteenth aspect of an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect, the third aspect, or any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.

[0058] In a possible implementation manner, the processor is coupled to the memory through an interface.

[0059] In a possible implementation, the chip system also includes a memory, in which a computer program or computer instructions are stored.

[0060] A fifteenth aspect of an embodiment of the present application provides a computer storage medium, which stores a computer program. When the program is executed by a computer, the computer implements the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect, the third aspect, or any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.

[0061] A sixteenth aspect of an embodiment of the present application provides a computer program product, which stores instructions, which, when executed by a computer, enable the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect, the third aspect, or any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.

[0062] In an embodiment of the present application, after acquiring the target image, the target image can be input into the first neural network to obtain the number of quantization bits used to compress the second neural network. The number of quantization bits is positively correlated with the amount of calculation required to process the target image. If the amount of calculation required to process the target image is large, the number of quantization bits is large. If the amount of calculation required to process the target image is small, the number of quantization bits is small. It can be seen that due to the different processing difficulties of the target image, the number of quantization bits output by the first neural network is different, so that the degree of quantization processing of the second neural network is also different. In this way, for target images with lower processing difficulty, the second neural network after quantization processing can perform image processing with lower precision. For target images with higher processing difficulty, the second neural network after quantization processing can perform image processing with higher precision, thereby ensuring the accuracy of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A structural diagram of the main framework of artificial intelligence;

[0064] Figure 2a A schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0065] Figure 2b Another structural schematic diagram of the image processing system provided in an embodiment of the present application;

[0066] Figure 2c A schematic diagram of an image processing device provided in an embodiment of the present application;

[0067] Figure 3a A schematic diagram of the system 100 architecture provided in an embodiment of the present application;

[0068] Figure 4 A schematic diagram of a flow chart of a neural network compression method provided in an embodiment of the present application;

[0069] Figure 5 A schematic diagram of an application example of the neural network compression method provided in an embodiment of the present application;

[0070] Figure 6Another schematic diagram of a process flow of a neural network compression method provided in an embodiment of the present application;

[0071] Figure 7 A schematic diagram of another application example of the neural network compression method provided in an embodiment of the present application;

[0072] Figure 8 A schematic diagram of a flow chart of a model training method provided in an embodiment of the present application;

[0073] Fig. 9 Another schematic diagram of the flow chart of the model training method provided in the embodiment of the present application;

[0074] Fig.10 A schematic diagram of the structure of a neural network compression device provided in an embodiment of the present application;

[0075] Fig.11 Another schematic diagram of the structure of the neural network compression device provided in an embodiment of the present application;

[0076] Fig.12 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0077] Fig.13 Another structural schematic diagram of the model training device provided in an embodiment of the present application;

[0078] Fig.14 A schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0079] Fig.15 A schematic diagram of the structure of the training device provided in the embodiment of the present application;

[0080] Fig.16 A schematic diagram of the structure of the chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The embodiments of the present application provide a neural network compression method and related equipment, which can enable the compressed neural network to perform image processing of different images with different precisions, thereby ensuring the accuracy of image processing.

[0082] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0083] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0084] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1 It is a structural diagram of the main framework of artificial intelligence. The following is an explanation of the above artificial intelligence theme framework from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.

[0085] (1) Infrastructure

[0086] The infrastructure provides computing power support for the AI ​​system, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0087] (2) Data

[0088] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0089] (3) Data processing

[0090] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0091] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0092] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0093] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0094] (4) General ability

[0095] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0096] (5) Smart products and industry applications

[0097] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0098] Next, several application scenarios of this application are introduced.

[0099] Figure 2a A schematic diagram of the structure of an image processing system provided in an embodiment of the present application, the image processing system includes a user device and a data processing device. The user device includes an intelligent terminal such as a mobile phone, a personal computer or an information processing center. The user device is the initiator of image processing, and as the initiator of the image processing request, the request is usually initiated by the user through the user device.

[0100] The above-mentioned data processing device can be a device or server with data processing function such as a cloud server, a network server, an application server and a management server. The data processing device receives the image enhancement request from the intelligent terminal through the interactive interface, and then performs image processing in the form of machine learning, deep learning, search, reasoning, decision-making, etc. through the memory for storing data and the processor for data processing. The memory in the data processing device can be a general term, including local storage and a database for storing historical data. The database can be on the data processing device or on other network servers.

[0101] exist Figure 2a In the image processing system shown, the user device can receive the user's instructions. For example, the user device can obtain an image input / selected by the user, and then initiate a request to the data processing device, so that the data processing device executes the image semantic segmentation application for the image obtained by the user device, thereby obtaining the corresponding processing result for the image. Exemplarily, the user device can obtain an image to be processed input by the user, and then initiate an image processing request to the data processing device, so that the data processing device executes the image processing application (for example, image classification, object detection and image segmentation, etc.) on the image, thereby obtaining a processed image.

[0102] exist Figure 2a In the embodiment of the present application, the data processing device can execute the neural network compression method and model training method of the embodiment of the present application.

[0103] Figure 2b Another structural diagram of the image processing system provided in the embodiment of the present application is shown in FIG. Figure 2b In the process, the user device directly acts as a data processing device. The user device can directly obtain input from the user and directly process it by the hardware of the user device itself. The specific process is similar to Figure 2a Similarly, please refer to the above description and will not repeat them here.

[0104] exist Figure 2bIn the image processing system shown, the user device can receive instructions from the user. For example, the user device can obtain an image to be processed selected by the user in the user device, and then the user device itself executes an image processing application (for example, image classification, target detection, and image segmentation, etc.) on the image, thereby obtaining a corresponding processing result for the image.

[0105] exist Figure 2b In the embodiment of the present application, the user device itself can execute the neural network compression method and model training method of the embodiment of the present application.

[0106] Figure 2c A schematic diagram of image processing related equipment provided in an embodiment of the present application.

[0107] Above Figure 2a and Figure 2b The user equipment in the example may specifically be Figure 2c The local device 301 or the local device 302 in Figure 2a The data processing device in the embodiment may be Figure 2c The execution device 210 in the embodiment, wherein the data storage system 250 can store the data to be processed by the execution device 210, and the data storage system 250 can be integrated on the execution device 210, or can be set on the cloud or other network servers.

[0108] Figure 2a and Figure 2b The processor in the image processing apparatus can perform data training / machine learning / deep learning through a neural network model or other models (for example, a model based on a support vector machine), and use the model finally trained or learned from the data to execute image processing applications on the image, thereby obtaining corresponding processing results.

[0109] In addition, the embodiment of the present application also provides another image processing system, which includes a data processing device (such as a cloud server, a network server, an application server, a management server, etc., a device or server with data processing functions) and a user device. The data processing device executes the model training method in the embodiment of the present application, and obtains the first neural network, the second neural network, and the third neural network based on the method, and deploys them in the user device, so that the user device executes the neural network compression method in the embodiment of the present application.

[0110] Figure 3a A schematic diagram of the system 100 architecture provided in an embodiment of the present application, in Figure 3aIn the embodiment, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with an external device. The user can input data to the I / O interface 112 through the client device 140. The input data may include: various tasks to be scheduled, callable resources and other parameters in the embodiment of the application.

[0111] When the execution device 110 preprocesses the input data, or when the computing module 111 of the execution device 110 performs calculation and other related processing (such as implementing the function of the neural network in the present application), the execution device 110 can call the data, code, etc. in the data storage system 150 for the corresponding processing, and can also store the data, instructions, etc. obtained by the corresponding processing in the data storage system 150.

[0112] Finally, the I / O interface 112 returns the processing result to the client device 140 so as to provide it to the user.

[0113] It is worth noting that the training device 120 can generate corresponding target models / rules based on different training data for different goals or tasks, and the corresponding target models / rules can be used to achieve the above goals or complete the above tasks, thereby providing the user with the desired results. The training data can be stored in the database 130 and come from the training samples collected by the data collection device 160.

[0114] exist Figure 3a In the case shown in the figure, the user can manually give input data, and the manual giving can be operated through the interface provided by the I / O interface 112. In another case, the client device 140 can automatically send input data to the I / O interface 112. If the client device 140 is required to automatically send input data and needs to obtain the user's authorization, the user can set the corresponding authority in the client device 140. The user can view the results output by the execution device 110 on the client device 140, and the specific presentation form can be a specific method such as display, sound, action, etc. The client device 140 can also be used as a data acquisition terminal to collect the input data of the input I / O interface 112 and the output results of the output I / O interface 112 as shown in the figure as new sample data, and store them in the database 130. Of course, it is also possible not to collect through the client device 140, but the I / O interface 112 directly stores the input data of the input I / O interface 112 and the output results of the output I / O interface 112 as new sample data in the database 130.

[0115] It is worth noting that Figure 3a This is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 3a In the embodiment, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110. Figure 3a As shown, a neural network can be obtained by training according to the training device 120.

[0116] The present application also provides a chip including a neural network processor NPU. The chip can be arranged in Figure 3a The execution device 110 shown in FIG. 1 is used to complete the calculation work of the calculation module 111. The chip can also be set in Figure 3a The training device 120 shown is used to complete the training work of the training device 120 and output the target model / rule.

[0117] Neural network processor NPU, NPU is mounted on the main central processing unit (CPU) (host CPU) as a coprocessor, and the main CPU assigns tasks. The core part of NPU is the operation circuit, and the controller controls the operation circuit to extract data from the memory (weight memory or input memory) and perform operations.

[0118] In some implementations, the arithmetic circuit includes multiple processing units (process engines, PEs) inside. In some implementations, the arithmetic circuit is a two-dimensional systolic array. The arithmetic circuit can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit is a general-purpose matrix processor.

[0119] For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory and performs matrix operations with matrix B. The partial results or final results of the matrix are stored in the accumulator.

[0120] The vector computing unit can further process the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector computing unit can be used for network calculations of non-convolutional / non-FC layers in neural networks, such as pooling, batch normalization, local response normalization, etc.

[0121] In some implementations, the vector computation unit can store the processed output vector to a unified buffer. For example, the vector computation unit can apply a nonlinear function to the output of the computation circuit, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector computation unit generates a normalized value, a merged value, or both. In some implementations, the processed output vector can be used as an activation input to the computation circuit, such as for use in a subsequent layer in a neural network.

[0122] The unified memory is used to store input data and output data.

[0123] The weight data is directly transferred from the external memory to the input memory and / or the unified memory through the direct memory access controller (DMAC), the weight data in the external memory is stored in the weight memory, and the data in the unified memory is stored in the external memory.

[0124] The bus interface unit (BIU) is used to implement interaction between the main CPU, DMAC and instruction fetch memory through the bus.

[0125] An instruction fetch buffer connected to the controller, used to store instructions used by the controller;

[0126] The controller is used to call the instructions cached in the memory to control the working process of the computing accelerator.

[0127] Generally, the unified memory, input memory, weight memory and instruction fetch memory are all on-chip memories, and the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDRSDRAM), a high bandwidth memory (HBM) or other readable and writable memory.

[0128] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0129] (1) Neural Network

[0130] A neural network may be composed of neural units, and a neural unit may refer to an operation unit with xs and intercept 1 as input, and the output of the operation unit may be:

[0131]

[0132] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the predicted label of the local receptive field. The local receptive field can be an area composed of several neural units.

[0133] The work of each layer in the neural network can be described by the mathematical expression y=a(Wx+b): From a physical level, the work of each layer in the neural network can be understood as completing the transformation from the input space to the output space (i.e., the row space to the column space of the matrix) through five operations on the input space (the set of input vectors). These five operations include: 1. Dimension increase / reduction; 2. Zoom in / out; 3. Rotation; 4. Translation; 5. "Bending". Among them, operations 1, 2, and 3 are completed by Wx, operation 4 is completed by +b, and operation 5 is implemented by a(). The word "space" is used here because the classified object is not a single thing, but a class of things, and space refers to the collection of all individuals of this class of things. Among them, W is a weight vector, and each value in the vector represents the weight value of a neuron in the neural network of this layer. The vector W determines the spatial transformation from the input space to the output space described above, that is, the weight W of each layer controls how to transform the space. The purpose of training a neural network is to finally obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by many layers of vectors W). Therefore, the training process of a neural network is essentially about learning how to control spatial transformations, or more specifically, learning the weight matrix.

[0134] Because we want the output of the neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the target value we really want, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the neural network). For example, if the network's predicted value is high, adjust the weight vector to make it predict a lower value, and keep adjusting until the neural network can predict the target value we really want. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network becomes a process of minimizing this loss as much as possible.

[0135] (2) Back propagation algorithm

[0136] Neural networks can use the error back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the parameters in the initial neural network model are updated by back propagating the error loss information, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, which aims to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0137] The following describes the method provided in this application from the training side of the neural network and the application side of the neural network.

[0138] The model training method provided in the embodiment of the present application involves image processing, and can be specifically applied to data processing methods such as data training, machine learning, and deep learning, to perform symbolic and formalized intelligent information modeling, extraction, preprocessing, and training on the training data (such as the image to be trained in the present application), and finally obtain a trained neural network; and the neural network compression method provided in the embodiment of the present application can use the above-mentioned trained neural network to input the input data (such as the target image in the present application) into the trained neural network to obtain output data (such as the number of quantization bits and the characteristics of the target image in the present application). It should be noted that the model training method and the neural network compression method provided in the embodiment of the present application are inventions based on the same concept, and can also be understood as two parts in a system, or two stages of an overall process: such as the model training stage and the model application stage.

[0139] Figure 4 A flowchart of a neural network compression method provided in an embodiment of the present application. Figure 4 As shown, the method includes:

[0140] 401. Obtain a target image.

[0141] When it is necessary to perform image processing on the target image, the target image to be processed can be obtained first. Furthermore, the first neural network and the second neural network can also be obtained. Among them, the first neural network is used to compress the second neural network, that is, the first neural network is used to quantize the parameters of the second neural network, so as to reduce the device resources occupied by the parameters of the second neural network. The second neural network is used to perform image processing on the target image. For example, the second neural network can classify the target image to determine the category to which the object in the target image belongs. For another example, the second neural network can segment the target image to separate objects of different categories in the target image, and so on.

[0142] It should be understood that the first neural network can be any one of the models such as multi-layer perceptron (MLP), convolutional neural networks (CNN), recursive neural network (RNN), etc., and the second neural network can also be any one of the models such as MLP, CNN, recursive neural network, RNN, etc., without limitation here.

[0143] It should also be understood that the first neural network and the second neural network in the embodiments of the present application are both trained neural network models, and the training process of the first neural network and the second neural network will not be introduced in detail here.

[0144] 402. Input the target image into the first neural network to obtain the number of quantization bits of the second neural network, wherein the second neural network is used to perform image processing on the target image, and the number of quantization bits of the second neural network is positively correlated with the amount of calculation required for image processing.

[0145] For target images containing different contents, the difficulty of image processing is also different. For example, the image processing of the target image is used to identify the object of the target image. Suppose there are images A and B, the object in image A is a dog, and the object in image B is a dragonfly resting on a flower. Since the dog in image A is easy to identify, and the dragonfly in image B is not easy to identify, the recognition difficulty of image A is lower than that of image A.

[0146] It can be seen that if the image processing difficulty of the target image is relatively low, even if the amount of calculation required by the second neural network to process the target image is relatively small (i.e., the image processing accuracy is relatively low), the accuracy of image processing can be guaranteed. If the image processing difficulty of the target image is relatively high, the amount of calculation required by the second neural network to process the target image needs to be relatively large (i.e., the image processing accuracy is relatively high), thereby ensuring the accuracy of image processing. Based on this, when the parameters of the second neural network are quantized, they need to change with the changes in the image processing difficulty of the target image. If the parameters of the second neural network are represented and stored as high-bit numerical values, the image processing accuracy that can be achieved by the second neural network is relatively high. If the parameters of the second neural network are represented and stored as low-bit numerical values, the image processing accuracy that can be achieved by the second neural network is relatively low.

[0147] Specifically, after obtaining the target image, the target image can be input into the first neural network so that the first neural network performs image processing on the target image to obtain the number of quantization bits of the second neural network. The number of quantization bits of the second neural network is associated with the amount of computation required for the second neural network to perform image processing on the target image (i.e., the difficulty of the second neural network performing image processing on the target image). Generally, if the image processing difficulty of the target image is relatively small, the number of quantization bits of the second neural network is relatively small, and if the image processing difficulty of the target image is relatively large, the number of quantization bits of the second neural network is relatively large. Still as in the above example, if image A is input into the first neural network, the number of quantization bits of the second neural network output by the first neural network is 2 bits, and if image B is input into the second neural network, the number of quantization bits of the second neural network output by the first neural network is 4 bits.

[0148] Furthermore, the number of quantization bits of the second neural network may include the number of quantization bits of M layers of the second neural network (for example, M convolutional layers of the second neural network), where M is a positive integer. For the second neural network, the number of quantization bits of the M layers of the network may be completely the same or partially the same. Then, the first neural network may obtain the number of quantization bits of the M layers of the network in the second neural network in the following manner:

[0149] The target image is input into the first neural network to obtain the probability of the candidate bit number of the i-th layer network in the second neural network, where i = 1, 2, ..., M. Then, according to the size of the probability of the candidate bit number of the i-th layer network, the quantization bit number of the i-th layer network is selected from the candidate bit number of the i-th layer network. Therefore, the quantization bit number of each layer network in the M-layer network of the second neural network can be obtained.

[0150] In order to further understand the above process, Figure 5 For further explanation. Figure 5A schematic diagram of an application example of the neural network compression method provided in the embodiment of the present application is shown as follows: Figure 5 As shown, assume that there are image A and image B (the image processing difficulty of image A is lower than that of image B), and the number of candidate bits of each convolutional layer of the second neural network is 2 bits, 3 bits, 4 bits and 5 bits.

[0151] After inputting image A into the first neural network, the probability of the candidate bit numbers of each convolutional layer of the second neural network can be obtained, among which, among the probabilities of the candidate bit numbers of the first convolutional layer, the probability of 2 bits is the largest, among the probabilities of the candidate bit numbers of the second convolutional layer, the probability of 3 bits is the largest, among the probabilities of the candidate bit numbers of the third convolutional layer, the probability of 2 bits is the largest, and so on. Therefore, it can be determined that the quantization bit number of the first convolutional layer of the second neural network is 2 bits, the quantization bit number of the second convolutional layer is 3 bits, the quantization bit number of the third convolutional layer is 2 bits, …, the quantization bit number of the Mth convolutional layer is 2 bits.

[0152] After inputting image B into the first neural network, the probability of the candidate bit numbers of each convolutional layer of the second neural network can be obtained, among which, among the probabilities of the candidate bit numbers of the first convolutional layer, the probability of 5 bits is the largest, among the probabilities of the candidate bit numbers of the second convolutional layer, the probability of 4 bits is the largest, among the probabilities of the candidate bit numbers of the third convolutional layer, the probability of 4 bits is the largest, and so on. Therefore, it can be determined that the quantization bit number of the first convolutional layer of the second neural network is 5 bits, the quantization bit number of the second convolutional layer is 4 bits, the quantization bit number of the third convolutional layer is 4 bits, …, the quantization bit number of the Mth convolutional layer is 5 bits.

[0153] 403. Perform quantization processing on the parameters of the second neural network according to the quantization bit number of the second neural network to obtain the quantized second neural network.

[0154] After obtaining the number of quantization bits of the second neural network, the parameters of the second neural network can be quantized according to the number of quantization bits of the second neural network to obtain the quantized second neural network. Specifically, for an M-layer network of the second neural network, the parameters of the i-th layer network can be quantized according to the number of quantization bits of the i-th layer network in the second neural network to obtain the quantized second neural network, where i=1, 2, ..., M, and M is a positive integer.

[0155] Still like Figure 5As shown in the example, for image A, the quantization bit number of the first convolution layer of the second neural network is determined to be 2 bits, the quantization bit number of the second convolution layer is 3 bits, the quantization bit number of the third convolution layer is 2 bits, ..., the quantization bit number of the Mth convolution layer is 2 bits, then the parameters of the first convolution layer of the second neural network are represented by a 2-bit value, the parameters of the second convolution layer are represented by a 3-bit value, the parameters of the third convolution layer are represented by a 2-bit value, ..., the parameters of the Mth convolution layer are represented by a 2-bit value, thereby completing the quantization processing of the second neural network and obtaining the second neural network after quantization processing.

[0156] For image B, determine that the quantization bit number of the first convolution layer of the second neural network is 5 bits, the quantization bit number of the second convolution layer is 4 bits, the quantization bit number of the third convolution layer is 4 bits, ..., after the quantization bit number of the Mth convolution layer is 5 bits, the parameters of the first convolution layer of the second neural network are represented by a 5-bit value, the parameters of the second convolution layer are represented by a 4-bit value, the parameters of the third convolution layer are represented by a 4-bit value, ..., the parameters of the Mth convolution layer are represented by a 5-bit value, thereby completing the quantization processing of the second neural network and obtaining the second neural network after quantization processing.

[0157] In addition, in this embodiment, the parameters of each layer of the second neural network generally refer to the weight of the layer. While the weight of each layer of the network is quantized, the input of each layer may also be quantized accordingly. For example, while the weight of the first convolutional layer is represented by a 2-bit value, the input of the first convolutional layer may also be represented by a 2-bit value.

[0158] 404. Input the target image into the second neural network after quantization processing to obtain the characteristics of the target image.

[0159] After obtaining the second neural network after quantization processing, the target image can be input into the second neural network after quantization processing, so that the second neural network after quantization processing performs image processing (for example, image classification, target detection and image segmentation, etc.) on the target image, thereby obtaining the characteristics of the target image.

[0160] In this embodiment, after acquiring the target image, the target image can be input into the first neural network to obtain the number of quantization bits used to compress the second neural network. The number of quantization bits is positively correlated with the amount of calculation required to process the target image. If the amount of calculation required to process the target image is large, the number of quantization bits is large. If the amount of calculation required to process the target image is small, the number of quantization bits is small. It can be seen that due to the different processing difficulties of the target image, the number of quantization bits output by the first neural network is different, so that the degree of quantization processing of the second neural network is also different. In this way, for target images with lower processing difficulty, the second neural network after quantization processing can perform image processing with lower precision. For target images with higher processing difficulty, the second neural network after quantization processing can perform image processing with higher precision, thereby ensuring the accuracy of image processing.

[0161] In addition, the performance of the neural network in the embodiment of the present application can be compared with that of the neural network obtained by other compression methods (such as the neural network compression method in the background art). The comparison results are shown in Table 1 (corresponding to the first image data set):

[0162] Table 1

[0163]

[0164] Other compression methods (including method 1 and method 2) use a static fixed bit number method (i.e., the number of bits of the parameters of the neural network is fixed), while the present application uses a dynamic bit number adjustment method (i.e., the number of bits of the parameters of the neural network is automatically adjusted as the image changes). Based on Table 1, it can be seen that the compression method provided in the embodiment of the present application can ensure that the neural network can achieve higher accuracy while occupying a lower amount of computation when processing images in the first type of image data set.

[0165] Furthermore, the performance of the neural network in the embodiment of the present application can be compared with that of the neural network obtained by other compression methods (such as the neural network compression method in the background art). The comparison results are shown in Table 2 (corresponding to the second image data set):

[0166] Table 2

[0167]

[0168] Based on Table 2, it can be seen that the compression method provided in the embodiment of the present application can ensure that the neural network can achieve higher accuracy while occupying a lower amount of computation when processing images in the second category of image data sets.

[0169] Figure 6 Another flowchart of the neural network compression method provided in the embodiment of the present application is shown in FIG. Figure 6 As shown, the method includes:

[0170] 601. Acquire a target image.

[0171] When it is necessary to perform image processing on the target image, the target image to be processed can be obtained first. Furthermore, a third neural network and a second neural network can also be obtained. Among them, the third neural network is used to compress the input of the second neural network, that is, the third neural network is used to adjust the resolution of the target image, thereby reducing the device resources occupied by the second neural network when processing the image target. The second neural network is used to perform image processing on the target image. For example, the second neural network can classify the target image to determine the category to which the object in the target image belongs. For another example, the second neural network can segment the target image to separate objects of different categories in the target image, and so on.

[0172] It should be understood that the third neural network can be any one of the models such as MLP, CNN, recursive neural network, RNN, etc., and the second neural network can also be any one of the models such as MLP, CNN, recursive neural network, RNN, etc., without limitation here.

[0173] It should also be understood that the third neural network and the second neural network in the embodiments of the present application are both trained neural network models, and the training process of the third neural network and the second neural network will not be introduced in detail here.

[0174] 602. Input the target image into the third neural network to obtain a target resolution corresponding to the second neural network, wherein the second neural network is used to perform image processing on the target image, and the target resolution corresponding to the second neural network is positively correlated with the amount of calculation required for image processing.

[0175] For target images containing different contents, the difficulty of image processing is also different. For example, the image processing of the target image is used to identify the object of the target image. Suppose there are images A and B, the object in image A is a dog, and the object in image B is a dragonfly resting on a flower. Since the dog in image A is easy to identify, and the dragonfly in image B is not easy to identify, the recognition difficulty of image A is lower than that of image A.

[0176] It can be seen from this that if the image processing difficulty of the target image is relatively low, even if the amount of calculation required by the second neural network to process the target image is relatively small (i.e., the image processing accuracy is relatively low), the accuracy of image processing can be guaranteed. If the image processing difficulty of the target image is relatively high, the amount of calculation required by the second neural network to process the target image is relatively high (i.e., the image processing accuracy is relatively high), thereby ensuring the accuracy of image processing. Based on this, when the target image is input into the second neural network, the resolution of the target image can be changed as the image processing difficulty of the target image changes. If the resolution of the input target image is relatively low, the amount of calculation required by the second neural network to implement image processing is relatively low. If the resolution of the input target image is relatively high, the amount of calculation required by the second neural network to implement image processing is relatively high.

[0177] Specifically, after obtaining the target image, the target image is input into the third neural network to obtain the target resolution corresponding to the second neural network, wherein the target resolution corresponding to the second neural network is associated with the amount of computation required for the second neural network to process the target image (i.e., the difficulty of the second neural network to process the target image). Generally, if the image processing difficulty of the target image is relatively small, the target resolution corresponding to the second neural network (the target resolution of the target image input to the second neural network) is relatively small, and if the image processing difficulty of the target image is relatively large, the target resolution corresponding to the second neural network is relatively large. Still as in the above example, if image A is input into the first neural network, the target resolution corresponding to the second neural network output by the first neural network is 168×168, and if image B is input into the second neural network, the target resolution corresponding to the second neural network output by the first neural network is 224×224.

[0178] Furthermore, the first neural network can obtain the target resolution corresponding to the second neural network in the following manner:

[0179] The target image is input into the third neural network to obtain the probability of the candidate resolutions corresponding to the second neural network. According to the size of the probability of the candidate resolutions corresponding to the second neural network, the target resolution corresponding to the second neural network is selected from the candidate resolutions corresponding to the second neural network.

[0180] In order to further understand the above process, Figure 7 For further explanation. Figure 7 A schematic diagram of another application example of the neural network compression method provided in the embodiment of the present application is as follows: Figure 7 As shown, assume that there are image A and image B (the image processing difficulty of image A is lower than that of image B), and the candidate resolutions corresponding to the second neural network are 168×168, 200×200, and 224×224.

[0181] After inputting image A into the first neural network, the probabilities of the candidate resolutions corresponding to the second neural network can be obtained, that is, the probabilities of 168×168, 200×200 and 224×224. Among them, the probability of 168×168 is the largest, so 168×168 can be determined as the target resolution corresponding to the second neural network.

[0182] After inputting image B into the first neural network, the probabilities of the candidate resolutions corresponding to the second neural network can be obtained, that is, the probabilities of 168×168, 200×200 and 224×224 can be obtained. Among them, the probability of 224×224 is the largest, so 224×224 can be determined as the target resolution corresponding to the second neural network.

[0183] 603. Adjust the resolution of the target image to the target resolution to obtain the target image with the target resolution.

[0184] After obtaining the target resolution corresponding to the second neural network, the target image can be adjusted from the original resolution to the target resolution to obtain the target image with the target resolution. For example, if the target resolution is 168×168 and the original resolution of the target image is 400×400, the resolution of the target image is adjusted from 400×400 to 168×168 to obtain a target image with a resolution of 168×168.

[0185] 604. Input the target image of the target resolution into the second neural network to obtain features of the target image of the target resolution.

[0186] After obtaining the target image of the target resolution, the target image of the target resolution can be input into the second neural network so that the second neural network performs image processing (for example, image classification, target detection, image segmentation, etc.) on the target image of the target resolution, thereby obtaining the features of the target image of the target resolution.

[0187] In this embodiment, after acquiring the target image, the target image can be input into the third neural network to obtain the target resolution corresponding to the second neural network, which is used to compress the input of the second neural network, that is, the target image. The target resolution is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the target resolution is large. If the amount of calculation required for image processing of the target image is small, the target resolution is small. It can be seen that due to the different processing difficulties of the target image, the target resolution output by the third neural network is different, so that the target resolution of the adjusted target image is also different. In this way, for target images with lower processing difficulty, the resolution can be made smaller, so that the second neural network performs image processing with lower precision. For target images with higher processing difficulty, the resolution can be made larger, so that the second neural network performs image processing with higher precision, thereby ensuring the accuracy of image processing.

[0188] It is worth noting that the combination Figure 4 The embodiments shown and Figure 6 In an embodiment, after obtaining the target image of the target resolution, the target image of the target resolution can also be input into the second neural network after quantization processing, so that the second neural network after quantization processing performs image processing on the target image of the target resolution, thereby obtaining the characteristics of the target image of the target resolution.

[0189] In addition, the performance of the neural network in the embodiment of the present application can be compared with the neural network obtained by other compression methods. The comparison results are shown in Table 3 (corresponding to the first image data set):

[0190] Table 3

[0191]

[0192] Based on Table 3, it can be seen that the neural network provided in the embodiment of the present application can achieve higher accuracy while occupying a lower amount of computation when performing image processing.

[0193] The above is a detailed description of the neural network compression method provided in the embodiment of the present application. The following is an introduction to the model training method provided in the embodiment of the present application. Figure 8 A flow chart of the model training method provided in the embodiment of the present application is as follows: Figure 8 As shown, the method includes:

[0194] 801. Obtain an image to be trained.

[0195] When model training is to be performed, a training image for model training may be obtained first. It should be noted that the real features of the training image are known. Furthermore, a first model to be trained and a second model to be trained may be obtained to jointly train the two models.

[0196] 802. Input the image to be trained into the first model to be trained to obtain the number of quantization bits of the second model to be trained.

[0197] After obtaining the image to be trained and the first model to be trained, the image to be trained may be input into the first model to be trained to obtain the number of quantization bits of the second model to be trained.

[0198] Specifically, the number of quantization bits of the second model to be trained includes the number of quantization bits of the M-layer network in the second model to be trained, where M is a positive integer. Then, the first model to be trained can obtain the number of quantization bits of the M-layer network in the second model to be trained in the following manner:

[0199] The image to be trained is input into the first model to be trained to obtain the probability of the candidate bit number of the i-th layer network in the second model to be trained; according to the size of the probability of the candidate bit number of the i-th layer network in the second model to be trained, the quantization bit number of the i-th layer network is selected from the candidate bit number of the i-th layer network, i = 1, 2, ..., M.

[0200] 803. Perform quantization processing on the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained to obtain the quantized second model to be trained.

[0201] After obtaining the number of quantization bits of the second model to be trained, the parameters of the second model to be trained can be quantized according to the number of quantization bits of the second model to be trained to obtain the second model to be trained after quantization. Specifically, the parameters of the i-th layer network can be quantized according to the number of quantization bits of the i-th layer network in the second model to be trained to obtain the second model to be trained after quantization.

[0202] 804. Input the image to be trained into the second model to be trained after quantization processing to obtain features of the image to be trained.

[0203] After obtaining the second model to be trained after quantization processing, the image to be trained can be input into the second model to be trained after quantization processing, so that the second model to be trained after quantization processing performs image processing on the image to be trained, thereby obtaining the features (prediction features) of the image to be trained.

[0204] For instructions on steps 802 to 804, please refer to Figure 4 The relevant description parts of step 402 to step 404 in the illustrated embodiment will not be repeated here.

[0205] 805. Obtain a target loss according to a deviation between the number of quantization bits of the second model to be trained and a preset number of bits, and a deviation between the features of the image to be trained and the real features of the image to be trained.

[0206] After obtaining the number of quantization bits of the second model to be trained and the features of the image to be trained, the target loss can be obtained according to the deviation between the number of quantization bits of the second model to be trained and the preset number of bits, and the deviation between the features of the image to be trained and the real features of the image to be trained. Specifically, the target loss is obtained by constructing a standard cross entropy loss and introducing a regularization term, wherein the standard cross entropy loss is used to indicate the deviation between the features of the image to be trained and the real features of the image to be trained, and the regularization term is used to indicate the deviation between the number of quantization bits of the i-th layer network in the second model to be trained and the preset number of bits (i.e., the variation between the number of quantization bits of each layer network and the threshold number of bits in the M-layer network of the second model to be trained).

[0207] The target loss in this embodiment can be obtained by formula (2):

[0208]

[0209] In the above formula, L is Figure 8 The target loss, L, in the embodiment shown cls is the standard cross entropy loss, which is generated based on the features of the image to be trained and the real features of the image to be trained. α is a preset hyperparameter, and B i is the amount of computation corresponding to the number of quantization bits of the i-th layer network (i.e., the amount of computation occupied when the parameters of the i-th layer network are expressed in the corresponding number of quantization bits), B tar is the amount of calculation corresponding to the preset number of bits.

[0210] 806. Update the parameters of the first model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met, thereby obtaining the first neural network and the second neural network.

[0211] After obtaining the target loss, it can be determined whether the target loss has converged. If the target loss has not converged, the parameters of the first model to be trained and the parameters of the second model to be trained (the second model to be trained without quantization) are updated, and the first model to be trained and the second model to be trained are retrained jointly with the new training image until the target loss converges, that is, the model training condition is met, and the target loss is obtained. Figure 4 A first neural network and a second neural network in the illustrated embodiment.

[0212] In this embodiment, the first neural network and the second neural network can be obtained by jointly training the first model to be trained and the second model to be trained. The first neural network obtained by this embodiment can accurately obtain the number of quantization bits of the second neural network based on the input target image, and the second neural network obtained by this embodiment can accurately process the target image.

[0213] Fig. 9 Another flowchart of the model training method provided in the embodiment of the present application is as follows: Fig. 9 As shown, the method includes:

[0214] 901. Obtain an image to be trained;

[0215] When model training is to be performed, the image to be trained for model training can be obtained first. It should be noted that the real features of the image to be trained at the target resolution are known. Furthermore, the third model to be trained and the second model to be trained can also be obtained to jointly train the two models.

[0216] 902. Input the image to be trained into the third model to be trained to obtain a target resolution corresponding to the second model to be trained.

[0217] After obtaining the image to be trained and the third model to be trained, the image to be trained is input into the third model to be trained to obtain the target resolution corresponding to the second model to be trained.

[0218] Specifically, the third model to be trained can obtain the target resolution corresponding to the second model to be trained in the following manner:

[0219] The image to be trained is input into the third model to be trained to obtain the probability of the candidate resolutions corresponding to the second model to be trained; according to the size of the probability of the candidate resolutions corresponding to the second model to be trained, the target resolution corresponding to the second model to be trained is selected from the candidate resolutions corresponding to the second model to be trained.

[0220] 903. Adjust the resolution of the image to be trained to the target resolution to obtain the image to be trained with the target resolution.

[0221] After obtaining the target resolution corresponding to the second model to be trained, the image to be trained may be adjusted from the original resolution to the target resolution to obtain the image to be trained with the target resolution.

[0222] 904. Input the image to be trained at the target resolution into the second model to be trained to obtain features of the image to be trained at the target resolution.

[0223] After obtaining the image to be trained with the target resolution, the image to be trained with the target resolution can be input into the second model to be trained, so that the second model to be trained performs image processing on the image to be trained with the target resolution, thereby obtaining the features (prediction features) of the image to be trained with the target resolution.

[0224] For instructions on steps 902 to 904, please refer to Figure 6 The relevant description parts of steps 602 to 604 in the illustrated embodiment will not be repeated here.

[0225] 905. Obtain a target loss according to a deviation between an expected value corresponding to the target resolution and a preset expected value, and a deviation between features of the to-be-trained image at the target resolution and actual features of the to-be-trained image at the target resolution.

[0226] After obtaining the target resolution corresponding to the second model to be trained and the features of the target image at the target resolution, the target loss can be obtained according to the deviation between the expected value corresponding to the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution. Specifically, the target loss is obtained by constructing a standard cross entropy loss and introducing a regularization term, wherein the standard cross entropy loss is used to indicate the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution, and the regularization term is used to indicate the deviation between the expected value of the probability according to the target resolution and the preset expected value, and the deviation between the expected value of the probability of the remaining candidate resolutions except the target resolution and the preset expected value.

[0227] The target loss in this embodiment can be obtained by formula (3):

[0228] L0=L ce +ηL reg

[0229]

[0230] In the above formula, L0 is Fig. 9 The target loss, L, in the embodiment shown ce is the standard cross entropy loss, which is generated based on the features of the training image at the target resolution and the real features of the training image at the target resolution, η is the preset hyperparameter, β is the preset expected value (preset penalty coefficient), E(h j ) is the expected value (mathematical expectation) of the probability of the jth candidate resolution, and N is the number of candidate resolutions. It can be seen that if the expected value of the probability of a candidate resolution is less than β, then L reg Will increase.

[0231] 906. Update the parameters of the third model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met, thereby obtaining the third neural network and the second neural network.

[0232] After obtaining the target loss, it can be determined whether the target loss has converged. If the target loss has not converged, the parameters of the third model to be trained and the parameters of the second model to be trained are updated, and the third model to be trained and the second model to be trained are retrained jointly with the new image to be trained until the target loss converges, that is, the model training conditions are met, and the target loss is obtained. Figure 6 The third neural network and the second neural network in the illustrated embodiment.

[0233] In this embodiment, the third neural network and the second neural network can be obtained by jointly training the third model to be trained and the second model to be trained. The third neural network obtained by this embodiment can accurately obtain the target resolution corresponding to the second neural network based on the input target image, and the second neural network obtained by this embodiment can accurately process the target image.

[0234] The above is a detailed description of the model training method provided in the embodiment of the present application. The following is an introduction to the neural network compression device provided in the embodiment of the present application. Fig.10 A schematic diagram of the structure of a neural network compression device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the device includes: an acquisition module 1001 and a processing module 1002;

[0235] An acquisition module 1001 is used to acquire a target image;

[0236] The processing module 1002 is used to input the target image into the first neural network to obtain the quantization bit number of the second neural network, wherein the second neural network is used to perform image processing on the target image, and the quantization bit number of the second neural network is positively correlated with the amount of calculation required for the image processing;

[0237] The processing module 1002 is further used to quantize the parameters of the second neural network according to the quantization bit number of the second neural network to obtain the quantized second neural network.

[0238] It can be seen from the above device that after acquiring the target image, the target image can be input into the first neural network to obtain the number of quantization bits used to compress the second neural network. The number of quantization bits is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the number of quantization bits is large, and if the amount of calculation required for image processing of the target image is small, the number of quantization bits is small. It can be seen that due to the different processing difficulties of the target image, the number of quantization bits output by the first neural network is different, so that the degree of quantization processing of the second neural network is also different. In this way, for target images with lower processing difficulty, the second neural network after quantization processing can perform image processing with lower precision, and for target images with higher processing difficulty, the second neural network after quantization processing can perform image processing with higher precision, thereby ensuring the accuracy of image processing.

[0239] In one possible implementation, the quantization bit number of the second neural network includes the quantization bit number of the M-layer network in the second neural network, and the processing module 1002 is specifically used to quantize the parameters of the i-th layer network according to the quantization bit number of the i-th layer network in the second neural network to obtain the quantized second neural network, where i=1, 2,…, M, and M is a positive integer.

[0240] In one possible implementation, the processing module 1002 is specifically used to: input the target image into the first neural network to obtain the probability of the candidate bit number of the i-th layer network in the second neural network; and select the quantization bit number of the i-th layer network from the candidate bit number of the i-th layer network according to the size of the probability.

[0241] In a possible implementation, the processing module 1002 is further configured to input the target image into the second neural network after quantization processing to obtain features of the target image.

[0242] Fig.11 Another structural diagram of the neural network compression device provided in the embodiment of the present application is as follows Fig.11 As shown, the device includes an acquisition module 1101 and a processing module 1102;

[0243] An acquisition module 1101 is used to acquire a target image;

[0244] The processing module 1102 is used to input the target image into the third neural network to obtain the target resolution corresponding to the second neural network, wherein the second neural network is used to perform image processing on the target image, and the target resolution corresponding to the second neural network is positively correlated with the amount of calculation required for the image processing;

[0245] The processing module 1102 is further configured to adjust the resolution of the target image to the target resolution to obtain the target image with the target resolution.

[0246] It can be seen from the above device that after acquiring the target image, the target image can be input into the third neural network to obtain the target resolution corresponding to the second neural network, which is used to compress the input of the second neural network, that is, the target image. The target resolution is positively correlated with the amount of calculation required for image processing of the target image. If the amount of calculation required for image processing of the target image is large, the target resolution is large. If the amount of calculation required for image processing of the target image is small, the target resolution is small. It can be seen that due to the different processing difficulties of the target image, the target resolution output by the third neural network is different, so that the target resolution of the adjusted target image is also different. In this way, for target images with lower processing difficulty, the resolution can be made smaller, so that the second neural network performs image processing with lower precision. For target images with higher processing difficulty, the resolution can be made larger, so that the second neural network performs image processing with higher precision, thereby ensuring the accuracy of image processing.

[0247] In one possible implementation, the processing module 1102 is specifically used to: input the target image into the third neural network to obtain the probability of the candidate resolutions corresponding to the second neural network; and select the target resolution corresponding to the second neural network from the candidate resolutions corresponding to the second neural network according to the size of the probability.

[0248] In a possible implementation, the processing module 1102 is further configured to input the target image of the target resolution into the second neural network to obtain features of the target image of the target resolution.

[0249] The above is a detailed description of the neural network compression device provided in the embodiment of the present application. The following is an introduction to the model training device provided in the embodiment of the present application. Fig.12 A structural diagram of a model training device provided in an embodiment of the present application, such as Fig.12 As shown, the device includes an acquisition module 1201 and a training module 1202;

[0250] An acquisition module 1201 is used to acquire an image to be trained;

[0251] The training module 1202 is used to input the image to be trained into the first model to be trained to obtain the number of quantization bits of the second model to be trained;

[0252] The training module 1202 is further used to quantize the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained to obtain the quantized second model to be trained;

[0253] The training module 1202 is further used to input the image to be trained into the second model to be trained after quantization processing to obtain the features of the image to be trained;

[0254] The training module 1202 is also used to update the parameters of the first model to be trained and the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained and the characteristics of the image to be trained, until the model training conditions are met to obtain the first neural network and the second neural network.

[0255] It can be seen from the above device that the first neural network and the second neural network can be obtained by jointly training the first model to be trained and the second model to be trained. The first neural network obtained by the device can accurately obtain the number of quantization bits of the second neural network based on the input target image, and the second neural network obtained by the device can accurately process the target image.

[0256] In one possible implementation, the training module 1202 is specifically used to: obtain a target loss based on a deviation between the number of quantization bits of the second model to be trained and a preset number of bits, and a deviation between features of the image to be trained and real features of the image to be trained; update the parameters of the first model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met, thereby obtaining a first neural network and a second neural network.

[0257] In one possible implementation, the number of quantization bits of the second model to be trained includes the number of quantization bits of the M-layer network in the second model to be trained. The training module 1202 is specifically used to quantize the parameters of the i-th layer network according to the number of quantization bits of the i-th layer network in the second model to be trained to obtain the second model to be trained after quantization, where i=1, 2,…, M, and M is a positive integer.

[0258] In one possible implementation, the training module 1202 is specifically used to: input the image to be trained into the first model to be trained to obtain the probability of the candidate bit number of the i-th layer network in the second model to be trained; and select the quantization bit number of the i-th layer network from the candidate bit number of the i-th layer network according to the size of the probability.

[0259] In a possible implementation, the deviation between the number of quantization bits of the second model to be trained and the preset number of bits includes the deviation between the number of quantization bits of the i-th layer network in the second model to be trained and the preset number of bits.

[0260] Fig.13 Another structural diagram of the model training device provided in the embodiment of the present application is as follows Fig.13 As shown, the device includes an acquisition module 1301 and a training module 1302;

[0261] An acquisition module 1301 is used to acquire an image to be trained;

[0262] A training module 1302 is used to input the image to be trained into the third model to be trained to obtain a target resolution corresponding to the second model to be trained;

[0263] The training module 1302 is further used to adjust the resolution of the image to be trained to the target resolution, so as to obtain the image to be trained with the target resolution;

[0264] The training module 1302 is further used to input the image to be trained at the target resolution into the second model to be trained to obtain the features of the image to be trained at the target resolution;

[0265] The training module 1302 is also used to update the parameters of the third model to be trained and the parameters of the second model to be trained according to the target resolution and the features of the image to be trained with the target resolution until the model training conditions are met to obtain the third neural network and the second neural network.

[0266] It can be seen from the above device that by jointly training the third model to be trained and the second model to be trained, a third neural network and a second neural network can be obtained. The third neural network obtained by the device can accurately obtain the target resolution corresponding to the second neural network based on the input target image, and the second neural network obtained by the device can accurately process the target image.

[0267] In one possible implementation, the training module 1302 is specifically used to: obtain the target loss based on the deviation between the expected value corresponding to the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution; update the parameters of the third model to be trained and the parameters of the second model to be trained according to the target loss until the model training conditions are met, so as to obtain the third neural network and the second neural network.

[0268] In one possible implementation, the training module 1302 is specifically used to: input the image to be trained into the third model to be trained to obtain the probability of the candidate resolution corresponding to the second model to be trained; and select the target resolution corresponding to the second model to be trained from the candidate resolutions corresponding to the second model to be trained according to the size of the probability.

[0269] In one possible implementation, the training module 1302 is specifically used to obtain the target loss based on the deviation between the expected value of the probability of the target resolution and the preset expected value, the deviation between the expected value of the probability of the remaining candidate resolutions except the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the actual features of the image to be trained at the target resolution.

[0270] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the embodiment of the present application, and will not be repeated here.

[0271] The embodiment of the present application also relates to an execution device, Fig.14 A schematic diagram of the structure of the execution device provided in the embodiment of the present application. Fig.14 As shown, the execution device 1400 can be specifically a mobile phone, a tablet, a laptop, a smart wearable device, a server, etc., which is not limited here. Fig.10 or Fig.11 The neural network compression device described in the corresponding embodiment is used to implement Figure 4 or Figure 6 The neural network compression function and the image processing function in the corresponding embodiment. Specifically, the execution device 1400 includes: a receiver 1401, a transmitter 1402, a processor 1403 and a memory 1404 (wherein the number of the processor 1403 in the execution device 1400 can be one or more, Fig.14 In the example of FIG. 1403 , the processor 1403 may include an application processor 14031 and a communication processor 14032. In some embodiments of the present application, the receiver 1401, the transmitter 1402, the processor 1403 and the memory 1404 may be connected via a bus or other means.

[0272] The memory 1404 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1403. A portion of the memory 1404 may also include a non-volatile random access memory (NVRAM). The memory 1404 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0273] The processor 1403 controls the operation of the execution device. In a specific application, the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0274] The method disclosed in the above embodiment of the present application can be applied to the processor 1403, or implemented by the processor 1403. The processor 1403 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1403. The above processor 1403 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1403 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1404, and the processor 1403 reads the information in the memory 1404 and completes the steps of the above method in combination with its hardware.

[0275] The receiver 1401 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1402 can be used to output digital or character information through the first interface; the transmitter 1402 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1402 can also include a display device such as a display screen.

[0276] In one embodiment of the present application, the processor 1403 is configured to execute Figure 4 or Figure 6 The neural network compression method in the corresponding embodiment.

[0277] The present application also relates to a training device. Fig.15 A schematic diagram of the structure of the training device provided in the embodiment of the present application. Fig.15As shown, the training device 1500 is implemented by one or more servers. The training device 1500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1515 (for example, one or more processors) and memory 1532, one or more storage media 1530 (for example, one or more mass storage devices) storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be short-term storage or permanent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the training device. Furthermore, the central processing unit 1515 can be configured to communicate with the storage medium 1530 to execute a series of instruction operations in the storage medium 1530 on the training device 1500.

[0278] The training device 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558; or, one or more operating systems 1541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0279] Specifically, the training device can perform Figure 8 or Fig. 9 The steps in the corresponding embodiments.

[0280] An embodiment of the present application also relates to a computer storage medium, in which a program for signal processing is stored. When the program is run on a computer, the computer executes the steps executed by the aforementioned execution device, or the computer executes the steps executed by the aforementioned training device.

[0281] An embodiment of the present application also relates to a computer program product, which stores instructions, which, when executed by a computer, enable the computer to execute the steps executed by the aforementioned execution device, or enable the computer to execute the steps executed by the aforementioned training device.

[0282] The execution device, training device or terminal device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0283] For details, please refer to Fig.16 , Fig.16 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 1600, NPU 1600 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 1603, which is controlled by the controller 1604 to extract matrix data from the memory and perform multiplication operations.

[0284] In some implementations, the operation circuit 1603 includes multiple processing units (Process Engine, PE) inside. In some implementations, the operation circuit 1603 is a two-dimensional systolic array. The operation circuit 1603 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1603 is a general-purpose matrix processor.

[0285] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1602 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1601 and performs matrix operation with matrix B, and the partial result or final result of the matrix is ​​stored in the accumulator 1608.

[0286] The unified memory 1606 is used to store input data and output data. The weight data is directly transferred to the weight memory 1602 through the direct memory access controller (DMAC) 1605. The input data is also transferred to the unified memory 1606 through the DMAC.

[0287] BIU stands for Bus Interface Unit 1610 , which is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (IFB) 1609 .

[0288] The bus interface unit 1610 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1609 to obtain instructions from the external memory, and is also used for the storage unit access controller 1605 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0289] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1606 or to transfer weight data to the weight memory 1602 or to transfer input data to the input memory 1601.

[0290] The vector calculation unit 1607 includes multiple operation processing units, and further processes the output of the operation circuit 1603 when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of feature planes, etc.

[0291] In some implementations, the vector calculation unit 1607 can store the processed output vector to the unified memory 1606. For example, the vector calculation unit 1607 can apply a linear function; or a nonlinear function to the output of the operation circuit 1603, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 1607 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1603, for example, for use in a subsequent layer in a neural network.

[0292] An instruction fetch buffer 1609 connected to the controller 1604 is used to store instructions used by the controller 1604;

[0293] Unified memory 1606, input memory 1601, weight memory 1602 and instruction fetch memory 1609 are all On-Chip memories. External memories are private to the NPU hardware architecture.

[0294] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.

[0295] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0296] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0297] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0298] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A neural network compression method, characterized in that: The method comprises: Get the target image; Inputting the target image into a first neural network to obtain a quantization bit number of a second neural network, wherein the second neural network is used to perform image processing on the target image, and the quantization bit number of the second neural network is positively correlated with the amount of calculation required for the image processing; According to the number of quantization bits of the second neural network, the parameters of the second neural network are quantized to obtain the quantized second neural network.

2. The method according to claim 1, characterized in that The quantization bit number of the second neural network includes the quantization bit number of the M-layer network in the second neural network, and the quantization processing of the parameters of the second neural network according to the quantization bit number of the second neural network to obtain the quantized second neural network specifically includes: According to the number of quantization bits of the i-th layer network in the second neural network, the parameters of the i-th layer network are quantized to obtain the second neural network after quantization, wherein i=1, 2, ..., M, and M is a positive integer.

3. The method according to claim 2, characterized in that The step of inputting the target image into the first neural network to obtain the number of quantization bits of the second neural network specifically includes: Input the target image into the first neural network to obtain the probability of the number of candidate bits of the i-th layer network in the second neural network; According to the size of the probability, the number of quantization bits of the i-th layer network is selected from the candidate number of bits of the i-th layer network.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The target image is input into the second neural network after the quantization process to obtain the features of the target image.

5. A neural network compression method, characterized in that: The method comprises: Get the target image; Inputting the target image into a third neural network to obtain a target resolution corresponding to the second neural network, wherein the second neural network is used to perform image processing on the target image, and the target resolution corresponding to the second neural network is positively correlated with the amount of calculation required for the image processing; The resolution of the target image is adjusted to the target resolution to obtain the target image of the target resolution.

6. The method according to claim 5, characterized in that Inputting the target image into the third neural network to obtain the target resolution corresponding to the second neural network specifically includes: Inputting the target image into a third neural network to obtain a probability of a candidate resolution corresponding to the second neural network; According to the size of the probability, a target resolution corresponding to the second neural network is selected from candidate resolutions corresponding to the second neural network.

7. The method according to claim 5 or 6, characterized in that: After adjusting the resolution of the target image to the target resolution to obtain the target image with the target resolution, the method further includes: The target image of the target resolution is input into the second neural network to obtain features of the target image of the target resolution.

8. A model training method, characterized in that: The method comprises: Get the image to be trained; Inputting the image to be trained into the first model to be trained to obtain the number of quantization bits of the second model to be trained; quantizing the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained to obtain the quantized second model to be trained; Inputting the image to be trained into the second model to be trained after the quantization process to obtain features of the image to be trained; According to the number of quantization bits of the second model to be trained and the features of the image to be trained, the parameters of the first model to be trained and the parameters of the second model to be trained are updated until the model training conditions are met, thereby obtaining the first neural network and the second neural network.

9. The method according to claim 8, characterized in that The updating of the parameters of the first model to be trained and the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained and the features of the image to be trained until the model training conditions are met to obtain the first neural network and the second neural network specifically includes: Obtaining a target loss according to a deviation between the number of quantization bits of the second model to be trained and a preset number of bits, and a deviation between the features of the image to be trained and the real features of the image to be trained; The parameters of the first model to be trained and the parameters of the second model to be trained are updated according to the target loss until the model training conditions are met, thereby obtaining the first neural network and the second neural network.

10. The method according to claim 8 or 9, characterized in that: The number of quantization bits of the second model to be trained includes the number of quantization bits of the M-layer network in the second model to be trained, and the quantization processing of the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained to obtain the quantized second model to be trained specifically includes: According to the number of quantization bits of the i-th layer network in the second model to be trained, the parameters of the i-th layer network are quantized to obtain the second model to be trained after quantization, where i=1, 2, ..., M, and M is a positive integer.

11. The method according to claim 10, characterized in that The step of inputting the image to be trained into the first model to be trained to obtain the number of quantization bits of the second model to be trained specifically includes: Inputting the image to be trained into the first model to be trained, and obtaining the probability of the number of candidate bits of the i-th layer network in the second model to be trained; According to the size of the probability, the number of quantization bits of the i-th layer network is selected from the candidate number of bits of the i-th layer network.

12. The method according to claim 9, characterized in that The deviation between the number of quantization bits of the second model to be trained and the preset number of bits includes the deviation between the number of quantization bits of the i-th layer network in the second model to be trained and the preset number of bits.

13. A model training method, characterized in that: The method comprises: Get the image to be trained; Inputting the image to be trained into the third model to be trained to obtain a target resolution corresponding to the second model to be trained; Adjusting the resolution of the image to be trained to the target resolution to obtain the image to be trained with the target resolution; Inputting the image to be trained at the target resolution into the second model to be trained to obtain features of the image to be trained at the target resolution; According to the target resolution and the features of the image to be trained at the target resolution, the parameters of the third model to be trained and the parameters of the second model to be trained are updated until the model training conditions are met, thereby obtaining the third neural network and the second neural network.

14. The method according to claim 13, characterized in that The updating of the parameters of the third model to be trained and the parameters of the second model to be trained according to the target resolution and the features of the image to be trained of the target resolution until the model training conditions are met, and obtaining the third neural network and the second neural network specifically includes: Obtaining a target loss according to a deviation between an expected value corresponding to the target resolution and a preset expected value, and a deviation between a feature of the to-be-trained image at the target resolution and a real feature of the to-be-trained image at the target resolution; The parameters of the third model to be trained and the parameters of the second model to be trained are updated according to the target loss until the model training conditions are met, thereby obtaining the third neural network and the second neural network.

15. The method according to claim 13 or 14, characterized in that Inputting the to-be-trained image into the third to-be-trained model to obtain the target resolution corresponding to the second to-be-trained model specifically includes: Inputting the image to be trained into the third model to be trained to obtain the probability of the candidate resolution corresponding to the second model to be trained; According to the size of the probability, a target resolution corresponding to the second model to be trained is selected from candidate resolutions corresponding to the second model to be trained.

16. The method according to claim 14, characterized in that The obtaining of the target loss according to the deviation between the expected value corresponding to the target resolution and the preset expected value, and the deviation between the feature of the image to be trained at the target resolution and the real feature of the image to be trained at the target resolution specifically includes: The target loss is obtained according to the deviation between the expected value of the probability of the target resolution and the preset expected value, the deviation between the expected value of the probability of the remaining candidate resolutions except the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution.

17. A neural network compression device, characterized in that: The device comprises an acquisition module and a processing module; The acquisition module is used to acquire the target image; The processing module is used to input the target image into a first neural network to obtain a quantization bit number of a second neural network, wherein the second neural network is used to perform image processing on the target image, and the quantization bit number of the second neural network is positively correlated with the amount of calculation required for the image processing; The processing module is further used to quantize the parameters of the second neural network according to the quantization bit number of the second neural network to obtain the quantized second neural network.

18. The device according to claim 17, characterized in that The quantization bit number of the second neural network includes the quantization bit number of the M-layer network in the second neural network. The processing module is specifically used to quantize the parameters of the i-th layer network according to the quantization bit number of the i-th layer network in the second neural network to obtain the second neural network after quantization, wherein i=1, 2, ..., M, and M is a positive integer.

19. The device according to claim 18, characterized in that The processing module is specifically used for: Input the target image into the first neural network to obtain the probability of the number of candidate bits of the i-th layer network in the second neural network; According to the size of the probability, the number of quantization bits of the i-th layer network is selected from the candidate number of bits of the i-th layer network.

20. The device according to any one of claims 17 to 19, characterized in that The processing module is further used to input the target image into the second neural network after the quantization process to obtain the characteristics of the target image.

21. A neural network compression device, characterized in that: The device comprises an acquisition module and a processing module; The acquisition module is used to acquire the target image; The processing module is used to input the target image into a third neural network to obtain a target resolution corresponding to the second neural network, wherein the second neural network is used to perform image processing on the target image, and the target resolution corresponding to the second neural network is positively correlated with the amount of calculation required for the image processing; The processing module is further used to adjust the resolution of the target image to a target resolution to obtain a target image with a target resolution.

22. The device according to claim 21, characterized in that The processing module is specifically used for: Inputting the target image into a third neural network to obtain a probability of a candidate resolution corresponding to the second neural network; According to the size of the probability, a target resolution corresponding to the second neural network is selected from candidate resolutions corresponding to the second neural network.

23. The device according to claim 21 or 22, characterized in that The processing module is further used to input the target image of the target resolution into the second neural network to obtain features of the target image of the target resolution.

24. A model training device, characterized in that: The device comprises an acquisition module and a training module; The acquisition module is used to acquire the image to be trained; The training module is used to input the image to be trained into the first model to be trained to obtain the number of quantization bits of the second model to be trained; The training module is further used to quantize the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained to obtain the second model to be trained after quantization; The training module is further used to input the image to be trained into the second model to be trained after the quantization process to obtain the features of the image to be trained; The training module is also used to update the parameters of the first model to be trained and the parameters of the second model to be trained according to the number of quantization bits of the second model to be trained and the characteristics of the image to be trained, until the model training conditions are met to obtain the first neural network and the second neural network.

25. The device according to claim 24, characterized in that The training module is specifically used for: Obtaining a target loss according to a deviation between the number of quantization bits of the second model to be trained and a preset number of bits, and a deviation between the features of the image to be trained and the real features of the image to be trained; The parameters of the first model to be trained and the parameters of the second model to be trained are updated according to the target loss until the model training conditions are met, thereby obtaining the first neural network and the second neural network.

26. The device according to claim 24 or 25, characterized in that The number of quantization bits of the second model to be trained includes the number of quantization bits of the M-layer network in the second model to be trained. The training module is specifically used to quantize the parameters of the i-th layer network according to the number of quantization bits of the i-th layer network in the second model to be trained to obtain the second model to be trained after quantization, wherein i=1, 2,..., M, and M is a positive integer.

27. The device according to claim 26, characterized in that The training module is specifically used for: Inputting the image to be trained into the first model to be trained, and obtaining the probability of the number of candidate bits of the i-th layer network in the second model to be trained; According to the size of the probability, the number of quantization bits of the i-th layer network is selected from the candidate number of bits of the i-th layer network.

28. The device according to claim 25, characterized in that The deviation between the number of quantization bits of the second model to be trained and the preset number of bits includes the deviation between the number of quantization bits of the i-th layer network in the second model to be trained and the preset number of bits.

29. A model training device, characterized in that: The device comprises an acquisition module and a training module; The acquisition module is used to acquire the image to be trained; The training module is used to input the image to be trained into the third model to be trained to obtain the target resolution corresponding to the second model to be trained; The training module is further used to adjust the resolution of the image to be trained to a target resolution, so as to obtain an image to be trained with the target resolution; The training module is further used to input the to-be-trained image of the target resolution into the second to-be-trained model to obtain features of the to-be-trained image of the target resolution; The training module is also used to update the parameters of the third model to be trained and the parameters of the second model to be trained according to the target resolution and the characteristics of the image to be trained with the target resolution, until the model training conditions are met to obtain the third neural network and the second neural network.

30. The device according to claim 29, characterized in that The training module is specifically used for: Obtaining a target loss according to a deviation between an expected value corresponding to the target resolution and a preset expected value, and a deviation between a feature of the to-be-trained image at the target resolution and a real feature of the to-be-trained image at the target resolution; The parameters of the third model to be trained and the parameters of the second model to be trained are updated according to the target loss until the model training conditions are met, thereby obtaining the third neural network and the second neural network.

31. The device according to claim 29 or 30, characterized in that The training module is specifically used for: Inputting the image to be trained into the third model to be trained to obtain the probability of the candidate resolution corresponding to the second model to be trained; According to the size of the probability, a target resolution corresponding to the second model to be trained is selected from candidate resolutions corresponding to the second model to be trained.

32. The device according to claim 30, characterized in that The training module is specifically used to obtain the target loss based on the deviation between the expected value of the probability of the target resolution and the preset expected value, the deviation between the expected value of the probability of the remaining candidate resolutions except the target resolution and the preset expected value, and the deviation between the features of the image to be trained at the target resolution and the real features of the image to be trained at the target resolution.

33. A neural network compression device, characterized in that: It includes a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the neural network compression device performs the method as described in any one of claims 1 to 7.

34. A model training device, characterized in that: It includes a memory and a processor; the memory stores codes, and the processor is configured to execute the codes. When the codes are executed, the model training device executes the method as described in any one of claims 8 to 16.

35. A computer storage medium, characterized in that The computer storage medium stores a computer program, which, when executed by a computer, enables the computer to implement the method according to any one of claims 1 to 16.

36. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, enable the computer to implement the method according to any one of claims 1 to 16.

Images