Neural network compression method and device

Abstract

The present invention provides a neural network compression method and device, which includes: using the neural network loaded in the target accelerator as the environment to determine the compression hyperparameter combination; using the compression hyperparameter combination as the action to compress the neural network to determine the reward value after the neural network is compressed; updating the environment, compression hyperparameter combination, action and reward value according to the compressed neural network, and stopping the update when the preset conditions are met; compressing the neural network according to the compression hyperparameter combination after the last update. The device is used to execute the above method. The neural network compression method and device provided by the present invention analyze the accelerator to obtain the compression hyperparameter combination suitable for the acceleration architecture, thereby more effectively reducing the computational overhead of the neural network on the accelerator than traditional non-adaptive compression, and improving the energy efficiency of the reasoning process of the neural network on the accelerator.

Description

Technical Field

[0001] The present invention relates to the field of computer information processing technology, and in particular to a neural network compression method and device. Background Art

[0002] In recent years, with the huge demand for edge neural network applications, dedicated ASIC (Application Specific Integrated Circuit) neural network accelerators have seen rapid development. Relying on neural network accelerators, most computations can be performed locally at the edge. Compared to cloud-based solutions, edge solutions based on network accelerators offer high energy efficiency, high real-time performance, and privacy protection.

[0003] However, in pursuit of stronger generalization capabilities, existing cutting-edge models such as the third-generation General Pre-trained Transformer-3 (GPT-3) and AlphaFold have adopted increasingly larger network scales. For example, the GPT-3 model for natural language processing uses more than 10,000 GPUs for training, and the total parameter size reaches 175 billion. Even the common ResNet34 model has a model size of 83.2MB. For existing ASIC accelerators, their on-chip storage space often does not exceed 1MB. Excessive model size will lead to severe challenges such as excessive chip area, excessive power consumption of external DRAM access, and memory wall.

[0004] Neural network compression, including model pruning and model quantization, effectively reduces model size. Model pruning significantly reduces computational and storage overhead by forcing redundant neurons in the network to zero, thereby skipping these neurons during inference. Model quantization effectively reduces computational power consumption by converting floating-point operations to fixed-point operations. However, in practice, compression algorithms often conflict with accelerator architectures, preventing them from fully realizing their performance. Summary of the Invention

[0005] The neural network compression method provided by the present invention is used to overcome the above-mentioned problems existing in the prior art. By analyzing the accelerator, a compression hyperparameter combination suitable for the acceleration architecture is obtained, thereby more effectively reducing the computational overhead of the neural network on the accelerator than traditional non-adaptive compression and improving the energy efficiency of the neural network inference process on the accelerator.

[0006] The present invention provides a neural network compression method, comprising:

[0007] Use the neural network loaded into the target accelerator as an environment to determine the compression hyperparameter combination;

[0008] Compressing the neural network using the compression hyperparameter combination as an action to determine a reward value after compressing the neural network;

[0009] updating the environment, the compressed hyperparameter combination, the action, and the reward value according to the compressed neural network, and stopping the updating when a preset condition is met;

[0010] The neural network is compressed according to the last updated compression hyperparameter combination.

[0011] According to a neural network compression method provided by the present invention, the neural network loaded in the target accelerator is used as an environment to determine a compression hyperparameter combination, including:

[0012] Taking a neural network loaded in a target accelerator as an environment, determining a quantization parameter and a sparsity parameter for compressing the neural network by analyzing a data stream of the target accelerator;

[0013] The compression hyperparameter combination is determined according to the quantization parameter and the sparsity parameter.

[0014] According to a neural network compression method provided by the present invention, analyzing the data stream of the target accelerator to determine the quantization parameter for compressing the neural network includes:

[0015] Determining a target quantization method for compressing the neural network based on an array of processing engines (PEs) in the data stream;

[0016] Determining a quantization bit width of the target quantization mode based on an access bandwidth and an access mode of a storage unit in the data stream;

[0017] The quantization parameter is determined according to the target quantization mode and the quantization bit width.

[0018] According to a neural network compression method provided by the present invention, analyzing the data stream of the target accelerator to determine the sparse parameters for compressing the neural network includes:

[0019] Determining a target sparsity method for compressing the neural network based on an interaction between a target storage and a PE array in the data stream;

[0020] Determining a sparse size of the target sparse mode based on the PE array parallelism in the data stream;

[0021] The sparsity parameter is determined according to the target quantization mode and the sparsity size.

[0022] According to a neural network compression method provided by the present invention, the target quantization method includes at least any one of the following methods:

[0023] Symmetrical quantization method, asymmetric quantization method and zero offset method.

[0024] According to a neural network compression method provided by the present invention, the target sparse mode includes at least any one of the following modes:

[0025] Regular sparse mode and irregular sparse mode.

[0026] According to a neural network compression method provided by the present invention, the preset conditions include:

[0027] The updated reward value reaches the preset reward value threshold; or

[0028] The number of updates reaches the preset update threshold.

[0029] The present invention also provides a neural network compression device, comprising: a parameter determination module, a compression reward module, a parameter update module, and a network compression module;

[0030] The parameter determination module is used to use the neural network loaded in the target accelerator as an environment to determine a compression hyperparameter combination;

[0031] The compression reward module is configured to use the compression hyperparameter combination as an action to compress the neural network to determine a reward value after compressing the neural network;

[0032] The parameter updating module is used to update the environment, the compressed hyperparameter combination, the action and the reward value according to the compressed neural network, and stop updating when a preset condition is met;

[0033] The network compression module is used to compress the neural network according to the compression hyperparameter combination after the last update.

[0034] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the above-described neural network compression methods are implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described neural network compression methods.

[0036] The neural network compression method and device provided by the present invention analyze the accelerator to obtain a compression hyperparameter combination suitable for the acceleration architecture, thereby more effectively reducing the computational overhead of the neural network on the accelerator than traditional non-adaptive compression and improving the energy efficiency of the neural network's inference process on the accelerator. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 It is a flowchart of the neural network compression method provided by the present invention;

[0039] Figure 2 It is a structural schematic diagram of the neural network compression device provided by the present invention;

[0040] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0041] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0042] To implement applications such as image recognition and face recognition on edge devices, a common solution is to use an application-specific integrated circuit (ASIC) neural network accelerator to perform the inference phase of the neural network on the edge device. However, most existing neural networks have high computational complexity. Therefore, network compression technology (pruning and quantization) is also needed to reduce the amount of computation and power consumption of the neural network on the accelerator. Through model compression, the accelerator can ignore redundant calculations in the network when executing the network inference process and allows the accelerator to replace high-power floating-point operators with low-power fixed-point operators, thereby improving the energy efficiency of the network inference process on the accelerator.

[0043] First, for different accelerator architectures, the most suitable compression scheme for the architecture is obtained through the architecture analysis module, so that the compressed model can play a more effective role on the target accelerator. In comparison, the compressed model obtained by the traditional compression method has problems such as the pruning pattern is too large or too small, the pruning pattern is irregularly distributed, and the quantization bit width is not compatible. Secondly, the neural network compression algorithm provided by the present invention can adjust the compression hyperparameters at different positions in the neural network more accurately and finely according to the architectural characteristics. For example, through the reinforcement learning module, we can adjust the network compression rate and quantization bit width between different layers of the network and even different convolution blocks. Thereby, the network calculation complexity can be more fully reduced under the premise of maintaining higher network detection accuracy (such as the accuracy of face detection). The specific implementation is as follows:

[0044] Figure 1 It is a flowchart of the neural network compression method provided by the present invention, such as Figure 1 As shown, the method includes:

[0045] S1 uses the neural network loaded in the target accelerator as an environment to determine the compression hyperparameter combination;

[0046] S2. Compress the neural network using the compression hyperparameter combination as an action to determine the reward value after compressing the neural network;

[0047] S3. Update the environment, compressed hyperparameter combination, action, and reward value based on the compressed neural network, and stop updating when the preset conditions are met;

[0048] S4. Compress the neural network according to the last updated compression hyperparameter combination.

[0049] It should be noted that the execution subject of the above method may be a computer device.

[0050] Optionally, for the neural network loaded into the target accelerator, specifically a full-precision floating-point model (the neural network model before compression), the reinforcement learning agent iteratively searches for the optimal compression scheme. Specifically, the neural network serves as the reinforcement learning environment, and at each iteration, the agent generates a set of quantization hyperparameters as the compression hyperparameter combination for compressing the neural network. The compression hyperparameter combination can include the sparsity pattern, sparsity, and quantization bit width of each layer.

[0051] Compression hyperparameter combinations are used as actions, which are applied to the neural network and generate corresponding rewards. Generally speaking, rewards consist of two main components: the relative accuracy loss after the neural network model is compressed, and optimization metrics such as target accelerator energy, energy efficiency, and floating-point operations per second (FLOPS).

[0052] For example, accelerator energy is used as an optimization metric. In order to improve the algorithm's running speed, rewards are obtained by quickly evaluating and establishing energy models in the specific implementation. Specifically, based on the actions generated by the intelligent agent, the neural network will be compressed and briefly fine-tuned with Finetune. The main purpose of fine-tuning is that the model weights have changed significantly due to model compression, but network parameters such as the mean and variance in the batch normalization layer remain at their original values. This results in the accuracy of the neural network model in the compressed accelerator being disproportionate to the final model accuracy. Finetune is required to adjust parameters (such as the weights of the neural network in the target accelerator) to adapt to the changes brought about by model compression.

[0053] As for the energy model, the main focus is on reducing the huge power consumption overhead of the actual hardware test model. The core idea is to analyze the power consumption overhead of each key module of the hardware platform through DC simulation and other means, and combine the data stream obtained by analyzing the target accelerator to establish a relationship model between hardware power consumption and compression hyperparameter combinations such as compression ratio and quantization bit width, so that hardware power consumption can be quickly evaluated in actual iterations.

[0054] According to the compressed neural network, the environment is updated, and the compressed hyperparameter combination is updated according to the updated environment. The updated compressed hyperparameter combination is used as the updated action, and the neural network is compressed again to obtain the reward value Reward obtained after the compression of the neural network. Through the feedback of the reward, the reinforcement learning agent adjusts and optimizes the generated compressed hyperparameter combination. The above process is repeated until the preset conditions are met, and the updating of the environment, compressed hyperparameters, actions and reward values ​​is stopped. The above neural network is compressed according to the compressed hyperparameter combination updated after the last iteration.

[0055] The optimal compression hyperparameter combination will be selected, and based on the selected compression hyperparameter combination (compression scheme), the above neural network will be actually compressed and trained for a sufficient time to restore the accuracy of the compressed model as much as possible.

[0056] The neural network compression method provided by the present invention analyzes the accelerator to obtain a compression hyperparameter combination suitable for the acceleration architecture, thereby more effectively reducing the computational overhead of the neural network on the accelerator than traditional non-adaptive compression and improving the energy efficiency of the neural network's inference process on the accelerator.

[0057] Furthermore, in one embodiment, step S1 may specifically include:

[0058] S11, using the neural network loaded in the target accelerator as an environment, and determining the quantization parameters and sparsity parameters of the compressed neural network by analyzing the data stream of the target accelerator;

[0059] S12. Determine a compression hyperparameter combination based on the quantization parameter and the sparsity parameter.

[0060] Optionally, the data flow of the target accelerator is analyzed, for example, by an architecture analysis module, where the data flow may include on-chip (target accelerator) and off-chip storage interactions, on-chip storage interactions with the PE computing array, and PE array computing processes. This allows the key constraints of the target accelerator in terms of key hyperparameters such as sparsity pattern (i.e., the size and shape of the minimum zero-value neuron module), sparsity ratio, and quantization bit width to be summarized. These key constraints can effectively constrain the search space for compressed neural networks, thereby converging to a compression scheme that is more suitable for the neural network in the target accelerator.

[0061] It should be noted that the above key constraints can be specifically: the allowable quantization bit width range, the allowable sparsity range and the allowable sparse pattern (i.e., sparse size). For example, some accelerators only support a C×K×K size data block along the N dimension to be 0 or non-zero at the same time for an N×C×K×K weight matrix, while others allow finer granularity such as 8 consecutive elements. These key constraints are used to determine the quantization parameters and sparsity parameters for compressing the above neural network.

[0062] According to the obtained quantization parameters and sparsity parameters, a compression hyperparameter combination for compressing the above neural network can be obtained.

[0063] The neural network compression method provided by the present invention determines the quantization parameters and sparsity parameters of the compressed neural network by analyzing the data stream of the accelerator. When facing different accelerators, the quantization parameters and sparsity parameters suitable for the neural networks loaded in different accelerators can be obtained through specific analysis of the data streams of different accelerators. The neural networks loaded in different accelerators can be compressed in a targeted manner, thereby obtaining the optimal compression scheme for different accelerators.

[0064] Furthermore, in one embodiment, step S11 may specifically include:

[0065] S111, determining a target quantization method used by the compression neural network based on the processing engine PE array in the data stream;

[0066] S112, determining a quantization bit width of a target quantization method based on an access bandwidth and an access method of a storage unit in a data stream;

[0067] S113: Determine a quantization parameter according to a target quantization mode and a quantization bit width.

[0068] Furthermore, in one embodiment, step S11 may further specifically include:

[0069] S114, determining a target sparsity method used by the compression neural network based on the interaction between the target storage and the PE array in the data stream;

[0070] S115, determining the sparse size of the target sparse mode based on the PE array parallelism in the data stream;

[0071] S116: Determine the sparsity parameters according to the target quantization mode and sparsity size.

[0072] Optionally, for neural network model quantization, a target quantization method for compressing the neural network is selected based on the processing engine PE array, such as whether to use a symmetric or asymmetric quantization method, or whether to use a zero offset quantization method. Furthermore, the quantization bit width of the target quantization method is determined based on the access bandwidth and access method of the storage unit in the target accelerator, such as determining a range of quantization bit width values, and obtaining a final quantization bit width based on the range of quantization bit width values. Finally, the quantization parameters of the compressed neural network are obtained based on the target quantization method and quantization bit width.

[0073] For neural network model sparsity, the target sparsity method used in the compressed neural network, such as regular sparsity or irregular sparsity, is determined based on the interaction between the target storage and the PE array. Specifically, the target storage can include on-chip storage (typically implemented with dynamic random access memory (SRAM)) and off-chip storage (typically implemented with static random access memory (DRAM). The interaction between PE arrays can include interaction between PE and on-chip storage, and interaction between on-chip storage and off-chip storage.

[0074] The sparse size of the target sparsity mode is determined based on the PE array parallelism in the data stream (specifically, how many operations the PE can perform in one cycle), and the sparse parameters are obtained according to the obtained target sparsity mode and sparse size.

[0075] The neural network compression method provided by the present invention obtains the quantization parameters and sparsity parameters of the compressed neural network by analyzing the data stream of the accelerator, laying the foundation for the subsequent determination of the compression hyperparameter combination. At the same time, by analyzing the data streams of different accelerators, the corresponding quantization parameters and sparsity parameters can be adaptively adjusted, thereby fully reducing the network calculation complexity while maintaining higher network detection accuracy.

[0076] Furthermore, in one embodiment, the target quantization method in step S111 includes at least any one of the following methods:

[0077] Symmetrical quantization method, asymmetric quantization method and zero offset method.

[0078] Optionally, the target quantization method used to compress the above neural network can be any one of the following methods: a symmetric quantization method, an asymmetric quantization method, and a zero-point offset method.

[0079] Neural network model quantization is a problem of memory capacity and memory bandwidth in the neural network inference process. It is a model compression technology that converts the floating-point type of neural network parameters into integer storage.

[0080] For asymmetric quantization:

[0081] The quantization formula from floating point to fixed point is as follows:

[0082]

[0083] Among them, R represents the real floating-point value, Q represents the quantized fixed-point value, Z represents the quantized fixed-point value corresponding to the floating-point value of 0, and S represents the minimum scale that can be represented after fixed-point quantization. The calculation formulas of S and Z are as follows:

[0084]

[0085] Among them, R max Represents the maximum floating point value, R min Represents the smallest floating point value, Q max Represents the maximum fixed-point value, Q min Represents the smallest fixed-point value.

[0086] Z=Q max -R max ÷S

[0087] The dequantization method from fixed point to floating point is as follows:

[0088] R=(QZ)×S

[0089] For example, when quantizing int8, the data range is [-128, 127], and the maximum and minimum values ​​are Xmax ,X min , X q Represents the quantified data, X f Represents floating-point data.

[0090] X q =X f / scale+zero

[0091] scale=(X max -X min ) / (127-(-128))

[0092] zero=0-round(X min / scale)

[0093] For symmetric quantization, it is only necessary to change the range of the original floating point number from [X min , X max ] Expand to [-X max , X max ], and then convert it according to the above quantization formula.

[0094] The zero-point offset method can be used to map the floating-point 0 to a number between 0 and 255 after quantization, making the quantized number more accurate.

[0095] The neural network compression method provided by the present invention adopts a symmetric quantization method, an asymmetric quantization method and a zero-point offset method, which can exchange an acceptable loss of accuracy for less storage, bandwidth requirements, computing speed and lower energy consumption.

[0096] Furthermore, in one embodiment, the target sparse mode in step S114 includes at least any one of the following modes:

[0097] Regular sparse mode and irregular sparse mode.

[0098] Alternatively, the sparsity of the neural network can be achieved using either regular or irregular sparsity. Irregular sparsity refers to a smaller sparsity size, while regular sparsity refers to a larger sparsity size. For example, a weight matrix of size N×C×K×K only supports a C×K×K block of data along the N dimension that is either 0 or non-0 at the same time. Some allow finer granularity, such as 8 consecutive elements that are either 0 or non-0 at the same time. In this case, a sparsity processing method with less than or equal to 8 elements that are either 0 or non-0 at the same time is an irregular sparsity method, and vice versa, it is a regular sparsity method.

[0099] It should be noted that the number of the above-mentioned specific elements can be freely set according to the dimension of the weight matrix of the neural network to be sparse, and the present invention does not impose any specific limitation on this.

[0100] The neural network compression method provided by the present invention can realize automatic feature selection of the neural network reasoning process by performing sparse processing on the neural network, thereby improving the accuracy of the reasoning process.

[0101] Furthermore, in one embodiment, the preset conditions in step S3 may specifically include:

[0102] The updated reward value reaches the preset reward value threshold; or

[0103] The number of updates reaches the preset update threshold.

[0104] Optionally, the intelligent agent updates the compression hyperparameter combination according to a preset update number threshold. When the update number reaches the preset update number threshold, the updating of the compression hyperparameter combination is stopped, and the above-mentioned neural network is compressed based on the compression hyperparameter combination after the last update.

[0105] Alternatively, based on the relationship between the reward value generated by the compression hyperparameter combination acting on the neural network during the update process and the preset reward value threshold, if the reward value of the updated compression hyperparameter combination acting on the neural network is greater than or equal to the preset reward value threshold, it is considered that the compression convergence condition is met, and the above-mentioned neural network is compressed based on the compression hyperparameter combination after the last update.

[0106] In practical applications, the proposed neural network compression algorithm has been validated for image classification through algorithm design and actual chip testing. This validation was conducted on the STICKER sparse neural network accelerator chip manufactured using TSMC's 65nm process. For VGG16 and ResNet18 networks on the Cifar10 and Cifar100 datasets, the algorithm achieved a 10-fold reduction in energy consumption with no loss of accuracy. Furthermore, compared to existing non-hardware-adaptive independent sparsification and quantization methods, the algorithm achieved a 25% reduction in energy consumption at the same compression ratio.

[0107] The neural network compression method provided by the present invention can enable the compression hyperparameter combination to converge quickly. If accelerator energy consumption is used as an optimization indicator and a reward value is obtained, the accelerator energy consumption loss can be quickly evaluated when the compression hyperparameter combination is actually updated.

[0108] The neural network compression device provided by the present invention is described below. The neural network compression device described below and the neural network compression method described above can be referenced to each other.

[0109] Figure 2 is a schematic diagram of the structure of the neural network compression device provided by the present invention, such as Figure 2As shown, it includes: a parameter determination module 210, a compression reward module 211, a parameter update module 212 and a network compression module 213;

[0110] a parameter determination module 210 for determining a compression hyperparameter combination using the neural network loaded in the target accelerator as an environment;

[0111] A compression reward module 211 is configured to compress the neural network using a combination of compression hyperparameters as an action to determine a reward value after the neural network is compressed;

[0112] A parameter updating module 212 is configured to update the environment, compressed hyperparameter combinations, actions, and reward values ​​based on the compressed neural network, and stop updating when a preset condition is met;

[0113] The network compression module 213 is used to compress the neural network according to the last updated compression hyperparameter combination.

[0114] The neural network compression device provided by the present invention analyzes the accelerator to obtain a compression hyperparameter combination suitable for the acceleration architecture, thereby more effectively reducing the computational overhead of the neural network on the accelerator than traditional non-adaptive compression and improving the energy efficiency of the neural network's inference process on the accelerator.

[0115] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 311, a memory 312, and a bus 313, wherein the processor 310, the communication interface 311, and the memory 312 communicate with each other via the bus 313. The processor 310 may call the logic instructions in the memory 312 to execute the following method:

[0116] Use the neural network loaded into the target accelerator as an environment to determine the compression hyperparameter combination;

[0117] Compress the neural network using the compression hyperparameter combination as an action to determine the reward value after the compression of the neural network;

[0118] According to the compressed neural network, the environment, compressed hyperparameter combination, action and reward value are updated, and the update is stopped when the preset conditions are met;

[0119] Compress the neural network based on the last updated compression hyperparameter combination.

[0120] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer power screen (which can be a personal computer, server, or network power screen, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0121] Furthermore, the present invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the neural network compression method provided by the above-mentioned method embodiments, for example, including:

[0122] Use the neural network loaded into the target accelerator as an environment to determine the compression hyperparameter combination;

[0123] Compress the neural network using the compression hyperparameter combination as an action to determine the reward value after the compression of the neural network;

[0124] According to the compressed neural network, the environment, compressed hyperparameter combination, action and reward value are updated, and the update is stopped when the preset conditions are met;

[0125] Compress the neural network based on the last updated compression hyperparameter combination.

[0126] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the neural network compression method provided in each of the above embodiments is implemented, for example, including:

[0127] Use the neural network loaded into the target accelerator as an environment to determine the compression hyperparameter combination;

[0128] Compress the neural network using the compression hyperparameter combination as an action to determine the reward value after the compression of the neural network;

[0129] According to the compressed neural network, the environment, compressed hyperparameter combination, action and reward value are updated, and the update is stopped when the preset conditions are met;

[0130] Compress the neural network based on the last updated compression hyperparameter combination.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer power screen (which can be a personal computer, a server, or a network power screen, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A neural network compression method, characterized in that: include: Use the neural network loaded into the target accelerator as an environment to determine the compression hyperparameter combination; Compressing the neural network using the compression hyperparameter combination as an action to determine a reward value after compressing the neural network; updating the environment, the compressed hyperparameter combination, the action, and the reward value according to the compressed neural network, and stopping the updating when a preset condition is met; Compressing the neural network according to the last updated compression hyperparameter combination; The neural network loaded in the target accelerator is used as an environment to determine the compression hyperparameter combination, including: Taking a neural network loaded in a target accelerator as an environment, determining a quantization parameter and a sparsity parameter for compressing the neural network by analyzing a data stream of the target accelerator; Determining the compression hyperparameter combination according to the quantization parameter and the sparsity parameter; The analyzing the data stream of the target accelerator to determine the quantization parameter for compressing the neural network includes: Determining a target quantization method for compressing the neural network based on an array of processing engines (PEs) in the data stream; Determining a quantization bit width of the target quantization mode based on an access bandwidth and an access mode of a storage unit in the data stream; Determining the quantization parameter according to the target quantization mode and the quantization bit width; The analyzing the data stream of the target accelerator to determine the sparse parameters for compressing the neural network includes: Determining a target sparsity method for compressing the neural network based on an interaction between a target storage and a PE array in the data stream; Determining a sparse size of the target sparse mode based on the PE array parallelism in the data stream; The sparsity parameter is determined according to the target quantization mode and the sparsity size.

2. The neural network compression method according to claim 1, characterized in that: The target quantification method includes at least one of the following methods: Symmetrical quantization method, asymmetric quantization method and zero offset method.

3. The neural network compression method according to claim 1, characterized in that The target sparse mode includes at least one of the following modes: Regular sparse mode and irregular sparse mode.

4. The neural network compression method according to claim 1, characterized in that The preset conditions include: The updated reward value reaches the preset reward value threshold; or The number of updates reaches the preset update threshold.

5. A neural network compression device, characterized in that: include: Parameter determination module, compression reward module, parameter update module and network compression module; The parameter determination module is used to use the neural network loaded in the target accelerator as an environment to determine a compression hyperparameter combination; The compression reward module is configured to use the compression hyperparameter combination as an action to compress the neural network to determine a reward value after compressing the neural network; The parameter updating module is used to update the environment, the compressed hyperparameter combination, the action and the reward value according to the compressed neural network, and stop updating when a preset condition is met; The network compression module is used to compress the neural network according to the last updated compression hyperparameter combination; The neural network loaded in the target accelerator is used as an environment to determine the compression hyperparameter combination, including: Taking a neural network loaded in a target accelerator as an environment, determining a quantization parameter and a sparsity parameter for compressing the neural network by analyzing a data stream of the target accelerator; Determining the compression hyperparameter combination according to the quantization parameter and the sparsity parameter; The analyzing the data stream of the target accelerator to determine the quantization parameter for compressing the neural network includes: Determining a target quantization method for compressing the neural network based on an array of processing engines (PEs) in the data stream; Determining a quantization bit width of the target quantization mode based on an access bandwidth and an access mode of a storage unit in the data stream; Determining the quantization parameter according to the target quantization mode and the quantization bit width; The analyzing the data stream of the target accelerator to determine the sparse parameters for compressing the neural network includes: Determining a target sparsity method for compressing the neural network based on an interaction between a target storage and a PE array in the data stream; Determining a sparse size of the target sparse mode based on the PE array parallelism in the data stream; The sparsity parameter is determined according to the target quantization mode and the sparsity size.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the neural network compression method according to any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the neural network compression method according to any one of claims 1 to 4 are implemented.

Images