Neural network quantification method, apparatus and device, and readable storage medium
By dynamically adjusting the quantization accuracy and optimizing the configuration in the neural network accelerator, the performance loss problem caused by single low-precision quantization is solved, and efficient computing and performance maintenance of the neural network are achieved.
Patent Information
- Application Number
- CN202510911946.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-02
- Publication Date
- 2025-10-10
Smart Images
Figure CN120764600A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network technology, and in particular to a neural network quantization method, device, equipment and readable storage medium. Background Art
[0002] Quantization technology significantly reduces computational complexity and storage requirements by reducing neural network parameters from high-precision floating-point numbers to low-precision ones. However, single low-precision quantization can lead to performance loss, especially in deep neural networks or when features have a large dynamic range.
[0003] It can be seen that how to improve the computational efficiency of neural networks while maintaining their performance is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a neural network quantization method, device, equipment and readable storage medium to solve the technical problem of low computational efficiency of neural networks in the prior art.
[0005] To solve the above technical problems, the present invention provides a neural network quantization method, comprising:
[0006] Obtaining a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantized computing;
[0007] Determine the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of different layers of the target neural network;
[0008] Determining a quantization accuracy range for each layer based on the quantization sensitivity, and determining a configuration of a target neural network based on the quantization accuracy range and the neural network accelerator, to obtain a plurality of quantization configurations, each quantization configuration having a different quantization calculation combination determined based on the reconfigurable computing unit; the plurality of quantization configurations being a quantization configuration having a number greater than one;
[0009] The multiple quantization configurations are used as input, the performance of the target neural network and the performance of the neural network accelerator are set as targets, and optimization is performed among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
[0010] On the one hand, the neural network accelerator includes the reconfigurable computing unit and the nonlinear computing unit, the reconfigurable computing unit realizes the accelerated calculation of the linear part of the target neural network, and the nonlinear computing unit includes an activation function acceleration unit and a normalization acceleration unit.
[0011] On the one hand, the neural network accelerator includes a first set number of first reconfigurable computing units and a second set number of second reconfigurable computing units; wherein, the first set number is smaller than the second set number, the quantization calculation supported by the first reconfigurable computing unit is within the first quantization number, the quantization calculation supported by the second reconfigurable computing unit is within the second quantization number, and the first quantization number is greater than the second quantization number; the quantization calculation types supported by the first reconfigurable computing unit more than those supported by the second reconfigurable computing unit are all high-precision quantization calculations, and the high-precision quantization calculations are quantization technologies with quantization accuracy greater than the minimum set quantization accuracy threshold.
[0012] On the one hand, the multiple quantization configurations are used as input, the performance of the target neural network and the performance of the neural network accelerator are set as targets, and the optimization is performed among the multiple quantization configurations to obtain the optimal quantization accuracy configuration, including:
[0013] Obtaining the plurality of quantization configurations, each quantization configuration specifying a quantization precision type for each layer of the neural network;
[0014] Measuring a neural network performance index and an accelerator performance index corresponding to each of the quantization configurations to obtain a measured neural network performance index and an accelerator performance index;
[0015] establishing a dominance relationship based on the measured neural network performance indicator and the accelerator performance indicator;
[0016] The dominance relationship is used to select a non-dominated configuration as a dominant quantization configuration to obtain a selected quantization configuration; wherein, for any two quantization configurations, if the neural network performance index of the first quantization configuration is not lower than that of the second quantization configuration, and the accelerator performance index of the first quantization configuration is better than that of the second quantization configuration, then the first quantization configuration dominates the second quantization configuration; the non-dominated configuration is a configuration that is not dominated by any other quantization configuration among all quantization configurations of the current generation;
[0017] performing a crossover operation on the screened quantized configurations to generate offspring configurations;
[0018] Perform a mutation operation on the child configuration, and when a termination condition is met, determine the optimal quantization accuracy configuration; wherein the termination condition includes that the performance of the current quantization configuration meets the performance of the set target neural network and the performance of the neural network accelerator.
[0019] On the one hand, a quantization accuracy range of each layer is determined based on the quantization sensitivity, and a configuration of a target neural network is determined based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, including:
[0020] Based on the quantization accuracy range and the neural network accelerator, a dynamic quantization method is used to determine the configuration of the target neural network to obtain the multiple quantization configurations; wherein, the dynamic quantization method is a method that can parallelize the quantization accuracy and / or merge the quantization accuracy of multiple sub-computing units in the reconfigurable computing unit.
[0021] On the one hand, after taking the multiple quantization configurations as input, setting the target neural network performance and the neural network accelerator performance as targets, and optimizing among the multiple quantization configurations to obtain the optimal quantization accuracy configuration, the method further includes:
[0022] Adjusting the target neural network based on the optimal quantization accuracy configuration to obtain an adjusted target neural network;
[0023] The adjusted target neural network is trained, and parameters are iteratively updated to obtain a quantized target neural network.
[0024] On the one hand, the method is applied in data center or edge computing scenarios.
[0025] An embodiment of the present invention further provides a neural network quantization device, comprising:
[0026] A neural network accelerator acquisition module, configured to acquire a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantized computing;
[0027] A quantization sensitivity determination module, configured to determine the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of the different layers of the target neural network;
[0028] a multiple quantization configuration determination module, configured to determine a quantization accuracy range for each layer based on the quantization sensitivity, and determine a configuration of a target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each quantization configuration having a different quantization calculation combination determined based on the reconfigurable computing unit; the multiple quantization configurations being a quantization configuration having a number greater than one;
[0029] The quantization module is used to take the multiple quantization configurations as input, set the performance of the target neural network and the performance of the neural network accelerator as targets, and perform optimization among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
[0030] An embodiment of the present invention further provides a neural network quantization device, including:
[0031] memory for storing computer programs;
[0032] A processor is used to execute the computer program to implement the steps of the above-mentioned neural network quantization method.
[0033] An embodiment of the present invention further provides a computer-readable storage medium (readable storage medium), on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned neural network quantization method are implemented.
[0034] An embodiment of the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned neural network quantization method when executed by a processor.
[0035] To solve the above technical problems, an embodiment of the present invention provides a neural network quantization method, which may include: obtaining a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, and each reconfigurable computing unit supports a different type of quantization calculation; based on the accuracy requirements of different layers of the target neural network, determining the quantization sensitivity of different layers of the target neural network; determining the quantization accuracy range of each layer based on the quantization sensitivity, and determining the configuration of the target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each quantization configuration having a different combination of quantization calculations determined based on the reconfigurable computing units; the multiple quantization configurations are quantization configurations with a number greater than 1; using the multiple quantization configurations as input, and setting the performance of the target neural network and the performance of the neural network accelerator as targets, and performing optimization among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
[0036] It can be seen from the above technical solution that the beneficial effect of the present invention is that the present invention can perform a specific analysis based on the quantization accuracy range of each layer and determine the quantization method corresponding to each layer, so that each layer can be quantized with a different quantization method while meeting the set performance goals, thereby achieving a balance between neural network performance and computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0038] Figure 1 A flowchart of a neural network quantization method provided by an embodiment of the present invention;
[0039] Figure 2 A schematic diagram of a CNN-Transformer accelerator provided by an embodiment of the present invention;
[0040] Figure 3 A schematic diagram of a reconfigurable computing unit provided in an embodiment of the present invention;
[0041] Figure 4 A flowchart illustrating a neural network quantization method according to an embodiment of the present invention;
[0042] Figure 5 A schematic diagram of the structural framework of a neural network quantization device provided by an embodiment of the present invention;
[0043] Figure 6 A schematic diagram of the structural framework of a neural network quantization device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0045] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.
[0046] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0047] Next, a neural network quantization method provided by an embodiment of the present invention is introduced in detail. Figure 1 A flowchart of a neural network quantization method provided in an embodiment of the present invention may include:
[0048] S101, obtaining a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, and each reconfigurable computing unit supports a different type of quantitative computing.
[0049] The executor of this embodiment is an electronic device, which may be a computer, a mobile phone, or the like. This embodiment can be applied to data centers or edge computing scenarios. The neural network accelerator in this embodiment is a device that includes a reconfigurable computing unit and constructs the quantization accuracy of each layer of the target neural network based on the reconfigurable computing unit. This embodiment does not limit the specific preset number, and can be set according to demand, or computing units including various types of quantization calculations can be constructed. The types of quantization calculations in this embodiment are different. For example, the reconfigurable computing unit 1 supports X types of quantization calculations (such as FP4 (four-bit floating point), FP8 (eight-bit floating point), and FP16 (half-precision floating point), and the reconfigurable computing unit 2 supports Y types of quantization calculations (such as FP4).
[0050] It should be further explained that, to enhance the comprehensiveness of the quantization-based acceleration types supported by the neural network accelerator, the aforementioned neural network accelerator includes a reconfigurable computation unit (CCU) and a nonlinear computation unit (NCU). The CCU accelerates the linear portion of the target neural network, while the NCU includes an activation function accelerator and a normalization accelerator. The activation function accelerator supports computations for functions such as Softmax (normalized exponential function), ReLU (rectified linear unit), and GELU (Gaussian error linear unit), and can be accelerated using approximate computation techniques such as lookup tables. By pre-calculating and storing a lookup table of commonly used nonlinear function outputs offline, complex function computations can be replaced with simple table indexing operations, enabling efficient and low-latency nonlinear computations with low hardware overhead. The normalization accelerator supports computations such as batch normalization (BN) and layer normalization (LN). On-chip memory includes an input buffer, a parameter buffer, and an output buffer, which respectively store the input, weight, and output parameters of the CNN-Transformer (convolutional neural network-transformer hybrid architecture) neural network. The input and weight parameters are fed into the computation array, while the output parameters are retrieved from the computation array. For easier understanding, please refer to Figure 2 , Figure 2 A schematic diagram of a CNN-Transformer accelerator provided for an embodiment of the present invention includes a computing array and on-chip storage. In the computing array, the accelerated computing of the linear part and the nonlinear part of the CNN-Transformer neural network are respectively implemented through a reconfigurable computing part and a nonlinear computing part. The reconfigurable computing part includes a variety of reconfigurable computing units, all of which can implement the calculation of two different types of neural networks, CNN and Transformer (such as the convolution layer of CNN, the self-attention layer of Transformer, etc.), and can implement mixed-precision quantization calculations. Multiple reconfigurable computing units perform parallel calculations, effectively improving computing efficiency.
[0051] It should be further explained that, in order to achieve the best match of hardware resources and maximize energy efficiency, the neural network accelerator includes a first set number of first reconfigurable computing units and a second set number of second reconfigurable computing units; wherein, the first set number is smaller than the second set number, the quantization calculation supported by the first reconfigurable computing unit is in the first quantization number, the quantization calculation supported by the second reconfigurable computing unit is in the second quantization number, and the first quantization number is larger than the second quantization number; the quantization calculation types supported by the first reconfigurable computing unit more than the second reconfigurable computing unit are all high-precision quantization calculations, and high-precision quantization calculations are quantization technologies with quantization accuracy greater than the minimum set quantization accuracy threshold. Figure 2 As shown in the figure, the difference between reconfigurable computing unit 1 and reconfigurable computing unit 2 is the type of quantization calculation they support. Reconfigurable computing unit 1 supports X types of quantization calculation (such as FP4, FP8, FP16), and reconfigurable computing unit 2 supports Y types of quantization calculation (such as FP4), where X>Y, and the quantization calculation types supported by reconfigurable computing unit 1 more than reconfigurable computing unit 2 are all high-precision. The number of reconfigurable computing units 1 and 2 is N and M respectively, where N <M。通过这样的设置可以提高量化配置的全面性,避免量化的重复,从而实现硬件资源的最优匹配与能效最大化。为便于理解,请参考 Figure 3 , Figure 3 A schematic diagram of a reconfigurable computing unit provided by an embodiment of the present invention, Figure 3 As can be seen, the reconfigurable computing unit includes a control unit and multiple computing units (FP4, FP8, FP16, BF16, INT4, and INT8). The reconfigurable computing unit is dynamically configured through the control unit. When executing different types of operations, the data path is reconfigured, thereby enabling flexible calculation of various types of operators in the CNN-Transformer neural network. The reconfigurable computing unit includes multiple computing units and uses dynamic quantization switching technology to implement various quantization types of calculations. In addition, the computing unit includes multiple sub-computational units and can be flexibly configured. For example, when executing INT4 quantization calculations, multiple sub-computational units can execute more operation instructions in parallel; when switching to INT8 quantization calculations, multiple sub-computational units are merged to achieve higher-precision calculations.
[0052] S102, determining the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of different layers of the target neural network.
[0053] The accuracy requirements for different layers in this embodiment are determined based on the data processing requirements of each layer. For example, the deep convolution layer of a CNN (convolutional neural network) extracts local features from the input feature map. If the quantization precision is too low, it can lead to significant performance degradation, thereby affecting the quality of subsequent feature extraction. Therefore, the quantization sensitivity is at a higher level (for example, greater than the set minimum quantization sensitivity is higher) and a higher quantization precision (such as INT8). The point-by-point convolution layer performs feature channel fusion and can try a lower quantization precision (such as INT4, which is less than the set minimum quantization sensitivity). The self-attention layer of the Transformer is used to capture global correlations between feature sequences and can consider choosing a higher quantization precision (such as INT8). The target neural network in this embodiment is a CNN-Transformer neural network, which is composed of a combination of CNN and Transformer. These two neural networks have significant architectural differences, and different types of neural network layers (such as deep convolution layers, point-by-point convolution layers, self-attention layers, and fully connected layers) have different sensitivities to quantization precision. A convolutional neural network (CNN) is a neural network model used for image data processing, primarily consisting of convolutional layers, pooling layers, and fully connected layers. CNNs reduce computational complexity, improve training efficiency, and enhance model generalization through local receptive fields and weight sharing. The convolutional layer, the fundamental building block of a CNN, consists of multiple convolution kernels. Each convolution kernel is typically a small weight matrix, such as 1×1, 3×3, or 5×5, responsible for extracting specific features from the input data. Convolution is performed by sliding the convolution kernel over the input data, gradually covering different regions of the input. At each sliding position, the kernel is element-wise multiplied with the input data in the covered area. All products are then summed to produce a single value, which serves as the value for the corresponding position in the output feature map. This process is repeated for all sliding positions to generate the complete output feature map. Pooling layers downsample the feature map, reducing spatial dimensionality, parameters, and computational complexity, thereby enhancing the robustness of CNNs to small transformations such as translation. Common pooling methods include max pooling and average pooling. Max pooling selects the maximum value within each pooling window, while average pooling calculates the average of all values within the pooling window. Fully connected layers are typically located at the end of a CNN and are used to map features extracted by the convolutional and pooling layers to the final output category space. The Transformer architecture includes self-attention layers, fully connected layers, layer normalization, and residual connections. In the self-attention layers, a multi-head self-attention mechanism is used to integrate contextual semantics, allowing the dependency between any two words in a sequence to be directly modeled without relying on traditional recurrent structures, thereby better addressing long-range dependencies in text.Fully connected layers are used to perform more complex transformations on the representation of each word in the input text sequence. Layer normalization is applied to the output representation sequences of the attention and fully connected layers, performing layer normalization operations on the representation sequences and stabilizing optimization. Residual connections are direct paths connecting the input and output of the attention or fully connected layers, facilitating model optimization.
[0054] S103, determining the quantization accuracy range of each layer based on the quantization sensitivity, and determining the configuration of the target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each of which has a different quantization calculation combination determined based on the reconfigurable computing unit; the multiple quantization configurations are quantization configurations whose number is greater than 1.
[0055] This embodiment can determine the quantization precision range (optional quantization configuration range) of each layer of the neural network based on the quantization sensitivity, thereby constructing multiple quantization configurations based on the quantization precision range of each layer and the quantization precision types supported by the neural network accelerator. The quantization precision range of each layer is set in combination with the quantization calculation type supported by the neural network accelerator, such as FP4, FP8, FP16, INT4, INT8, etc.
[0056] It should be further explained that, based on any of the above embodiments, the above-mentioned determination of the quantization accuracy range of each layer based on the quantization sensitivity, and determination of the configuration of the target neural network based on the quantization accuracy range and the neural network accelerator to obtain multiple quantization configurations may include: determining the configuration of the target neural network based on the quantization accuracy range and the neural network accelerator using a dynamic quantization method to obtain multiple quantization configurations; wherein the dynamic quantization method is a method that can parallelize the quantization accuracy and / or merge the quantization accuracy of multiple sub-computing units in the reconfigurable computing unit. This embodiment can merge the various quantization accuracies, so that when high precision is required, a higher precision quantization method can be used for quantization, and multiple quantization accuracy methods can be executed in parallel, which can improve the efficiency of quantization execution.
[0057] S104, taking multiple quantization configurations as input, setting the performance of the target neural network and the performance of the neural network accelerator as targets, and searching for the best among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
[0058] The embodiment does not limit the specific method of optimization. For example, the embodiment can be an intelligent algorithm such as a genetic algorithm, a particle swarm algorithm, a reinforcement learning algorithm, etc. When the optimization method is a particle swarm algorithm: multiple quantization configurations are taken as inputs, and the performance of the target neural network and the performance of the neural network accelerator are taken as optimization targets. A particle swarm is initialized, where each particle represents a quantization configuration and has a position and a speed; a fitness function is defined for evaluating the performance of the neural network and the performance of the neural network accelerator under the quantization configuration represented by the particle; in each iteration, the particle is guided to update the speed and the position according to the individual extremum of the particle and the global extremum of the particle swarm, so as to search for a better solution; the position and the speed of the particle are repeatedly updated, and the fitness is evaluated until a preset algorithm termination condition is met; finally, the required quantization configuration is obtained according to the global optimal particle. When the optimization algorithm is a reinforcement learning algorithm, multiple quantization configurations are taken as inputs, and the performance of the target neural network and the performance of the neural network accelerator are taken as optimization targets. Each quantization configuration is regarded as a state in the environment, and the reinforcement learning agent selects an action according to the current strategy to change the quantization configuration; the environment feeds back a reward value according to the performance of the neural network and the performance of the neural network accelerator under the selected quantization configuration; the agent constantly updates the strategy function or the value function according to the reward value; through multiple rounds of interaction, the agent gradually converges to the optimal strategy, and finally selects the quantization configuration with the maximum cumulative reward as the optimal solution.
[0059] It needs to be further explained that, in order to improve the efficiency of optimization, the above-mentioned taking multiple quantization configurations as inputs, taking the performance of the target neural network and the performance of the neural network accelerator as targets, and performing optimization among the multiple quantization configurations to obtain the optimal quantization precision configuration can include:
[0060] S1041, obtaining multiple quantization configurations, each of which specifies the quantization precision type of each layer of the neural network;
[0061] S1042, measuring the neural network performance index and the accelerator performance index corresponding to each quantization configuration to obtain the measured neural network performance index and the measured accelerator performance index;
[0062] S1043, establishing a dominance relationship based on the measured neural network performance index and the measured accelerator performance index;
[0063] S1044, screening non-dominated configurations as dominant quantization configurations by using the dominance relationship to obtain screened quantization configurations; wherein, for any two quantization configurations, if the neural network performance index of the first quantization configuration is not lower than that of the second quantization configuration, and the accelerator performance index of the first quantization configuration is better than that of the second quantization configuration, then the first quantization configuration dominates the second quantization configuration; the non-dominated configuration is a configuration that is not dominated by any other quantization configuration among all quantization configurations in the current generation;
[0064] S1045, performing a crossover operation on the screened quantized configuration to generate a child configuration;
[0065] S1045, performing a mutation operation on the offspring configuration, and determining the optimal quantization accuracy configuration when a termination condition is met; wherein the termination condition includes that the performance of the current quantization configuration meets the performance of the set target neural network and the performance of the neural network accelerator.
[0066] This embodiment provides a specific process for determining the optimal quantization accuracy configuration based on multiple quantization configurations. Since the model can be used without training in this process and the optimal solution is directly output through non-domination, the efficiency of optimization can be improved.
[0067] It should be further explained that, to improve the accuracy of the target neural network, after taking multiple quantization configurations as input, setting the performance of the target neural network and the performance of the neural network accelerator as targets, and optimizing among the multiple quantization configurations to obtain the optimal quantization precision configuration, the process may further include: adjusting the target neural network based on the optimal quantization precision configuration to obtain an adjusted target neural network; training the adjusted target neural network and iteratively updating parameters to obtain a quantized target neural network. After obtaining the optimal quantization precision configuration, this embodiment uses post-training fine-tuning to retrain the neural network parameters to further improve the performance of the CNN-Transformer neural network. During this process, the neural network iteratively updates parameters under the mixed-precision quantization configuration (optimal quantization precision configuration), thereby compensating for errors introduced by quantization and ensuring that the quantized neural network approaches the performance level of a full-precision neural network as closely as possible. For ease of understanding, this embodiment provides a data processing process based on the quantized target neural network, which may include: Parameter processing: quantizing the original neural network parameters (typically FP32) to the precision format required by the network (such as FP16, INT8, etc.). This step typically includes dynamic range calculation, determining the scale factor and zero point, and then performing quantization. Quantized Computation: For each layer, the appropriate compute unit is selected from the reconfigurable compute units based on its quantization precision type. For example, if a layer is configured to use INT8, convolution or matrix multiplication operations are performed using integer compute units; if another layer is configured to use FP4, FP4 compute units are used. Computation at each layer involves: a. Calculation using quantized weights and quantized inputs (i.e., activation outputs from the previous layer). b. Conversion between different precisions may be required during the computation (for example, if the output of the previous layer is FP8 and the weights of the current layer are INT8, conversion is required). c. Obtaining the computation results of the current layer (typically quantized values in integer or floating-point format). Inter-layer Precision Conversion (if necessary): If adjacent layers use different quantization formats, data conversion is required between layers. For example, if the previous layer uses INT8 and the next layer uses FP4, the INT8 outputs need to be converted to FP4 format for the next layer's input. Operations such as activation functions and pooling also need to be performed in quantized precision. For example, the ReLU activation function is implemented in integer or floating-point quantization formats. Dynamic precision adjustment: When the edge device is running, the precision of some layers may be dynamically adjusted based on current resources (such as battery power, temperature, etc.). For example, when the temperature is too high, the precision of some layers may be switched from FP16 to FP8 or FP4.
[0068] The neural network quantization method provided by an embodiment of the present invention may include: S101, obtaining a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantization calculation; S102, determining the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of different layers of the target neural network; S103, determining the quantization accuracy range of each layer based on the quantization sensitivity, and determining the configuration of the target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each quantization configuration having a different combination of quantization calculations determined based on the reconfigurable computing units; the multiple quantization configurations being quantization configurations with a number greater than one; S104, using the multiple quantization configurations as input, the performance of the target neural network and the performance of the neural network accelerator as targets, and optimizing among the multiple quantization configurations to obtain the optimal quantization accuracy configuration. Current quantization technology significantly reduces computational complexity and storage requirements by reducing neural network parameters from high-precision floating-point numbers to low-precision numbers. However, single low-precision quantization may lead to performance loss, especially in deep neural networks or in the case of large feature dynamic range. The present invention can perform a specific analysis based on the quantization precision range of each layer to determine the quantization method corresponding to each layer, so that each layer can be quantized with a different quantization method while meeting the set performance goals, thereby achieving a balance between neural network performance and computational efficiency.
[0069] In order to make the present invention easier to understand, please refer to Figure 4 , Figure 4 An example flow chart of a neural network quantization method provided in an embodiment of the present invention may specifically include:
[0070] S201. Design a neural network accelerator; the neural network accelerator includes a reconfigurable computing unit and a nonlinear computing unit;
[0071] S202: Determine multiple reconfigurable computing subunits included in the reconfigurable computing unit, and an activation function acceleration unit and a normalization acceleration unit in the nonlinear computing unit; the computing accuracy of each reconfigurable computing unit is different; each reconfigurable computing subunit includes a control unit and different types of computing units; the control unit is used to implement dynamic configuration based on the computing unit;
[0072] S203, determining the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of different layers of the target neural network;
[0073] S204, determining a quantization accuracy range based on the quantization sensitivity;
[0074] S205. Determine a configuration of the neural network based on the quantization accuracy range and the neural network accelerator, and obtain multiple quantization configurations, each of which has a different combination of computing units.
[0075] S206. Taking multiple quantization configurations as input, setting the performance of the target neural network and the performance of the neural network accelerator as targets, searching for the best among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
[0076] This paper proposes a CNN-Transformer neural network accelerator. By configuring the reconfigurable computing unit in the neural network accelerator, it can implement mixed-precision computation of the CNN-Transformer neural network. Combined with the quantization computation types supported by the neural network accelerator, it optimizes the mixed-precision quantization of the neural network. Through hardware and software collaborative design, it achieves efficient computation of the CNN-Transformer neural network while maintaining its high performance.
[0077] The following is an introduction to a neural network quantization device provided by an embodiment of the present invention. The neural network quantization device described below and the neural network quantization method described above can be referenced to each other.
[0078] Figure 5 A schematic diagram of the structural framework of a neural network quantization device provided in an embodiment of the present invention may include:
[0079] A neural network accelerator acquisition module 100 is used to acquire a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantized computing;
[0080] A quantization sensitivity determination module 200 is configured to determine the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of the different layers of the target neural network;
[0081] a multiple quantization configuration determination module 300, configured to determine a quantization accuracy range for each layer based on the quantization sensitivity, and determine a configuration of a target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each quantization configuration having a different quantization calculation combination determined based on the reconfigurable computing unit; the multiple quantization configurations being a quantization configuration having a number greater than one;
[0082] The quantization module 400 is used to take the multiple quantization configurations as input, set the performance of the target neural network and the performance of the neural network accelerator as targets, and perform optimization among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
[0083] Further, based on the above embodiment, the neural network accelerator includes the reconfigurable computing unit and the nonlinear computing unit, the reconfigurable computing unit realizes the accelerated calculation of the linear part of the target neural network, and the nonlinear computing unit includes an activation function acceleration unit and a normalization acceleration unit.
[0084] Further, based on any of the above embodiments, the neural network accelerator includes a first set number of first reconfigurable computing units and a second set number of second reconfigurable computing units; wherein, the first set number is smaller than the second set number, the quantization calculation supported by the first reconfigurable computing unit is within the first quantization number, the quantization calculation supported by the second reconfigurable computing unit is within the second quantization number, and the first quantization number is greater than the second quantization number; the quantization calculation types supported by the first reconfigurable computing unit more than those supported by the second reconfigurable computing unit are all high-precision quantization calculations, and the high-precision quantization calculations are quantization technologies with quantization accuracy greater than the minimum set quantization accuracy threshold.
[0085] Furthermore, based on any of the above embodiments, the multiple quantization configurations are used as input, the performance of the target neural network and the performance of the neural network accelerator are set as targets, and optimization is performed among the multiple quantization configurations to obtain the optimal quantization accuracy configuration, including:
[0086] a multiple quantization configuration determining unit, configured to obtain the multiple quantization configurations, each quantization configuration specifying a quantization precision type for each layer of the neural network;
[0087] A measured neural network performance index and accelerator performance index determination unit, configured to measure the neural network performance index and accelerator performance index corresponding to each of the quantization configurations to obtain the measured neural network performance index and accelerator performance index;
[0088] a dominance relationship determining unit, configured to establish a dominance relationship based on the measured neural network performance index and accelerator performance index;
[0089] a screened quantization configuration determining unit, configured to use the dominance relationship to screen a non-dominated configuration as a dominant quantization configuration, thereby obtaining a screened quantization configuration; wherein, for any two quantization configurations, if the neural network performance index of a first quantization configuration is not lower than that of a second quantization configuration, and the accelerator performance index of the first quantization configuration is better than that of the second quantization configuration, then the first quantization configuration dominates the second quantization configuration; and the non-dominated configuration is a configuration that is not dominated by any other quantization configuration among all quantization configurations of the current generation;
[0090] a child configuration generating unit, configured to perform a crossover operation on the screened quantized configurations to generate child configurations;
[0091] An optimal quantization precision configuration determination unit is used to perform a mutation operation on the offspring configuration and determine the optimal quantization precision configuration when a termination condition is met; wherein the termination condition includes that the performance of the current quantization configuration meets the performance of the set target neural network and the performance of the neural network accelerator.
[0092] Further, based on any of the above embodiments, the multiple quantization configuration determination module 300 may include:
[0093] A multiple quantization configuration determination unit is used to determine the configuration of the target neural network using a dynamic quantization method based on the quantization accuracy range and the neural network accelerator to obtain the multiple quantization configurations; wherein the dynamic quantization method is a method that can parallelize quantization accuracy and / or merge quantization accuracy for multiple sub-computing units in the reconfigurable computing unit.
[0094] Furthermore, based on any of the above embodiments, the neural network quantization device may further include:
[0095] An adjusted target neural network determination module is configured to adjust the target neural network based on the optimal quantization accuracy to obtain an adjusted target neural network;
[0096] The iterative update module is used to train the adjusted target neural network and iteratively update parameters to obtain a quantized target neural network.
[0097] Furthermore, based on any of the above embodiments, the above neural network quantization device is applied to a data center or edge computing scenario.
[0098] It should be noted that the order of the modules and units in the above-mentioned neural network quantization device can be changed without affecting the logic.
[0099] Figure 5 The description of the features in the corresponding embodiment can be found in Figure 5 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.
[0100] The neural network quantization device provided by an embodiment of the present invention may include: a neural network accelerator acquisition module 100 for acquiring a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantization calculation; a quantization sensitivity determination module 200 for determining the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of different layers of the target neural network; a multiple quantization configuration determination module 300 for determining the quantization accuracy range of each layer based on the quantization sensitivity, and determining the configuration of the target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each quantization configuration having a different quantization calculation combination determined based on the reconfigurable computing unit; the multiple quantization configurations being quantization configurations with a number greater than 1; and a quantization module 400 for taking the multiple quantization configurations as input, the performance of the set target neural network and the performance of the neural network accelerator as targets, and performing optimization among the multiple quantization configurations to obtain the optimal quantization accuracy configuration. It can be seen that the present invention can perform a specific analysis based on the quantization accuracy range of each layer to determine the quantization method corresponding to each layer, so that each layer can be quantized using different quantization methods while meeting the set performance target, thereby achieving a balance between neural network performance and computational efficiency.
[0101] A neural network quantization device provided by an embodiment of the present invention is introduced below. The neural network quantization device described below and the neural network quantization method described above can be referenced to each other.
[0102] Figure 6 A schematic diagram of the structural framework of a neural network quantization device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the neural network quantization device includes: a memory 60 for storing a computer program;
[0103] The processor 61 is configured to implement the steps of the neural network quantization method in the above embodiment when executing a computer program.
[0104] The neural network quantization device provided in this embodiment may include but is not limited to a smartphone, a tablet computer, a laptop computer, or a desktop computer.
[0105] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing content required to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor for handling computational operations related to machine learning.
[0106] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the neural network quantization method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include but is not limited to data required for the neural network quantization method, etc.
[0107] In some embodiments, the neural network quantization device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .
[0108] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation to the neural network quantization device, and may include more or fewer components than shown in the figure.
[0109] It can be understood that if the neural network quantization method in the above embodiments is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and performs all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic disk or an optical disk, and various media that can store program codes.
[0110] Based on this, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize the steps of the neural network quantization method.
[0111] The above describes a neural network quantization method, device, equipment and readable storage medium provided by the present application in detail. The principles and implementation manners of the present application are described by applying specific examples in this paper. The above embodiment is only used to help understand the method and core idea of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, the present application can be improved and modified in several ways. These improvements and modifications also fall within the scope of protection of the present application.
[0112] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this paper can be realized by electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
Claims
1. A neural network quantization method, characterized in that: include; Obtaining a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantized computing; Determine the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of different layers of the target neural network; Determining a quantization accuracy range for each layer based on the quantization sensitivity, and determining a configuration of a target neural network based on the quantization accuracy range and the neural network accelerator, to obtain a plurality of quantization configurations, each quantization configuration having a different quantization calculation combination determined based on the reconfigurable computing unit; the plurality of quantization configurations being a quantization configuration having a number greater than one; The multiple quantization configurations are used as input, the performance of the target neural network and the performance of the neural network accelerator are set as targets, and optimization is performed among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
2. The neural network quantization method according to claim 1, characterized in that The neural network accelerator includes the reconfigurable computing unit and the nonlinear computing unit. The reconfigurable computing unit realizes the accelerated calculation of the linear part of the target neural network. The nonlinear computing unit includes an activation function acceleration unit and a normalization acceleration unit.
3. The neural network quantization method according to claim 1, characterized in that The neural network accelerator includes a first set number of first reconfigurable computing units and a second set number of second reconfigurable computing units; wherein, the first set number is smaller than the second set number, the quantization calculation supported by the first reconfigurable computing unit is within the first quantization number, the quantization calculation supported by the second reconfigurable computing unit is within the second quantization number, and the first quantization number is greater than the second quantization number; the quantization calculation types supported by the first reconfigurable computing unit more than those supported by the second reconfigurable computing unit are all high-precision quantization calculations, and the high-precision quantization calculations are quantization technologies with quantization accuracy greater than a minimum set quantization accuracy threshold.
4. The neural network quantization method according to any one of claims 1 to 3, characterized in that: Taking the multiple quantization configurations as input, the performance of the target neural network and the performance of the neural network accelerator as targets, optimizing the multiple quantization configurations to obtain the optimal quantization accuracy configuration, including: Obtaining the plurality of quantization configurations, each quantization configuration specifying a quantization precision type for each layer of the neural network; Measuring a neural network performance index and an accelerator performance index corresponding to each of the quantization configurations to obtain a measured neural network performance index and an accelerator performance index; establishing a dominance relationship based on the measured neural network performance indicator and the accelerator performance indicator; The dominance relationship is used to select a non-dominated configuration as a dominant quantization configuration to obtain a selected quantization configuration; wherein, for any two quantization configurations, if the neural network performance index of the first quantization configuration is not lower than that of the second quantization configuration, and the accelerator performance index of the first quantization configuration is better than that of the second quantization configuration, then the first quantization configuration dominates the second quantization configuration; the non-dominated configuration is a configuration that is not dominated by any other quantization configuration among all quantization configurations of the current generation; performing a crossover operation on the screened quantized configurations to generate offspring configurations; Perform a mutation operation on the child configuration, and when a termination condition is met, determine the optimal quantization accuracy configuration; wherein the termination condition includes that the performance of the current quantization configuration meets the performance of the set target neural network and the performance of the neural network accelerator.
5. The neural network quantization method according to claim 1, characterized in that: A quantization accuracy range of each layer is determined based on the quantization sensitivity, and a configuration of a target neural network is determined based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, including: Based on the quantization accuracy range and the neural network accelerator, a dynamic quantization method is used to determine the configuration of the target neural network to obtain the multiple quantization configurations; wherein, the dynamic quantization method is a method that can parallelize the quantization accuracy and / or merge the quantization accuracy of multiple sub-computing units in the reconfigurable computing unit.
6. The neural network quantization method according to claim 1, characterized in that: After taking the multiple quantization configurations as input, setting the target neural network performance and the neural network accelerator performance as targets, and optimizing the multiple quantization configurations to obtain the optimal quantization accuracy configuration, the method further includes: Adjusting the target neural network based on the optimal quantization accuracy configuration to obtain an adjusted target neural network; The adjusted target neural network is trained, and parameters are iteratively updated to obtain a quantized target neural network.
7. The neural network quantization method according to claim 1, characterized in that: The method is applied in data center or edge computing scenarios.
8. A neural network quantization device, characterized in that: include; A neural network accelerator acquisition module, configured to acquire a neural network accelerator; wherein the neural network accelerator includes a preset number of reconfigurable computing units, each reconfigurable computing unit supporting a different type of quantized computing; A quantization sensitivity determination module, configured to determine the quantization sensitivity of different layers of the target neural network based on the accuracy requirements of the different layers of the target neural network; a multiple quantization configuration determination module, configured to determine a quantization accuracy range for each layer based on the quantization sensitivity, and determine a configuration of a target neural network based on the quantization accuracy range and the neural network accelerator, to obtain multiple quantization configurations, each quantization configuration having a different quantization calculation combination determined based on the reconfigurable computing unit; the multiple quantization configurations being a quantization configuration having a number greater than one; The quantization module is used to take the multiple quantization configurations as input, set the performance of the target neural network and the performance of the neural network accelerator as targets, and perform optimization among the multiple quantization configurations to obtain the optimal quantization accuracy configuration.
9. A neural network quantization device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the neural network quantization method according to any one of claims 1 to 7.
10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the neural network quantization method according to any one of claims 1 to 7.