Neural network quantization method and apparatus, chip, electronic device, and storage medium

Abstract

The present disclosure provides a neural network quantization method, device, chip, electronic equipment and storage medium. The method comprises: obtaining a quantized neural network corresponding to a to-be-processed neural network; processing first test data based on initialized parameter information and the quantized neural network to obtain a test result corresponding to at least one parameter group in the initialized parameter information; wherein each parameter group comprises an initialized parameter value corresponding to at least one target processing layer in the quantized neural network; determining a target parameter corresponding to at least one target processing layer in the quantized neural network based on a reference result corresponding to the first test data and the test result corresponding to the at least one parameter group; wherein the target parameter is used to represent a bit width of a multiplier used in a quantization process of the target processing layer; and the reference result is obtained by processing the first test data based on the to-be-processed neural network.

Description

Technical Field

[0001] This disclosure relates to the field of chip technology, and more specifically, to a neural network quantization method, apparatus, chip, electronic device, and storage medium. Background Technology

[0002] With the continuous development of artificial intelligence technology, neural network technology, represented by convolutional neural networks, has permeated various industries and is applied in a variety of scenarios, such as facial recognition, speech recognition, and semantic detection. Many of these scenarios place high demands on the power consumption and performance of the chips running neural networks. Therefore, a method to reduce the power consumption of AI chips running neural networks is particularly important. Summary of the Invention

[0003] In view of this, the present disclosure provides at least one neural network quantization method, apparatus, chip, electronic device and storage medium.

[0004] In a first aspect, this disclosure provides a neural network quantization method, including:

[0005] Obtain the quantized neural network corresponding to the neural network to be processed;

[0006] Based on the initialized parameter information and the quantized neural network, the first test data is processed to obtain the test result corresponding to at least one parameter group in the initialized parameter information; wherein, each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network;

[0007] Based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

[0008] In a second aspect, this disclosure provides a chip, including: a memory and a computing device, the computing device including a dot productr, an accumulator and a multiplier, wherein the bit width of the multiplier is determined based on the target parameters in the neural network quantization method described in the first aspect or any embodiment;

[0009] The memory is used to store the quantized input feature data and quantized weight feature data corresponding to any target processing layer in the quantized neural network.

[0010] The dot product is used to retrieve the quantized input feature data and the quantized weight feature data from the memory, perform a dot product operation on the quantized input feature data and the quantized weight feature data to obtain multiple dot product values, and transmit the multiple dot product values ​​to the accumulator.

[0011] The accumulator is used to perform an accumulation operation on the multiple dot product values ​​to obtain an accumulated value;

[0012] The multiplier is used to perform a multiplication operation on the accumulated value and a determined quantization coefficient to obtain the quantized output feature value corresponding to the quantized neural network, wherein the quantization coefficient matches the bit width corresponding to the target processing layer.

[0013] Here, since the bit width of the multiplier is determined based on the target parameter in the neural network quantization method described in the above embodiment, the determination of the target parameter is relatively accurate, so that the bit width of the multiplier running different target processing layers can be different. For example, the bit width of the multiplier may be 32×8 or 32×16. Compared with the 32×32 bit width method of multipliers in related technologies, the power consumption of the chip can be reduced while ensuring the performance of the neural network.

[0014] Thirdly, this disclosure provides a neural network quantization device, comprising:

[0015] The acquisition module is used to acquire the quantized neural network corresponding to the neural network to be processed.

[0016] The first processing module is used to process the first test data based on the initialized parameter information and the quantized neural network to obtain the test result corresponding to at least one parameter group in the initialized parameter information; wherein each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network.

[0017] The determination module is used to determine the target parameters corresponding to at least one target processing layer in the quantized neural network based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

[0018] Fourthly, this disclosure provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, they perform the steps of the neural network quantization method as described in the first aspect or any of the embodiments above.

[0019] Fifthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the neural network quantization method as described in the first aspect or any of the embodiments above.

[0020] In this application, a quantized neural network with introduced initial parameter information is used to process the first test data to obtain test results corresponding to at least one parameter group in the initial parameter information. By comparing the benchmark results with the test results corresponding to the parameter groups, the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined more quickly and accurately, i.e., the target parameters corresponding to at least one target processing layer in the quantized neural network are obtained. These target parameters characterize the bit width of the multiplier used in the quantization process of the target processing layer. Multiple target processing layers can correspond to different multiplier bit widths, achieving optimized adaptation of the multiplier bit width of the target processing layers. Compared with related technologies where each target processing layer uses a large fixed bit width, this reduces the multiplication power consumption of the multiplier. Therefore, this disclosure can reduce the bit width of the multiplier used in the target processing layer during inference while ensuring neural network performance, thus achieving the purpose of power compression.

[0021] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0023] Figure 1 A schematic diagram illustrating the power compression principle provided in the embodiments of this disclosure is shown;

[0024] Figure 2A flowchart illustrating a neural network quantization method provided in an embodiment of this disclosure is shown.

[0025] Figure 3 A flowchart illustrating a neural network quantization method provided in an embodiment of this disclosure is shown.

[0026] Figure 4 A flowchart illustrating a neural network quantization method provided in an embodiment of this disclosure is shown.

[0027] Figure 5 This diagram illustrates the structure of a chip provided in an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of the architecture of a neural network quantization device provided in an embodiment of this disclosure is shown;

[0029] Figure 7a A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown;

[0030] Figure 7b A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0032] In the chip industry, the Tops / W metric is highly emphasized. This metric refers to the computing power a chip can provide per unit of power consumption and is generally used to evaluate the performance of a processor. This metric can typically be improved through quantization methods and sparsity techniques in neural networks. For example, quantization techniques can quantize feature data, such as weight feature data and input feature data, from single-precision floating-point to integer types, such as int8 (simply referred to as int8), thereby significantly improving the Tops / W metric.

[0033] Although the data has already been quantized to int8, during inference, the intermediate results of the neural network model may expand to int32 type (abbreviated as int32). Therefore, it is necessary to further quantize the intermediate results from int32 to int8. The process of quantizing the intermediate results from int32 to int8 requires the use of a multiplier with a larger bit width, resulting in a certain amount of power consumption in the chip. To alleviate the above problems, embodiments of this disclosure provide a neural network quantization method, apparatus, electronic device, and storage medium.

[0034] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0035] To facilitate understanding of the embodiments of this disclosure, a detailed description of a neural network quantization method disclosed in this disclosure will be provided first. The execution entity of the neural network quantization method provided in this disclosure is generally a computer device with a certain computing capability. This computer device may include, for example, a terminal device, a server, or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant (PDA), etc. In some possible implementations, this neural network quantization method can be implemented by a processor calling computer-readable instructions stored in memory.

[0036] Research has found that the requantization process involving int32 to int8 in neural network quantization techniques generally occurs in network processing layers such as convolutional layers and fully connected layers that require dot product operations. The calculation process of this network processing layer can be abstracted as: y = ∑w*x. After quantizing this calculation process, the quantized calculation process can be abstracted as: in, The output data after quantization is of type int 8; The weights are quantized and their data type is int 8. The input data after quantization is of type int8; This is denoted as the cumulative sum, and the result is an int 32; s1 is the quantization coefficient of the weight; s2 is the quantization coefficient of the input data; s3 is the quantization coefficient of the output data.

[0037] See Figure 1 The diagram shown illustrates the principle of power compression, which aims to reduce the power consumption used for computation. The int32 accumulator is converted to quantized output data. The int 8 requires a multiplier with a very large bit width to calculate. It consumes a lot of power and has high computational costs. However, during the chip implementation process, the values ​​of M and j can be calculated so that s1s2 / s3 can be approximated as m / 2. j Where m is a decimal integer, i.e., a fixed-point real number, and j is the number of times m is divided by 2 when converting it to a binary number using the divide-by-two remainder method. The above number system conversion process then becomes... Where M = m / 2 j cumulative sum The number of bits is 32.

[0038] In related technologies, the number of bits of M in each network processing layer is set to a fixed value, such as 32 bits, and a 32-bit × 32-bit multiplier is selected to process each network processing layer in the quantized neural network to obtain the output result of that network processing layer. In this disclosure, in order to reduce computational complexity... To address the power consumption during the process, a neural network quantization method is proposed. In this method, a parameter h(i) is introduced to control the number of bits in M. For example, M can be selected from four cases: 8 bits, 16 bits, 24 bits, and 32 bits. The number of bits in M ​​after quantization varies for different network processing layers. For example, when the number of bits in M ​​is 8 bits, the multiplier width required for calculation is 32 bits × 8 bits. Compared with a 32 bits × 32 bits multiplier, the multiplication power consumption can be reduced. Therefore, by determining the multiplier width used by each network processing layer in the inference process of the neural network, this disclosure can reduce the multiplication power consumption of at least some network processing layers compared with related technologies, thereby improving the computing power of the chip per unit power consumption.

[0039] See Figure 2 The diagram shown is a flowchart of a neural network quantization method provided in this embodiment of the present disclosure. The method includes steps S201-S203, wherein:

[0040] S201, Obtain the quantized neural network corresponding to the neural network to be processed.

[0041] S202, based on the initialized parameter information and the quantized neural network, the first test data is processed to obtain the test results corresponding to at least one parameter group in the initialized parameter information; wherein, each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network.

[0042] S203, based on the benchmark results corresponding to the first test data and the test results corresponding to at least one parameter group, determine the target parameters corresponding to at least one target processing layer in the quantized neural network; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

[0043] In the above method, a quantized neural network with introduced initial parameter information is used to process the first test data to obtain test results corresponding to at least one parameter group in the initial parameter information. By comparing the benchmark results with the test results corresponding to the parameter groups, the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined quickly and accurately, i.e., the target parameters corresponding to at least one target processing layer in the quantized neural network are obtained. These target parameters characterize the bit width of the multiplier used in the quantization process of the target processing layer. Multiple target processing layers can correspond to different multiplier bit widths, achieving optimized adaptation of the multiplier bit width of the target processing layers. Compared with related technologies where each target processing layer uses a large fixed bit width, this reduces the multiplication power consumption of the multiplier. Therefore, this disclosure can reduce the bit width of the multiplier used in the target processing layer during inference while ensuring neural network performance, thus achieving the purpose of power compression.

[0044] The following provides a detailed explanation of S201-S203.

[0045] For S201, obtain the quantized neural network corresponding to the neural network to be processed:

[0046] In implementation, the neural network to be processed can be linearly quantized to obtain the quantized neural network. The weight feature data and input feature data of the original neural network can be of higher precision data types, such as float32. The weight feature data and input feature data of the quantized neural network can be of lower precision data types, such as int8.

[0047] The neural network to be processed can be applied to various scenarios, such as autonomous driving, regional security checks, information recognition, and liveness detection. The neural network to be processed can be any type of neural network with any function; for example, it could be a neural network for face recognition or for semantic segmentation. The structure of the neural network can also be customized according to actual needs.

[0048] For S202, based on the initialized parameter information and the quantized neural network, the first test data is processed to obtain the test results corresponding to at least one parameter group in the initialized parameter information:

[0049] The target processing layer is the network processing layer that requires the use of multipliers during the quantization process. For example, the target processing layer can be a convolutional layer, a fully connected layer, etc.

[0050] The initialization parameter information includes at least one parameter group, and each parameter group includes initialization parameter values ​​corresponding to at least one target processing layer in the quantized neural network. For example, when the initialization parameter information includes parameter group 1, parameter group 2, and parameter group 3, and the quantized neural network includes target processing layer 1 and target processing layer 2, then parameter group 1 may include initialization parameter value 11 corresponding to target processing layer 1 and initialization parameter value 12 corresponding to target processing layer 2; parameter group 2 includes initialization parameter value 21 corresponding to target processing layer 1 and initialization parameter value 22 corresponding to target processing layer 2; and parameter group 3 includes initialization parameter value 31 corresponding to target processing layer 1 and initialization parameter value 32 corresponding to target processing layer 2. The initialization parameter values ​​corresponding to each target processing layer can be obtained through initialization, and the parameter values ​​included in different parameter groups are at least partially different.

[0051] In implementation, the first test data can be processed based on the initialized parameter information and the quantized neural network to generate test results corresponding to at least one parameter group in the initialized parameter information. For example, if the initialized parameter information includes parameter group 1 and parameter group 2, the first test data can be processed using the quantized neural network with parameter group 1 introduced to generate test results corresponding to parameter group 1; similarly, the first test data can be processed using the quantized neural network with parameter group 2 introduced to generate test results corresponding to parameter group 2. The first test data can be test data, sample data, or scene data collected within the application scenario, such as image data or video data including face regions in a security check scenario requiring facial recognition; or, in an autonomous driving scenario, 3D point cloud data or image data collected on the driving road.

[0052] In practice, simulation tools can be used to simulate the processing flow of the quantized neural network. Specifically, the simulation tool can simulate the computational logic of the quantized neural network, process the first test data based on the initialized parameter information, and obtain test results corresponding to at least one parameter group from the initialized parameter information. Furthermore, the simulation tool can be used to simulate the computational logic of the neural network to be processed, process the first test data, and obtain benchmark results.

[0053] For S203, based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined:

[0054] The target parameter characterizes the bit width of the multiplier used in the quantization process of the target processing layer. For example, if the target parameter for the target processing layer is 0, the bit width of the multiplier used in the quantization process can be 32 bits × 8 bits; if the target parameter is 1, the bit width can be 32 bits × 16 bits; if the target parameter is 2, the bit width can be 32 bits × 24 bits; and if the target parameter is 3, the bit width can be 32 bits × 32 bits. The mapping relationship between the target parameter and the bit width of the multiplier can be specifically set as needed.

[0055] In implementation, the target parameters for at least one target processing layer in the quantized neural network can be determined based on the baseline results corresponding to the first test data and the test results corresponding to at least one parameter group. For example, the test results corresponding to parameter group A and parameter group B can be compared with the baseline results corresponding to the first test data to determine the test result with the smallest deviation from the baseline result, and the parameter group corresponding to the test result with the smallest deviation can be determined as the target parameter group. Alternatively, the parameter group with lower power consumption and better test results in parameter A and parameter group B can be selected as the target parameter group. That is, the target parameters for at least one target processing layer in the quantized neural network are obtained based on the parameter values ​​included in the target parameter group.

[0056] In one possible implementation, based on the benchmark results corresponding to the first test data and the test results corresponding to at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined, including:

[0057] S2031, for each parameter group in the at least one parameter group, based on the benchmark result corresponding to the first test data and the test result corresponding to the parameter group, determine the fitness information corresponding to at least one parameter group; wherein, the fitness information is used to characterize the similarity between the test result and the benchmark result.

[0058] S2032, based on the fitness information corresponding to at least one parameter group, determine the target parameters corresponding to at least one target processing layer in the quantized neural network.

[0059] Here, the fitness information corresponding to the parameter group is determined based on the benchmark results corresponding to the first test data and the test results corresponding to the parameter group. This fitness information is used to characterize the similarity between the test results and the benchmark results. Through the fitness information of the parameter group, the target parameters of at least one target processing layer in the quantized neural network can be determined relatively easily.

[0060] Based on the benchmark results corresponding to the first test data and the test results corresponding to at least one parameter group, the fitness information corresponding to at least one parameter group is determined. The fitness information is used to characterize the similarity between the test results and the benchmark results. For example, the fitness information corresponding to the parameter group can be determined according to the following formula (1):

[0061] F=mean(cosinedist(f0,f1))+λeval(∑h(i)) (1)

[0062] Wherein, cosinedist() is the cosine distance calculation function; f0 is the output data of the neural network to be processed (i.e., the baseline result); f1 is the output data of the neural network after quantization with the parameter set (i.e., the test result corresponding to the parameter set); mean() is the mean calculation function; λ is an empirical value, and different neural networks can correspond to different λ values; the calculation formula of the eval() function is: (1 / n)×∑(-h(i)), where n is the number of the target processing layer; i is a positive integer less than or equal to n, and h(i) is the parameter value in the parameter set that matches the i-th target processing layer.

[0063] In implementation, a processing neural network can be used to process the first test data to obtain the benchmark result corresponding to the first test data. Then, a quantized neural network with any set of parameters can be used to process the first test data to obtain the test result corresponding to that set of parameters. The benchmark result can include multiple benchmark output data, i.e., f0 includes f... 01 and f 02 Accordingly, the test results can also include multiple test output data, that is, f1 includes f 11 and f 12 For example, when the neural network to be processed includes classification and regression tasks, f 01 or f 11 The output data can be f, which corresponds to the classification task. 02 or f 12 This can be the output data corresponding to the regression task.

[0064] f 01 with f 11 Inputting the cosine distance into the function yields the cosine distance 1. Then, f... 02 with f 12 The input is fed into the cosine distance calculation function to obtain cosine distance 2. Then, the mean of cosine distance 1 and cosine distance 2 is determined according to the mean calculation function. The sum of the mean of cosine distance 1 and cosine distance 2 and the mean of the parameter values ​​corresponding to each target processing layer can be determined as the fitness information corresponding to the parameter group.

[0065] Then, based on the fitness information corresponding to at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined. For example, if the initialized parameter information includes one parameter group, the fitness information corresponding to that parameter group can be determined based on the test results corresponding to that parameter group and the benchmark results corresponding to the first test data. If the fitness information corresponding to that parameter group is greater than or equal to the set threshold, then the parameter values ​​included in that parameter group are determined as the target parameters corresponding to at least one target processing layer in the quantized neural network.

[0066] Alternatively, if the initial parameter information includes multiple parameter groups, the fitness information corresponding to each parameter group can be determined, and the parameter group with the largest fitness information can be selected as the target parameter group. Based on the parameter values ​​included in the target parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined.

[0067] For example, if the initial parameter information includes multiple parameter groups, then based on the fitness information corresponding to at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined, which may include:

[0068] Step A1: Determine multiple parameter groups as multiple current parameter groups, use the fitness information corresponding to multiple parameter groups as current fitness information, and determine the first target number of candidate parameter groups from multiple current parameter groups based on the current fitness information corresponding to multiple current parameter groups.

[0069] Step A2: Transform the candidate parameter set for the first number of targets to generate the transformed parameter set for the second number of targets, and determine the transformed fitness information corresponding to the transformed parameter set; wherein, the number of the second target is greater than the number of the first target.

[0070] Step A3: If the fitness change value between the transformed fitness information and the current fitness information is greater than or equal to the fitness threshold, the transformed parameter group of the second target number is determined as the updated multiple current parameter groups, the transformed fitness information is determined as the updated current fitness information, and the process returns to the step of determining the candidate parameter group of the first target number from the multiple current parameter groups based on the current fitness information corresponding to the multiple current parameter groups, until the fitness change value is less than the fitness threshold.

[0071] Step A4: If the fitness change value between the transformed fitness information and the current fitness information is less than the fitness threshold, determine the target parameters corresponding to at least one target processing layer in the quantized neural network based on the transformed parameter group of the second target number.

[0072] In the above embodiments, by iterating multiple times, a better parameter set can be determined from multiple candidate parameter sets, thereby enabling more accurate determination of the target parameters corresponding to at least one target processing layer in the quantized neural network.

[0073] For step A1, multiple parameter groups can be defined as multiple current parameter groups, and the fitness information corresponding to these parameter groups can be used as current fitness information. Based on the current fitness information of these current parameter groups, candidate parameter groups for the first target number can be determined from these current parameter groups. For example, for 10,000 parameter groups, all 10,000 parameter groups can be defined as current parameter groups, and the fitness information corresponding to these 10,000 parameter groups can be defined as current fitness information. The 10,000 current fitness values ​​are sorted in descending order of fitness value. When the first target number is 2,000, the parameter groups corresponding to the top 2,000 current fitness values ​​can be selected as candidate parameter groups.

[0074] For example, based on the current fitness information corresponding to multiple current parameter groups, candidate parameter groups for the first target number are determined from the multiple current parameter groups, specifically including:

[0075] Step A11: Based on the current fitness information corresponding to each current parameter group, sort the multiple current parameter groups to obtain the sequence number corresponding to each current parameter group.

[0076] Step A12: Based on the sequence number corresponding to each current parameter group, determine the selection probability corresponding to each current parameter group, wherein the sequence number is negatively correlated with the selection probability.

[0077] Step A13: Based on the selection probability corresponding to each current parameter group, select a first target number of candidate parameter groups from multiple current parameter groups, wherein the selection probability is used to characterize the probability that the current parameter group is selected as a candidate parameter group.

[0078] Based on the current fitness information corresponding to each current parameter group, multiple current parameter groups can be sorted to obtain a sequence number for each current parameter group. For example, the sequence number of the current parameter group ranked first after sorting could be 1, the sequence number of the current parameter group ranked second could be 2, and so on. Based on the sequence numbers of each current parameter group, the selection probability of each current parameter group can be determined. For example, the reciprocal of the sequence number of the current parameter group can be used to determine the selection probability of the current parameter group. For instance, the selection probability of the current parameter group with sequence number k could be 1 / k, where k is a positive integer.

[0079] Furthermore, according to the selection probability corresponding to each parameter group, the first target number of candidate parameter groups can be randomly selected from multiple current parameter groups. The selection probability is used to characterize the probability that the current parameter group is selected as a candidate parameter group. For example, if the selection probability of the current parameter group with the sequence number 2 is 1 / 2, then the probability that the current parameter group is selected as a candidate parameter group is 1 / 2.

[0080] In the above implementation, multiple current parameter groups are sorted according to the current fitness information to obtain the sequence number corresponding to each current parameter group. When the current fitness information of the current parameter group is large, it indicates that the test result obtained by the current parameter group is more similar to the benchmark result. Therefore, its corresponding sequence number can be small, and the selection probability is large, so that the probability of the current parameter group being selected as a candidate parameter group is higher. This allows for a more flexible and accurate selection of the first target number of candidate parameter groups from multiple current parameter groups based on the selection probability.

[0081] For step A2, the candidate parameter set for the first target number is transformed to generate the transformed parameter set for the second target number, and the transformed fitness information corresponding to the transformed parameter set is determined. The transformation process includes, but is not limited to, mutation and crossover. The number of the second target is greater than the number of the first target. The number of the first and second targets can be set as needed.

[0082] For example, one or more candidate parameter groups can be randomly selected from the first target number of candidate parameter groups, and one or more parameter values ​​in the selected candidate parameter groups can be updated and transformed. For example, the parameter value corresponding to the target processing layer 1 in the selected candidate parameter group can be updated and transformed from 1 to 2 to obtain the transformed parameter group.

[0083] In one possible implementation, the candidate parameter set for a first target number is transformed to generate a transformed parameter set for a second target number, including:

[0084] Step A21: Based on the first target number of candidate parameter groups, construct a parameter group set, and select two candidate parameter groups from the parameter group set.

[0085] Step A22: Swap the parameter values ​​of at least one target processing layer in the two selected candidate parameter groups to generate two swapped candidate parameter groups.

[0086] Step A23: Update the parameter set to obtain a new parameter set, wherein the new parameter set includes: other candidate parameter sets besides the selected candidate parameter sets; or, other candidate parameter sets besides the selected candidate parameter sets and the two swapped candidate parameter sets; return to the step of selecting two candidate parameter sets from the parameter set until the swap cutoff condition is met, wherein the swap cutoff condition includes at least one of the following: the number of selections is equal to the number of selections threshold, the swap ratio is greater than or equal to the ratio threshold, and the number of unselected candidate parameter sets in the parameter set is less than a preset value.

[0087] Step A24: Based on the current candidate parameter group of the first target number obtained after the last exchange process, generate the transformed parameter group of the second target number, wherein the current candidate parameter group of the first target number includes the exchanged candidate parameter group, or includes the exchanged candidate parameter group and the candidate parameter group that has not undergone exchange processing.

[0088] Based on a first target number of candidate parameter groups, a parameter set is constructed, meaning the parameter set includes the first target number of candidate parameter groups. Two candidate parameter groups are randomly selected from the parameter set. The parameter values ​​of at least one target processing layer in the two selected candidate parameter groups are swapped, generating two swapped candidate parameter groups. For example, for candidate parameter group 1 and candidate parameter group 2, the parameter values ​​of target processing layer 1 in candidate parameter group 1 can be swapped with the parameter values ​​of target processing layer 1 in the candidate parameter group, resulting in swapped candidate parameter group 1 and swapped candidate parameter group 2.

[0089] The parameter set is updated to obtain a new parameter set. During implementation, two candidate parameter sets can be selected from the first target number of candidate parameter sets, using either replacement or no replacement. If the selection method is no replacement, the new parameter set includes all candidate parameter sets except the selected ones. If the selection method is replacement, the new parameter set includes all candidate parameter sets except the selected ones, and the two swapped candidate parameter sets.

[0090] After obtaining the new parameter set, the process returns to the step of selecting two candidate parameter sets from the parameter set (i.e., the new parameter set) until the exchange cutoff condition is met. The exchange cutoff condition includes at least one of the following: the number of selections equals a selection threshold, the exchange ratio is greater than or equal to a ratio threshold, or the number of unselected candidate parameter sets in the parameter set is less than a preset value. For example, in a selection method with replacement, a selection threshold can be set. When the number of selections of two candidate parameter sets from a first target number of candidate parameter sets exceeds the selection threshold, the exchange cutoff condition is met, and the exchange process ends. Alternatively, a ratio threshold can be set. When the exchange ratio of the exchanged candidate parameter sets included in the parameter set is greater than or equal to the ratio threshold, the exchange cutoff condition is met. Or, the cutoff condition is met when the number of unselected candidate parameter sets in the parameter set is less than a preset value.

[0091] If the selection method is without replacement, the exchange cutoff condition is met and the exchange process ends when there is no unselected candidate parameter group in the candidate parameter group of the first target quantity, or when there is only one unselected candidate parameter group (the preset value is 2). Alternatively, the preset value can be determined based on the exchange ratio and the first target quantity. For example, when the exchange ratio is 80% and the first target quantity is 1000, the preset value can be (1-80%)×1000=200.

[0092] After the last exchange process, a second target number of transformed parameter groups can be generated based on the current candidate parameter group of the first target number obtained after this process. The current candidate parameter group of the first target number may include only the exchanged candidate parameter group, or it may include both the exchanged candidate parameter group and the candidate parameter group that has not undergone the exchange process.

[0093] The following is an illustrative explanation of the swapping process. For example, if either of the two selected candidate parameter groups includes m parameter values ​​corresponding to the target processing layer, then for each target processing layer, a swapping probability p can be randomly assigned to that target processing layer, where p is greater than or equal to 0 and less than or equal to 1. If the swapping probability p corresponding to the target processing layer is less than the swapping probability threshold, then the parameter values ​​of that target processing layer are not swapped; if the swapping probability p corresponding to the target processing layer is greater than or equal to the swapping probability threshold, then the parameter values ​​of that target processing layer in the two selected candidate parameter groups are swapped, until all m target processing layers have been traversed, resulting in two swapped candidate parameter groups.

[0094] Through the above exchange processing operation, a first target number of current candidate parameter groups can be obtained. Then, based on the first target number of current candidate parameter groups, a second target number of transformed parameter groups can be generated. For example, the current candidate parameter groups can be copied, and at least one parameter value in the copied current candidate parameter groups can be updated and transformed to obtain a new parameter group. The new parameter group is used as the transformed parameter group. Repeating the above process can obtain multiple transformed parameter groups.

[0095] In the above implementation, by exchanging the parameter values ​​of at least one target processing layer in each candidate parameter group, a new candidate parameter group is generated. Then, based on the current candidate parameter group of the first target number, a new parameter group of the second target number is generated. This achieves the optimization and update of the parameter group, so that the test results of the quantized neural network after introducing each new parameter group are generally closer to the baseline results of the first test data, so that the target parameters can be determined more accurately in the future.

[0096] In step A24, based on the current candidate parameter set for the first target number obtained after the last exchange process, a transformed parameter set for the second target number is generated, specifically including:

[0097] Step A241: Repeat the following steps N times to generate N adjusted candidate parameter groups, where N is a positive integer greater than 1, and the value of N is consistent with the difference between the number of the second target and the number of the first target.

[0098] Step A242: Select any current candidate parameter group from the current candidate parameter groups of the first target number.

[0099] Step A243: Adjust the parameter values ​​of at least one target processing layer in any current candidate parameter group to generate an adjusted candidate parameter group.

[0100] Step A244: Based on the N adjusted candidate parameter groups and the current candidate parameter group of the first target number, construct the transformed parameter group of the second target number.

[0101] During implementation, from the candidate parameter groups after the exchange of the first target number, any one of the current candidate parameter groups is selected; the parameter values ​​of at least one target processing layer in any current candidate parameter group are adjusted to generate an adjusted candidate parameter group. For example, the parameter values ​​of one or more target processing layers in the selected current candidate parameter group can be adjusted to generate an adjusted candidate parameter group. The number of target processing layers to be adjusted (i.e., the number of mutations) can be determined by a preset mutation rate. For example, if the mutation rate is 0.08 and the total number of target processing layers is 100, then the number of mutations for the target processing layers to be adjusted can be 100 × 0.08 = 8, meaning the parameter values ​​of 8 target processing layers in the current candidate parameter group are adjusted. After the number of mutations for the parameter values ​​is determined, the parameter values ​​of the target processing layers with the specified number of mutations can be randomly selected from the selected current candidate parameter group for adjustment.

[0102] Repeating the above process N times generates N adjusted candidate parameter sets. Based on the N adjusted candidate parameter sets and the current candidate parameter sets for the first target number, a transformed parameter set for the second target number is constructed.

[0103] In the above implementation, by selecting any current candidate parameter group from the first target number of current candidate parameter groups, and adjusting the parameter values ​​of at least one target processing layer in the selected current candidate parameter group, an adjusted candidate parameter group is generated. Multiple adjustments can yield multiple adjusted candidate parameter groups. These multiple adjusted candidate parameter groups, together with the first target number of current candidate parameter groups, constitute a transformed parameter group. This enriches the quantity and content of the transformed parameter group, providing more options for determining the target parameters corresponding to at least one target processing layer in the quantized neural network based on the second target number of transformed parameter groups, thereby improving the accuracy of the target parameters.

[0104] In step A2, after generating the transformed parameter set of the second target number, the transformed fitness information corresponding to the transformed parameter set can be determined.

[0105] In one possible implementation, determining the transformed fitness information corresponding to the transformed parameter set includes: processing the first test data based on the transformed parameter set and the quantized neural network to obtain the transformed test results corresponding to the transformed parameter set; and determining the transformed fitness information corresponding to the transformed parameter set based on the benchmark results corresponding to the first test data and the transformed test results corresponding to the transformed parameter set.

[0106] Based on the transformed parameter set and the quantized neural network, the first test data is processed to obtain the transformed test results corresponding to the transformed parameter set. For example, simulation tools can be used to simulate the operational logic of the quantized neural network, and the first test data can be processed based on the transformed parameter set to obtain the transformed test results. Furthermore, based on the benchmark results corresponding to the first test data and the transformed test results corresponding to the transformed parameter set, the transformed fitness information corresponding to the transformed parameter set is determined.

[0107] For example, based on the test results and benchmark results after each transformation, the transformed fitness information corresponding to each transformed parameter group is determined according to formula (1) described above. The process of determining the transformed fitness information can be referred to the description of S2031 above, and will not be detailed here.

[0108] In the above implementation, based on the benchmark results corresponding to the first test data and the transformed test results corresponding to the transformed parameter group, the transformed fitness information corresponding to the transformed parameter group is determined, providing data support for subsequently determining the target parameters corresponding to at least one target processing layer in the quantized neural network.

[0109] After completing step A2, you can also execute steps A3 and A4.

[0110] After determining the fitness information after transformation, the fitness change value between the transformed fitness information and the current fitness information can be determined. It is then determined whether the fitness change value is greater than or equal to the fitness threshold. If yes, step A3 is executed; otherwise, step A4 is executed.

[0111] In step A3, if the fitness change between the transformed fitness information and the current fitness information is greater than or equal to the fitness threshold, the transformed parameter group for the second target number is determined as the updated multiple current parameter groups, and the transformed fitness information is determined as the updated current fitness information. The process returns to step A1, which involves determining the candidate parameter group for the first target number from the multiple current parameter groups based on their corresponding current fitness information, until the fitness change is less than the fitness threshold. The fitness threshold can be set as needed.

[0112] When there are multiple transformed fitness information and multiple current fitness information, they are sorted in descending order of size. Intermediate change values ​​are determined by sequence number. For example, the intermediate change value between the transformed fitness information with sequence number 1 and the current fitness information with sequence number 1 can be determined. Similarly, intermediate change values ​​corresponding to multiple sequence numbers can be obtained, and then the average of these intermediate change values ​​is taken to obtain the fitness change value.

[0113] In step A4, if the change in fitness between the transformed fitness information and the current fitness information is less than the fitness threshold, it is determined that the transformed parameter set meets the requirements. Based on the second target number of transformed parameter sets, the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined. For example, the transformed parameter set with the largest transformed fitness information can be selected as the target parameter set, and the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined based on the parameter values ​​included in the target parameter set.

[0114] In one possible implementation, step A4, determining the target parameters corresponding to at least one target processing layer in the quantized neural network based on the transformed parameter set of the second target number, may include:

[0115] Step A41: Based on the transformed fitness information corresponding to the transformed parameter groups of the second target quantity, select the intermediate parameter group of the third target quantity from the transformed parameter group.

[0116] Step A42: For each intermediate parameter group, the second test data is processed based on the intermediate parameter group and the quantized neural network to obtain the intermediate test results corresponding to the intermediate parameter group.

[0117] Step A43: Based on the sample results corresponding to the second test data and the intermediate test results corresponding to the intermediate parameter group, determine the cosine distance information corresponding to the intermediate parameter group.

[0118] Step A44: Determine the target parameter set based on the cosine distance information corresponding to each intermediate parameter set.

[0119] Step A45: Based on the target parameter set, determine the target parameters corresponding to at least one target processing layer in the quantized neural network.

[0120] Based on the transformed fitness information corresponding to the transformed parameter groups of the second target number, an intermediate parameter group of the third target number is selected from the transformed parameter groups. For example, the transformed parameter groups can be sorted in descending order of their transformed fitness information, and the transformed parameter groups of the third target number before sorting can be selected as the intermediate parameter groups. The third target number can be determined according to the actual situation; for example, the third target number can be 5, 10, etc.

[0121] The second test data can then be processed using a neural network to obtain the sample results corresponding to the second test data. For each intermediate parameter group, the second test data can be processed using a quantized neural network that incorporates the intermediate parameter group to obtain the intermediate test results corresponding to each intermediate parameter group. Based on the sample results corresponding to the second test data and the intermediate test results corresponding to each intermediate parameter group, the cosine distance information corresponding to each intermediate parameter group can be determined according to the cosine distance calculation function in formula (1). The intermediate parameter group corresponding to the cosine distance information with the largest value among the cosine distance information is determined as the target parameter group. The second test data can be partially different from or completely different from the first test data. The second test data can be any scenario data matching the application scenario, such as test data.

[0122] Based on the target parameter set, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined. That is, the parameter values ​​of each target processing layer included in the target parameter set can be used as the target parameters corresponding to that target processing layer.

[0123] In the above embodiments, the cosine distance information corresponding to the intermediate parameter group is determined based on the sample results corresponding to the second test data and the intermediate test results corresponding to the intermediate parameter group. The cosine distance information can characterize the similarity between the intermediate test results and the sample results. Based on the cosine distance information, the target parameter group can be determined more accurately. Furthermore, based on the target parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined more accurately.

[0124] See Figure 3 The diagram shown illustrates a flowchart of a neural network quantization method. Figure 3 The above method will be illustrated by example. Specifically:

[0125] Step 301: Initialize multiple parameter groups.

[0126] Obtain initialization parameter information, which includes at least one parameter group, and each parameter group includes initialization parameter values ​​corresponding to at least one target processing layer in the quantized neural network. Multiple initialization parameter groups can be defined as multiple current parameter groups.

[0127] Step 302: Calculate the fitness information for each parameter group.

[0128] For example, the current fitness information corresponding to multiple current parameter groups can be determined according to formula (1).

[0129] Step 303: Select the parameter group that meets the fitness criteria.

[0130] For example, based on the current fitness information of each current parameter group obtained in step 302, a candidate parameter group for the first target number is determined from multiple current parameter groups.

[0131] Step 304: Crossover / mutation generates a new set of parameters.

[0132] For example, from multiple candidate parameter groups, two candidate parameter groups can be selected using either no-replacement or replacement methods. These two selected candidate parameter groups can then be swapped to obtain swapped candidate parameter groups, thus generating new parameter groups using a crossover method. Alternatively, the parameter values ​​of at least one target processing layer in the candidate parameter group (or the swapped candidate parameter group) can be adjusted to generate adjusted candidate parameter groups, thus generating new parameter groups using a mutation method.

[0133] Step 305: Calculate the fitness information for each new parameter group.

[0134] For each new parameter group generated in step 304, calculate the fitness information for each new parameter group to obtain the fitness information corresponding to each new parameter group.

[0135] Step 306: Determine whether the termination condition is met.

[0136] For example, determine the fitness change value between the fitness information corresponding to each new parameter group and the current fitness information. Determine whether the fitness change value is greater than or equal to the fitness threshold. If the fitness change value is greater than or equal to the fitness threshold, then determine each new parameter group as the updated multiple current parameter groups, determine the fitness information corresponding to each new parameter group as the updated current fitness information, and return to step 303.

[0137] If the fitness change value is less than the fitness threshold, that is, if the fitness change value between the fitness information corresponding to each new parameter group and the current fitness information is less than the fitness threshold, then continue to execute step 307.

[0138] Step 307: Determine the target parameters.

[0139] Based on the new parameter sets that satisfy the termination condition in step 306, determine the target parameters corresponding to at least one target processing layer in the quantized neural network. For example, the parameter set with the largest fitness information among the new parameter sets can be determined as the target parameter set, and then the target parameters corresponding to at least one target processing layer in the quantized neural network can be determined based on this target parameter set.

[0140] In one possible implementation, after determining the target parameters corresponding to at least one target processing layer in the quantized neural network, the method further includes: quantizing the inference process of the quantized neural network based on the target parameters corresponding to each target processing layer.

[0141] Based on the target parameters corresponding to each target processing layer, the bit width of the multiplier used in the inference process of the quantized neural network is determined, and then the inference process of the quantized neural network is quantized based on the chip containing the multiplier.

[0142] For example, if the quantized neural network includes target processing layers x, y, and z, where the target parameter for target processing layer x is 0, the target parameter for target processing layer y is 2, and the target parameter for target processing layer z is 1, then the bit width of the multiplier used in the quantization process of target processing layer x can be determined to be 32-bit × 8-bit; the bit width of the multiplier used in the quantization process of target processing layer y can be determined to be 32-bit × 24-bit; and the bit width of the multiplier used in the quantization process of target processing layer z can be determined to be 32-bit × 16-bit. Based on a chip including multipliers with determined bit widths, the inference process of the quantized neural network is quantized.

[0143] Here, after determining the target parameters of the target processing layer, the inference process of the quantized neural network can be quantized based on the target parameters corresponding to each target processing layer. While ensuring the performance of the neural network, the bit width of the multipliers used by each target processing layer in the inference process is reduced, thereby reducing the use of computing resources and reducing the power consumption of the chip running the quantization process.

[0144] Combination Figure 4 As shown, an illustrative explanation of neural network quantization methods and their applications is provided. Figure 4 The neural network to be processed is quantized using a common quantization method to generate a quantized neural network. Simulation tools are then used to simulate the operational logic of both the neural network to be processed and the quantized neural network to process the first test data, obtaining test results and benchmark results. A genetic algorithm is then used to iterate multiple times on multiple parameter sets based on the test results and benchmark results until convergence, generating a target parameter set. During each iteration, fitness information corresponding to multiple parameter sets can be calculated based on the test results and benchmark results. This results in an adjusted neural network including the target parameter set, enabling subsequent quantization processing of the quantized neural network based on this target parameter set.

[0145] For example, when this method is applied to a face recognition scenario, the neural network used for face recognition can be used as the neural network to be processed. The target parameters of the target processing layer in the quantized neural network corresponding to the neural network to be processed are determined by the method proposed in this disclosure. Based on the target parameters corresponding to each target processing layer in the quantized neural network, the multiplier bit width matching the target processing layer is determined, thereby realizing the quantization processing of the inference process of the quantized neural network, reducing the power consumption consumed by the quantization processing, and improving the processing efficiency of the quantized neural network.

[0146] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0147] Based on the same technical concept, this disclosure also provides a chip, including: a memory 501 and a computing device 502. The computing device includes a dot productr 5021, an accumulator 5022 and a multiplier 5023, wherein the bit width of the multiplier is determined based on the target parameters in the neural network quantization method described in the above embodiments.

[0148] The memory 501 is used to store the quantized input feature data and quantized weight feature data corresponding to any target processing layer in the quantized neural network.

[0149] The dot product unit 5021 is used to retrieve quantized input feature data and quantized weight feature data from memory, perform dot product operation on the quantized input feature data and quantized weight feature data to obtain multiple dot product values, and transmit the multiple dot product values ​​to the accumulator.

[0150] The accumulator 5022 is used to accumulate multiple dot product values ​​to obtain the accumulated value.

[0151] Multiplier 5023 is used to multiply the accumulated value with a determined quantization coefficient to obtain the quantized output feature value of the quantized neural network, wherein the quantization coefficient matches the bit width of the target processing layer.

[0152] In implementation, the memory can be internal or external. The memory can store quantized input feature data and quantized weight feature data corresponding to any target processing layer in the quantized neural network, as well as quantized output feature data corresponding to the quantized neural network.

[0153] The computing device retrieves quantized input feature data and quantized weight feature data from memory. The dot product unit in the computing device performs a dot product operation on the retrieved quantized input feature data and quantized weight feature data. For example, it can perform a dot product operation on the feature values ​​at the same feature positions in the quantized input feature data and quantized weight feature data to obtain the dot product value, thus obtaining the dot product value corresponding to each feature position. The dot product unit then transmits multiple dot product values ​​to the accumulator.

[0154] The accumulator performs an accumulation operation on the received dot product values ​​to obtain an accumulated value, which is then transmitted to the multiplier.

[0155] The bit width of the multiplier can be determined by the target parameters in the neural network quantization method described in the above embodiments. For example, the mapping relationship between the target parameters and the bit width of the multiplier can be:

[0156]

[0157] Where h(i) is the target parameter corresponding to the i-th target processing layer.

[0158] For example, when the target parameter of the i-th target processing layer is 0, the bit width of the multiplier corresponding to the target processing layer is 32bit × 8bit.

[0159] This multiplier performs a multiplication operation between the accumulated value and a determined quantization coefficient to obtain the quantized output feature value of the neural network. The quantization coefficient is matched to the bit width of the target processing layer.

[0160] Here, since the bit width of the multiplier is determined based on the target parameter in the neural network quantization method described in the above embodiment, the determination of the target parameter is relatively accurate, so that the bit width of the multiplier running different target processing layers can be different. For example, the bit width of the multiplier may be 32×8 or 32×16. Compared with the method of using 32×32 bit width for multipliers in related technologies, the power consumption of the chip can be reduced while ensuring the performance of the neural network.

[0161] Based on the same concept, this disclosure also provides a neural network quantization device, see [link to relevant documentation]. Figure 6 The diagram shown is an architectural schematic of a neural network quantization device provided in an embodiment of this disclosure, including an acquisition module 601, a first processing module 602, and a generation module 603. Specifically:

[0162] The acquisition module 601 is used to acquire the quantized neural network corresponding to the neural network to be processed.

[0163] The first processing module 602 is used to process the first test data based on the initialized parameter information and the quantized neural network to obtain the test result corresponding to at least one parameter group in the initialized parameter information; wherein, each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network.

[0164] The generation module 603 is used to determine the target parameters corresponding to at least one target processing layer in the quantized neural network based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

[0165] In one possible implementation, after determining the target parameters corresponding to at least one target processing layer in the quantized neural network, the apparatus further includes:

[0166] The second processing module 604 is used to quantize the inference process of the quantized neural network based on the target parameters corresponding to each of the target processing layers.

[0167] In one possible implementation, the generation module 603, when determining the target parameters corresponding to at least one target processing layer in the quantized neural network based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group, is used to:

[0168] For each parameter group in the at least one parameter group, fitness information corresponding to the at least one parameter group is determined based on the benchmark result corresponding to the first test data and the test result corresponding to the parameter group; wherein, the fitness information is used to characterize the similarity between the test result and the benchmark result;

[0169] Based on the fitness information corresponding to the at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined.

[0170] In one possible implementation, the generation module 603, when initializing parameter information including multiple parameter groups, determines the target parameters corresponding to at least one target processing layer in the quantized neural network based on the fitness information corresponding to the at least one parameter group, is used for:

[0171] The plurality of parameter groups are determined as a plurality of current parameter groups, and the fitness information corresponding to the plurality of parameter groups is used as the current fitness information. Based on the current fitness information corresponding to the plurality of current parameter groups, a first target number of candidate parameter groups are determined from the plurality of current parameter groups.

[0172] The candidate parameter set for the first target number is transformed to generate a transformed parameter set for the second target number, and the transformed fitness information corresponding to the transformed parameter set is determined; wherein, the second target number is greater than the first target number;

[0173] If the fitness change value between the transformed fitness information and the current fitness information is greater than or equal to the fitness threshold, the transformed parameter group of the second target number is determined as the updated multiple current parameter groups, the transformed fitness information is determined as the updated current fitness information, and the process returns to the step of determining the candidate parameter group of the first target number from the multiple current parameter groups based on the current fitness information corresponding to the multiple current parameter groups, until the fitness change value is less than the fitness threshold;

[0174] If the fitness change value between the transformed fitness information and the current fitness information is less than the fitness threshold, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined based on the transformed parameter group of the second target number.

[0175] In one possible implementation, the generation module 603, when determining the transformed fitness information corresponding to the transformed parameter set, is used to:

[0176] Based on the transformed parameter set and the quantized neural network, the first test data is processed to obtain the transformed test result corresponding to the transformed parameter set;

[0177] Based on the benchmark results corresponding to the first test data and the transformed test results corresponding to the transformed parameter group, the transformed fitness information corresponding to the transformed parameter group is determined.

[0178] In one possible implementation, the generation module 603, when determining a first target number of candidate parameter groups from the plurality of current parameter groups based on the current fitness information corresponding to the plurality of current parameter groups, is configured to:

[0179] Based on the current fitness information corresponding to each current parameter group, the multiple current parameter groups are sorted to obtain the sequence number corresponding to each current parameter group;

[0180] Based on the sequence number corresponding to each current parameter group, the selection probability corresponding to each current parameter group is determined, wherein the sequence number is negatively correlated with the selection probability.

[0181] Based on the selection probability corresponding to each of the current parameter groups, a first target number of candidate parameter groups are selected from the plurality of current parameter groups, wherein the selection probability is used to characterize the probability that the current parameter group is selected as a candidate parameter group.

[0182] In one possible implementation, the generation module 603, when transforming the first target number of candidate parameter sets to generate the second target number of transformed parameter sets, is used to:

[0183] Based on the first target number of candidate parameter groups, a parameter group set is constructed, and two candidate parameter groups are selected from the parameter group set;

[0184] In the two selected candidate parameter groups, the parameter values ​​of at least one target processing layer are swapped to generate two swapped candidate parameter groups.

[0185] The parameter set is updated to obtain a new parameter set, wherein the new parameter set includes: other candidate parameter sets besides the selected candidate parameter sets; or, other candidate parameter sets besides the selected candidate parameter sets and the two swapped candidate parameter sets.

[0186] Return to the step of selecting two candidate parameter groups from the parameter group set until the exchange cutoff condition is met, wherein the exchange cutoff condition includes at least one of the following: the number of selections is equal to the number of selections threshold, the exchange ratio is greater than or equal to the ratio threshold, and the number of unselected candidate parameter groups in the parameter group set is less than a preset value.

[0187] Based on the current candidate parameter set of the first target number obtained after the last exchange process, a transformed parameter set of the second target number is generated, wherein the current candidate parameter set of the first target number includes the exchanged candidate parameter set, or includes the exchanged candidate parameter set and the candidate parameter set that has not undergone exchange process.

[0188] In one possible implementation, when generating a transformed parameter set for a second target number based on the current candidate parameter set for the first target number obtained after the last exchange process, the generation module 603 is used to:

[0189] Repeat the following steps N times to generate N adjusted candidate parameter groups, where N is a positive integer greater than 1, and the value of N is consistent with the difference between the second target quantity and the first target quantity:

[0190] Select any current candidate parameter group from the current candidate parameter groups of the first target number;

[0191] Adjust the parameter values ​​of at least one target processing layer in any current candidate parameter group to generate an adjusted candidate parameter group;

[0192] Based on the N adjusted candidate parameter groups and the current candidate parameter group of the first target number, a transformed parameter group of the second target number is constructed.

[0193] In one possible implementation, when the generation module 603 determines the target parameters corresponding to at least one target processing layer in the quantized neural network based on the transformed parameter set of the second target quantity, it is used to:

[0194] Based on the transformed fitness information corresponding to the transformed parameter groups of the second target quantity, an intermediate parameter group of the third target quantity is selected from the transformed parameter groups;

[0195] For each intermediate parameter set, based on the intermediate parameter set and the quantized neural network, the second test data is processed to obtain the intermediate test results corresponding to the intermediate parameter set; and

[0196] Based on the sample results corresponding to the second test data and the intermediate test results corresponding to the intermediate parameter group, the cosine distance information corresponding to the intermediate parameter group is determined.

[0197] Based on the cosine distance information corresponding to each of the intermediate parameter groups, the target parameter group is determined;

[0198] Based on the target parameter set, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined.

[0199] In some embodiments, the functions or templates of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0200] Based on the same technical concept, this disclosure also provides an electronic device. (See also...) Figure 7aThe diagram shown is a schematic representation of the structure of an electronic device provided in this embodiment of the present disclosure, including a processor 701, a memory 702, and a bus 703. The memory 702 stores machine-readable instructions executable by the processor 701, as well as computational data in the processor 701. The processor 701 retrieves the machine-readable instructions and computational data from the memory for data processing. When the electronic device 700 is running, the processor 701 communicates with the memory 702 via the bus 703, causing the processor 701 to execute the following instructions:

[0201] Obtain the quantized neural network corresponding to the neural network to be processed;

[0202] Based on the initialized parameter information and the quantized neural network, the first test data is processed to obtain the test result corresponding to at least one parameter group in the initialized parameter information; wherein, each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network;

[0203] Based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

[0204] The specific processing flow of the processor 701 can be referred to the description in the above method embodiment, and will not be repeated here.

[0205] See Figure 7b As shown, the electronic device 700 may further include the aforementioned chip, which includes a memory and a computing device. The bit width of the multiplier included in the computing device is determined based on the target parameters to perform the operation process of the target processing layer in the quantized neural network.

[0206] Furthermore, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the neural network quantization method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0207] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the neural network quantization method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0208] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDM), etc.

[0209] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0210] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0212] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0213] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A neural network quantization method, characterized in that, include: Obtain the quantized neural network corresponding to the neural network to be processed; Based on the initialized parameter information and the quantized neural network, the first test data is processed to obtain the test result corresponding to at least one parameter group in the initialized parameter information; wherein, each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network; Based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

2. The method according to claim 1, characterized in that, After determining the target parameters corresponding to at least one target processing layer in the quantized neural network, the method further includes: The inference process of the quantized neural network is quantized based on the target parameters corresponding to each target processing layer.

3. The method according to claim 1 or 2, characterized in that, The step of determining the target parameters corresponding to at least one target processing layer in the quantized neural network based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group includes: For each parameter group in the at least one parameter group, fitness information corresponding to the parameter group is determined based on the benchmark result corresponding to the first test data and the test result corresponding to the parameter group; wherein, the fitness information is used to characterize the similarity between the test result and the benchmark result; Based on the fitness information corresponding to the at least one parameter group, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined.

4. The method according to claim 3, characterized in that, The initialized parameter information includes multiple parameter groups; determining the target parameters corresponding to at least one target processing layer in the quantized neural network based on the fitness information corresponding to the at least one parameter group includes: The plurality of parameter groups are determined as a plurality of current parameter groups, and the fitness information corresponding to the plurality of parameter groups is used as the current fitness information. Based on the current fitness information corresponding to the plurality of current parameter groups, a first target number of candidate parameter groups are determined from the plurality of current parameter groups. The candidate parameter set for the first target number is transformed to generate a transformed parameter set for the second target number, and the transformed fitness information corresponding to the transformed parameter set is determined; wherein, the second target number is greater than the first target number; If the fitness change value between the transformed fitness information and the current fitness information is greater than or equal to the fitness threshold, the transformed parameter group of the second target number is determined as the updated multiple current parameter groups, the transformed fitness information is determined as the updated current fitness information, and the process returns to the step of determining the candidate parameter group of the first target number from the multiple current parameter groups based on the current fitness information corresponding to the multiple current parameter groups, until the fitness change value is less than the fitness threshold; If the fitness change value between the transformed fitness information and the current fitness information is less than the fitness threshold, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined based on the transformed parameter group of the second target number.

5. The method according to claim 4, characterized in that, Determining the transformed fitness information corresponding to the transformed parameter set includes: Based on the transformed parameter set and the quantized neural network, the first test data is processed to obtain the transformed test result corresponding to the transformed parameter set; Based on the benchmark results corresponding to the first test data and the transformed test results corresponding to the transformed parameter group, the transformed fitness information corresponding to the transformed parameter group is determined.

6. The method according to claim 4 or 5, characterized in that, The step of determining a first number of candidate parameter groups from the plurality of current parameter groups based on the current fitness information corresponding to the plurality of current parameter groups includes: Based on the current fitness information corresponding to each current parameter group, the multiple current parameter groups are sorted to obtain the sequence number corresponding to each current parameter group; Based on the sequence number corresponding to each current parameter group, the selection probability corresponding to each current parameter group is determined, wherein the sequence number is negatively correlated with the selection probability. Based on the selection probability corresponding to each of the current parameter groups, a first target number of candidate parameter groups are selected from the plurality of current parameter groups, wherein the selection probability is used to characterize the probability that the current parameter group is selected as a candidate parameter group.

7. The method according to claim 4 or 5, characterized in that, The step of transforming the candidate parameter set for the first target number to generate the transformed parameter set for the second target number includes: Based on the first target number of candidate parameter groups, a parameter group set is constructed, and two candidate parameter groups are selected from the parameter group set; In the two selected candidate parameter groups, the parameter values ​​of at least one target processing layer are swapped to generate two swapped candidate parameter groups. The parameter set is updated to obtain a new parameter set, wherein the new parameter set includes: other candidate parameter sets besides the selected candidate parameter sets; or, other candidate parameter sets besides the selected candidate parameter sets and the two swapped candidate parameter sets. Return to the step of selecting two candidate parameter groups from the parameter group set until the exchange cutoff condition is met, wherein the exchange cutoff condition includes at least one of the following: the number of selections is equal to the number of selections threshold, the exchange ratio is greater than or equal to the ratio threshold, and the number of unselected candidate parameter groups in the parameter group set is less than a preset value. Based on the current candidate parameter set of the first target number obtained after the last exchange process, a transformed parameter set of the second target number is generated, wherein the current candidate parameter set of the first target number includes the exchanged candidate parameter set, or includes the exchanged candidate parameter set and the candidate parameter set that has not undergone exchange process.

8. The method according to claim 7, characterized in that, The process of generating a transformed parameter set for a second target number based on the current candidate parameter set for the first target number obtained after the last exchange process includes: Repeat the following steps N times to generate N adjusted candidate parameter groups, where N is a positive integer greater than 1, and the value of N is consistent with the difference between the second target quantity and the first target quantity: Select any current candidate parameter group from the current candidate parameter groups of the first target number; Adjust the parameter values ​​of at least one target processing layer in any current candidate parameter group to generate an adjusted candidate parameter group; Based on the N adjusted candidate parameter groups and the current candidate parameter group of the first target number, a transformed parameter group of the second target number is constructed.

9. The method according to claim 4 or 5, characterized in that, The transformation parameter set based on the second target quantity determines the target parameters corresponding to at least one target processing layer in the quantized neural network, including: Based on the transformed fitness information corresponding to the transformed parameter groups of the second target quantity, an intermediate parameter group of the third target quantity is selected from the transformed parameter groups; For each intermediate parameter set, based on the intermediate parameter set and the quantized neural network, the second test data is processed to obtain the intermediate test results corresponding to the intermediate parameter set; and Based on the sample results corresponding to the second test data and the intermediate test results corresponding to the intermediate parameter group, the cosine distance information corresponding to the intermediate parameter group is determined. Based on the cosine distance information corresponding to each of the intermediate parameter groups, the target parameter group is determined; Based on the target parameter set, the target parameters corresponding to at least one target processing layer in the quantized neural network are determined.

10. A chip, characterized in that, include: The memory and computing device, the computing device including a dot productr, an accumulator and a multiplier, wherein the bit width of the multiplier is determined based on the target parameters in the neural network quantization method according to any one of claims 1 to 9; The memory is used to store the quantized input feature data and quantized weight feature data corresponding to any target processing layer in the quantized neural network. The dot product is used to retrieve the quantized input feature data and the quantized weight feature data from the memory, perform a dot product operation on the quantized input feature data and the quantized weight feature data to obtain multiple dot product values, and transmit the multiple dot product values ​​to the accumulator. The accumulator is used to perform an accumulation operation on the multiple dot product values ​​to obtain an accumulated value; The multiplier is used to perform a multiplication operation on the accumulated value and a determined quantization coefficient to obtain the quantized output feature value corresponding to the quantized neural network, wherein the quantization coefficient matches the bit width corresponding to the target processing layer.

11. A neural network quantization device, characterized in that, include: The acquisition module is used to acquire the quantized neural network corresponding to the neural network to be processed. The first processing module is used to process the first test data based on the initialized parameter information and the quantized neural network to obtain the test result corresponding to at least one parameter group in the initialized parameter information; wherein, each parameter group includes the initialized parameter value corresponding to at least one target processing layer in the quantized neural network; The determination module is used to determine the target parameters corresponding to at least one target processing layer in the quantized neural network based on the benchmark results corresponding to the first test data and the test results corresponding to the at least one parameter group; wherein, the target parameters are used to characterize the bit width of the multiplier used in the quantization process of the target processing layer; the benchmark results are obtained by processing the first test data based on the neural network to be processed.

12. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and the processor communicates with the memory via the bus when the electronic device is in operation, and the machine-readable instructions, when executed by the processor, perform the steps of the neural network quantization method as described in any one of claims 1 to 9. Alternatively, the chip as described in claim 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the neural network quantization method as described in any one of claims 1 to 9.

Images