Quantization method, device and system of neural network data and readable storage medium

Abstract

The application provides a quantization method, device and system of neural network data and a readable storage medium. The method comprises the following steps: obtaining calibration output data of a neural network; selecting a quantization boundary value according to probability distribution information of the calibration output data, and determining a target quantization range of the neural network; and quantizing the output of the neural network according to the target quantization range. By using the above method, the loss of quantization precision of the output data of the neural network can be reduced.

Description

Technical Field

[0001] This invention belongs to the field of data quantization, and specifically relates to a method, apparatus, system, and readable storage medium for quantizing neural network data. Background Technology

[0002] This section is intended to provide background or context for embodiments of the invention as set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] Quantization of a neural network model refers to converting the parameters of the neural network model from high-precision data to low-precision data, thereby reducing the size of the neural network model, increasing its running speed, and maintaining accuracy similar to that before quantization. Neural network model quantization includes two stages: quantization of weight data and quantization of output data. A commonly used quantization technique is Int8 quantization, which maps neural network data (e.g., double-precision (64-bit), single-precision (32-bit), or half-precision (16-bit)) and input data (double-precision, single-precision, or half-precision) to 8-bit quantized data using linear or non-linear methods.

[0004] In existing technologies, uniform affine quantization is commonly used to quantize neural network data. This involves determining the original quantization range based on the maximum and minimum floating-point values ​​of the original floating-point data, and then uniformly mapping the original floating-point data within the original quantization range to uniformly distributed integer data using a scaling factor. However, in practice, uniform affine quantization often leads to significant quantization errors in the neural network data, reducing the accuracy of the neural network model.

[0005] Therefore, how to reduce the quantization error of neural network data is an urgent problem to be solved. Summary of the Invention

[0006] To address the problems existing in the prior art, a method, apparatus, system, and computer-readable storage medium for quantizing neural network data are proposed. Using this method, apparatus, system, and computer-readable storage medium, the aforementioned problems can be solved.

[0007] The present invention provides the following solutions.

[0008] In a first aspect, a method for quantizing neural network data is provided, comprising: acquiring calibration output data of the neural network; selecting quantization boundary values ​​based on the probability distribution information of the calibration output data to determine the target quantization range of the neural network; and quantizing the output of the neural network according to the target quantization range.

[0009] In one implementation, determining the target quantization range of the neural network further includes: obtaining a probability density function based on the probability distribution information of the calibration output data; selecting a quantization boundary value based on the cumulative distribution function of the probability density function to determine the target quantization range of the neural network.

[0010] In one implementation, selecting the quantization boundary value based on the cumulative distribution function of the probability density function further includes: determining the quantization boundary value based on the slope of the cumulative distribution function.

[0011] In one implementation, the slope of each point is calculated point by point from the endpoints of the cumulative distribution function toward the center; when a target point with a slope exceeding a preset threshold is detected, a quantization boundary value is determined based on the target point.

[0012] In one implementation, the method further includes: calculating the slope of each point from the endpoint of the cumulative distribution function toward the center; when the slope of N points is detected to exceed a preset threshold, determining the target point among the N points that is closer to the endpoint, and determining the quantization boundary value based on the target point; where N is an integer greater than 1.

[0013] In one implementation, the method further includes: if the quantization is symmetric quantization, calculating the slope of each point from any endpoint of the cumulative distribution function toward the center.

[0014] In one implementation, the method further includes: if the quantization is asymmetric quantization, calculating the slope of each point from both ends of the cumulative distribution function toward the center point by point.

[0015] In one implementation, determining the quantization boundary value based on the slope of the cumulative distribution function further includes: forward searching point by point from the minimum endpoint of the cumulative distribution function for the minimum target point whose slope exceeds a preset threshold, and determining the minimum quantization boundary value of the target quantization range based on the minimum target point; and / or, backward searching point by point from the maximum endpoint of the cumulative distribution function for the maximum target point whose slope exceeds a preset threshold, and determining the maximum quantization boundary value of the target quantization range based on the maximum target point.

[0016] In one embodiment, the method further includes: normalizing the cumulative distribution function; and removing the maximum value range and / or minimum value range of the cumulative distribution function according to a set proportion range.

[0017] In one implementation, the method further includes: applying a smooth window to the probability density function as a sliding filter to eliminate glitches in the probability density function.

[0018] In one implementation, the neural network includes multiple layers, and the method further includes: determining a target quantization range corresponding to each layer based on the calibration output data of each layer; and quantizing the output of each layer according to the target quantization range corresponding to each layer.

[0019] In one implementation, the method further includes: acquiring a calibration dataset; inputting the calibration dataset into a neural network to obtain calibration output data from the neural network.

[0020] In a second aspect, a quantization apparatus for neural network data is provided for performing the method of the first aspect. The apparatus includes: an acquisition module for acquiring calibration output data of the neural network; a quantization range module for selecting quantization boundary values ​​based on the probability distribution information of the calibration output data to determine the target quantization range of the neural network; and a quantization module for quantizing the output of the neural network according to the target quantization range.

[0021] Thirdly, a system for quantizing neural network data is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform: the method of the first aspect.

[0022] Fourthly, a computer-readable storage medium is provided, which stores a program that, when executed by a multi-core processor, causes the multi-core processor to perform the method of the first aspect.

[0023] One of the advantages of the above implementation method is that it can reduce the quantization accuracy error of the output and improve the accuracy of the quantized neural network model.

[0024] Other advantages of the present invention will be explained in more detail below with reference to the accompanying drawings.

[0025] It should be understood that the above description is merely an overview of the technical solution of the present invention, so as to enable a clearer understanding of the technical means of the present invention, and thus to implement it in accordance with the contents of the specification. To make the above and other objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are illustrated below. Attached Figure Description

[0026] By reading the detailed description of the exemplary embodiments below, those skilled in the art will understand the advantages and benefits of this document, as well as other advantages and benefits. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0027] Figure 1 This is a schematic diagram of the structure of a neural network data quantization device according to an embodiment of the present invention;

[0028] Figure 2 This is a flowchart illustrating a method for quantizing neural network data according to an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the quantization of neural network data according to an embodiment of the present invention;

[0030] Figure 4 This is a flowchart illustrating a method for quantizing neural network data according to an embodiment of the present invention.

[0031] Figure 5 This is a schematic diagram of the probability density function of calibration output data according to an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of the cumulative distribution function of calibration output data according to an embodiment of the present invention;

[0033] Figure 7 This is a schematic diagram of the structure of a neural network data quantization device according to an embodiment of the present invention. Detailed Implementation

[0034] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0035] In the description of embodiments of this application, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, numbers, steps, behaviors, components, portions or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence of one or more other features, numbers, steps, behaviors, components, portions or combinations thereof.

[0036] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0037] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0038] To save storage space and improve computational efficiency, it is usually necessary to quantize the output data of neural network models. Specifically, the quantization process can be to quantize the output data of each network layer of the neural network from floating-point data (e.g., 32-bit floating-point data, hereinafter referred to as FP32) to a lower-bit fixed-point number (e.g., 8-bit fixed-point data, hereinafter referred to as INT8), thereby reducing the number of computing resources used.

[0039] As mentioned above, uniform affinity quantization is commonly used in traditional schemes. Its specific principle is as follows:

[0040] Assume the output of this neural network layer includes floating-point data. First, determine the minimum floating-point value V of the overall output of this neural network layer. min Floating-point maximum value V max When it is necessary to uniformly map and quantize the output of the floating-point neural network layer to N times a specific point-type data V... quant At that time, fixed-point data V quant This represents the fixed-point value of floating-point data R after quantization to N bits. The minimum fixed-point value can be 0, and the maximum fixed-point value can be 2. N -1.

[0041] In one embodiment, the quantization formula used when quantizing floating-point values ​​to fixed-point values ​​is as follows:

[0042] V quant =Q*(V x -V min )

[0043] Where Q = S / R, R = V max -V min S = 1 << quant bit -1

[0044] Among them, V x V represents the floating-point value to be quantized. quant V represents the quantized fixed-point value. min and V max quant represents the minimum and maximum floating-point values. bit This indicates the number of bits after quantization; for example, INT8 represents 8 fixed-point bits. Besides the quantization methods mentioned above, other quantization methods exist in the prior art, which will not be elaborated upon here.

[0045] For uniformly distributed data, V min and V maxYou can simply take the maximum and minimum values ​​of the real floating-point numbers. However, the activation data of neural networks usually exhibits a shape close to a Gaussian distribution, with a large amount of data concentrated around the mean. Directly taking the maximum and minimum values ​​of the real floating-point numbers will cause most of the intermediate data to lose too much precision due to quantization.

[0046] To at least partially address the problem of excessive precision loss due to quantization, an exemplary embodiment of this disclosure proposes a scheme for neural network weight quantization. In this scheme, calibration output data (a set of multiple floating-point data points) of the neural network is acquired. Quantization boundary values ​​are selected based on the probability distribution information of the calibration output data; specifically, two values ​​within the data range of the calibration output data are selected as the quantization boundary values ​​for the target quantization range. This probability distribution information is specifically generated, for example, based on the cumulative distribution function (CDF) of the calibration output data. Subsequently, the actual activation output of the neural network can be quantized according to the target quantization range, directly mapping values ​​outside the target quantization range to fixed-point minimum or maximum values. This preserves higher precision for most data points.

[0047] In this way, the minimum floating-point value V can be corrected. min The maximum floating-point value V max The correspondence between the quantized fixed-point value and the quantized value improves quantization performance and reduces the loss of quantization accuracy.

[0048] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] First see Figure 1 The illustration schematically shows an environment 100 in which an exemplary implementation according to the present disclosure can be used.

[0050] Figure 1 A schematic diagram of an example of a computing device 100 according to an embodiment of the present disclosure is shown. It should be noted that... Figure 1 This is a schematic diagram of the hardware operating environment for a method of quantizing neural network data. The quantization device for neural network data in this invention can be a PC, a portable computer, or other terminal device.

[0051] like Figure 1As shown, the quantization system for the neural network data may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The quantization device structure for neural network data shown does not constitute a limitation on the quantization device for neural network data. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a quantization program for neural network data. The operating system is a program that manages and controls the hardware and software resources of the neural network data quantization device, supporting the execution of the neural network data quantization program and other software or programs.

[0054] exist Figure 1 In the quantization system for neural network data shown, the user interface 1003 is mainly used to receive requests and data sent by the first terminal, the second terminal, and the monitoring terminal; the network interface 1004 is mainly used to connect the backend server to the backend server for data communication; and the processor 1001 can be used to call the quantization program for neural network data stored in the memory 1005 and perform the following operations: obtain the calibration output data of the neural network; select the quantization boundary value according to the probability distribution information of the calibration output data to determine the target quantization range of the neural network; and quantize the output of the neural network according to the target quantization range.

[0055] Therefore, the target quantization range of the neural network can be quickly and easily searched, which can effectively and significantly reduce the loss of quantization accuracy of the neural network.

[0056] Figure 2 A flowchart is shown of a method for performing quantization of neural network data according to an embodiment of the present disclosure. This method can be, for example, by... Figure 1The method is executed by the computing device 100 shown. It should be understood that the method 300 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0057] Step 210: Obtain the calibration output data of the neural network;

[0058] The calibration output data refers to the activation output data of each layer of the neural network after the input calibration data. Specifically, the calibration output data can be collected by inputting specific calibration data into the neural network. Alternatively, the activation output data of the neural network during actual operation can be collected as the calibration output data. Preferably, the activation output of the neural network in various operating time periods can also be dynamically collected as calibration output data, and the target quantization range of the neural network can be dynamically adjusted periodically based on the activation output of each time period to adapt to changes in the neural network's data. This embodiment does not impose specific limitations in this regard.

[0059] Step 220: Select quantization boundary values ​​based on the probability distribution information of the calibration output data to determine the target quantization range of the neural network;

[0060] Specifically, the calibration output data actually includes a large number of floating-point values ​​output by the neural network. This embodiment does not simply select the maximum and minimum values ​​from this large number of floating-point values ​​as quantization boundary values, but first obtains the probability distribution information of this large number of floating-point values. For example, if the probability distribution information of the calibration output data shows a Gaussian probability distribution, the quantization boundary values ​​can be selected based on its distribution divergence, thereby determining the narrowed target quantization range. If the data in the calibration output dataset exhibits a uniform probability distribution, the maximum and minimum values ​​from this large number of floating-point values ​​can be selected as quantization boundary values.

[0061] Step 230: Quantize the output of the neural network according to the target quantization range.

[0062] Specifically, in subsequent runs of the neural network, the output of the neural network will be quantized based on the adjusted target quantization range [-|T|, |T|]. (See reference) Figure 3 As can be seen, assuming the original quantization range obtained directly from the minimum and maximum values ​​of a large number of floating-point values ​​is [-|T|', |T|], several floating-point numbers near the minimum value are significantly more dispersed. Using this quantization range for quantization would lead to a large loss of quantization precision. However, by adjusting the target quantization range to [-|T|, |T|], the loss of quantization precision can be reduced to some extent.

[0063] Neural networks consist of multiple layers, each with a different task, resulting in significant differences in the distribution of activation and output across the layers.

[0064] In one implementation, to achieve quantization with less precision loss for each layer, a target quantization range corresponding to each layer is independently determined based on the calibration output data of each layer in the neural network. Then, the output of each layer is quantized according to the target quantization range corresponding to each layer. In this way, a more accurate quantization effect can be achieved even when the activation outputs of each layer differ.

[0065] Alternatively, calibration output data from two or more layers can be integrated together, and a target quantization range can be output for those two or more layers. This reduces quantization complexity. In particular, if the high similarity of the activation output distributions of two or more layers can be detected in advance, the quantization strategies of those two layers can be bound together.

[0066] In one embodiment, before step 210 above, in order to obtain the calibration data of the neural network, a calibration dataset can be obtained first, and the calibration dataset can be input into the neural network to obtain the output data of each layer of the neural network as the calibration output data.

[0067] Specifically, the calibration dataset can be collected based on the functions implemented by the neural network. For example, if the neural network is a face recognition neural network, a large number of real face images can be collected to form the calibration dataset. If the neural network is a semantic recognition neural network, a large amount of text data can be collected to form the calibration dataset, and so on. It can be understood that the activation output data of the neural network is highly correlated with the content of the data it processes. By collecting actual data samples that need to be processed subsequently based on the functions implemented by the neural network as the calibration dataset, a more accurate quantization range can be obtained.

[0068] Figure 4 This is a flowchart illustrating a method for quantizing neural network data according to another exemplary embodiment of the present invention. Figure 2 Based on the embodiment shown, the process of step 220 above will be described in further detail.

[0069] like Figure 4 As shown, in step 220 above, in order to more accurately determine the target quantization range of the neural network, the following steps can be performed:

[0070] Step 221: Obtain the probability density function based on the probability distribution information of the calibration output data;

[0071] Step 222: Select the quantization boundary value based on the cumulative distribution function of the probability density function to determine the target quantization range of the neural network.

[0072] The probability density function (PDF) is a function that describes the likelihood of the output value of a continuous random variable near a certain point in time. Typical continuous probability distributions include the normal distribution and the exponential distribution. Specifically, in step 221, this probability density function can be obtained by obtaining the histogram of the distribution of the calibration output data.

[0073] The cumulative distribution function (CDF), also known as the distribution function, is the integral of the probability density function and can completely describe the probability distribution of a real random variable X. For example, when the probability distribution function of a calibration output data is... Figure 5 When the distribution is normal, its cumulative distribution function is as follows: Figure 6 As shown.

[0074] It is understandable that the scheme of selecting quantization boundary values ​​by using the probability density function (PDF) and cumulative distribution function (CDF) of the calibration output data provides a novel quantization boundary value search scheme, which can quickly calculate the appropriate quantization boundary values ​​in the calibration output data.

[0075] In one specific implementation, the quantization boundary value is selected and determined based on the slope of the cumulative distribution function (CDF) of the probability density function. It can be understood that, based on the characteristics of the CDF, a small slope indicates slow probability accumulation and a corresponding small probability density function, making it suitable as a boundary point. Thus, by directly calculating the slope of the CDF, a suitable quantization boundary value can be found more quickly.

[0076] For example, from Figure 6 The slope of each point is calculated in the cumulative distribution function shown. Finally, the minimum and maximum points among several points whose slopes exceed a preset threshold are selected as quantization boundary values. Alternatively, the second minimum and second maximum points among several points whose slopes exceed a preset threshold can be selected as quantization boundary values. This application does not impose specific restrictions on this.

[0077] In one specific implementation, the slope of each point is calculated point by point from the endpoints of the cumulative distribution function toward the center; when a target point with a slope exceeding a preset threshold is detected, a quantization boundary value is determined based on the target point.

[0078] For example, from Figure 6The cumulative distribution function shown calculates the slope of each point from its left endpoint to the right. The initial slope is small, and it gradually increases at an accelerating rate as the cumulative probability increases. When the slope exceeds a preset threshold for the first time, the calculation can be stopped, and this point is determined as the minimum value of the quantization boundary value. The selection of the maximum value of the quantization boundary value is similar and will not be described in detail in this embodiment.

[0079] Alternatively, the slope can be calculated from the endpoints of the cumulative distribution function toward the center at a certain step size, thereby saving computation time and speeding up the search.

[0080] In one specific implementation, the minimum target point with a slope exceeding a preset threshold can be searched point by point in the forward direction from the minimum endpoint of the cumulative distribution function, and the minimum quantization boundary value of the target quantization range can be determined based on the minimum target point; and / or, the maximum target point with a slope exceeding a preset threshold can be searched point by point in the reverse direction from the maximum endpoint of the cumulative distribution function, and the maximum quantization boundary value of the target quantization range can be determined based on the maximum target point.

[0081] The above-mentioned forward search is from small to large, and the above-mentioned reverse search is from large to small.

[0082] In one possible implementation, the slope of each point can be calculated point by point from the endpoint of the cumulative distribution function toward the center; when the slope of N points is detected to exceed a preset threshold, a target point closer to the endpoint among the N points is determined, and a quantization boundary value is determined based on the target point; N is an integer greater than 1.

[0083] For example, refer to Figure 6 When searching from the left endpoint to the right, if the slope of a certain point exceeds the preset threshold, the search continues to the right until the slope of three consecutive values ​​is greater than the preset threshold. Only then is the first point found used as the quantization boundary value.

[0084] In this embodiment, the first point whose slope exceeds the preset value is not directly used as the quantization boundary value. Instead, the first point is used as the quantization boundary value only after multiple points whose slopes have exceeded the preset threshold are detected consecutively. This avoids the adverse effects of concentrated data noise.

[0085] In one implementation, if quantization is symmetric quantization, the slope of each point is calculated point by point from either endpoint of the cumulative distribution function toward the center. It can be understood that with symmetric quantization, the distribution at both ends is symmetrical, thus requiring only the calculation of a single quantization boundary value, saving search time and increasing speed.

[0086] In one implementation, if the quantization is asymmetric, the slope of each point is calculated point by point from both ends of the cumulative distribution function toward the center. It can be understood that with asymmetric quantization, the distributions at both ends are inconsistent, thus requiring the separate calculation of two quantization boundary values.

[0087] In one implementation, in step 220, the cumulative distribution function (CDF) can also be normalized; the maximum value range and / or minimum value range of the cumulative distribution function can be removed according to the set proportion range of the cumulative distribution function.

[0088] For example, still refer to Figure 6 The minimum 5% interval can be removed from the left and the maximum 5% interval can be removed from the right. Since the data on both sides are usually discrete data, this processing can speed up the point-by-point search and improve the search speed.

[0089] In one implementation, the method further includes applying a smoothing window to the distribution histogram using a sliding filter to eliminate spikes in the distribution histogram. It is understood that the amount of data in the calibration dataset may be small, which can cause numerous data spikes in the cumulative probability distribution function (CDF), thus affecting the slope calculation. The smoothing window can reduce these data spikes, lowering the probability of errors in the slope calculation.

[0090] In the description of this specification, the references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same implementation or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0091] The method flowcharts for embodiments of this application describe certain operations as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0092] Based on the same technical concept, embodiments of the present invention also provide a quantization apparatus for neural network data, used to execute the quantization method for neural network data provided in any of the above embodiments. Figure 7 A schematic diagram of a neural network data quantization device provided for an embodiment of the present invention.

[0093] like Figure 7 As shown, the quantization device 700 for neural network data includes:

[0094] The acquisition module 710 is used to acquire the calibration output data of the neural network;

[0095] The quantization range module 720 is used to select quantization boundary values ​​based on the probability distribution information of the calibration output data, and to determine the target quantization range of the neural network.

[0096] The quantization module 730 is used to quantize the output of the neural network according to the target quantization range.

[0097] It should be noted that the apparatus in the embodiments of this application can implement each process of the aforementioned method and achieve the same effect and function, which will not be elaborated here.

[0098] According to some embodiments of this application, a non-volatile computer storage medium is provided for a method of quantizing neural network data, having stored thereon computer-executable instructions configured to be executed when run by a processor: the method of the above embodiments.

[0099] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the descriptions of the apparatus, device, and computer-readable storage medium embodiments are simplified because they are substantially similar to the method embodiments; relevant details can be found in the descriptions of the method embodiments.

[0100] The apparatus, system, and computer-readable storage medium provided in this application correspond one-to-one with the method. Therefore, the apparatus, system, and computer-readable storage medium also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus, device, and computer-readable storage medium will not be repeated here.

[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus (devices or systems), or computer-readable storage media. Therefore, the present invention can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, the present invention can take the form of a computer-readable storage medium embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0102] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices or systems), and computer-readable storage media according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for quantizing neural network data, characterized in that, include: Obtain the calibration output data of the neural network. If the neural network is an image recognition neural network, the calibration dataset input to the neural network consists of real human face images; if the neural network is a semantic recognition neural network, the calibration dataset input to the neural network consists of text data. Selecting quantization boundary values ​​based on the probability distribution information of the calibration output data and determining the target quantization range of the neural network includes: obtaining a probability density function based on the probability distribution information of the calibration output data; The quantization boundary value is selected based on the slope of the cumulative distribution function of the probability density function to determine the target quantization range of the neural network; The output of the neural network is quantized according to the target quantization range.

2. The method according to claim 1, characterized in that, Calculate the slope of each point from the endpoints of the cumulative distribution function toward the center; When a target point with a slope exceeding a preset threshold is detected, the quantization boundary value is determined based on the target point.

3. The method according to claim 1, characterized in that, Also includes: Calculate the slope of each point from the endpoints of the cumulative distribution function toward the center; When the slope of N consecutive points exceeds a preset threshold, a target point closer to the endpoint is determined among the N points, and the quantization boundary value is determined based on the target point; where N is an integer greater than 1.

4. The method according to claim 2 or 3, characterized in that, Also includes: If the quantization is symmetric quantization, the slope of each point is calculated point by point from any endpoint of the cumulative distribution function toward the center.

5. The method according to claim 2 or 3, characterized in that, Also includes: If the quantization is asymmetric quantization, the slope of each point is calculated point by point from both ends of the cumulative distribution function toward the center.

6. The method according to claim 1, characterized in that, Determining the quantization boundary value based on the slope of the cumulative distribution function further includes: Starting from the minimum endpoint of the cumulative distribution function, a forward point-by-point search is performed for the minimum target point whose slope exceeds a preset threshold; and the minimum quantization boundary value of the target quantization range is determined based on the minimum target point; and / or, Starting from the endpoint of the maximum value of the cumulative distribution function, search backward point by point for the maximum target point whose slope exceeds a preset threshold, and determine the maximum quantization boundary value of the target quantization range based on the maximum target point.

7. The method according to claim 1, characterized in that, Also includes: The cumulative distribution function is normalized. According to the set proportion range of the cumulative distribution function, remove the maximum value range and / or minimum value range of the cumulative distribution function.

8. The method according to claim 1, characterized in that, Also includes: A smoothing window is used to perform sliding filtering on the probability density function to eliminate glitches in the probability density function.

9. The method according to claim 1, characterized in that, The neural network comprises multiple layers, and the method further includes: Based on the calibration output data of each layer, determine the target quantization range corresponding to each layer; The output of each layer is quantized according to the target quantization range corresponding to each layer.

10. The method according to claim 1, characterized in that, Also includes: Obtain the calibration dataset; The calibration dataset is input into the neural network to obtain the calibration output data of the neural network.

11. A quantization device for neural network data, characterized in that, The apparatus for performing the method as described in any one of claims 1-10 includes: The acquisition module is used to acquire the calibration output data of the neural network; The quantization range module is used to select quantization boundary values ​​based on the probability distribution information of the calibration output data, and to determine the target quantization range of the neural network. The quantization module is used to quantize the output of the neural network according to the target quantization range.

12. A quantization system for neural network data, characterized in that, include: At least one processor; And a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to perform: the method as described in any one of claims 1-10.

13. A computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to perform the method as described in any one of claims 1-10.

Images