Neural network quantization, processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a neural network quantization method and device, electronic equipment and storage medium, the method comprising: obtaining a neural network to be quantized; for any network processing layer to be quantized in the neural network to be quantized, determining a target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and a quantization error expression, wherein the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data; so as to quantize the operation feature data of the network processing layer in the neural network to be quantized based on the target segmentation point corresponding to the network processing layer to be quantized in the inference process of the neural network to be quantized.

Description

Technical Field

[0001] This disclosure relates to the field of deep learning technology, and more specifically, to a neural network quantization, processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, deep neural networks (DNNs) have been widely used in computer vision fields such as image classification and object detection. Deeper or wider network structures offer higher accuracy, but at the cost of higher computational complexity and increased memory requirements, limiting their application on resource-constrained embedded devices.

[0003] Generally, a feasible approach to deploying deep neural networks on embedded devices is to quantize the weights and activations of the full-precision network, thereby reducing the number of discrete values ​​by decreasing the bit width required to store the data. Therefore, it is particularly important to propose a high-precision quantization method. Summary of the Invention

[0004] In view of this, the present disclosure provides at least one neural network quantization, processing method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, this disclosure provides a neural network quantization method, including:

[0006] Obtain the neural network to be quantized;

[0007] For any network processing layer in the neural network to be quantized, a target segmentation point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data of the network processing layer to be quantized and the quantization error expression; wherein, the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data; so that during the inference process of the neural network to be quantized, the neural network to be quantized is quantized based on the target segmentation point corresponding to the network processing layer to be quantized.

[0008] In the above method, after obtaining the neural network to be quantized, for any network processing layer in the neural network to be quantized, the target segment point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data and the quantization error expression of the network processing layer to be quantized. Since the quantization error expression can be used to determine the error between the feature data and the quantized data corresponding to the feature data, the error of the selected target segment point is small, making the determination of the target segment point more accurate. Therefore, based on the target segment point corresponding to the network processing layer to be quantized, the neural network to be quantized can be quantized more accurately, improving the quantization accuracy while ensuring the quantization rate.

[0009] In one possible implementation, determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression includes:

[0010] The initial iteration count is determined as the current iteration count, the target feature value is determined as the historical segmentation point, and the initialization error is determined as the error threshold. Based on the historical segmentation point and the total iteration count, the current segmentation point corresponding to the current iteration count is determined; wherein, the target feature value is the feature value with the largest absolute value in the feature data;

[0011] Based on the current segmentation point, the feature data, and the quantization error expression, determine the quantization error corresponding to the current segmentation point;

[0012] If the quantization error is less than the error threshold, increment the current iteration number by one to obtain the updated current iteration number; determine the quantization error as the updated error threshold, determine the current segment point as the updated historical segment point, and return to the step of determining the current segment point corresponding to the current iteration number based on the historical segment point and the total iteration number, until the updated current iteration number is greater than the total iteration number;

[0013] The historical segmentation points obtained after the last iteration are determined as the target segmentation points corresponding to the feature data of the network processing layer to be quantized.

[0014] In this embodiment of the disclosure, by using multiple iterations, the target segment point corresponding to the feature data of the network processing layer to be quantized can be determined more accurately based on the quantization error of the current segment point determined in each iteration.

[0015] In one possible implementation, the method further includes:

[0016] If the quantization error corresponding to the current segment point is greater than or equal to the error threshold, the current iteration number is incremented by one to obtain the updated current iteration number. Then, the process returns to the step of determining the current segment point corresponding to the current iteration number based on the historical segment points and the total number of iterations, until the updated current iteration number is greater than the total number of iterations.

[0017] The current segment point obtained after the last iteration is determined as the target segment point corresponding to the feature data of the network processing layer to be quantized.

[0018] In one possible implementation, determining the quantization error corresponding to the current segment point based on the current segment point, the feature data, and the quantization error expression includes:

[0019] Based on the current segmentation point and the set quantization expression, determine the quantized feature value corresponding to each feature value in the feature data;

[0020] Based on each feature value in the feature data, the quantized feature value corresponding to each feature value, and the quantization error expression, the quantization error corresponding to the current segment point is determined.

[0021] During implementation, based on the current segmentation point and the set quantization expression, the quantized feature value corresponding to each feature value in the feature data at the current segmentation point can be determined. Then, based on each feature value in the feature data, the quantized feature value corresponding to each feature value, and the quantization error expression, the quantization error corresponding to the current segmentation point can be determined more accurately, providing a basis for determining the target segmentation point in the future.

[0022] In one possible implementation, determining the quantized feature value corresponding to each feature value in the feature data based on the current segmentation point and the set quantization expression includes:

[0023] Based on the current segmentation point, a first segmentation interval and a second segmentation interval corresponding to the feature data are determined; wherein, the first segmentation interval includes: the interval between the negative value of the current segmentation point and the positive value of the current segmentation point; the second segmentation interval includes: the interval between the positive value of the current segmentation point and the positive value of the target feature value, and the interval between the negative value of the target feature value and the negative value of the current segmentation point;

[0024] For each feature value in the feature data, if the feature value is located in the first segment interval, the quantized feature value corresponding to the feature value is determined by using the first quantization expression corresponding to the first segment interval.

[0025] When the feature value is located in the second segmented interval, the quantized feature value corresponding to the feature value is determined by using the second quantization expression corresponding to the second segmented interval.

[0026] Here, based on the current segmentation point, the first segmentation interval and the second segmentation interval corresponding to the feature data are determined. Then, based on the quantization expressions corresponding to the first segmentation interval and the second segmentation interval respectively, the quantized feature value corresponding to each feature value is determined, providing data support for the subsequent determination of quantization error.

[0027] In one possible implementation, when the feature data includes input feature data and / or output feature data, before determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression, the method further includes:

[0028] Acquire calibration data, wherein the calibration data includes at least one calibration image;

[0029] For each calibration image, based on the calibration image and the neural network to be quantized, feature data corresponding to the network processing layer to be quantized that matches the calibration image is determined.

[0030] When the feature data includes input feature data and / or output feature data, since the input feature data and output feature data are related to the inference process of the neural network, in order to predetermine the target segmentation points corresponding to the input feature data and output feature data, calibration data can be obtained, and for each calibration image, based on the calibration image and the neural network to be quantized, feature data corresponding to the network processing layer to be quantized that matches the calibration image can be determined, so as to realize the determination of the target segmentation points based on the feature data corresponding to the calibration image.

[0031] In one possible implementation, determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression includes:

[0032] For each calibration image, based on the quantization error expression and the feature data corresponding to the network processing layer to be quantized that matches the calibration image, candidate segmentation points corresponding to the feature data of the calibration image are determined;

[0033] Based on the candidate segment points corresponding to the feature data of each of the calibration images, the target segment point corresponding to the network processing layer to be quantized is determined.

[0034] In this embodiment of the disclosure, when there are multiple calibration images, candidate segment points corresponding to each calibration image can be determined. Then, based on the candidate segment points corresponding to the feature data of each calibration image, the target segment point corresponding to the network processing layer to be quantized can be determined more accurately.

[0035] Secondly, this disclosure provides a neural network processing method, including:

[0036] Obtain the target segmentation point corresponding to at least one network processing layer in the neural network to be quantized;

[0037] Based on the target segment point corresponding to the network processing layer to be quantized, the feature data of the network processing layer to be quantized is quantized to obtain the quantized feature data.

[0038] Based on the quantized feature data, the inference result of the neural network to be quantized is determined.

[0039] The effects of the following devices, electronic equipment, etc., are described in the instructions above and will not be repeated here.

[0040] Thirdly, this disclosure provides a neural network quantization device, comprising:

[0041] The first acquisition module is used to acquire the neural network to be quantized;

[0042] The first determining module is configured to, for any network processing layer in the neural network to be quantized, determine the target segment point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression; wherein, the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data; so that during the inference process of the neural network to be quantized, the neural network to be quantized is quantized based on the target segment point corresponding to the network processing layer to be quantized.

[0043] In one possible implementation, the first determining module, when determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data and quantization error expression of the network processing layer to be quantized, is used to:

[0044] The initial iteration count is determined as the current iteration count, the target feature value is determined as the historical segmentation point, and the initialization error is determined as the error threshold. Based on the historical segmentation point and the total iteration count, the current segmentation point corresponding to the current iteration count is determined; wherein, the target feature value is the feature value with the largest absolute value in the feature data;

[0045] Based on the current segmentation point, the feature data, and the quantization error expression, determine the quantization error corresponding to the current segmentation point;

[0046] If the quantization error is less than the error threshold, increment the current iteration number by one to obtain the updated current iteration number; determine the quantization error as the updated error threshold, determine the current segment point as the updated historical segment point, and return to the step of determining the current segment point corresponding to the current iteration number based on the historical segment point and the total iteration number, until the updated current iteration number is greater than the total iteration number;

[0047] The historical segmentation points obtained after the last iteration are determined as the target segmentation points corresponding to the feature data of the network processing layer to be quantized.

[0048] In one possible implementation, the first determining module is further configured to:

[0049] If the quantization error corresponding to the current segment point is greater than or equal to the error threshold, the current iteration number is incremented by one to obtain the updated current iteration number. Then, the process returns to the step of determining the current segment point corresponding to the current iteration number based on the historical segment points and the total number of iterations, until the updated current iteration number is greater than the total number of iterations.

[0050] The current segment point obtained after the last iteration is determined as the target segment point corresponding to the feature data of the network processing layer to be quantized.

[0051] In one possible implementation, the first determining module, when determining the quantization error corresponding to the current segment point based on the current segment point, the feature data, and the quantization error expression, is configured to:

[0052] Based on the current segmentation point and the set quantization expression, determine the quantized feature value corresponding to each feature value in the feature data;

[0053] Based on each feature value in the feature data, the quantized feature value corresponding to each feature value, and the quantization error expression, the quantization error corresponding to the current segment point is determined.

[0054] In one possible implementation, the first determining module, when determining the quantized feature value corresponding to each feature value in the feature data based on the current segmentation point and the set quantization expression, is used to:

[0055] Based on the current segmentation point, a first segmentation interval and a second segmentation interval corresponding to the feature data are determined; wherein, the first segmentation interval includes: the interval between the negative value of the current segmentation point and the positive value of the current segmentation point; the second segmentation interval includes: the interval between the positive value of the current segmentation point and the positive value of the target feature value, and the interval between the negative value of the target feature value and the negative value of the current segmentation point;

[0056] For each feature value in the feature data, if the feature value is located in the first segment interval, the quantized feature value corresponding to the feature value is determined by using the first quantization expression corresponding to the first segment interval.

[0057] When the feature value is located in the second segmented interval, the quantized feature value corresponding to the feature value is determined by using the second quantization expression corresponding to the second segmented interval.

[0058] In one possible implementation, when the feature data includes input feature data and / or output feature data, before determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression, the method further includes: an extraction module, configured to:

[0059] Acquire calibration data, wherein the calibration data includes at least one calibration image;

[0060] For each calibration image, based on the calibration image and the neural network to be quantized, feature data corresponding to the network processing layer to be quantized that matches the calibration image is determined.

[0061] In one possible implementation, the first determining module, when determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data and quantization error expression of the network processing layer to be quantized, is used to:

[0062] For each calibration image, based on the quantization error expression and the feature data corresponding to the network processing layer to be quantized that matches the calibration image, candidate segmentation points corresponding to the feature data of the calibration image are determined;

[0063] Based on the candidate segment points corresponding to the feature data of each of the calibration images, the target segment point corresponding to the network processing layer to be quantized is determined.

[0064] Fourthly, this disclosure provides a neural network processing apparatus, comprising:

[0065] The second acquisition module is used to acquire the target segmentation point corresponding to at least one network processing layer in the neural network to be quantized.

[0066] The quantization module is used to quantize the feature data of the network processing layer to be quantized based on the target segmentation point corresponding to the network processing layer to be quantized, so as to obtain the quantized feature data.

[0067] The second determining module is used to determine the inference result of the neural network to be quantized based on the quantized feature data.

[0068] Fifthly, this disclosure provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, they perform the steps of the neural network quantization method as described in the first aspect or any embodiment above, or perform the steps of the neural network processing method as described in the second aspect above.

[0069] In a sixth aspect, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the neural network quantization method as described in the first aspect or any embodiment above, or performs the steps of the neural network processing method as described in the second aspect above.

[0070] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0071] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0072] Figure 1 A flowchart illustrating a neural network quantization method provided in an embodiment of this disclosure is shown.

[0073] Figure 2 A flowchart illustrating a neural network processing method provided in an embodiment of this disclosure is shown.

[0074] Figure 3 A schematic diagram of the architecture of a neural network quantization device provided in an embodiment of this disclosure is shown;

[0075] Figure 4 A schematic diagram of the architecture of a neural network processing device provided in an embodiment of this disclosure is shown;

[0076] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0078] One feasible approach to deploying deep neural networks on embedded devices is to quantize the weights and activations of a full-precision network, thereby reducing the number of discrete values ​​by decreasing the bit width required to store the data.

[0079] In studying convolutional neural networks (CNNs) with various tasks and structures, it has been found that the weight features or intermediate features of convolutional layers (such as input and output features of network processing layers) exhibit a bell-shaped distribution. This means that there are more feature values ​​near the value 0 and fewer feature values ​​at the two boundaries. Linear quantization functions, however, provide a uniform quantization interval across all ranges, making them generally inadequate for describing this bell-shaped distribution. To describe this bell-shaped feature distribution, a non-linear quantization function has been proposed for quantizing CNNs. However, this non-linear quantization method requires a large number of quantization parameters and consumes significant computational resources, making it unsuitable for hardware.

[0080] In another approach, piecewise linear quantization can be used. This piecewise quantization method only requires adding a hyperparameter for the segmentation point location compared to linear quantization. In this method, determining the segmentation point location is crucial to the quantization accuracy. Generally, the parameter distribution of the convolutional neural network can be assumed to be Gaussian or Laplace, and the optimal segmentation point location can be derived using the distribution characteristics. However, the above method of determining the segmentation point makes strong prior assumptions about the neural network distribution. In practice, the parameter distribution of each network and each layer of the network is different. Methods based on assumed distributions make it difficult to guarantee quantization accuracy.

[0081] To alleviate the above problems, embodiments of this disclosure provide a neural network quantization method, apparatus, electronic device, and storage medium. In this method, for each network processing layer, a target segment point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data of the network processing layer to be quantized and the quantization error expression. The quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data. For example, a segment point with a smaller error can be selected as the target segment point. The determination of the target segment point is more accurate. Therefore, based on the target segment point corresponding to the network processing layer to be quantized, the neural network to be quantized can be quantized more accurately, thus improving the quantization accuracy.

[0082] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0083] To facilitate understanding of the embodiments of this disclosure, a detailed description of a neural network quantization method disclosed in this disclosure will be provided first. The execution entity of the neural network quantization method provided in this disclosure is generally a computer device with a certain computing capability. This computer device may include, for example, a terminal device, a server, or other processing device. The terminal device may be a user equipment (UE), a mobile device, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, this neural network quantization method can be implemented by a processor calling computer-readable instructions stored in memory.

[0084] See Figure 1 The diagram shown is a flowchart of a neural network quantization method provided in this embodiment of the present disclosure. The method includes steps S101-S102, wherein:

[0085] S101, Obtain the neural network to be quantized;

[0086] S102, for any network processing layer in the neural network to be quantized, determine the target segment point corresponding to the feature data of the network processing layer to be quantized based on the feature data and the quantization error expression of the network processing layer to be quantized; wherein, the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data; so that during the inference process of the neural network to be quantized, the neural network to be quantized is quantized based on the target segment point corresponding to the network processing layer to be quantized.

[0087] In the above method, after obtaining the neural network to be quantized, for any network processing layer in the neural network to be quantized, the target segment point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data and the quantization error expression of the network processing layer to be quantized. Since the quantization error expression can be used to determine the error between the feature data and the quantized data corresponding to the feature data, the error of the selected target segment point is small, making the determination of the target segment point more accurate. Therefore, based on the target segment point corresponding to the network processing layer to be quantized, the neural network to be quantized (e.g., the operational feature data of the network processing layer in the neural network to be quantized) can be quantized more accurately, improving the quantization accuracy while ensuring the quantization rate.

[0088] The following provides a detailed explanation of S101 and S102.

[0089] Regarding S101:

[0090] Here, the neural network to be quantized can be any trained neural network. For example, the neural network to be quantized can be a neural network used for semantic segmentation tasks, a neural network used for classification tasks, a neural network used for regression tasks, etc.

[0091] The network structure of the neural network to be quantized can be set as needed. For example, the neural network to be quantized may include convolutional processing layers, fully connected processing layers, pooling processing layers, activation processing layers, etc.

[0092] Regarding S102:

[0093] After obtaining the neural network to be quantized, the network processing layers within that neural network can be determined. For example, each network processing layer in the neural network can be identified as the network processing layer to be quantized; alternatively, a subset of the network processing layers can also be identified as the network processing layers to be quantized. The network processing layers to be quantized can be configured as needed.

[0094] After determining the network processing layer to be quantized, for any network processing layer to be quantized, the target segmentation point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data of the network processing layer to be quantized and the quantization error expression; wherein, the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data.

[0095] In one implementation, multiple initial segmentation points can be determined first, and then target segmentation points can be selected from these initial segmentation points based on the quantization error expression. For example, firstly, multiple initial segmentation points can be determined from the feature intervals where each feature value of the feature data is located. For instance, the feature value with the largest absolute value among all feature values ​​of the feature data can be determined, and the feature interval can be determined based on this feature value; assuming the feature value with the largest absolute value is m, then the feature interval can be [-m, m]. Then, multiple initial segmentation points are determined from the feature interval. For example, a preset number of initial segmentation points can be selected uniformly within the feature interval, or a preset number of initial segmentation points can be selected non-uniformly within the feature interval. For example, fewer initial segmentation points can be selected at the boundary positions of the feature interval (close to m and -m), and more initial segmentation points can be selected at the middle position of the feature interval (close to 0).

[0096] Secondly, for each initial segmentation point, the initial quantization error corresponding to that initial segmentation point is determined based on the initial segmentation point, the feature data, and the quantization error expression. In practice, the feature data can be linearly quantized based on the initial segmentation point to obtain the quantized data corresponding to that initial segmentation point; then, using the quantization error expression, the quantization error between the feature data and the quantized data is determined, thus obtaining the initial quantization error corresponding to that initial segmentation point. Therefore, the initial quantization error corresponding to each initial segmentation point can be obtained.

[0097] Furthermore, based on the initial quantization error corresponding to each initial segmentation point, the target segmentation point corresponding to the feature data of the network processing layer to be quantized is determined. For example, the initial segmentation point with the smallest initial quantization error can be selected as the target segmentation point corresponding to the feature data of the network processing layer to be quantized.

[0098] In another implementation, the target segmentation point corresponding to the feature data of the network processing layer to be quantized can be determined iteratively based on the feature data of the network processing layer to be quantized and the quantization error expression.

[0099] In S102, based on the feature data of the network processing layer to be quantized and the quantization error expression, the target segmentation points corresponding to the feature data of the network processing layer to be quantized are determined, specifically including:

[0100] Step A1: Determine the initial iteration count as the current iteration count, the target feature value as the historical segmentation point, and the initialization error as the error threshold. Based on the historical segmentation point and the total iteration count, determine the current segmentation point corresponding to the current iteration count. The target feature value is the feature value with the largest absolute value in the feature data.

[0101] Step A2: Based on the current segment point, feature data, and quantization error expression, determine the quantization error corresponding to the current segment point.

[0102] Step A3: If the quantization error is less than the error threshold, increment the current iteration number by one to obtain the updated current iteration number; determine the quantization error as the updated error threshold, determine the current segment point as the updated historical segment point, and return to the step of determining the current segment point corresponding to the current iteration number based on the historical segment point and the total iteration number, until the updated current iteration number is greater than the total iteration number.

[0103] Step A4: If the quantization error corresponding to the current segment point is greater than or equal to the error threshold, increment the current iteration number by one to obtain the updated current iteration number. Return to the step of determining the current segment point corresponding to the current iteration number based on the historical segment points and the total number of iterations, until the updated current iteration number is greater than the total number of iterations.

[0104] Step A5: Determine the historical segmentation points obtained after the last iteration as the target segmentation points corresponding to the feature data of the network processing layer to be quantized.

[0105] In step A1, the initial iteration count is set; for example, the initial iteration count can be the first iteration. An initialization error is set; for example, the initialization error can be a preset value (such as 1000). Then, from the various feature values ​​included in the feature data, the target feature value is determined, that is, the feature value with the largest absolute value is determined as the target feature value. For example, the largest and smallest feature values ​​among the various feature values ​​included in the feature data can be determined first, the first absolute value corresponding to the largest feature value and the second absolute value corresponding to the smallest feature value can be determined, and the maximum value between the first and second absolute values ​​is determined as the target feature value.

[0106] The initial iteration count can be set as the current iteration count, the target feature value can be set as the historical segmentation point, the initialization error can be set as the error threshold, and the iterative processing corresponding to the current iteration count can be performed.

[0107] In implementation, the current segment point corresponding to the current iteration number can be determined based on historical segment points and the total number of iterations. For example, the current segment point corresponding to the current iteration number can be determined according to the following formula (1):

[0108]

[0109] Where n is the total number of iterations, i is the current iteration number, and α i Let α be the current segmentation point. i-1 These are historical segmentation points.

[0110] In step A2, after obtaining the current segment point corresponding to the current iteration number, the feature data is linearly quantized based on the current segment point to obtain the quantized data corresponding to the feature data at the current segment point; then, using the quantization error expression, the quantization error corresponding to the current segment point is determined based on the feature data and the quantized data.

[0111] In one optional implementation, the quantization error corresponding to the current segment point is determined based on the current segment point, feature data, and quantization error expression, specifically including:

[0112] Step A21: Based on the current segmentation point and the set quantization expression, determine the quantized feature value corresponding to each feature value in the feature data.

[0113] Step A22: Based on each feature value in the feature data, the quantized feature value corresponding to each feature value, and the quantization error expression, determine the quantization error corresponding to the current segment point.

[0114] During implementation, based on the current segmentation point and the set quantization expression, the quantized feature value corresponding to each feature value in the feature data at the current segmentation point can be determined. Then, based on each feature value in the feature data, the quantized feature value corresponding to each feature value, and the quantization error expression, the quantization error corresponding to the current segmentation point can be determined more accurately, providing a basis for determining the target segmentation point in the future.

[0115] In step A21, the quantization expression is a linear quantization expression. After obtaining the current segment point, the target expression corresponding to the current segment point can be determined based on the set quantization expression; each feature value in the feature data is input into the target expression to obtain the quantized feature value corresponding to that feature value; the quantized data corresponding to the feature data is constructed based on each quantized feature value.

[0116] In one optional implementation, based on the current segmentation point and the set quantization expression, the quantized feature value corresponding to each feature value in the feature data is determined, including:

[0117] Step 1: Based on the current segmentation point, determine the first segmentation interval and the second segmentation interval corresponding to the feature data; wherein, the first segmentation interval includes: the interval between the negative value of the current segmentation point and the positive value of the current segmentation point; the second segmentation interval includes: the interval between the positive value of the current segmentation point and the positive value of the target feature value, and the interval between the negative value of the target feature value and the negative value of the current segmentation point.

[0118] Step 2: For each feature value in the feature data, if the feature value is located in the first segment interval, use the first quantization expression corresponding to the first segment interval to determine the quantized feature value corresponding to the feature value.

[0119] Step 3: When the feature value is located in the second segment interval, use the second quantization expression corresponding to the second segment interval to determine the quantized feature value corresponding to the feature value.

[0120] If the current segmentation point is α, then the first segmentation interval corresponding to the feature data can include [-α, α], and the second segmentation interval can include: [-m, -α) ∪ (α, m). Where m is the target feature value.

[0121] After obtaining the first segment interval and the second segment interval, the first quantization expression corresponding to the first segment interval and the second quantization expression corresponding to the second segment interval can be determined based on the set quantization expression. The first quantization expression and the second quantization expression are shown in the following formula (2):

[0122]

[0123] Where p(x) is the quantized eigenvalue; sign() represents the sign function, and uni() represents the linear quantization function. The linear quantization function is shown in formula (3) below:

[0124]

[0125]

[0126] Here, `round()` is the rounding function, and `b` represents the quantization bit width. h The maximum value within the segmented interval is x, which is defined in formula (2). h Let α, x l For the minimum value within the segmented interval, that is, in formula (2), x l z is 0. z is the zero point, that is, z is 0 in formula (2). x is the eigenvalue in the eigendata, which corresponds to |x| in formula (2).

[0127] Furthermore, since the purpose of quantization is to reduce computation time and energy consumption, but this computation is implemented in hardware, quantization is hardware-dependent. Therefore, the quantization bit width can be determined based on the integrated circuit (chip) running the neural network to be quantized. As a hardware-constrained device, the precision of the data types supported by the chip is often lower than that of non-hardware-constrained devices (such as servers). For example, when the chip supports the integer int8 data type (i.e., the data type of the quantized feature value is int8), in order to ensure that the quantized feature value is within the data range of the data type supported by the chip, such as int8, it is necessary to use a scaling factor s to scale the value obtained by the rounding function. Therefore, the value of b is related to the data type; thus, when the data type is int8, the value of b can be set to 8.

[0128] For each feature value in the feature data, when the feature value is located in the first segment interval (i.e., the feature value is in [-α, α]), the first quantization expression is used to determine the quantized feature value corresponding to the feature value. When the feature value is located in the second segment interval (i.e., the feature value is in [-m, -α)∪(α, m]), the second quantization expression is used to determine the quantized feature value corresponding to the feature value.

[0129] Here, based on the current segmentation point, the first segmentation interval and the second segmentation interval corresponding to the feature data are determined. Then, based on the quantization expressions corresponding to the first segmentation interval and the second segmentation interval respectively, the quantized feature value corresponding to each feature value is determined, providing data support for the subsequent determination of quantization error.

[0130] In step A22, the quantization error is determined using the quantization error expression, based on each feature value in the feature data and the quantized feature value corresponding to each feature value. The quantization error expression is shown in formula (4) below:

[0131]

[0132] Where ω is the feature value located at position x in the feature data. p is the quantized eigenvalue corresponding to this eigenvalue, where p is the norm, such as 1, 2, etc.

[0133] After obtaining the quantization error corresponding to the current segment, compare the magnitude of this quantization error with the current error threshold. If the quantization error corresponding to the current segment is less than the error threshold, proceed to step A3; if the quantization error corresponding to the current segment is greater than or equal to the error threshold, proceed to step A4.

[0134] In step A3, if the quantization error is less than the error threshold, the current iteration number is incremented by one to obtain the updated current iteration number; for example, if the current iteration number is the first iteration (i.e., i = 1), then the updated current iteration number is the second iteration (i.e., i = 2).

[0135] The quantization error is determined as the updated error threshold, which is the error threshold corresponding to the updated current iteration number. The current segment point is also determined as the updated historical segment point, meaning the updated historical segment point is the historical segment point corresponding to the updated current iteration number. The process then returns to the step of determining the current segment point corresponding to the current iteration number based on the historical segment point and the total number of iterations, until the updated current iteration number is greater than the total number of iterations.

[0136] In step A4, if the quantization error is greater than or equal to the error threshold, the current iteration number is incremented by one to obtain the updated current iteration number. The error threshold and historical segment points are not updated; the process directly returns to the step of determining the current segment point corresponding to the current iteration number based on the historical segment points and the total number of iterations, until the updated current iteration number is greater than the total number of iterations.

[0137] In step A5, the historical segmentation points obtained after the last iteration can be determined as the target segmentation points corresponding to the feature data of the network processing layer to be quantized.

[0138] For example, if the total number of iterations is 100, after obtaining the current segment point corresponding to the 100th iteration, the quantization error of this current segment point is determined. If the quantization error is less than the error threshold corresponding to the 100th iteration, the current segment point corresponding to the 100th iteration is determined as the historical segment point obtained after the last iteration. If the quantization error is greater than or equal to the error threshold corresponding to the 100th iteration, the historical segment point used in the 100th iteration is determined as the historical segment point obtained after the last iteration. That is, through the above 100 iterations, the target segment point with the minimum quantization error is determined, and this target segment point is the selected optimal segment point.

[0139] In this embodiment of the disclosure, by using multiple iterations, the target segment point corresponding to the feature data of the network processing layer to be quantized can be determined more accurately based on the quantization error of the current segment point determined in each iteration.

[0140] In practice, the feature data of the network processing layer to be quantized includes at least one of weight feature data, input feature data, and output feature data.

[0141] When the feature data includes weight feature data, the weight feature data of the network processing layer can be obtained, and based on this weight feature data and the quantization error expression, the target segment point corresponding to the weight feature data of the network processing layer to be quantized can be determined. The determination process can refer to steps A1 to A5, and will not be detailed here.

[0142] When the feature data includes input feature data and / or output feature data, before determining the target segment point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression, the method further includes: acquiring calibration data, wherein the calibration data includes at least one calibration image; and for each calibration image, determining the feature data corresponding to the network processing layer to be quantized that matches the calibration image based on the calibration image and the neural network to be quantized.

[0143] Calibration data can be selected from the test samples corresponding to the neural network to be quantized. This calibration data includes at least one calibration image. Each calibration image is input into the neural network to be quantized, and each processing layer of the neural network extracts features from the calibration image, obtaining input and output feature data corresponding to each processing layer that matches the calibration image. This yields the feature data corresponding to the processing layer to be quantized that matches the calibration image.

[0144] When the feature data includes input feature data and / or output feature data, since the input feature data and output feature data are related to the inference process of the neural network, in order to predetermine the target segmentation points corresponding to the input feature data and output feature data, calibration data can be obtained, and for each calibration image, based on the calibration image and the neural network to be quantized, feature data corresponding to the network processing layer to be quantized that matches the calibration image can be determined, so as to realize the determination of the target segmentation points based on the feature data corresponding to the calibration image.

[0145] In one optional implementation, the target segmentation point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data of the network processing layer to be quantized and the quantization error expression, including:

[0146] For each calibration image, based on the quantization error expression and the feature data of the network processing layer to be quantized that matches the calibration image, candidate segment points corresponding to the feature data of the calibration image are determined; based on the candidate segment points corresponding to the feature data of each calibration image, the target segment point corresponding to the network processing layer to be quantized is determined.

[0147] Taking output feature data as an example, for each calibration image, the output feature data corresponding to the network processing layer to be quantized and matching the calibration image are determined. Then, based on the quantization error expression and the output feature data corresponding to the network processing layer to be quantized and matching the calibration image, candidate segmentation points corresponding to the output feature data of the calibration image are determined.

[0148] For example, for the output feature data of each calibration image at the network processing layer, we can first determine the output feature range of the output feature data, and select multiple output segment points from this range. Then, for each output segment point, we perform linear quantization on the output feature data based on the segment point (e.g., using formulas 2 and 3) to obtain the quantized output data corresponding to that segment point. Then, using the quantization error expression (e.g., formula 4), we determine the quantization error between the output feature data and the quantized output data, obtaining the initial quantization error corresponding to that segment point. Therefore, we can obtain the initial quantization error corresponding to each output segment point. Based on the initial quantization error, we then determine candidate segment points from the multiple output segment points; for example, we select the output segment point with the smallest initial quantization error as a candidate segment point.

[0149] For example, candidate segmentation points can be determined through multiple iterations. Specifically, this can include: Step 1, determining the current iteration number as the number of the first iteration, determining the feature value with the largest absolute value in the output feature data as the target output feature value, and determining this target output feature value as a historical segmentation point, determining the initialization error as the error threshold, and determining the current output segmentation point corresponding to the current iteration number based on the historical segmentation points and the total number of iterations. Step 2, determining the output quantization error corresponding to the current output segmentation point based on the current output segmentation point, the output feature data, and the quantization error expression.

[0150] Step 3: If the output quantization error is less than the error threshold, increment the current iteration number to obtain the updated current iteration number. Set the output quantization error as the updated error threshold, and the current output segment point as the updated historical segment point. Return to the step of determining the current output segment point corresponding to the current iteration number based on the historical segment point and the total iteration number, until the updated current iteration number is greater than the total iteration number. Step 4: If the output quantization error corresponding to the current output segment point is greater than or equal to the error threshold, increment the current iteration number to obtain the updated current iteration number. Return to the step of determining the current output segment point corresponding to the current iteration number based on the historical segment point and the total iteration number, until the updated current iteration number is greater than the total iteration number. Step 5: Determine the historical segment point obtained after the last iteration as the candidate segment point corresponding to the output feature data of the network processing layer to be quantized.

[0151] The process of determining candidate segmentation points (i.e., steps 1 to 5 above) can be referred to in the specific description of steps A1 to A5, and will not be detailed here.

[0152] Furthermore, based on the candidate segment points corresponding to the output feature data of each calibration image, the target segment point corresponding to the output feature data of the network processing layer to be quantized can be determined. For example, the target segment point can be obtained by averaging the candidate segment points; or, a candidate segment point can be randomly selected from multiple candidate segment points as the target segment point; or, multiple candidate segment points can be clustered to obtain at least one segment point set (each segment point set includes at least one candidate segment point), then the target segment point set with the largest number of candidate segment points can be determined, the average value of multiple candidate segment points in the target segment point set can be determined, and this average value can be determined as the target segment point.

[0153] In this embodiment of the disclosure, when there are multiple calibration images, candidate segment points corresponding to each calibration image can be determined. Then, based on the candidate segment points corresponding to the feature data of each calibration image, the target segment point corresponding to the network processing layer to be quantized can be determined more accurately.

[0154] After the execution entity, such as a server or terminal device, determines the target segmentation point of the feature data, if the feature data includes weight feature data, the target segmentation point corresponding to the weight feature data of the network processing layer to be quantized can be obtained. Based on the target segmentation point corresponding to the weight feature data of the network processing layer to be quantized, the weight feature data of the network processing layer of the neural network to be quantized is quantized to obtain the quantized weight feature data. In the inference process of the neural network, the quantized weight feature data is used for data processing to obtain the output feature data of the network processing layer. The quantization process of the weight feature data can be performed in real time during the inference process of the neural network, or it can be performed in advance before the inference process of the neural network. Among them, formulas (2) and (3) can be used to quantize the weight feature data of the network processing layer of the neural network to be quantized. Here, α in formula (2) is the target segmentation point.

[0155] If the feature data includes the input feature data, then the target segment point corresponding to the input feature data of the network processing layer to be quantized can be obtained. Based on the target segment point corresponding to the input feature data of the network processing layer to be quantized, the input feature data of the network processing layer of the neural network to be quantized is quantized to obtain the quantized input feature data. In the inference process of the neural network, the quantized input feature data can be used for data processing to obtain the output feature data of the network processing layer. The quantization process of the input feature data can be performed in real time during the inference process of the neural network. Among them, formulas (2) and (3) can be used to quantize the input feature data of the network processing layer of the neural network to be quantized. Here, α in formula (2) is the target segment point.

[0156] If the feature data includes weight feature data and input feature data, then the target segment point 1 corresponding to the weight feature data of the network processing layer to be quantized and the target segment point 2 corresponding to the input feature data can be obtained. Based on the target segment point 1 and target segment point 2 corresponding to the network processing layer to be quantized, the weight feature data and input feature data used by the network processing layer during the inference process of the neural network to be quantized are quantized respectively to obtain quantized weight feature data and quantized input feature data. Then, feature processing is performed using the quantized weight feature data and quantized input feature data. For example, if the network processing layer is a convolutional layer, then the quantized weight feature data and quantized input feature data are convolved to obtain the output feature data of the network processing layer.

[0157] In practice, after determining the target segmentation points of the feature data of the network processing layer to be quantized, such as the target segmentation points of the weight feature data and the target segmentation points of the output feature data, the inference process of the neural network to be quantized can be run on the chip (e.g., an artificial intelligence (AI) chip) based on the target segmentation points.

[0158] For example, during the inference process of a neural network to be quantized running on a chip, for the network processing layer of the neural network to be quantized, the chip can determine the target segmentation point of the feature data (such as weight feature data, input feature data, etc.) of that network processing layer, or it can receive the target segmentation point of the feature data of that network processing layer. Then, using the quantization operation module in the chip's processor that performs quantization processing, the chip performs linear quantization on the weight feature data based on the target segmentation point of the weight feature data to obtain the quantized weight feature data. For example, the data type of the weight feature data is quantized from floating-point number float32 to integer int8 to obtain the quantized weight feature data. Similarly, based on the target segmentation point of the input feature data, the chip performs linear quantization on the input feature data to obtain the quantized input feature data.

[0159] The feature computation module within the chip's processor, which executes the network processing layer's computation process, then performs operations on the quantized weight feature data and the quantized input feature data to obtain output feature data. Assume the data type of this output feature data is int8 (i.e., the quantized output feature data). This output feature data is then stored in memory, or it can be used as input feature data for the next network processing layer for subsequent processing. The quantization computation module can include the computational devices required to perform quantization processing, such as adders and multipliers. Similarly, the feature computation module can include the computational devices required to execute the network processing layer's computation process; for example, when the network processing layer is a convolutional layer, the feature computation module can include adders, multipliers, accumulators, etc.

[0160] By determining the target segmentation points of the feature data, the chip can accurately quantize the feature data, obtaining quantized feature data. While ensuring data accuracy, the reduced precision of the feature data allows for a smaller bit width required to store the quantized feature data on the chip. Furthermore, processing the quantized feature data on the chip reduces computational resources, power consumption, and computation speed, ultimately resulting in higher efficiency in the processing of the neural network to be quantized.

[0161] Based on the same concept, this disclosure also provides a neural network processing method, which can be applied to chips such as AI chips. The method includes steps S201-S203, wherein:

[0162] S201, Obtain the target segmentation point corresponding to at least one network processing layer in the neural network to be quantized;

[0163] S202, based on the target segmentation point corresponding to the network processing layer to be quantized, the feature data of the network processing layer to be quantized is quantized to obtain the quantized feature data;

[0164] S203, Based on the quantized feature data, determine the inference result of the neural network to be quantized.

[0165] Here, at least one target segment point of the network processing layer to be quantized in the neural network is obtained. This target segment point can be determined using the neural network quantization method described above. For example, it can include target segment points of the weight features of the network processing layer, target segment points of the input features, etc. Then, based on the corresponding target segment points of the network processing layer to be quantized, the feature data of the network processing layer to be quantized is quantized to obtain quantized feature data. For example, the weight feature data and the input feature data can be quantized to obtain quantized weight feature data and quantized input feature data.

[0166] Based on the quantized feature data, the inference result of the neural network to be quantized is determined. For example, the image to be detected can be input into the neural network to be quantized, and each processing layer in the neural network can detect the image to be detected, obtaining output feature data. Finally, the inference result can be determined based on the output feature data of the last processing layer in the neural network to be quantized; or, the output feature data of the last processing layer can be used as the inference result. For example, when the neural network to be quantized is a neural network used for face recognition, the inference result can be the face recognition result.

[0167] Here, by obtaining the target segment point of the network processing layer to be quantized, which is determined according to the neural network quantization method described above, the target segment point is relatively accurate. Based on this target segment point, the feature data of the network processing layer to be quantized can be quantized more accurately to obtain quantized feature data. Based on the quantized feature data, the inference result of the neural network to be quantized can be determined more accurately. While ensuring the inference accuracy, by reducing the data precision of the feature data, the processing efficiency of the execution entity running the neural network to be quantized is improved, and the computing resources are reduced.

[0168] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0169] Based on the same concept, this disclosure also provides a neural network quantization device, see [link to relevant documentation]. Figure 3 The diagram shown is an architectural schematic of a neural network quantization device provided in an embodiment of this disclosure, including a first acquisition module 301 and a first determination module 302. Specifically:

[0170] The first acquisition module 301 is used to acquire the neural network to be quantized;

[0171] The first determining module 302 is used to determine, for any network processing layer to be quantized in the neural network to be quantized, a target segmentation point corresponding to the feature data of the network processing layer to be quantized, based on the feature data of the network processing layer to be quantized and a quantization error expression; wherein, the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data; so that during the inference process of the neural network to be quantized, the computational feature data of the network processing layer in the neural network to be quantized is quantized based on the target segmentation point corresponding to the network processing layer to be quantized.

[0172] In one possible implementation, the first determining module 302, when determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data and quantization error expression of the network processing layer to be quantized, is used to:

[0173] The initial iteration count is determined as the current iteration count, the target feature value is determined as the historical segmentation point, and the initialization error is determined as the error threshold. Based on the historical segmentation point and the total iteration count, the current segmentation point corresponding to the current iteration count is determined; wherein, the target feature value is the feature value with the largest absolute value in the feature data;

[0174] Based on the current segmentation point, the feature data, and the quantization error expression, determine the quantization error corresponding to the current segmentation point;

[0175] If the quantization error is less than the error threshold, increment the current iteration number by one to obtain the updated current iteration number; determine the quantization error as the updated error threshold, determine the current segment point as the updated historical segment point, and return to the step of determining the current segment point corresponding to the current iteration number based on the historical segment point and the total iteration number, until the updated current iteration number is greater than the total iteration number;

[0176] The historical segmentation points obtained after the last iteration are determined as the target segmentation points corresponding to the feature data of the network processing layer to be quantized.

[0177] In one possible implementation, the first determining module 302 is further configured to:

[0178] If the quantization error corresponding to the current segment point is greater than or equal to the error threshold, the current iteration number is incremented by one to obtain the updated current iteration number. Then, the process returns to the step of determining the current segment point corresponding to the current iteration number based on the historical segment points and the total number of iterations, until the updated current iteration number is greater than the total number of iterations.

[0179] The current segment point obtained after the last iteration is determined as the target segment point corresponding to the feature data of the network processing layer to be quantized.

[0180] In one possible implementation, the first determining module 302, when determining the quantization error corresponding to the current segment point based on the current segment point, the feature data, and the quantization error expression, is used to:

[0181] Based on the current segmentation point and the set quantization expression, determine the quantized feature value corresponding to each feature value in the feature data;

[0182] Based on each feature value in the feature data, the quantized feature value corresponding to each feature value, and the quantization error expression, the quantization error corresponding to the current segment point is determined.

[0183] In one possible implementation, the first determining module 302, when determining the quantized feature value corresponding to each feature value in the feature data based on the current segmentation point and the set quantization expression, is used to:

[0184] Based on the current segmentation point, a first segmentation interval and a second segmentation interval corresponding to the feature data are determined; wherein, the first segmentation interval includes: the interval between the negative value of the current segmentation point and the positive value of the current segmentation point; the second segmentation interval includes: the interval between the positive value of the current segmentation point and the positive value of the target feature value, and the interval between the negative value of the target feature value and the negative value of the current segmentation point;

[0185] For each feature value in the feature data, if the feature value is located in the first segment interval, the quantized feature value corresponding to the feature value is determined by using the first quantization expression corresponding to the first segment interval.

[0186] When the feature value is located in the second segmented interval, the quantized feature value corresponding to the feature value is determined by using the second quantization expression corresponding to the second segmented interval.

[0187] In one possible implementation, when the feature data includes input feature data and / or output feature data, before determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression, the method further includes: an extraction module 303, configured to:

[0188] Acquire calibration data, wherein the calibration data includes at least one calibration image;

[0189] For each calibration image, based on the calibration image and the neural network to be quantized, feature data corresponding to the network processing layer to be quantized that matches the calibration image is determined.

[0190] In one possible implementation, the first determining module 302, when determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data and quantization error expression of the network processing layer to be quantized, is used to:

[0191] For each calibration image, based on the quantization error expression and the feature data corresponding to the network processing layer to be quantized that matches the calibration image, candidate segmentation points corresponding to the feature data of the calibration image are determined;

[0192] Based on the candidate segment points corresponding to the feature data of each of the calibration images, the target segment point corresponding to the network processing layer to be quantized is determined.

[0193] Based on the same concept, this disclosure also provides a neural network processing apparatus, see [link to relevant documentation]. Figure 4The diagram shown is an architectural schematic of a neural network processing device provided in an embodiment of this disclosure, including a second acquisition module 401, a quantization module 402, and a second determination module 403. Specifically:

[0194] The second acquisition module 401 is used to acquire the target segmentation point corresponding to at least one network processing layer to be quantized in the neural network to be quantized.

[0195] Quantization module 402 is used to quantize the feature data of the network processing layer to be quantized based on the target segment point corresponding to the network processing layer to be quantized, so as to obtain quantized feature data.

[0196] The second determining module 403 is used to determine the inference result of the neural network to be quantized based on the quantized feature data.

[0197] In some embodiments, the functions or templates of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0198] Based on the same technical concept, embodiments of this disclosure also provide an electronic device. (Refer to...) Figure 5 The diagram shows the structure of an electronic device provided in this embodiment, including a processor 501, a memory 502, and a bus 503. The memory 502 stores execution instructions and includes a main memory 5021 and an external memory 5022. The main memory 5021, also called internal memory, is used to temporarily store computational data in the processor 501 and data exchanged with external memory such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the main memory 5021. When the electronic device 500 is running, the processor 501 and the memory 502 communicate through the bus 503, causing the processor 501 to execute the following instructions:

[0199] Obtain the neural network to be quantized;

[0200] For any network processing layer in the neural network to be quantized, a target segmentation point corresponding to the feature data of the network processing layer to be quantized is determined based on the feature data of the network processing layer to be quantized and the quantization error expression; wherein, the quantization error expression is used to determine the error between the feature data and the quantized data corresponding to the feature data; so that during the inference process of the neural network to be quantized, the operational feature data of the network processing layer in the neural network to be quantized is quantized based on the target segmentation point corresponding to the network processing layer to be quantized.

[0201] Alternatively, processor 501 may execute the following instructions:

[0202] Obtain the target segmentation point corresponding to at least one network processing layer in the neural network to be quantized;

[0203] Based on the target segment point corresponding to the network processing layer to be quantized, the feature data of the network processing layer to be quantized is quantized to obtain the quantized feature data.

[0204] Based on the quantized feature data, the inference result of the neural network to be quantized is determined.

[0205] The specific processing flow of the processor 501 can be referred to the description in the above method embodiment, and will not be repeated here.

[0206] Furthermore, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the neural network quantization method and neural network processing method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0207] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the neural network quantization method and neural network processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0208] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0209] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0210] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0212] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0213] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0214] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A neural network quantization method, characterized in that, The method comprises the following steps: acquiring a neural network to be quantized; wherein the neural network to be quantized is a neural network used for performing a semantic segmentation task; for any network processing layer to be quantized in the neural network to be quantized, determining a target segmentation point corresponding to feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and a quantization error expression, wherein the quantization error expression is used to determine an error between the feature data and quantized data corresponding to the feature data; and the target segmentation point is used as input to a chip running the neural network to be quantized, so that the chip performs quantization processing on the neural network to be quantized based on the target segmentation point corresponding to the network processing layer to be quantized in an inference process of running the neural network to be quantized; the step of determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression comprises the following steps: determining a current iteration number as an initialization iteration number, a target feature value as a historical segmentation point, and an initialization error as an error threshold, and determining a current segmentation point corresponding to the current iteration number based on the historical segmentation point and a total iteration number; wherein the target feature value is a feature value with the largest absolute value in the feature data; determining a quantization error corresponding to the current segmentation point based on the current segmentation point, the feature data, and the quantization error expression; in a case where the quantization error is less than the error threshold, performing a plus one operation on the current iteration number to obtain an updated current iteration number, determining the quantization error as an updated error threshold, determining the current segmentation point as an updated historical segmentation point, and returning to the step of determining the current segmentation point corresponding to the current iteration number based on the historical segmentation point and the total iteration number until the updated current iteration number is greater than the total iteration number; determining the historical segmentation point obtained after the last iteration as the target segmentation point corresponding to the feature data of the network processing layer to be quantized; in a case where the quantization error corresponding to the current segmentation point is greater than or equal to the error threshold, performing a plus one operation on the current iteration number to obtain an updated current iteration number, and returning to the step of determining the current segmentation point corresponding to the current iteration number based on the historical segmentation point and the total iteration number until the updated current iteration number is greater than the total iteration number; determining the current segmentation point obtained after the last iteration as the target segmentation point corresponding to the feature data of the network processing layer to be quantized; in a case where the feature data comprises input feature data and / or output feature data, before the step of determining the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data of the network processing layer to be quantized and the quantization error expression, the method further comprises the following step: acquiring calibration data, wherein the calibration data comprises at least one calibration image. For each of the calibration images, based on the calibration image and the neural network to be quantized, determine the feature data corresponding to the calibration image matching the network processing layer to be quantized.

2. The method of claim 1, wherein, The determination of the quantization error corresponding to the current segmentation point based on the current segmentation point, the feature data and the quantization error expression comprises: Based on the current segmentation point and the set quantization expression, determine the quantized feature value corresponding to each feature value in the feature data; Based on each of the feature values in the feature data, the quantized feature value corresponding to each of the feature values, and the quantization error expression, determine the quantization error corresponding to the current segmentation point.

3. The method of claim 2, wherein, The determination of the quantized feature value corresponding to each feature value in the feature data based on the current segmentation point and the set quantization expression comprises: Based on the current segmentation point, determine the first and second segmentation intervals corresponding to the feature data; wherein the first segmentation interval comprises the interval between the negative value of the current segmentation point and the positive value of the current segmentation point; the second segmentation interval comprises the interval between the positive value of the current segmentation point and the positive value of the target feature value, and the interval between the negative value of the target feature value and the negative value of the current segmentation point; For each feature value in the feature data, if the feature value is in the first segmentation interval, determine the quantized feature value corresponding to the feature value using the first quantization expression corresponding to the first segmentation interval; If the feature value is in the second segmentation interval, determine the quantized feature value corresponding to the feature value using the second quantization expression corresponding to the second segmentation interval.

4. The method of claim 1, wherein, The determination of the target segmentation point corresponding to the feature data of the network processing layer to be quantized based on the feature data and the quantization error expression of the network processing layer to be quantized comprises: For each of the calibration images, based on the quantization error expression and the feature data corresponding to the calibration image matching the network processing layer to be quantized, determine the candidate segmentation point corresponding to the feature data of the calibration image; Based on the candidate segmentation points corresponding to the feature data of each of the calibration images, determine the target segmentation point corresponding to the network processing layer to be quantized.

5. A neural network processing method, comprising: Applied to a chip, comprising: Obtain the target segmentation point corresponding to at least one network processing layer to be quantized in the neural network to be quantized; wherein the neural network to be quantized is a neural network used for semantic segmentation task; Based on the target segmentation point corresponding to the network processing layer to be quantized, quantize the feature data of the network processing layer to be quantized to obtain quantized feature data, wherein the quantization process is realized by using the neural network quantization method of any one of claims 1-4; Based on the quantized feature data, determine the inference result of the neural network to be quantized.

6. A neural network quantization apparatus, characterized by comprising: Comprise: The first acquisition module is used for acquiring the neural network to be quantized; Wherein the neural network to be quantized is a neural network used for semantic segmentation task; The first determining module is configured to determine, for any to-be-quantized network processing layer in the to-be-quantized neural network, a target segmentation point corresponding to feature data of the to-be-quantized network processing layer based on the feature data of the to-be-quantized network processing layer and a quantization error expression; the quantization error expression is used to determine an error between the feature data and quantized data corresponding to the feature data; and the target segmentation point is used as an input to a chip running the to-be-quantized neural network, so that the chip performs quantization processing on the to-be-quantized neural network based on the target segmentation point corresponding to the to-be-quantized network processing layer during an inference process of running the to-be-quantized neural network. The first determining module is configured to, when determining the target segmentation point corresponding to the feature data of the to-be-quantized network processing layer based on the feature data of the to-be-quantized network processing layer and the quantization error expression: determine an initial iteration number as a current iteration number, determine a target feature value as a historical segmentation point, determine an initial error as an error threshold, and determine a current segmentation point corresponding to the current iteration number based on the historical segmentation point and a total iteration number; the target feature value is a feature value with the largest absolute value in the feature data; determine a quantization error corresponding to the current segmentation point based on the current segmentation point, the feature data, and the quantization error expression; in a case where the quantization error is less than the error threshold, perform a plus-one operation on the current iteration number to obtain an updated current iteration number, determine the quantization error as an updated error threshold, determine the current segmentation point as an updated historical segmentation point, and return to the step of determining the current segmentation point corresponding to the current iteration number based on the historical segmentation point and the total iteration number until the updated current iteration number is greater than the total iteration number; determine the historical segmentation point obtained after the last iteration as the target segmentation point corresponding to the feature data of the to-be-quantized network processing layer; in a case where the quantization error corresponding to the current segmentation point is greater than or equal to the error threshold, perform a plus-one operation on the current iteration number to obtain an updated current iteration number, and return to the step of determining the current segmentation point corresponding to the current iteration number based on the historical segmentation point and the total iteration number until the updated current iteration number is greater than the total iteration number; determine the current segmentation point obtained after the last iteration as the target segmentation point corresponding to the feature data of the to-be-quantized network processing layer; in a case where the feature data includes input feature data and / or output feature data, before the step of determining the target segmentation point corresponding to the feature data of the to-be-quantized network processing layer based on the feature data of the to-be-quantized network processing layer and the quantization error expression, the method further includes the following step of a extracting module: obtain calibration data, wherein the calibration data includes at least one calibration image; For each of the calibration images, based on the calibration image and the neural network to be quantized, determine the feature data corresponding to the network processing layer to be quantized and matching the calibration image.

7. A neural network processing apparatus, comprising: Applied to a chip, comprising: A second acquisition module is configured to acquire a target segmentation point corresponding to at least one network processing layer to be quantized in a neural network to be quantized; wherein the neural network to be quantized is a neural network used for semantic segmentation task; A quantization module is configured to perform quantization processing on the feature data of the network processing layer to be quantized based on the target segmentation point corresponding to the network processing layer to be quantized, to obtain quantized feature data, wherein the quantization processing process is realized by using the neural network quantization method of any one of claims 1-4; A second determination module is configured to determine the inference result of the neural network to be quantized based on the quantized feature data.

8. An electronic device, comprising: Comprising: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the steps of the neural network quantization method of any one of claims 1-4; or execute the steps of the neural network processing method of claim 5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is executed by the processor to execute the steps of the neural network quantization method of any one of claims 1-4; or execute the steps of the neural network processing method of claim 5.

Images