Lightweight neural network accelerator and quantization method

Abstract

The application discloses a kind of lightweight neural network accelerator and quantification method, lightweight neural network accelerator, comprising: instruction scheduling module, convolution calculation module, wherein;The instruction scheduling module is connected with the convolution calculation module, for sending operation instruction to the convolution calculation module and executing convolution calculation to input data;The convolution calculation module is used to receive operation instruction, and according to operation instruction, convolution calculation is executed to the input data, and convolution inference result is output;Wherein, the convolution calculation module includes shifter, each convolution layer of the convolution calculation module executes convolution calculation to the input data, and 2 power n shift is carried out through the shifter, n is natural number greater than or equal to 0.The model calculation architecture of accelerator can be ensured to be efficient, low power consumption, high precision by the application.

Description

Technical Field

[0001] This invention relates to the field of network accelerator technology, and in particular to a lightweight neural network accelerator and quantization method. Background Technology

[0002] Artificial intelligence has been applied across various industries. In the field of image processing, convolutional neural networks are widely used due to their superior performance and generalization capabilities. Thanks to the rapid development of AI hardware, many AI algorithms can now be deployed and used on edge devices and lightweight devices.

[0003] Lightweight neural network accelerators are characterized by low system computing power (0.1 TOPS to 2 TOPS), low power consumption, and low system latency. They can perform mainstream vision tasks (detection, classification, tracking, key points, etc.).

[0004] The main computation in visual tasks is convolution, accounting for over 90% of the overall inference computation. The core of convolution is that it is equivalent to multiplication and addition, and the scale of the input image and the convolution kernel can be extracted before the operation.

[0005] Lightweight neural network accelerators, limited by computing resources and power consumption, typically do not use float32 floating-point calculations, but rather 8 / 4-bit or other lower-bit precision, resulting in a certain loss of precision. This process is called quantization. Different quantization algorithms will affect the performance of the final model deployed on edge devices.

[0006] Existing accelerators and quantization methods have the following main shortcomings:

[0007] To support more general tasks, complex computational methods and hardware structures are often used, making it impossible to achieve high efficiency and low power consumption (TFlite) in lightweight neural network accelerators.

[0008] If complex computational methods and hardware structures are used to ensure the quantization accuracy of various tasks, then no targeted optimization is made for lightweight networks, so the quantization performance cannot be maximized.

[0009] If an overly simplified computation process is used, the quantization effect of lightweight neural networks will be poor, and the loss of accuracy after quantization will be significant (TensorRT).

[0010] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0011] The purpose of this invention is to provide a lightweight neural network accelerator and quantization method that can ensure that the accelerator's model computing architecture is efficient, low-power, and highly accurate.

[0012] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0013] This invention provides a lightweight neural network accelerator, comprising: an instruction scheduling module and a convolution calculation module, wherein;

[0014] The instruction scheduling module is connected to the convolution calculation module and is used to send operation instructions to the convolution calculation module to perform convolution calculation on the input data.

[0015] The convolution calculation module is used to receive operation instructions, perform convolution calculations on the input data according to the operation instructions, and output the convolution inference results.

[0016] The convolution calculation module includes a shifter. When each convolutional layer of the convolution calculation module performs convolution calculation on the input data, it performs a power of 2 shift through the shifter, where n is a natural number greater than or equal to 0.

[0017] Optionally, it also includes: an internal memory for storing the input data transmitted from the external memory, for the instruction scheduling module and the convolution calculation module to read.

[0018] Optionally, it further includes: the convolution calculation module further includes a multiplier and an adder. When each convolutional layer of the convolution calculation module performs convolution calculation on the input data, it first multiplies the data through the multiplier and sums the results through the adder, and then performs a power of 2 shift through the shifter.

[0019] Optionally, when performing convolution calculation, the convolution calculation module converts floating-point operations into fixed-point operations and uses the following formula for approximate calculation:

[0020] A≈Scale_A*QA;

[0021] B≈Scale_B*QB;

[0022] A*B≈Scale_A*Scale_B*QA*QB;

[0023] Scale_C*QC≈C=A*B≈Scale_A*Scale_B*QA*QB;

[0024] A is the floating-point matrix representation of the input data, QA is the fixed-point matrix representation of the input data, and Scale_A is the fixed-point position selected for the input data;

[0025] B is the floating-point matrix representation of the convolutional layer, QA is the fixed-point matrix representation of the convolutional layer, and Scale_A is the fixed-point position selected for the convolutional layer.

[0026] C represents the floating-point matrix representation of the output data of this convolutional layer, i.e., the convolution result of this layer; QC represents the fixed-point matrix representation of the convolution result of this convolutional layer; and Scale_C represents the fixed-point position selected for the output data of this convolutional layer, i.e., the convolution result of this layer.

[0027] Among them, the fixed position Scale_C and the product of fixed position Scale_A and fixed position Scale_B, Scale_A*Scale_B, maintain a power of 2 relationship, that is, Scale_C = Scale_A*Scale_B / 2^n.

[0028] Optionally, the convolutional inference result includes the target confidence level.

[0029] Optionally, it further includes: a post-processing module connected to the instruction scheduling module; the convolution calculation module is also used to send the convolution inference result to the instruction scheduling module after the convolution inference of all convolutional layers has been executed, and the instruction scheduling module performs task type detection based on the convolution inference result.

[0030] Optionally, when the instruction scheduling module detects a specific task type, it sends a scheduling instruction including the convolution calculation result to the post-processing module, and the post-processing module outputs the convolution post-processing result based on the convolution inference result.

[0031] Optionally, when the specific task type is an object detection task, the post-convolution processing result includes object confidence and object coordinates.

[0032] Optionally, the post-processing module includes a coordinate conversion unit and a sorting unit. The coordinate conversion unit is used to convert the coordinate values ​​of the predicted region from floating-point representation to fixed-point representation. The sorting unit is used to determine the final selected target region when multiple predicted regions predict the same object and multiple target regions appear, based on the overlap of the multiple target regions and the confidence score of each target region.

[0033] Optionally, the convolution calculation module and the post-processing module use 8-bit calculation.

[0034] Optionally, the positioning positions Scale_B[i] between different channels of each convolutional layer in the convolutional calculation module maintain a 2^n relationship.

[0035] Optionally, the difference in the power of 2 caused by the fixed position Scale_B of different channels in each convolutional layer is compensated for by shifting in the convolution calculation module.

[0036] The present invention also provides a quantization method for a lightweight neural network accelerator, comprising:

[0037] Send operation instructions to perform convolution calculations on the input data;

[0038] The input data is subjected to convolution calculation according to the operation instructions, and the convolution inference result is output; wherein, each convolutional layer is also shifted to the power of 2 when performing convolution calculation.

[0039] Optionally, it also includes storing the input data transmitted from outside for performing the convolution calculation.

[0040] Optionally, it further includes: when performing convolution calculation on the input data, performing multiplication operation on the input data first, then performing addition operation, and then performing a power of 2 shift.

[0041] Optionally, when performing convolution calculations, floating-point operations are converted to fixed-point operations, and the following formula is used for approximate calculation:

[0042] A≈Scale_A*QA;

[0043] B≈Scale_B*QB;

[0044] A*B≈Scale_A*Scale_B*QA*QB;

[0045] Scale_C*QC≈C=A*B≈Scale_A*Scale_B*QA*QB;

[0046] A is the floating-point matrix representation of the input data, QA is the fixed-point matrix representation of the input data, and Scale_A is the fixed-point position selected for the input data;

[0047] B is the floating-point matrix representation of the convolutional layer, QA is the fixed-point matrix representation of the convolutional layer, and Scale_A is the fixed-point position selected for the convolutional layer.

[0048] C represents the floating-point matrix representation of the output data of this convolutional layer, i.e., the convolution result of this layer; QC represents the fixed-point matrix representation of the convolution result of this convolutional layer; and Scale_C represents the fixed-point position selected for the output data of this convolutional layer, i.e., the convolution result of this layer.

[0049] Among them, the fixed position Scale_C and the product of fixed position Scale_A and fixed position Scale_B, Scale_A*Scale_B, maintain a power of 2 relationship.

[0050] Optionally, it also includes: performing task type detection based on the convolutional inference result.

[0051] Optionally, when a specific task type is detected, post-processing is performed based on the convolution inference results and the convolution post-processing results are output.

[0052] Optionally, when the specific task type is an object detection task, the post-convolution processing result includes object confidence and object coordinates.

[0053] Optionally, post-processing of the convolutional inference result further includes:

[0054] Convert the coordinate values ​​of the predicted region from floating-point representation to fixed-point representation;

[0055] When multiple prediction regions predict the same object and multiple target regions appear, the final target region is determined based on the overlap of the multiple target regions and the confidence score of each target region.

[0056] Optionally, the coordinates of the target region are determined by the coordinates of the center point of the predicted region and the offset between the predicted region and the target region.

[0057] Optionally, the calculation of the coordinates of the target area also includes converting floating-point operations to fixed-point operations, and obtaining them using the following formula:

[0058] Box_True≈Scale_Box*Q_Box=S*Q_Box=Decode(Priors,Loc)

[0059] =[P0–L0,P1–L1,P0+L2,P1+L3];

[0060] =[P0–S_L*Loc0,P1–S_L*Loc1,P0+S_L*Loc2,P1+S_L*Loc3];=S[QP0+(S_L / S*Scale_Loc)*(-Q_Loc0),QP 1+S_L / S*Scale_Loc)*(-Q_Loc1),QP0+(S_L / S*Scale_Loc)*Q_Loc0,QP1+(S_L / S*Scale_Loc)*Q_Loc1];

[0061] Where, Priors are the center point coordinates P0 to P1 of the predicted region; Loc are the offsets L0 to L3 of the left, right, top, and bottom coordinates between the predicted region and the target region output by the convolutional layer model; Box_True are the coordinates of the actual decoded target region;

[0062] Specifically, the fixed-point positions Scale_Priors of the predicted region and Scale_Box of the target region are equal, and equal to the quotient S of the longer side of the input data image divided by the number of fixed-point bits; during decoding, the fixed-point position S_L of the offset Loc output by the convolutional layer model is normalized to S during the first addition / subtraction calculation in the convolution calculation. Optionally, the coordinates of the target region are determined by the center point coordinates and width and height of the predicted region, as well as the offset between the predicted region and the target region.

[0063] Optionally, the calculation of the coordinates of the target area also includes converting floating-point operations to fixed-point operations, and obtaining them using the following formula:

[0064] Box_True≈Scale_Box*Q_Box=S*Q_Box=Decode(Priors,Loc)

[0065] =[P0–e^Loc0*P2,P1–e^Loc1*P3,P1+e^Loc2*P2,P1+e^Loc3*P3]

[0066] =S[QP0+(e^(S_L*QL0))*(-QP2),QP1+(e^(S_L*QL1))*(-QP3),

[0067] QP0+(e^(S_L*QL2))*QP2,QP1+(e^(S_L*QL3))*QP3]

[0068] Where Priors are the center point coordinates and width and height of the predicted region, i.e., P0 to P3; Loc are the left, right, top, and bottom offsets L0 to L3 between the predicted region and the target region output by the model; Box_True are the coordinates of the actual decoded target region;

[0069] Specifically, the fixed-point positions Scale_Priors of the predicted region and Scale_Box of the target region are equal, and are equal to the quotient S of the long side of the input data image divided by the number of fixed-point bits; and during decoding, the fixed-point position S_L of the offset Loc output by the convolutional layer model is normalized to S for calculation during the first addition and subtraction calculation in the convolution calculation.

[0070] Optionally, the convolution calculation and post-processing are performed using 8-bit computation.

[0071] Optionally, the positioning positions Scale_B[i] between different channels of each convolutional layer maintain a 2^n relationship.

[0072] Optionally, the method for selecting the positioning position Scale_B[i] between different channels of each convolutional layer includes:

[0073] Based on the maximum absolute value of the floating-point numbers of all channels, B_max, and the absolute value of the floating-point number of the corresponding channel, abs(B[i]), the floating-point number of the corresponding channel is approximated as a floating-point number B'[i] that conforms to the power of 2, so as to determine the corresponding fixed-point position Scale_B[i].

[0074] Optionally, it further includes: approximating the floating-point number B[i] of the corresponding channel to a floating-point number B'[i] that conforms to the power of 2 using the Ceil function, in order to determine the corresponding fixed-point position Scale_B[i], the calculation formula is as follows:

[0075] B'[i]=2^ceil(log2(max(abs(B[i])) / B_max))*B_max;

[0076] Scale_B[i] = B'[i] * 2 / 255.

[0077] Optionally, it also includes: when the fixed point position Scale_A corresponding to the fixed point number QA of the input data is fixed, based on the similarity KL_i between the floating point number C of the output data and the product QC*Scale_C of the fixed point number and the location, while ensuring that Scale_C conforms to the power of 2 relationship, that is, Scale_C=Scale_B[i]*Scale_A / 2^n, finding the best fixed point position of Scale_C on all Scale_B[i]*Scale_A / 2^n, so as to determine the fixed point position Scale_C of the output data.

[0078] Optionally, it also includes: when the fixed-point position Scale_A of the input data and the fixed-point position Scale_C of the output data are not fixed, considering the similarity between the floating-point number A of the input data and the floating-point number C of the output data and the product QA*Scale_A and QC*Scale_C of their corresponding fixed-point number and location, respectively, while ensuring that Scale_A and Scale_C respectively conform to the power of 2 relationship.

[0079] Optionally, it also includes: first traversing the fixed-point position Scale_A, and the fixed-point position Scale_C is determined because it is related to Scale_A*Scale_B by a power of 2, and then finding the minimum point of KL_i on Scale_C that satisfies Scale_A*Scale_B / Scale_C=2^n, i.e. KL_A[i]+KL_C[i]=KL_i, and taking the fixed-point position determined by the minimum value among all KL_i as the final fixed-point position, so as to determine the fixed-point positions Scale_A and Scale_C corresponding to the fixed-point numbers of the input data and the output data.

[0080] Optionally, it also includes: the difference of 2 to the power of n caused by the fixed position Scale_B of different channels in each convolutional layer is compensated by shifting in the convolution calculation module.

[0081] The beneficial effects of the technical solution provided by the embodiments of the present invention are as follows:

[0082] The lightweight neural network accelerator and quantization method provided in this invention, through the instruction scheduling module, sends operation instructions to the convolution calculation module when data is sent from external memory to internal memory; the convolution calculation module receives the operation instructions, reads data from internal memory according to the operation instructions to perform convolution calculation, and outputs the results according to the task type, thereby ensuring that the accelerator's model calculation architecture is efficient, low-power, and highly accurate, and also improves the performance of the actual model execution on the hardware. Attached Figure Description

[0083] Figure 1 A block diagram of a lightweight neural network accelerator provided in an embodiment of the present invention;

[0084] Figure 2 yes Figure 1 A schematic diagram of the calculation process of the convolution calculation module;

[0085] Figure 3 yes Figure 1 A schematic diagram of the post-processing module's processing flow;

[0086] Figure 4 This is a flowchart illustrating the quantization method for a lightweight neural network accelerator provided in an embodiment of the present invention.

[0087] Figure 5 It is a schematic diagram of an image;

[0088] Figure 6 It is a schematic diagram of an image;

[0089] Figure 7 This is a schematic diagram of similarity curves. Detailed Implementation

[0090] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0091] This invention provides a lightweight neural network accelerator, comprising: an instruction scheduling module and a convolution calculation module, wherein: the instruction scheduling module is connected to the convolution calculation module and is used to send operation instructions to the convolution calculation module to perform convolution calculation on input data; the convolution calculation module is used to receive operation instructions, perform convolution calculation on the input data according to the operation instructions, and output convolution inference results; wherein, the convolution calculation module includes a shifter, and when each convolutional layer of the convolution calculation module performs convolution calculation on the input data, the shifter performs a power of 2 shift, where n is a natural number greater than or equal to 0. In one embodiment, the convolution inference result includes a target confidence level. In a preferred embodiment, the lightweight neural network accelerator further includes an internal memory for storing the input data transmitted from an external memory for the instruction scheduling module and the convolution calculation module to read. Furthermore, the convolution calculation module also includes a multiplier and an adder. When each convolutional layer of the convolution calculation module performs convolution calculation on the input data, it first multiplies the data using the multiplier and sums the results using the adder, and then performs a power of 2 shift using the shifter.

[0092] In a preferred embodiment, the lightweight neural network accelerator is connected to a post-processing module linked to the instruction scheduling module. The convolution calculation module is further configured to send the convolution inference results to the instruction scheduling module after executing convolution inference for all convolutional layers. The instruction scheduling module performs task type detection based on the convolution inference results. Further, when the instruction scheduling module detects a specific task type, it sends a scheduling instruction including the convolution calculation results to the post-processing module. The post-processing module outputs convolution post-processing results based on the convolution inference results. Further, when the specific task type is an object detection task, the convolution post-processing results include object confidence and object coordinates. In one embodiment, the post-processing module includes a coordinate conversion unit and a sorting unit. The coordinate conversion unit converts the coordinate values ​​of the predicted region from floating-point representation to fixed-point representation. The sorting unit determines the final selected target region based on the overlap of multiple target regions and the confidence score of each target region when multiple predicted regions predict the same object and multiple target regions appear.

[0093] Figure 1 This is a block diagram of a lightweight neural network accelerator provided in an embodiment of the present invention. The lightweight neural network accelerator ensures that the accelerator's model computation architecture is efficient, low-power, and highly accurate. Please refer to... Figure 1 The lightweight neural network accelerator in this embodiment includes an instruction scheduling module 11 and a convolution calculation module 12. Preferably, the lightweight neural network accelerator may further include a post-processing module 13.

[0094] Specifically, the instruction scheduling module 11 is connected to the convolution calculation module 12 and is used to send operation instructions to the convolution calculation module 12 when data is sent from the external memory to the internal memory.

[0095] The convolution calculation module 12 is connected to the post-processing module 13. It is used to receive operation instructions, read data from the internal memory according to the operation instructions to calculate the convolution, and output the results according to the task type.

[0096] When data is sent from external memory to internal memory, instruction scheduling module 11 can send operation instructions to convolution calculation module 12. Convolution calculation module 12 reads data from internal memory according to the operation instructions, performs convolution calculation, and outputs the results according to the task type.

[0097] Preferably, the convolution calculation module 12 is further configured to determine the number of convolutions to be calculated based on the number of layers in the lightweight neural network being executed, and to perform lightweight neural network inference. After the lightweight neural network inference is completed, depending on the task type, if the task type is an object detection task, a scheduling instruction is sent to the post-processing module 13. The post-processing module 13 then performs its work according to the scheduling instruction, such as... Figure 1 As shown.

[0098] Preferably, the convolution calculation module 12 is also used to sequentially calculate the convolutional layers (e.g., in a lightweight neural network) in the lightest neural network. Figure 2 The convolutional layers 1 to N in the neural network are used to perform a task. If the task is object detection, a scheduling instruction including the convolution calculation results is sent to the post-processing module 13. The post-processing module 13 converts the convolution calculation results into target confidence and target coordinates. If the task is not object detection, a lightweight neural network inference result is output (the lightweight neural network inference result only includes target confidence), such as... Figure 2 As shown.

[0099] The convolution calculation module 12 is also used to convert the floating-point operations of convolution into fixed-point operations, such as 8-bit fixed-point numbers (int 8), when calculating convolution.

[0100] This invention optimizes lightweight networks for key applications. In the lightweight algorithm, the Conv+BN layer (convolution function and normalization function) uses int 8 quantization, and the data after BN fusion is symmetrically distributed, eliminating the need for asymmetric offsets. Therefore, fixed-point biases Bias_A and Bias_B are eliminated, and the calculation result can be obtained with a single multiplier calculation, simplifying the computation method. Convolution can be equivalent to the dot product of two matrices and their summation; therefore, the convolution calculation module 12 converts floating-point operations to fixed-point operations using the following formula:

[0101] Assuming floating-point matrices A and B, fixed-point locations Scale_A and Scale_B, and fixed-point biases Bias_A and Bias_B, and fixed-point matrices QA and QB, then the following formula is used for approximate calculation:

[0102] A≈Scale_A*QA;

[0103] B≈Scale_B*QB;

[0104] A*B≈Scale_A*Scale_B*QA*QB;

[0105] Scale_C*QC≈C=A*B≈Scale_A*Scale_B*QA*QB;

[0106] A represents the floating-point matrix of the input data, QA represents the fixed-point matrix of the input data, and Scale_A represents the selected fixed-point position of the input data; B represents the floating-point matrix of the convolutional layer, QA represents the fixed-point matrix of the convolutional layer, and Scale_A represents the selected fixed-point position of the convolutional layer; C represents the floating-point matrix of the output data of this convolutional layer, i.e., the convolution result of this layer, QC represents the fixed-point matrix of the convolution result of this layer, and Scale_C represents the selected fixed-point position of the output data of this convolutional layer. Specifically, A is the input floating-point image data matrix, and B is the floating-point parameter matrix of the convolutional layer model trained by the lightweight network. A will vary depending on the different image data, while B is a fixed value, ranging from (-∞, +∞), and is an int 8 (8-bit fixed-point number). The convolution calculation module 12 can use a multiplier to implement the above formula.

[0107] For Conv+ReLU (convolution function + activation function), its output range is (0, +∞). If uint 8 (8-bit floating-point number) quantization is used instead, a higher quantization granularity than the original int 8 asymmetric algorithm can be obtained without increasing the number of quantization bits, and the required hardware resources remain basically unchanged.

[0108] In AScale_A*QA, type(QA) = uint 8, changing from int 8 to two quantization representations, int8 / uint8, can further improve accuracy.

[0109] Preferably, the multiplier in the convolution calculation module 12 is removed and replaced with a shifting method using powers of 2 (shiftcut), that is, the multiplier is replaced with a shifter.

[0110] The formula is as follows: Scale_C*QC≈C=A*B≈Scale_A*Scale_B*QA*QB, where QC is a fixed-point matrix and C is a floating-point matrix. C is also the input floating-point matrix A of the next layer in the convolutional layer. Fixed-point matrices QA, QB, and QC are all 8-bit fixed-point numbers. Therefore, the product of fixed-point positions Scale_C and Scale_A and Scale_B, Scale_A*Scale_B, maintains a power of 2 relationship, i.e., Scale_C=Scale_A*Scale_B / 2^n. When Scale_A*Scale_B satisfies a power of 2, multiplication is unnecessary; a direct shift is sufficient. That is, for floating-point matrix A (0-10), (0 255) is used, and it is a uint 8; for floating-point matrix A (-10-10), (-178-178) is used, and it is an int 8. This convolution calculation method can further improve the efficiency and power consumption of applications.

[0111] Because the inference result of a lightweight neural network can be a single number or a large number. For example, a single number represents the probability that an image region contains an apple, in which case no post-processing module is needed (i.e., object classification does not require a post-processing module). If there are a large number of numbers, representing the number of apples in the image region and the location of each apple, then a conversion from the output result to the final coordinates is required (i.e., object detection requires a post-processing module). The post-processing module 13 originally used floating-point numbers for calculations, but the hardware requires fixed-point numbers, typically 16 bits. However, this invention, based on the lightweight model, only requires 8 bits, and the calculation process needs optimization; therefore, an 8-bit calculation process and corresponding hardware are designed.

[0112] like Figure 3As shown, the post-processing module 13 may include a coordinate point conversion unit 130 and a sorting unit 131. The coordinate point conversion unit 130 is used to obtain the coordinate values ​​of the regions in the image based on the confidence data and coordinate data. The sorting unit is used to determine the final selected target region based on the overlap of the multiple regions when the same object is predicted to appear in multiple regions. A large overlap indicates that the same object is predicted. The final selected target region is determined based on the confidence score of each region (the output result includes the target confidence score and target coordinate value of the target region).

[0113] In summary, the lightweight neural network accelerator provided by this embodiment of the invention sends operation instructions to the convolution calculation module when data is sent from external memory to internal memory through the instruction scheduling module; the convolution calculation module receives the operation instructions, reads data from internal memory according to the operation instructions to perform convolution calculation, and outputs the results according to the task type, thereby ensuring that the accelerator's model calculation architecture is efficient, low-power, and highly accurate, and also improves the performance of the actual model execution on the hardware.

[0114] The following are method embodiments of the present invention. For details not described in detail in the method embodiments, please refer to the corresponding device embodiments described above.

[0115] Figure 4 This is a flowchart illustrating the quantization method for a lightweight neural network accelerator provided in an embodiment of the present invention. Please refer to... Figure 4 The quantization method of the lightweight neural network accelerator is applied to the lightweight neural network accelerator. The quantization method of the lightweight neural network accelerator includes the following steps: Step S401, sending an operation command to perform convolution calculation on the input data.

[0116] Step S403: Perform convolution calculation on the input data according to the operation instruction, and output the convolution inference result; wherein, during the convolution calculation of each convolutional layer, a power of 2 shift is also performed.

[0117] Optionally, before step S401, the method further includes: storing the input data transmitted from the outside for performing the convolution calculation. Optionally, step S403 further includes: when performing the convolution calculation on the input data, performing multiplication operations first, then addition operations on the input data, and then performing a power-of-2 shift.

[0118] Preferably, step S401 specifically includes:

[0119] The convolution calculation module also calculates the convolutional layers in the lightweight neural network in sequence. Depending on the task type, if the task type is an object detection task, the scheduling instruction including the convolution calculation result is sent to the post-processing module. The post-processing module converts the convolution calculation result into the target confidence and target coordinates. If the task type is not an object detection task, the lightweight neural network inference result including the target confidence is output.

[0120] Preferably, step S401 specifically includes:

[0121] Assuming the floating-point matrices are A and B, the fixed-point positions are Scale_A and Scale_B, and the fixed-point matrices are QA and QB, the convolution calculation module also converts the floating-point operations of convolution into fixed-point operations when calculating the convolution, and uses the following formula for approximate calculation:

[0122] A≈Scale_A*QA;

[0123] B≈Scale_B*QB;

[0124] A*B≈Scale_A*Scale_B*QA*QB;

[0125] Scale_C*QC≈C=A*B≈Scale_A*Scale_B*QA*QB;

[0126] A represents the floating-point matrix representation of the input data, QA represents the fixed-point matrix representation of the input data, and Scale_A represents the fixed-point position selected for the input data; B represents the floating-point matrix representation of the convolutional layer, QA represents the fixed-point matrix representation of the convolutional layer, and Scale_A represents the fixed-point position selected for the convolutional layer; C represents the floating-point matrix representation of the output data of this convolutional layer, i.e., the convolution result of this layer, QC represents the fixed-point matrix representation of the convolution result of this convolutional layer, and Scale_C represents the fixed-point position selected for the output data of this convolutional layer.

[0127] Preferably, the quantization method further includes:

[0128] The coordinate transformation unit of the post-processing module obtains the coordinate values ​​of the region. When the sorting unit of the post-processing module predicts that the same object appears in multiple regions, it determines the final target region based on the overlap of multiple regions and the confidence score of each region.

[0129] Preferably, the products of fixed-point positions Scale_C and Scale_A and Scale_B are related as powers of 2, where Scale_C*QC≈C=A*B≈Scale_A*Scale_B*QA*QB, QC is a fixed-point matrix, and C is a floating-point matrix. Preferably, step S401 may further include: the convolution calculation module determining the number of convolutions to calculate based on the number of lightweight neural network layers, and performing lightweight neural network inference. After the lightweight neural network inference is completed, according to the task type, if the task type is an object detection task, a scheduling instruction is sent to the post-processing module, and the post-processing module works according to the scheduling instruction.

[0130] This invention also provides a method for calculating target coordinates in a post-processing module. In existing technologies, to achieve the conversion from floating-point to fixed-point coordinates, common methods require multiple fixed-point coordinate conversions, which results in significant precision loss and uses 16-bit coordinates. The method of this invention requires only one fixed-point coordinate conversion, and since the input image is small, 8-bit coordinates can be used. The fixed-point coordinate conversion unit is used to obtain... Figure 5 The coordinates (Q_Box) of the target region 50; the sorting unit can be the NMS sorting unit. When multiple different prediction regions 51 predict the same object and appear in multiple target regions, it is necessary to determine the final selected target region 50 based on the overlap of multiple target regions. A large overlap indicates that the same object is predicted. Then, based on the confidence score of each target region, the final selected target region 50 is determined.

[0131] The post-processing module of this invention employs two parallel calculation methods.

[0132] The post-processing module transforms the output of the lightweight neural network (i.e., the output of the convolutional computation module) into actual bounding boxes and category information for object detection. Considering the practical application of lightweight networks, the image size is not large; therefore, the post-processing module 13 also uses 8-bit computation to save hardware resources while maintaining accuracy.

[0133] Since 8-bit computation is used, in order to improve the performance of the post-processing module as much as possible, the post-processing module needs to perform box decoding. All box decoding recovery uses a method of fusion calculation once to reduce the precision loss caused by multiple multiplications of 8-bit fixed-point numbers.

[0134] The principle of Box Decode is as follows: Figure 6As shown. For example, an original image becomes 3x3x2x8 after passing through a lightweight network. Each small 3x3 cell (60) corresponds to a block in the original image at the corresponding location. Box Decode is used to recover the true rectangular box (Box) of the large frame in the image from a block in the original image, combining the network's output and a preset first calculation method.

[0135] The first post-processing module calculation method: The coordinates of the target region 50 are determined by the coordinates of the center point of the prediction region 51 and the offset between the prediction region 51 and the target region 50. Further, the calculation of the coordinates of the target region 50 also includes converting floating-point arithmetic to fixed-point arithmetic, and obtaining it using the following formula:

[0136] Box_True≈Scale_Box*Q_Box=S*Q_Box=Decode(Priors,Loc)

[0137] =[P0–L0,P1–L1,P0+L2,P1+L3];

[0138] =[P0–S_L*Loc0,P1–S_L*Loc1,P0+S_L*Loc2,P1+S_L*Loc3];

[0139] =S[QP0+(S_L / S*Scale_Loc)*(-Q_Loc0),QP1+S_L / S*Scale_Loc)*(-Q_Loc1),QP0+(S_L / S*Scale_Loc)*Q_Loc0,QP1+(S_L / S*Scale_Loc)*Q_Loc1];

[0140] Wherein, Priors are the center point coordinates P0 to P1 of the predicted region 51; Loc are the offsets L0 to L3 of the left, right, top, and bottom coordinates between the predicted region 51 and the target region 50 output by the convolutional layer model; Box_True are the coordinates of the actual decoded target region; wherein, the fixed position Scale_Priors of the predicted region 51 and the fixed position Scale_Box corresponding to the target region 50 are equal, and equal to the quotient S of the long side of the input data image divided by the number of fixed-point bits; and during decoding, the fixed position S_L of the offset Loc output by the convolutional layer model is directly replaced with S in the first addition and subtraction calculation in the convolution calculation. In one embodiment, when the input data is a square image, the long side of the image is the image side length.

[0141] Multiplication following addition can be represented as multiplication of a fixed Quint 8 value and a Qint 8 value.

[0142] The second post - processing module calculation. The coordinate values of the target region are determined by the center point coordinates, width, and height of the prediction region 51 and the offset between the prediction region 51 and the target region 50. Further, the calculation of the coordinate values of the target region 50 also includes converting floating - point operations to fixed - point operations, and is obtained through the following formula:

[0143] Box_True≈Scale_Box*Q_Box=S*Q_Box=Decode(Priors,Loc)

[0144] =[P0–e^Loc0*P2,P1–e^Loc1*P3,P1+e^Loc2*P2,P1+e^Loc3*P3];

[0145] =S[QP0+(e^(S_L*QL0))*(-QP2),QP1+(e^(S_L*QL1))*(-QP3);QP0+(e^(S_L*QL2))*QP2,QP1+(e^(S_L*QL3))*QP3];

[0146] Where Priors are the center point coordinates, width, and height of the prediction region 51, i.e., P0 - P3; Loc is the offset L0 - L3 of the left, right, up, and down coordinates between the prediction region 51 output by the model and the target region 50; Box_True is the coordinate of the target region 50 after actual decoding. Among them, the fixed - point position Scale_Priors of the prediction region 51 is equal to the fixed - point position Scale_Box of the corresponding target region 50, and is equal to the quotient S obtained by dividing the long side of the input data image by the fixed - point number of bits. And during decoding, the fixed - point position S_L of the offset Loc output by the convolutional layer model is directly replaced by S for operation during the first addition and subtraction calculation in the convolutional calculation. In one embodiment, when the input data is a square image, the long side of the image is the side length of the image.

[0147] In one embodiment, the preset range of e^(S_L*QLi)) is between 1 / k and k (3 < k < 4). The multiplication (e^(S_L*QL0))*(-QP2) after the addition can be approximated as the multiplication of Quint8(e^(S_L*QL0)) and Qint8(-QP2) by the method of LookUpTable. Therefore, in the post - processing module of the lightweight neural network accelerator, during the actual decoding operation, only one Scale conversion of S1 / S*Scale_Loc or one table - look - up for (e^(S_L*QL0))*(-QP2) is involved to reduce the precision loss.

[0148] This invention also provides a method for selecting the fixed-point position Scale_A or Scale_B, which requires an algorithm to guide and adapt the hardware design, based on the selection by statistically analyzing the distribution of A and B. Specifically, for each channel, if C is different, Scale_C is the same; for each channel, if A is different, Scale_A is the same; for each channel, if B is different, Scale_B is different; and the Scale_B between different channels is also a power of 2, meaning that the positioning positions Scale_B[i] between different channels of each convolutional layer in the convolutional calculation module maintain a 2^n relationship.

[0149] This invention selects the fixed-point position based on the quantization algorithm of each channel. A floating-point number, such as 1.6217, can be approximated by 8 / 5, 81 / 50, or 82 / 50. Therefore, there are many approximation numbers. This invention minimizes the approximation error while ensuring that the denominator is the same. For example, if 50 is used as the denominator, then 1.62 = 81 / 50.

[0150] Here, KL is the similarity between two sets of numbers (C and Scale_C*QC). The more similar the two sets of numbers are, the smaller the KL. This invention requires a small KL while ensuring that Scale_C is a power of 2. Figure 7 In the similarity curve, the horizontal axis is Scale_C and the vertical axis is the similarity KL.

[0151] The quantization formula for each channel of the convolutional layer and the input data in this invention can be expressed as:

[0152] Scale_C*QC[i]≈C[i]=B[i]*A≈Scale_B[i]*Scale_A*QB[i]*QA;

[0153] QA, QB, and QC are all 8-bit fixed-point numbers.

[0154] Because the multiplier in the convolution calculation module 12 has been removed, the relationship between Scale_C and Scale_B[i]*Scale_A needs to be maintained at 2^n. Since Scale_C / Scale_A cannot be changed once selected, it also means that the different channels of Scale_B[i] need to maintain a relationship of 2^n.

[0155] In one implementation, the method for selecting the positioning position Scale_B[i] between different channels of each convolutional layer includes:

[0156] Based on the maximum absolute value of the floating-point numbers of all channels, B_max, and the absolute value of the floating-point number of the corresponding channel, abs(B[i]), the floating-point number of the corresponding channel is approximated as a floating-point number B'[i] that conforms to the power of 2, to determine the corresponding fixed-point position Scale_B[i]. Further, the floating-point number B[i] of the corresponding channel is approximated as a floating-point number B'[i] that conforms to the power of 2 using the Ceil function, to determine the corresponding fixed-point position Scale_B[i]. The calculation formula is as follows:

[0157] B'[i]=2^ceil(log2(max(abs(B[i])) / B_max))*B_max;

[0158] Scale_B[i] = B'[i] * 2 / 255.

[0159] For example, if B_max = 10 is the global maximum, and a certain channel B[i] = 4.8, in order to ensure that the power of 2 between each channel must be 5 or 2.5, then by using (2^ceil(log2(max(abs(B[i])) / A_max))*B_max), B[i] = 4.8 is transformed into a power of 2, i.e., B'[i] = 5.

[0160] In a single implementation room, the selection algorithm for Scale_A and Scale_C is as follows:

[0161] When the fixed-point position Scale_A corresponding to the fixed-point number QA of the input data is fixed, the optimal fixed-point position Scale_C of the output data is determined by finding the optimal fixed-point position Scale_C among all Scale_B[i]*Scale_A / 2^n, based on the similarity KL_i between the floating-point number C of the output data and the product QC*Scale_C of the fixed-point number and the fixed-point position. Simultaneously, Scale_C is guaranteed to conform to the power of 2 relationship, i.e., Scale_C = Scale_B[i]*Scale_A / 2^n. In one implementation, the fixed-point position Scale_A can be fixed in the following way: Based on the data distribution of A being (-∞, +∞), the maximum value of A is calculated. If the maximum value is 9.9, Scale_A = 9.9*2 / 255; if the data distribution of A is (0, +∞), the maximum value of A is calculated. If the maximum value is 9.9, Scale_A = 9.9 / 255.

[0162] Since Scale_A and Scale_B have been determined, the optimal fixed point of Scale_C is found on Scale_B[1]*Scale_A / 2^n according to the curve. That is, the smaller the KL is, the more Scale_C is required to ensure that Scale_C is a power of 2, i.e., Scale_C = Scale_B[1]*Scale_A / 2^n.

[0163] When the fixed-point positions Scale_A and Scale_C of the input data and output data are not fixed, the similarity between the floating-point numbers A and C of the input data and their corresponding fixed-point numbers and positions, QA*Scale_A and QC*Scale_C, is considered respectively. Simultaneously, Scale_A and Scale_C are guaranteed to satisfy a power of 2 relationship. Further, by first traversing the fixed-point positions Scale_A, and since the fixed-point position Scale_C is determined by maintaining a power of 2 relationship with Scale_A*Scale_B, the minimum point KL_i satisfying Scale_A*Scale_B / Scale_C = 2^n is found on Scale_C, i.e., KL_A[i] + KL_C[i] = KL_i. The fixed-point position determined by the minimum value among all KL_i is taken as the final fixed-point position to determine the fixed-point positions Scale_A and Scale_C corresponding to the fixed-point numbers of the input and output data. Specifically, in this method, there are two graphs, Scale_A and Scale_C (i.e., two types of graphs). Figure 3 The curve is obtained by first sliding Scale_A(Scale_A), and Scale_C is determined by maintaining a power of 2 relationship with Scale_B*Scale_A. Then, by using KL_B[i]+KL_C[j]=KL_i, the minimum value is found among all KL_i, thereby determining Scale_A and Scale_C.

[0164] In this embodiment of the invention, the difference in 2 to the power of n caused by the Scale_B of different channels is compensated by shifting at the end of the convolution calculation module 12.

[0165] In summary, the quantization method for the lightweight neural network accelerator provided by this invention sends operation instructions to the convolution calculation module when data is sent from external memory to internal memory via the instruction scheduling module. The convolution calculation module receives the operation instructions, reads data from internal memory according to the operation instructions to perform convolution calculation, and outputs the results according to the task type. This ensures that the accelerator's model calculation architecture is efficient, low-power, and highly accurate, and also improves the performance of the actual model execution on the hardware.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of the present invention may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0168] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0169] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0170] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A lightweight neural network accelerator, characterized in that, include: The instruction scheduling module and the convolution calculation module, among which; The instruction scheduling module is connected to the convolution calculation module and is used to send operation instructions to the convolution calculation module to perform convolution calculation on the input data; the input data is image data. The convolution calculation module is used to receive operation instructions, perform convolution calculations on the input data according to the operation instructions, and output the convolution inference results. The convolution calculation module includes a shifter. When each convolutional layer of the convolution calculation module performs convolution calculation on the input data, it performs a power of 2 shift through the shifter, where n is a natural number greater than or equal to 0. When performing convolution calculations, the convolution calculation module converts floating-point operations into fixed-point operations and uses the following formula for approximate calculation: A≈Scale_A QA; B ≈Scale_B QB; A B ≈Scale_A Scale_B QA QB; Scale_C QC ≈ C = A B ≈Scale_A Scale_B QA QB; A is the floating-point matrix representation of the input data, QA is the fixed-point matrix representation of the input data, and Scale_A is the fixed-point position selected for the input data; B is the floating-point matrix representation of the convolutional layer, QB is the fixed-point matrix representation of the convolutional layer, and Scale_B is the fixed-point position selected for the convolutional layer. C represents the floating-point matrix representation of the output data of this convolutional layer, i.e., the convolution result of this layer; QC represents the fixed-point matrix representation of the convolution result of this convolutional layer; and Scale_C represents the fixed-point position selected for the output data of this convolutional layer, i.e., the convolution result of this layer. Wherein, the fixed position Scale_C and the product Scale_A of the fixed position Scale_A and the fixed position Scale_B are... The relationships between Scale_B are powers of 2, i.e., Scale_C = Scale_A. Scale_B / 2^n.

2. The lightweight neural network accelerator according to claim 1, characterized in that, Also includes: The internal memory is used to store the input data transmitted from the external memory for the instruction scheduling module and the convolution calculation module to read.

3. The lightweight neural network accelerator according to claim 1, characterized in that, The convolution calculation module further includes a multiplier and an adder. When each convolutional layer of the convolution calculation module performs convolution calculation on the input data, it first multiplies the data using the multiplier and sums the results using the adder, and then performs a power of 2 shift using the shifter.

4. The lightweight neural network accelerator according to claim 1, characterized in that, The convolutional inference result includes the target confidence level.

5. The lightweight neural network accelerator according to claim 1, characterized in that, Also includes: A post-processing module connected to the instruction scheduling module; the convolution calculation module is also used to send the convolution inference result to the instruction scheduling module after the convolution inference of all convolutional layers has been executed, and the instruction scheduling module performs task type detection based on the convolution inference result.

6. The lightweight neural network accelerator according to claim 5, characterized in that, When the instruction scheduling module detects a specific task type, it sends a scheduling instruction including the convolution calculation result to the post-processing module. The post-processing module then outputs the convolution post-processing result based on the convolution inference result.

7. The lightweight neural network accelerator according to claim 6, characterized in that, When the specific task type is an object detection task, the post-convolution processing result includes object confidence and object coordinates.

8. The lightweight neural network accelerator according to claim 5, characterized in that, The post-processing module includes a coordinate conversion unit and a sorting unit. The coordinate conversion unit is used to convert the coordinate values ​​of the predicted region from floating-point representation to fixed-point representation. The sorting unit is used to determine the final selected target region when multiple predicted regions predict the same object and multiple target regions appear, based on the overlap of multiple target regions and the confidence score of each target region.

9. The lightweight neural network accelerator according to claim 5, characterized in that, The convolution calculation module and the post-processing module use 8-bit calculation.

10. The lightweight neural network accelerator according to claim 1, characterized in that, In the convolution calculation module, the positioning positions Scale_B[i] between different channels of each convolutional layer maintain a 2^n relationship.

11. The lightweight neural network accelerator according to claim 10, characterized in that, The difference of 2 to the power of n caused by the fixed position Scale_B of different channels in each convolutional layer is compensated for by shifting in the convolution calculation module.

12. A quantization method for a lightweight neural network accelerator, characterized in that, include; An operation command is sent to perform convolution calculations on the input data; the input data is image data. The input data is subjected to convolution calculation according to the operation instructions, and the convolution inference result is output; wherein, each convolutional layer is also shifted to the power of 2 when performing convolution calculation; When performing convolution calculations, floating-point operations are converted to fixed-point operations, and the following formula is used for approximate calculation: A≈ Scale_A QA; B≈ Scale_B QB; A B≈Scale_A Scale_B QA QB; Scale_C QC≈ C = A B≈Scale_A Scale_B QA QB; A is the floating-point matrix representation of the input data, QA is the fixed-point matrix representation of the input data, and Scale_A is the fixed-point position selected for the input data; B is the floating-point matrix representation of the convolutional layer, QB is the fixed-point matrix representation of the convolutional layer, and Scale_B is the fixed-point position selected for the convolutional layer. C represents the floating-point matrix representation of the output data of this convolutional layer, i.e., the convolution result of this layer; QC represents the fixed-point matrix representation of the convolution result of this convolutional layer; and Scale_C represents the fixed-point position selected for the output data of this convolutional layer, i.e., the convolution result of this layer. Wherein, the fixed position Scale_C and the product Scale_A of the fixed position Scale_A and the fixed position Scale_B are... The relationships between Scale_B are maintained as powers of 2.

13. The quantization method according to claim 12, characterized in that, Also includes: The input data transmitted from the outside is stored for performing the convolution calculation.

14. The quantization method according to claim 12, characterized in that, Also includes: When performing convolution calculation on the input data, the input data is first multiplied, then added, and then shifted to the power of 2.

15. The quantization method according to claim 12, characterized in that, Also includes: Task type detection is performed based on the convolutional inference results.

16. The quantization method according to claim 15, characterized in that, When a specific task type is detected, post-processing is performed based on the convolution inference results, and the convolution post-processing results are output.

17. The quantization method according to claim 16, characterized in that, When the specific task type is an object detection task, the post-convolution processing result includes object confidence and object coordinates.

18. The quantization method according to claim 16, characterized in that, Post-processing of the convolutional inference results also includes: Convert the coordinate values ​​of the predicted region from floating-point representation to fixed-point representation; When multiple prediction regions predict the same object and multiple target regions appear, the final target region is determined based on the overlap of the multiple target regions and the confidence score of each target region.

19. The quantization method according to claim 18, characterized in that, The coordinates of the target region are determined by the coordinates of the center point of the predicted region and the offset between the predicted region and the target region.

20. The quantization method according to claim 19, characterized in that, The calculation of the coordinates of the target area also includes converting floating-point operations to fixed-point operations, and obtaining the coordinates using the following formula: Box_True≈ Scale_Box Q_Box = S Q_Box = Decode(Priors, Loc) = [P0 –L0, P1 –L1, P0+ L2, P1+ L3]; = [P0 – S_L Loc0, P1 – S_L Loc1, P0+S_L Loc2, P1+S_L Loc3];= S[QP0 + (S_L / S Scale_Loc) (-Q_Loc0), QP1 + S_L / S Scale_Loc) (-Q_Loc1),QP0 + (S_L / S Scale_Loc) Q_Loc0, QP1 + (S_L / S Scale_Loc) Q_Loc1]; Where, Priors are the center point coordinates P0~P1 of the predicted region; Loc are the offsets L0~L3 of the left, right, top, and bottom coordinates between the predicted region and the target region output by the convolutional layer model; Box_True are the coordinates of the actual decoded target region; Specifically, the fixed-point positions Scale_Priors of the predicted region and Scale_Box of the target region are equal, and are equal to the quotient S of the long side of the input data image divided by the number of fixed-point bits; and during decoding, the fixed-point position S_L of the offset Loc output by the convolutional layer model is normalized to S for calculation during the first addition and subtraction calculation in the convolution calculation.

21. The quantization method according to claim 18, characterized in that, The coordinates of the target region are determined by the coordinates of the center point of the predicted region, its width and height, and the offset between the predicted region and the target region.

22. The quantization method according to claim 21, characterized in that, The calculation of the coordinates of the target area also includes converting floating-point operations to fixed-point operations, and obtaining the coordinates using the following formula: Box_True≈Scale_Box Q_Box = S Q_Box = Decode(Priors, Loc) = [P0 – e^Loc0 P2, P1 – e^Loc1 P3, P1+e^Loc2 P2, P1+e^Loc3 P3] = S[QP0 + (e^(S_L QL0)) (-QP2), QP1 + (e^(S_L QL1)) (-QP3), QP0 + (e^(S_L QL2)) QP2, QP1 +(e^(S_L QL3)) QP3] Where Priors are the center point coordinates and width and height of the predicted region, i.e., P0~P3; Loc are the offsets L0~L3 of the left, right, top, and bottom coordinates between the predicted region and the target region output by the model; Box_True are the coordinates of the actual decoded target region; Specifically, the fixed-point positions Scale_Priors of the predicted region and Scale_Box of the target region are equal, and are equal to the quotient S of the long side of the input data image divided by the number of fixed-point bits; and during decoding, the fixed-point position S_L of the offset Loc output by the convolutional layer model is normalized to S for calculation during the first addition and subtraction calculation in the convolution calculation.

23. The quantization method according to claim 16, characterized in that, The convolution calculation and post-processing are performed using 8-bit computation.

24. The quantization method according to claim 12, characterized in that, The positioning positions Scale_B[i] between different channels of each convolutional layer maintain a 2^n relationship.

25. The quantization method according to claim 24, characterized in that, The methods for selecting the positioning position Scale_B[i] between different channels of each convolutional layer include: Based on the maximum absolute value of the floating-point numbers of all channels, B_max, and the absolute value of the floating-point number of the corresponding channel, abs(B[i]), the floating-point number of the corresponding channel is approximated as a floating-point number B'[i] that conforms to the power of 2, so as to determine the corresponding fixed-point position Scale_B[i].

26. The quantization method according to claim 25, characterized in that, include: The floating-point number B[i] of the corresponding channel is approximated by the Ceil function to a floating-point number B'[i] that conforms to the power of 2, in order to determine the corresponding fixed-point position Scale_B[i]. The calculation formula is as follows: B’[i]= 2^ceil(log2(max(abs(B[i])) / B_max)) B_max; Scale_B[i]= B’[i] 2 / 255 。 27. The quantization method according to claim 25, characterized in that, It also includes: when the fixed-point position Scale_A corresponding to the fixed-point number QA of the input data is fixed, the product QC of the floating-point number C of the output data, the fixed-point number, and the positioning position is used. The similarity KL_i between Scale_C is calculated, while ensuring that Scale_C conforms to a power of 2 relationship, i.e., Scale_C = Scale_B[i]. Scale_A / 2^n, in all Scale_B[i] Find the optimal fixed point location of Scale_C on Scale_A / 2^n to determine the fixed point location Scale_C of the output data.

28. The quantization method according to claim 25, characterized in that, Also includes: When neither the fixed-point position Scale_A of the input data nor the fixed-point position Scale_C of the output data is fixed, consider the product QA of the floating-point number A of the input data and the floating-point number C of the output data with their corresponding fixed-point number and the positioning position. Scale_A, QC The similarity between Scale_C is calculated, while ensuring that Scale_A and Scale_C are respectively in a power of 2 relationship.

29. The quantization method according to claim 28, characterized in that, Also includes: First, iterate through the fixed point Scale_A, and then the fixed point Scale_C is related to Scale_A. Scale_B is determined by maintaining the relationship of powers of 2, and thus we search on Scale_C for the condition that Scale_A satisfies. The minimum point of KL_i when Scale_B / Scale_C=2^n, i.e., KL_A[i]+KL_C[i] = KL_i, is used as the final fixed point position determined by the minimum value among all KL_i, so as to determine the fixed point positions Scale_A and Scale_C corresponding to the fixed points of the input data and the output data.

30. The quantization method according to claim 24, characterized in that, Also includes: The difference of 2 to the power of n caused by the fixed position Scale_B of different channels in each convolutional layer is compensated for by shifting in the convolution calculation module.

Images