Method for optimization for deep learning network and operation system
The hybrid quantization method for deep learning networks improves prediction accuracy and reduces complexity by using power-of-two and dynamic fixed-point quantization, addressing the limitations of existing quantized models.
Patent Information
- Application Number
- JP2023171693
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-28
- Filing Date
- 2023-10-02
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-10-02
AI Technical Summary
Existing quantized deep learning models face challenges in achieving high prediction accuracy while maintaining a constant compression ratio.
A hybrid optimization method combining power-of-two quantization and dynamic fixed-point quantization is applied to deep learning networks, allowing for improved prediction accuracy and reduced model complexity.
The method enhances prediction accuracy and reduces model complexity by optimizing data formats through power-of-two and dynamic fixed-point quantization techniques.
Smart Images

Figure 2024159416000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to machine learning technology, and in particular to an optimization method and computing system for deep learning networks. [Background technology]
[0002] In recent years, with the continuous development of artificial intelligence (AI) technology, the amount of parameters and the computational complexity of neural networks are also increasing. As a result, compression techniques for deep learning networks have made great progress. It should be noted that quantization is an important technique for compressing models. However, the prediction accuracy and compression rate of existing quantized models still have room for improvement. Summary of the Invention [Problem to be solved by the invention]
[0003] In view of the above, one embodiment of the present invention provides an optimization method and computing system for deep learning networks that can improve prediction accuracy and maintain a constant compression ratio by using hybrid efficient coding and quantization. [Means for solving the problem]
[0004] In one embodiment of the present invention, an optimization method for a deep learning network includes (but is not limited to) obtaining first data; quantizing the first data into a first format or a second format through Power of Two Quantization, where the number of first values in the first format or the second format is different; using the first format or the second format as a target format; and performing an operation on the deep learning network by using the quantized first data based on the target format.
[0005] In one embodiment of the present invention, an optimization method for a deep learning network includes (but is not limited to) obtaining first data, quantizing the first data through power-of-two quantization, where the first data quantized through power-of-two quantization is in a first format or a second format, and a number of first values in the first format or the second format is different, obtaining second data, quantizing the second data through dynamic fixed-point quantization, and performing an operation related to the deep learning network on the quantized first data after power-of-two quantization and the quantized second data after dynamic fixed-point quantization.
[0006] In one embodiment of the present invention, a computing system for a deep learning network includes a memory and a processor. The memory is configured to store a program code. The processor is coupled to the memory. The processor is configured to load the program code to execute the steps of obtaining first data, quantizing the first data into a first format or a second format through power-of-two quantization, where a number of first values in the first format or the second format is different, using the first format or the second format as a target format, and performing a computation on the deep learning network by using the quantized first data based on the target format. Effect of the Invention
[0007] Based on the above, according to the optimization method and the operation system for deep learning network according to the embodiment of the present invention, two data are quantized into a specific format through power-of-two quantization and dynamic fixed-point quantization. In this way, the prediction accuracy can be improved and the model complexity can be reduced.
[0008] In order to make the above features and advantages of the present invention more comprehensible, specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a block diagram of components of a computing system according to one embodiment of the present invention. [Diagram 2] FIG. 1 is a flow diagram of an optimization method for deep learning networks according to one embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram of a quantization comparison according to one embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram of layer-by-layer quantization widths according to one embodiment of the present invention. [Diagram 5] FIG. 2 is a flow diagram of power-of-two quantization according to one embodiment of the present invention. [Figure 6] FIG. 2 is a schematic diagram of quantization levels for power-of-two quantization according to one embodiment of the present invention. [Figure 7] This is an example showing power-of-two quantization. [Figure 8] FIG. 1 is a flow diagram of an optimization method for deep learning networks according to one embodiment of the present invention. [Figure 9] 1 is an example illustrating dynamic fixed-point quantization. [Figure 10] This is another example of dynamic fixed-point quantization. [Figure 11A] This is an example showing the Multiply-Accumulate (MAC) operation for two-hot encoding. [Figure 11B] 1 is an example showing a MAC operation for two-hot encoding. [Figure 12] FIG. 2 is a flow diagram of an application scenario according to one embodiment of the present invention. [Figure 13] FIG. 2 is a flow diagram of model optimization according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Figure 1 is a block diagram of components of a computing system 10 according to one embodiment of the present invention. With reference to Figure 1, computing system 10 includes (but is not limited to) a memory 11 and a processor 12. Computing system 10 may be one or more desktop computers, notebook computers, smartphones, tablet computers, wearable devices, servers, or other electronic devices.
[0011] The memory 11 may be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar component. In one embodiment, the memory 11 is configured to store program code, software modules, configuration settings, data, or files (e.g., data, model parameters, or calculated values), as described in more detail in subsequent embodiments.
[0012] Processor 12 is coupled to memory 11. Processor 12 may be a central processing unit (CPU), a graphic processing unit (GPU), or other programmable general-purpose or application-specific microprocessor, a digital signal processor (DSP), a programmable controller, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a neural network accelerator, or other similar element, or a combination of the above elements. In one embodiment, processor 12 is configured to perform all or a portion of the operations of computing system 10 and may load and execute various program codes, software modules, files, and data stored in memory 11.
[0013] The following describes the method in the embodiment of the present invention in conjunction with various devices, components and modules in the computing system 10. Each process of the method may be adjusted according to the implementation, and the present invention is not limited thereto.
[0014] FIG. 2 is a flow diagram of an optimization method for a deep learning network according to one embodiment of the present invention. Referring to FIG. 2, the processor 12 obtains first data (step S210). In one embodiment, the first data is weights of a pre-trained model based on a deep learning network. It should be noted that in a deep learning network, a convolution operation is realized by sliding a convolution kernel on an input feature map (e.g., an image or an image feature set). The convolution operation involves a multiply-accumulate operation (also called a MAC operation), and the multiply-accumulate operation may be regarded as a sum of weighting operations. From the perspective of the weighting operation, the elements in the convolution kernel used in the convolution operation may be regarded as weights of the weighting operation, and the input feature map may be regarded as values corresponding to the weights.
[0015] The pre-trained model refers to an algorithm based on a deep learning network (e.g., YOLO, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), or Generative Adversarial Network (GAN), and a model trained by inputting training samples and corresponding labels or actual results). A model based on a deep learning network can derive rules from training samples and corresponding labels or actual results by analyzing the relationship between them, and predict unknown data through the rules. The pre-trained model is a machine learning model constructed after learning to infer evaluation data. It should be noted that this pre-trained model can be used for image recognition / classification, object detection, semantic analysis, or other inference, and the embodiments of the present invention do not limit the use of the pre-trained model. In some application scenarios, the trained pre-trained model may be able to meet a preset accuracy standard.
[0016] In other embodiments, the first data may be other data / parameters in a pre-trained model or associated algorithm-related parameters for network-based deep learning.
[0017] In one embodiment, the processor 12 may retrieve the first data from the memory 11, or may retrieve the first data from an external electronic device or storage device via a communication transceiver (e.g., a transceiver supporting Wi-Fi, a mobile network, or a fiber optic network) or a transmission interface (e.g., USB or Thunderbolt).
[0018] The processor 12 quantizes the first data into a first format or a second format through power-of-two quantization (step S220). Specifically, each layer of the pre-trained model has corresponding parameters (e.g., weights, input / output activation values / feature values / feature maps). It is believed that an excessive number of parameters requires high requirements in computation and storage, and the more complex the parameters are, the more computational effort is required. Quantization is one of the techniques used to reduce the complexity of neural networks. Quantization reduces the number of bits used to represent activations / features or weights.
[0019] There are many types of quantization methods, such as symmetric quantization, asymmetric quantization, and clipping. In addition, power-of-two quantization is a non-uniform quantization scheme proposed for bell-shaped and long-tailed weights. This scheme restricts all quantized values to sums of several powers of two (or powers of two) (i.e., power-of-two data sets), which can be computationally efficient and applicable to the numerical distribution of weights. A power-of-two data set is, for example, {0, 2 0 ,2 1 ,2 2 ,2 3 ,2 4 However, it may be a combination of other powers of 2, and the embodiment of the present invention is not limited thereto.
[0020] FIG. 3 is a schematic diagram of quantization comparison according to one embodiment of the present invention. Referring to FIG. 3, the non-uniform quantization level is finer and more uniform, which means that more quantization points are used for quantization when the floating-point value is between 0 and 1. However, the distribution of quantization points of power of 2 quantization is tighter in the area where the floating-point value is close to 0 than in other areas, and the distribution of quantization points in the area where the floating-point value is close to 1 is sparser than in other areas. As can be seen, the quantization method of encoding the values in the data to the power of 2 may cause some precision loss of the whole model. However, the power of 2 quantization is suitable for weight quantization because the sensitivity of weight is low and the numerical distribution is relatively concentrated.
[0021] The first format and the second format are different formats of the power of 2 dataset described above. The number of first values in the first format and / or the second format is different. The first value is, for example, "1". That is, in a power of 2 dataset, the first value is the number of values "1" corresponding to each element in the dataset. For example, the value quantized by a power of 2 is 2, and the dataset {0, 2 0 ,2 1 ,2 2 The values of {0,0,1,0} are {0,0,1,0}, and the number of first values "1" is one. In another example, for example, if the value quantized by a power of 2 is 3, the data set {0,2 0 ,2 1 ,2 2} has the value {0,1,1,0}, and the number of the first value "1" is two.
[0022] In one embodiment, the first format is one-hot encoding, i.e., the number of "1" values corresponding to the data set quantized by a power of two in the first format is one. For example, for the data set {0, 2 0 ,2 1 ,2 2The values of {0,0,1,0} are {0,0,1,0}, and the number of the first value "1" is one. On the other hand, the second format is two-hot encoding, which means that the number of the value "1" corresponding to the data set quantized by powers of two in the second format is two. For example, the data set {0,2 0 ,2 1 ,2 2} has the value {0,1,1,0}, and the number of the first value "1" is two.
[0023] 4 is a schematic diagram of layer-by-layer quantization width according to one embodiment of the present invention. Referring to FIG. 4, a pre-trained model based on a deep learning network is L1 to L N The layer includes N layers, where N is a positive integer. Each layer L1 to L N may be a convolution layer, a pooling layer, a fully connected layer, or other layers, and the embodiment of the present invention does not limit the type of layer. N These layers L1 to L N are the respective scale / scaling coefficients S1 to S N The scaling coefficients S1 to S N represents the proportional relationship between floating point and integer / quantized values. Taking linear quantization as an example, the formula is:
number
[0024] 5 is a flow diagram of power-of-two quantization according to one embodiment of the present invention. Referring to FIG. 5, the processor 12 may determine a scaling factor of the layer to which the first data belongs (step S510). The mathematical expression of the scaling factor is:
number
[0025] The processor 12 may determine the upper limit and the lower limit of the quantization value of the layer to which the first data belongs based on the scaling coefficient (step S520). The quantization value is exemplified as 8 bits, and the formulas for the upper and lower limits of the quantization value are as follows:
number
[0026]
number
[0027] 5, the processor 12 may determine a data set to be used for power-of-two quantization of the first layer of data based on an upper quantization value limit and a lower quantization value limit (step S530). The data set is used to define the quantization values in the first format and the second format. Specifically, the data set Q n The formula for is as follows:
number
[0028] Referring to FIG. 2, the processor 12 selects the first format or the second format as the target format (step S230). In one embodiment, the processor 12 quantizes the first data into the quantized data of the first format and the quantized data of the second format, and then selects one of the quantized data of the first format and the quantized data of the second format. The selected format is the target format. It is noteworthy that the complexity of using one-hot encoding is low, but the prediction accuracy of the model may be lost. On the other hand, using two-hot encoding increases the prediction accuracy, but the complexity increases. In comparison, the embodiment of the present invention can select one of the two formats based on a performance index (e.g., accuracy or complexity) (hereinafter referred to as "hybrid encoding") to meet application requirements.
[0029] In one embodiment, the processor 12 may determine one of the first format and the second format as the target format based on the quantization error. The quantization error is an error between the first data quantized to the first format or the second format by a power of two and the first data (original) that is not quantized by a power of two (e.g., Euclidean distance, Mean Square Error (MSE), Root-Mean-Square Error (RMSE), Least-Mean-Square Error (LMSE), Cosine Similarity, or Cosine Distance). That is, the performance index is the prediction accuracy. If the quantization error is small, the prediction accuracy may be maintained, or otherwise the prediction accuracy may be decreased.
[0030] In one embodiment, the processor 12 may select the format with the smallest quantization error as the target format. For example, if the error between the quantized data of the first format and the first data is smaller than the error between the quantized data of the second format and the first data, the target format is the first format, or vice versa, and the details are not repeated here. Take 0.361 as an example, the value after power-of-two quantization in one-hot encoding is 0.25 (i.e., 2 -2 ), and the value after power-of-two quantization in two-hot encoding is 0.375 (i.e. 2 -2 +2 -1 ) Therefore, the quantization error corresponding to two-hot encoding is relatively small. For example, if 0.243 is taken as an example, the value after power-of-two quantization in one-hot encoding is 0.25 (i.e. 2 -2 ), and the value after power-of-two quantization in two-hot encoding is 0.251953125 (i.e. 2 -2 +2 -9 ) Therefore, the quantization error associated with one-hot encoding is relatively small.
[0031] FIG. 7 is an example showing power-of-two quantization. Referring to FIG. 7, it is illustrated that the floating point of the first data is 32 bits. Assuming that the target format is two-hot encoding, after the power-of-two quantization of the two-hot encoding (step S710), the first data (i.e., 8-bit quantized data) is quantized according to the target format corresponding to the two-hot encoding. That is, the number of values "1" is two.
[0032] 2, the processor 12 uses the first data quantized based on the target format to perform an operation related to the deep learning network (step S240). Specifically, the operation may be a multiplication operation, a MAC operation, and / or an addition operation. For example, if the first data are weights, the quantized data may be used to perform a MAC operation with the (quantized) feature map to implement a convolutional layer, a fully connected layer, or other operation layer.
[0033] 8 is a flow diagram of an optimization method for a deep learning network according to one embodiment of the present invention. Referring to FIG. 8, the processor 12 obtains first data (step S810). Specifically, the details of step S810 can be referred to the description of step S210, and will not be repeated here.
[0034] The processor 12 quantizes the first data through power-of-two quantization (step S820). Specifically, the first data (i.e., the quantized data) after the power-of-two quantization is one of a first format and a second format, and the number of first values in the first format and the second format is different. For details of step S820, please refer to the description of step S220. In one embodiment, the processor 12 may further determine a target format based on step S220. In another embodiment, the processor 12 has previously determined the use of one of the first format and the second format for the power-of-two quantization.
[0035] Meanwhile, the processor 12 obtains second data (step S830). In one embodiment, the second data is a parameter calculated from the second data in the deep learning network. In one embodiment, the second data is a feature value / map or activation value of a pre-trained model based on the deep learning network. For details of the pre-trained model, please refer to the description of step S210, and will not be repeated here. In another embodiment, the second data may be other data / parameters of the pre-trained model, or related algorithm-related parameters for network-based deep learning.
[0036] In one embodiment, the processor 12 retrieves the second data from the memory 11, or retrieves the second data from a communication transceiver (e.g., a transceiver supporting Wi-Fi, a mobile network, or a fiber optic network) or an external electronic device or storage device via a transmission interface (e.g., USB or Thunderbolt).
[0037] The processor 12 quantizes the second data through dynamic fixed-point quantization (step S840). Specifically, dynamic fixed-point quantization is a quantization scheme provided to solve the problem of limited expressiveness of fixed-point quantization. "Dynamic" means that each layer in the pre-trained model has its own quantization parameter. The quantization parameter is, for example, fractional / fractional length, integer length, or bit width.
[0038] FIG. 9 is an example showing dynamic fixed point quantization. Referring to FIG. 9, among the 10 bits representing a value, excluding the sign bit 902, the mantissa 903 includes an integer part 904 and a fractional part 905. When the fraction / fraction length is 3 (i.e., fl=3), the fractional part 905 occupies 3 bits as shown. In some application scenarios, dynamic fixed point quantization is more suitable for hardware implementation than asymmetric quantization. The mathematical formula for dynamic fixed point quantization is as follows:
number
[0039] Figure 10 is another example of dynamic fixed point quantization. Referring to Figure 10, when the floating point of the second data is exemplified as 32 bits, the multi-scale dynamic fixed point quantization quantizes based on the dynamic length (for example, 3) obtained in the preliminary analysis (step S1010), thereby obtaining 8-bit quantized data.
[0040] In one embodiment, the processor 12 may quantize the second data through multi-scale dynamic fixed-point quantization. "Multi-scale" means that the quantizer corresponding to each layer of the pre-trained model uses different quantization parameters in different segments of the value range. The quantization parameters are, for example, fractional / fractional length, integer length, or mantissa. For example, the processor 12 allocates more bit width to the fractional length (FL) for the central segment with tighter value distribution, and the processor 12 allocates more bit width to the integer length (IL) for the tail segment with sparser value distribution, thus achieving "multi-scale".
[0041] 8, the processor 12 performs an operation related to a deep learning network on the first data after the power-of-two quantization and the second data after the dynamic fixed-point quantization (step S850). Specifically, the operation may be a multiplication operation, a MAC operation, and / or an addition operation. For example, when the first data is a weight and the second data is an input / output feature map / value, the quantized data corresponding to the two data may be subjected to a MAC operation to implement a convolution layer, a fully connected layer, or other operation layer.
[0042] In one embodiment, the operation is a multiplication operation, and the target format is one-hot encoding. To realize the multiplication operation, the processor 12 shifts the quantized second data through a shifter according to one position of the first value in the target format. Since only one bit has a value of 1 in one-hot encoding, the processor 12 can find the position of this one bit in the data combination, and shifts the quantized second data accordingly, thereby obtaining the result of the MAC operation. For example, based on the least significant bit being the 0th bit, if "1" is in the 1st bit, the processor 12 shifts the second data by one bit. It can be seen that the multiplication operation can be realized by shifting.
[0043] For example, the MAC operation used in the convolution operation is:
number
number
[0044] In another embodiment, the operation is a multiplication operation, and the target format is two-hot encoding. To realize the multiplication operation, the processor 12 shifts the quantized second data through a shifter according to two positions of the first value in the target format, and adds the shifted second data through an adder. Since the two-hot encoding has two bits with a value of 1, the processor 12 can find the positions of the two bits in the data combination, and shifts the quantized second data accordingly to perform addition, thereby obtaining the result of the MAC operation. It can be seen that the addition operation can be realized by shifting and adding.
[0045]
number
[0046] 11A and 11B are examples of two-hot encoding MAC operations. Referring to FIG. 11A, the quantized second data is [0 1 0 1 0 1 1 0], and the quantized first data is [0 0 0 0 1 0 1 0]. By multiplying the two data, the output data can be obtained as [0 1 0 1 1 1 0 0].
[0047] Referring to FIG. 11B, the quantized first data in FIG. 11A is based on two-hot encoding, so there are two values of "1". Based on the least significant bit being the 0th bit, the 1st bit and the 3rd bit are "1". Therefore, the processor 12 shifts the quantized second data by 1 bit and 3 bits respectively. Then, the processor 12 adds the 1-bit shifted second data and the 3-bit shifted second data, and the output data is still [0 1 0 1 1 1 0 0].
[0048] Since the computational complexity of a shifter and an adder is lower than that of a multiplier, the computational complexity can be reduced by combining a shifter and an adder through power-of-two quantization.
[0049] It should be noted that the above embodiment can be implemented layer by layer.
[0050] For example, Figure 12 is a flow diagram of an application scenario according to one embodiment of the present invention. Referring to Figure 12 showing a specific layer of a pre-trained model, the processor 12 obtains an input feature map of 32-bit floating-point values (step S1201), and performs multi-scale dynamic fixed-point quantization on the input feature map (step S1202), thereby quantizing the 32-bit output feature map into an 8-bit quantized input feature map. Meanwhile, the processor 12 obtains weights of 32-bit floating-point values (step S1204), and performs power-of-two quantization on the weights (step S1205), thereby quantizing the 32-bit floating-point weights into 8-bit quantized weights. The selection of one of the first format and the second format is called "hybrid encoding". The processor 12 inputs the quantized feature map and the quantized weights into the feature layer of the pre-trained model to perform corresponding operations (step S1207). Next, the processor 12 performs multi-scale dynamic fixed-point quantization on the calculated output feature map to quantize the output feature map into an 8-bit quantized output feature map (step S1209). The processor 12 may use the input feature map as the next layer and repeat the above steps.
[0051] FIG. 13 is a flow diagram of model optimization according to one embodiment of the present invention. Referring to FIG. 13, after the parameters of the model are quantized for each layer, the processor 12 may obtain a quantized model. The processor 12 may post-train the quantized model (step S1310). For example, the quantized model may be trained using training samples with labeled results. The processor 12 may re-train / fine-tune the trained quantized model (step S1320). In some application scenarios, post-training the trained model may reduce prediction accuracy. Therefore, the accuracy can be improved by fine-tuning. For example, a straight-through estimator with boundary constraint (STEBC) is used to determine the gradient of quantization where the parameter type is weight. Finally, the processor 12 may obtain a fine-tuned quantized model (step S1330). In this way, the prediction accuracy can be further improved.
[0052] Depending on the application scenario, the quantized model may be used for image recognition, object detection, semantic analysis, or other inference.
[0053] In summary, in the optimization method and operation system for deep learning network of the embodiment of the present invention, the first data that is one-hot encoded after power-of-two quantization is adopted, and the second data is quantized by multi-scale dynamic fixed-point quantization and input into a certain layer of the trained model for operation. In this way, the prediction accuracy can be improved (for example, up to 85% or more, or the difference is less than 3% compared with the unquantized model), and is applied to implementation through hardware.
[0054] Although the present invention has been disclosed above by the embodiments, it is not intended to limit the present invention thereto. Those skilled in the art may make some modifications and changes without departing from the spirit and scope of the present invention. Therefore, the scope protected by the present invention should be defined by the appended claims. [Industrial Applicability]
[0055] The optimization method and computing system for deep learning networks of the present invention can be applied to machine learning techniques. [Explanation of symbols]
[0056] 10: Calculation system 11: Memory 12: Processor S210~S240, S510~S30, S710, S810~S850, S1010, S1201~S1209, S1310~S1330: Step L1~L N :layer S1~S N : Scaling factor 902: Sign bit 903: Mantissa 904: Integer part 905: Decimal part fl:fraction length
Claims
1. Obtaining first data; quantizing the first data into a first format or a second format through power-of-two quantization, wherein a number of first values in the first format or the second format is different; using the first format or the second format as a target format; performing an operation on a deep learning network using the first data quantized based on the target format; Including, Optimization methods for deep learning networks.
2. Using the first format or the second format as the target format includes: determining one of the first format and the second format as the target format based on a quantization error, the quantization error being an error between the first data quantized into the first format or the second format by the power-of-two quantization and the first data not quantized by the power-of-two quantization. Including, Or, The first data belongs to one of a plurality of layers in a pre-trained model based on the deep learning network, and quantizing the first data into the first format or the second format through the power-of-two quantization includes: determining a scaling factor for the layer; determining an upper bound for a quantization value of the layer and a lower bound for the quantization value based on the scaling factor; determining a data set for the power of two quantization for the layer based on the upper limit of the quantization values and the lower limit of the quantization values, the data set being used to define quantization values in the first format and the second format; Including, 2. The optimization method for a deep learning network according to claim 1.
3. the first format is one-hot encoding, the second format is two-hot encoding, the first data are weights of a pre-trained model based on the deep learning network, the operation is a multiplication operation, the target format is the one-hot encoding, and performing the operation on the deep learning network using the first data quantized based on the target format; shifting second data through a shifter based on a position of the first value in the target format, the second data being a parameter for performing the operation on the first data in the deep learning network. Including, Or, the operation is a multiplication operation, the target format is two-hot encoding, and performing the operation on the deep learning network using the first data quantized based on the target format includes: shifting the second data through the shifter based on two positions of the first value in the target format; adding the shifted second data via an adder; Including, 2. The optimization method for a deep learning network according to claim 1.
4. Obtaining first data; quantizing the first data through power-of-two quantization, the first data quantized through the power-of-two quantization being in a first format or a second format, and a number of first values in the first format or the second format being different; Obtaining second data; and quantizing the second data through dynamic fixed point quantization; performing an operation related to a deep learning network on the first quantized data after the power-of-two quantization and the second quantized data after the dynamic fixed point quantization; Including, Optimization methods for deep learning networks.
5. a memory configured to store program code; coupled to the memory; Obtaining first data; quantizing the first data into a first format or a second format through power-of-two quantization, wherein a number of first values in the first format or the second format is different; using the first format or the second format as a target format; performing an operation on a deep learning network using the first data quantized based on the target format; a processor configured to load said program code to execute Including, A computing system for deep learning networks.
