Deep learning feature quantization method and device
By scaling and quantizing floating-point feature values, the impact of quantization distortion on machine vision feature encoding is resolved, thereby improving encoding efficiency and fidelity.
Patent Information
- Application Number
- CN202110216652.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-02-26
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-02-26
AI Technical Summary
Existing video coding methods cannot effectively reduce the impact of quantization distortion on machine vision feature encoding, resulting in low coding efficiency and high distortion rate.
A deep learning feature quantization method is adopted to reduce the impact of quantization distortion by scaling and quantizing the floating-point feature values and processing their integer and fractional parts separately.
It improves the efficiency and fidelity of machine vision feature encoding and reduces the distortion rate during the encoding process.
Smart Images

Figure CN114979639B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of signal coding in machine vision data communication, and in particular to a deep learning feature quantization method and device. Background Art
[0002] With the growth of machine learning applications, many intelligent platforms have been adopted in the fields of Internet of Vehicles, video surveillance, smart cities, etc., and these platforms have generated massive data communications with a large number of sensors. The increase in data volume directly leads to the inefficiency of previous coding methods for human vision, and it is difficult to meet the reality in terms of latency and scale. Feature coding for intelligent machines has been put on the agenda.
[0003] Transform coding is to reduce the spatial correlation of image signals from the perspective of frequency domain. It has achieved similar results as predictive coding in terms of reducing bit rate. After entering the 1980s, a set of hybrid coding schemes combining motion compensation and transform coding was gradually formed, which greatly promoted the development of digital video coding technology. In the early 1990s, ITU proposed the famous video coding recommendation H.261 for conference video applications, which was the first widely used hybrid coding scheme. Later, with the continuous improvement of video coding standards and recommendations such as H.264, MPEG1 / MPEG2 / MPEG4, and the introduction of H.264, hybrid coding technology gradually matured and became the most widely used digital video coding technology.
[0004] Transform coding does not directly encode the spatial image signal, but first maps the spatial image signal to another orthogonal vector space (transform domain or frequency domain), generates a batch of transform coefficients, and then encodes these transform coefficients. Transform coding is an indirect coding method. The key problem is that when describing in the time domain or spatial domain, the data has a large correlation and data redundancy. After being transformed and described in the transform domain, the data correlation is greatly reduced, the data redundancy is reduced, the parameters are independent, and the amount of data is small. In this way, quantization is performed and the coding can obtain a larger compression ratio. Typical quasi-optimal transforms include DCT (discrete cosine transform), DFT (discrete Fourier transform), WHT (Walsh Hadama transform), HrT (Haar transform), etc. Among them, the most commonly used is DCT transform.
[0005] The principle of DCT transformation for image compression is to reduce the high-frequency components in the image. The high frequency mainly corresponds to the detail information in the image, and our human eyes are not very sensitive to the detail information, so the high-frequency information can be removed.
[0006] The principle of quantization is to divide the changed DCT coefficient by a constant. The result after quantization is an integer multiple of the quantization step size or more zero values, thereby achieving the purpose of compression.
[0007] However, the key in machine learning is feature extraction. Model building, training, and prediction are inseparable from the feature information extracted from the original information. The purpose of visual information encoding for intelligent machines is no longer to compress and restore videos or images in human vision, but to compress and restore extracted features to meet the tasks of intelligent machine learning.
[0008] The existing video change coding is mainly based on the high-frequency components in the image, which mainly correspond to the details of the image. The human eye is not very sensitive to details. By changing from the spatial domain to the frequency domain, the data distribution is changed and the high-frequency information is removed. The deep learning features for machine vision are used for computer vision tasks. The larger the deep learning feature layer, the more 0 feature values there are. The maximum proportion of 0 values can reach more than 90%. For this type of feature value, the need to remove the correlation between features is not obvious.
[0009] Existing videos are based on RGB pixel information, and the numerical representation is an integer [0,255]. Deep learning features for machine vision are all floating point numbers, and the feature map is the result of the previous convolution. The size of the feature value is inversely proportional to the number of layers, and the maximum value of the feature ranges from hundreds to tens of thousands. Statistical analysis shows that the feature map feature values are highly sparse and weakly correlated. Compared with traditional video coding, feature coding data format and data range are very different.
[0010] The distortion of video encoding is caused by quantization, and the quantization of floating-point numbers will inevitably introduce new distortion. The impact of double distortion on eigenvalues has a drastic impact on machine vision tasks. Therefore, it is necessary to introduce a new quantization method to minimize the impact of distortion. Summary of the invention
[0011] A technical problem to be solved by the present disclosure is to provide a deep learning feature quantization method and device, which can reduce the impact of quantization distortion on subsequent encoding.
[0012] According to one aspect of the present disclosure, a deep learning feature quantization method is proposed, including: determining a scaling value corresponding to a feature value according to a scaling coefficient of the feature value, wherein the feature value is a floating point number; converting an integer part corresponding to the scaling value into a first binary number, and shifting the first binary number left by a predetermined number of bits to obtain a first pre-quantization value; obtaining a quantization step size according to a set decimal bit depth of the feature value quantization, dividing the decimal part corresponding to the scaling value by the quantization step size and rounding it down to a second binary number to obtain a second pre-quantization value; calculating the sum of the first pre-quantization value and the second pre-quantization value to obtain a third pre-quantization value; and setting a flag bit for the third pre-quantization value to obtain a quantization result of the feature value.
[0013] In some embodiments, determining the scaling factor of the eigenvalue includes: determining whether the eigenvalue is less than or equal to a first predetermined value of 2, if so, setting the scaling factor to 1, otherwise, setting the scaling factor to the reciprocal of a second predetermined value of 2, wherein the first predetermined value is the integer bit depth of the quantized eigenvalue, and the second predetermined value is the difference between the logarithm of the eigenvalue with base 2, rounded up, and the integer bit depth.
[0014] In some embodiments, shifting the first binary number left by a predetermined number of bits includes: shifting the first binary number left by a fractional bit depth bit.
[0015] In some embodiments, the third pre-quantization value setting flag is a binary number corresponding to the inverse of the scaling factor.
[0016] In some embodiments, obtaining the quantization result of the eigenvalue includes: shifting the binary number corresponding to the reciprocal of the scaling factor left by the sum of the integer bit depth and the decimal bit depth, and adding the binary number to the third pre-quantization value to obtain the quantization result of the eigenvalue.
[0017] In some embodiments, the quantization result of the eigenvalue is bitwise ANDed with the binary number corresponding to the inverse of the quantization step length minus 1 to obtain the quantization value of the decimal part of the scaling value; the quantization value of the decimal part of the scaling value is multiplied by the quantization step length to obtain the inverse quantization value of the decimal part of the scaling value; the quantization result of the eigenvalue is right-shifted by a predetermined number of bits, and bitwise ANDed with the binary number corresponding to the first predetermined value power of 2 minus 1 to obtain the quantization value of the integer part of the scaling value; the decimal number corresponding to the sum of the inverse quantization value of the decimal part of the scaling value and the quantization value of the integer part of the scaling value is calculated to obtain the scaling value; the quantization result of the eigenvalue is right-shifted by the sum of the set integer bit depth and the decimal bit depth to obtain the flag bit; and the decimal number corresponding to the flag bit and the scaling value are calculated to obtain the inverse quantized eigenvalue.
[0018] In some embodiments, after removing the flag bit of the quantization result of the eigenvalue, the binary number corresponding to the inverse of the quantization step minus 1 is bitwise ANDed to obtain the quantization value of the decimal part of the scaling value; the quantization value of the decimal part of the scaling value is multiplied by the quantization step to obtain the inverse quantization value of the decimal part of the scaling value; after removing the flag bit of the quantization result of the eigenvalue, the binary number corresponding to the first predetermined value power of 2 minus 1 is bitwise ANDed to obtain the quantization value of the integer part of the scaling value; the decimal number corresponding to the sum of the inverse quantization value of the decimal part of the scaling value and the quantization value of the integer part of the scaling value is calculated to obtain the scaling value; and the scaling value is multiplied by the decimal number corresponding to the flag bit to obtain the inverse quantization eigenvalue.
[0019] According to another aspect of the present disclosure, a deep learning feature quantization device is also proposed, including: an eigenvalue scaling unit, configured to determine a scaling value corresponding to a eigenvalue according to a scaling coefficient of the eigenvalue, wherein the eigenvalue is a floating point number; an integer part quantization unit, configured to convert the integer part corresponding to the scaling value into a first binary number, and shift the first binary number left by a predetermined number of bits to obtain a first pre-quantization value; a decimal part quantization unit, configured to obtain a quantization step size according to a set decimal bit depth of the eigenvalue quantization, divide the decimal part corresponding to the scaling value by the quantization step size and round it down to convert it into a second binary number to obtain a second pre-quantization value; and a quantization result determination unit, configured to calculate the sum of the first pre-quantization value and the second pre-quantization value to obtain a third pre-quantization value, set a flag bit for the third pre-quantization value, and obtain a quantization result of the eigenvalue.
[0020] In some embodiments, the third pre-quantization value setting flag is a binary number corresponding to the inverse of the scaling factor.
[0021] In some embodiments, the quantization result determination unit is further configured to left-shift the binary number corresponding to the inverse of the scaling factor by the sum of the set integer bit depth and the decimal bit depth, and then add the binary number to the third pre-quantization value to obtain the quantization result of the eigenvalue.
[0022] In some embodiments, the decimal part dequantization unit is configured to perform bitwise AND operation on the quantization result of the eigenvalue and the binary number corresponding to the reciprocal of the quantization step length minus 1 to obtain the quantization value of the decimal part of the scaling value, and multiply the quantization value of the decimal part of the scaling value by the quantization step length to obtain the decimal part of the scaling value; the integer part dequantization unit is configured to perform bitwise AND operation on the quantization result of the eigenvalue and the binary number corresponding to the first predetermined value power of 2 minus 1 to obtain the quantization value of the integer part of the scaling value after shifting the quantization result of the eigenvalue right by a predetermined number of bits, and obtain the decimal number corresponding to the decimal part of the scaling value; the scaling value calculation unit is configured to calculate the decimal number corresponding to the sum of the decimal part of the scaling value and the quantization value of the integer part of the scaling value to obtain the scaling value; and the decimal eigenvalue determination unit is configured to shift the quantization result of the eigenvalue right by the sum of the set integer bit depth and the decimal bit depth to obtain the flag bit, calculate the decimal number corresponding to the flag bit, and the product of the scaling value to obtain the decimal eigenvalue.
[0023] In some embodiments, the decimal part dequantization unit is configured to remove the flag bit of the quantization result of the eigenvalue, and perform bitwise AND calculation with the binary number corresponding to the inverse of the quantization step size minus 1 to obtain the quantization value of the decimal part of the scaling value, and multiply the quantization value of the decimal part of the scaling value by the quantization step size to obtain the decimal part of the scaling value; the integer part dequantization unit is configured to remove the flag bit of the quantization result of the eigenvalue, right shift it by a predetermined number of bits, and perform bitwise AND calculation with the binary number corresponding to the first predetermined value power of 2 minus 1 to obtain the quantization value of the integer part of the scaling value; the scaling value calculation unit is configured to calculate the decimal number corresponding to the sum of the decimal part of the scaling value and the quantization value of the integer part of the scaling value to obtain the scaling value; the decimal eigenvalue determination unit is configured to multiply the scaling value by the decimal number corresponding to the flag bit to obtain the decimal eigenvalue.
[0024] According to another aspect of the present disclosure, a deep learning feature quantization device is also proposed, including: a memory; and a processor coupled to the memory, the processor being configured to execute the deep learning feature quantization method as described above based on instructions stored in the memory.
[0025] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is also proposed, on which computer program instructions are stored, and when the instructions are executed by a processor, the deep learning feature quantization method as described above is implemented.
[0026] In the disclosed embodiment, by scaling the floating-point eigenvalues and then quantizing the integer and decimal parts of the scaled values respectively to obtain the quantization results of the eigenvalues, the impact of quantization distortion on subsequent encoding can be reduced, thereby improving the efficiency and high-fidelity goals of machine vision feature encoding.
[0027] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0029] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
[0030] Figure 1 The present invention is a flowchart of some embodiments of the deep learning feature quantization method disclosed in the present invention.
[0031] Figure 2 Schematic diagram of the flow chart of other embodiments of the deep learning feature quantization method disclosed in the present invention.
[0032] Figure 3 Schematic diagram of VGG-C5 eigenvalue distribution.
[0033] Figure 4 Schematic diagram of feature encoding.
[0034] Figure 5 Schematic diagram of the flow chart of other embodiments of the deep learning feature quantization method disclosed in the present invention.
[0035] Figure 6 Schematic diagram of the flow chart of other embodiments of the deep learning feature quantization method disclosed in the present invention.
[0036] Figure 7 Schematic diagram of the flow chart of other embodiments of the deep learning feature quantization method disclosed in the present invention.
[0037] Figure 8 It is a schematic diagram of the structure of some embodiments of the deep learning feature quantization device disclosed in the present invention.
[0038] Fig. 9 Schematic diagrams of the structures of other embodiments of the deep learning feature quantization device disclosed in the present invention.
[0039] Fig.10 Schematic diagrams of the structures of other embodiments of the deep learning feature quantization device disclosed in the present invention. DETAILED DESCRIPTION
[0040] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.
[0041] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0042] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0043] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.
[0044] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0045] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0046] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0047] Figure 1 The present invention is a flowchart of some embodiments of the deep learning feature quantization method disclosed in the present invention.
[0048] In step 110, a scaling value corresponding to the eigenvalue is determined according to a scaling coefficient of the eigenvalue, wherein the eigenvalue is a floating point number.
[0049] In some embodiments, it is determined whether the eigenvalue is less than or equal to the first predetermined value of 2. If so, the scaling factor is set to 1. Otherwise, the scaling factor is set to the reciprocal of the second predetermined value of 2, wherein the first predetermined value is the integer bit depth of the quantized eigenvalue, and the second predetermined value is the difference between the logarithm of the eigenvalue with 2 as the base, rounded up, and the integer bit depth. The integer bit depth is set in combination with the eigenvalue distribution range to ensure that the original value is retained in the high-density area as much as possible.
[0050] In some embodiments, if value≤2 BDI , where value is the eigenvalue and BDI is the set integer bit depth, the scaling factor is set to 1, and the scaling value value′ corresponding to the eigenvalue value is value. If value>2 BDI , the scaling factor is set to The scaling value value′ corresponding to the eigenvalue value is
[0051] In some embodiments, the characteristic value and the scaling value corresponding to the characteristic value are decimal numbers.
[0052] In step 120, the integer part corresponding to the scaling value is converted into a first binary number, and the first binary number is shifted left by a predetermined number of bits to obtain a first pre-quantized value.
[0053] In some embodiments, the predetermined number of bits is a decimal bit depth of a set eigenvalue quantization.
[0054] In step 130, a quantization step is obtained according to the set decimal bit depth of the eigenvalue quantization, and the decimal part corresponding to the scaling value is divided by the quantization step and rounded down to convert into a second binary number to obtain a second pre-quantization value.
[0055] In some embodiments, the quantization step size is set to Wherein, BDF is the decimal bit depth. When calculating the second pre-quantized value, that is, the quantized value of the decimal part of the scaling value, int(f*2 BDF ) is converted into a binary number to obtain the second pre-quantized value, where f is the decimal part corresponding to the scaling value.
[0056] In step 140, the sum of the first prequantization value and the second prequantization value is calculated to obtain a third prequantization value.
[0057] In step 150, a flag is set for the third pre-quantization value to obtain a quantization result of the characteristic value.
[0058] In some embodiments, the flag bit is a binary number that is the inverse of the scaling factor.
[0059] In some embodiments, the binary number of the reciprocal of the scaling factor is left-shifted by the sum of the integer bit depth and the decimal bit depth, and then added to the third pre-quantization value to obtain a quantization result of the eigenvalue.
[0060] In some embodiments, the flag bit may be set on the left side of the third pre-quantization value, or may be set on the right side of the third pre-quantization value.
[0061] In the above embodiment, by scaling the floating-point eigenvalue and then quantizing the integer part and the decimal part of the scaled value respectively to obtain the quantization result of the eigenvalue, the impact of quantization distortion on subsequent encoding can be reduced, thereby improving the efficiency and high-fidelity goal of machine vision feature encoding.
[0062] Figure 2 Schematic diagram of the flow chart of other embodiments of the deep learning feature quantization method disclosed in the present invention.
[0063] In step 210 , it is determined whether the characteristic value is 0. If so, a value of 0 is returned, otherwise, step 220 is executed.
[0064] In step 220 , it is determined whether the eigenvalue value is less than or equal to 2 raised to the power of BDI. If so, step 230 is executed; otherwise, step 240 is executed.
[0065] In step 230, the scaled value value' of the feature value value is determined to be the original value value.
[0066] In step 240, the scaling value value' of the feature value is determined as
[0067] For example, taking a 228x228x3 RGB image as input, under the VGG16 network model, in the feature map of the output conv5, taking the feature value 400.76 as an example, Figure 3As shown, 60% of the conv5 feature values are distributed between 0 and 32, where BDI is 5, 400.76>2 5 . is 1 / 16, then the scaled value of 400.76 is 25.0475.
[0068] In step 250, the integer part of the scaling value is converted into a binary number and left-shifted by BDF bits to obtain a first pre-quantized value.
[0069] The integer part of 25.0475 is 25, and the corresponding binary number is 11001. BDF is 2, and the first pre-quantization value is 1100100.
[0070] At step 260, the fractional part of the scaled value is divided by 1 / 2. BDF , which is then rounded down and converted into a binary number to obtain a second pre-quantized value.
[0071] The quantization step is 1 / 2 BDF =1 / 4, int(0.0475 / (1 / 4))=0, the second pre-quantization value is 0.
[0072] In step 270, the sum of the first prequantization value and the second prequantization value is calculated to obtain a third prequantization value.
[0073] 1100100+0=1100100, so the third pre-quantization value is 1100100.
[0074] In step 280, The corresponding binary number is shifted left by (BDF+BDI) bits and added to the third pre-quantized value to obtain a quantized result of the eigenvalue.
[0075] The corresponding binary number is 10000, which is shifted left by 7 bits to 1000000000000, and added to 1100100, we get 100001100100. The quantization result of the eigenvalue is 100001100100.
[0076] In some embodiments, Figure 4 As shown in the figure, for the eigenvalues of the image, the transmitter directly quantizes the eigenvalues, then reorders and encodes them in sequence, and sends the data to the receiver through the channel. After the receiver decodes and reverses the reordering, it also needs to perform the inverse quantization process. This avoids the DCT and IDCT transformations in the video encoding process and reduces the coding distortion rate. The inverse quantization process is shown in the figure. Figure 5 shown.
[0077] In step 510, the quantization result of the characteristic value and the binary number corresponding to the reciprocal of the quantization step length minus 1 are bit-wise ANDed to obtain the quantization value of the fractional part of the scaling value.
[0078] In some embodiments, the quantization step size is 1 / 2 BDF If BDF is 2, the binary number corresponding to the reciprocal of the quantization step minus 1 is 11, and the quantization result of the eigenvalue is 100001100100, then the quantization value of the decimal part of the scaling value is 0.
[0079] In step 520, the quantized value of the fractional part of the scaling value is multiplied by the quantization step size to obtain an inverse quantized value of the fractional part of the scaling value.
[0080] In some embodiments, the quantization value of the fractional part of the scaling value is 0, and the quantization step is 1 / 4, so the dequantization value of the fractional part of the scaling value is 0.
[0081] In step 530, after the quantized result of the characteristic value is right-shifted by a predetermined number of bits, an AND operation is performed bit by bit with the binary number corresponding to the first predetermined value power of 2 minus 1 to obtain the quantized value of the integer part of the scaling value.
[0082] In some embodiments, the predetermined number of bits is the decimal bit depth of the set eigenvalue quantization, the quantization result of the eigenvalue is 100001100100, the decimal bit depth is 2, then the quantization result of the eigenvalue is 1000011001 after being right-shifted by the predetermined number of bits, and the first predetermined value is the set integer bit depth, for example, 5, and the binary number corresponding to the first predetermined value of 2 minus 1 is 11111, then 1000011001 and 11111 are bit-wise ANDed to obtain 11001, which is the quantization value of the integer part of the scaling value.
[0083] In step 540, a decimal number corresponding to the sum of the inverse quantized value of the fractional part of the scaling value and the quantized value of the integer part of the scaling value is calculated to obtain the scaling value.
[0084] In some embodiments, the dequantized value of the fractional part of the scaling value is 0, the quantized value of the integer part of the scaling value is 11001, and the scaling value is 25.
[0085] In step 550, the quantized result of the eigenvalue is shifted right by the sum of the integer bit depth and the decimal bit depth to obtain a flag bit.
[0086] In some embodiments, 100001100100 is shifted right by 7 bits to obtain 10000.
[0087] In step 560, the product of the decimal number corresponding to the flag bit and the scaling value is calculated to obtain the inverse quantized eigenvalue.
[0088] In some embodiments, the decimal number corresponding to 10000 is 16, the scaling value is 25, and the dequantized eigenvalue is 400.
[0089] In the above embodiment, although the dequantization process has a slight loss, the impact on the eigenvalue is small.
[0090] Figure 6 Schematic diagram of the flow chart of other embodiments of the deep learning feature quantization method disclosed in the present invention.
[0091] In step 610 , it is determined whether the quantization result of the eigenvalue is 0. If so, the eigenvalue 0 is returned. Otherwise, step 620 is executed.
[0092] In step 620, the quantization result is compared with (2 BDF -1) is bitwise ANDed to obtain the decimal distribution quantization value of the scaling value.
[0093] At step 630, the fractional distribution quantization value of the scaled value is quantized with the quantization step size 1 / 2 BDF Then the decimal part of the scaled value is dequantized.
[0094] In step 640, the quantized result is right-shifted by BDF bits, and (2 BDI -1) is bitwise ANDed to obtain the quantized value of the integer part of the scaling value.
[0095] In step 650, the quantized value of the integer part of the scaling value is added to the decimal part of the scaling value and then converted into a decimal number to obtain the scaling value.
[0096] In step 660, the quantization result is right-shifted by (BDF+BDI) bits and multiplied by the scaling value to obtain an inverse quantized eigenvalue.
[0097] In some embodiments of the present disclosure, since the flag bit can be set on the left side or on the right side, the quantization method can also be as follows: Figure 7 shown.
[0098] In step 710, after removing the flag bit of the quantization result of the eigenvalue, the binary number corresponding to the reciprocal of the quantization step size minus 1 is bitwise ANDed to obtain the quantization value of the decimal part of the scaling value.
[0099] For example, the quantization result is 100001100100, or 110010010000, the flag is 10000, the quantization step is 1 / 4, and the binary number corresponding to the reciprocal of the quantization step minus 1 is 11, then the quantization value of the decimal part of the scaling value is 0.
[0100] In step 720, the quantized value of the fractional part of the scaling value is multiplied by the quantization step size to obtain an inverse quantized value of the fractional part of the scaling value.
[0101] In step 730, after removing the flag bit of the quantization result of the eigenvalue and right shifting by a predetermined number of bits, the binary number corresponding to the first predetermined value power of 2 minus 1 is bitwise ANDed to obtain the quantization value of the integer part of the scaling value.
[0102] The first predetermined value is 5, and the quantized value of the integer part of the scaling value is 11001.
[0103] In step 740, a decimal number corresponding to the sum of the inverse quantized value of the fractional part of the scaling value and the quantized value of the integer part of the scaling value is calculated to obtain the scaling value.
[0104] The decimal part of the scaling value has an inverse quantization value of 0, and the integer part of the scaling value has a quantization value of 11001, so the scaling value is 25.
[0105] In step 750, the scaling value is multiplied by the decimal number corresponding to the flag bit to obtain an inverse quantized eigenvalue.
[0106] Figure 8 The structure diagram of some embodiments of the deep learning feature quantization device disclosed in the present invention includes: a feature value scaling unit 810, an integer part quantization unit 820, a fractional part quantization unit 830 and a quantization result determination unit 840.
[0107] The eigenvalue scaling unit 810 is configured to determine a scaling value corresponding to the eigenvalue according to a scaling coefficient of the eigenvalue, wherein the eigenvalue is a floating point number.
[0108] In some embodiments, it is determined whether the eigenvalue is less than or equal to the first predetermined value of 2. If so, the scaling factor is set to 1. Otherwise, the scaling factor is set to the reciprocal of the second predetermined value of 2, wherein the first predetermined value is the integer bit depth of the quantized eigenvalue, and the second predetermined value is the difference between the logarithm of the eigenvalue with 2 as the base, rounded up, and the integer bit depth. The integer bit depth is set in combination with the eigenvalue distribution range.
[0109] In some embodiments, the characteristic value and the scaling value corresponding to the characteristic value are decimal numbers.
[0110] The integer part quantization unit 820 is configured to convert the integer part corresponding to the scaling value into a first binary number, and shift the first binary number left by a predetermined number of bits to obtain a first pre-quantized value.
[0111] In some embodiments, the predetermined number of bits is a decimal bit depth of a set eigenvalue quantization.
[0112] The fractional part quantization unit 830 is configured to obtain a quantization step size according to a set fractional bit depth of the eigenvalue quantization, divide the fractional part corresponding to the scaling value by the quantization step size and round it down to a second binary number to obtain a second pre-quantized value.
[0113] In some embodiments, the quantization step size is set to Wherein, BDF is the decimal bit depth. When calculating the second pre-quantized value, that is, the quantized value of the decimal part of the scaling value, int(f*2 BDF ) is converted into a binary number to obtain the second pre-quantized value, where f is the decimal part corresponding to the scaling value.
[0114] The quantization result determination unit 840 is configured to calculate the sum of the first pre-quantization value and the second pre-quantization value to obtain a third pre-quantization value, set a flag bit for the third pre-quantization value, and obtain a quantization result of the characteristic value.
[0115] In some embodiments, the flag bit is a binary number that is the inverse of the scaling factor.
[0116] In some embodiments, the binary number of the reciprocal of the scaling factor is left-shifted by the sum of the integer bit depth and the decimal bit depth, and then added to the third pre-quantization value to obtain a quantization result of the eigenvalue.
[0117] In some embodiments, the flag bit may be set on the left side of the third pre-quantization value, or may be set on the right side of the third pre-quantization value.
[0118] In the above embodiment, by scaling the floating-point eigenvalue and then quantizing the integer part and the decimal part of the scaled value respectively, the quantization result of the eigenvalue is obtained, which can reduce the impact of distortion and improve the efficiency and high-fidelity goal of machine vision feature encoding.
[0119] Fig. 9 The structure diagram of some other embodiments of the deep learning feature quantization device disclosed in the present invention further includes a fractional part dequantization unit 910, an integer part dequantization unit 920, a scaling value calculation unit 930 and a dequantization feature value determination unit 940.
[0120] The decimal part inverse quantization unit 910 is configured to perform bitwise AND calculation on the quantization result of the characteristic value and the binary number corresponding to the inverse of the quantization step minus 1 to obtain the quantization value of the decimal part of the scaling value, and multiply the quantization value of the decimal part of the scaling value by the quantization step to obtain the inverse quantization value of the decimal part of the scaling value.
[0121] The integer part dequantization unit 920 is configured to right-shift the quantized result of the characteristic value by a predetermined number of bits, and then perform bitwise AND operation with the binary number corresponding to the first predetermined power of 2 minus 1 to obtain the quantized value of the integer part of the scaling value.
[0122] The scaling value calculation unit 930 is configured to calculate a decimal number corresponding to the sum of the inverse quantized value of the fractional part of the scaling value and the quantized value of the integer part of the scaling value to obtain the scaling value.
[0123] The inverse quantization eigenvalue determination unit 940 is configured to right shift the quantization result of the eigenvalue by the sum of the integer bit depth and the decimal bit depth to obtain a flag bit, calculate the product of the decimal number corresponding to the flag bit and the scaling value to obtain the inverse quantization eigenvalue.
[0124] In other embodiments of the present disclosure, the decimal part inverse quantization unit 910 is configured to remove the flag bit of the quantization result of the characteristic value, and then perform bitwise AND calculation on the binary number corresponding to the inverse of the quantization step size minus 1 to obtain the quantization value of the decimal part of the scaled value, and multiply the quantization value of the decimal part of the scaled value by the quantization step size to obtain the inverse quantization value of the decimal part of the scaled value.
[0125] The integer part dequantization unit 920 is configured to remove the flag bit of the quantization result of the characteristic value, right shift it by a predetermined number of bits, and perform bitwise AND calculation with the binary number corresponding to the first predetermined value power of 2 minus 1 to obtain the quantization value of the integer part of the scaling value.
[0126] The scaling value calculation unit 930 is configured to calculate a decimal number corresponding to the sum of the inverse quantized value of the fractional part of the scaling value and the quantized value of the integer part of the scaling value to obtain the scaling value.
[0127] The inverse quantization characteristic value determining unit 940 is configured to multiply the scaling value by a decimal number corresponding to the flag bit to obtain an inverse quantization characteristic value.
[0128] The present disclosure can reduce the severe impact of double distortion on machine vision tasks.
[0129] Fig.10 1 is a schematic diagram of the structure of some other embodiments of the deep learning feature quantization device disclosed in the present invention. The device 1000 includes a memory 1010 and a processor 1020. The memory 1010 can be a disk, a flash memory or any other non-volatile storage medium. The memory is used to store Figure 1-7 The processor 1020 is coupled to the memory 1010 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 1020 is used to execute instructions stored in the memory.
[0130] In some embodiments, the processor 1020 is coupled to the memory 1010 via a BUS 1030. The device 1000 can also be connected to an external storage system 1050 via a storage interface 1040 to call external data, and can also be connected to a network or another computer system (not shown) via a network interface 1060. No further details will be given here.
[0131] In this embodiment, by storing data instructions in a memory and then processing the instructions through a processor, the impact of distortion can be reduced, thereby improving the efficiency and high-fidelity goal of machine vision feature encoding.
[0132] In some other embodiments, a computer readable storage medium stores computer program instructions, which when executed by a processor implement Figure 1-7 The steps of the method in the corresponding embodiment. Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0133] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0136] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.
[0137] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
[0138] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. A deep learning feature quantization method, include: Determine whether the image feature value is less than or equal to a first predetermined value of 2, if so, set the scaling factor to 1, otherwise, set the scaling factor to the reciprocal of a second predetermined value of 2, wherein the first predetermined value is the integer bit depth of the quantized image feature value, and the second predetermined value is the difference between the logarithm of the image feature value with 2 as the base, rounded up, and the integer bit depth; Determine, according to a scaling factor of an image feature value, a scaling value corresponding to the image feature value, wherein the image feature value is a floating point number; Converting the integer part corresponding to the scaling value into a first binary number, and shifting the first binary number left by a predetermined number of bits to obtain a first pre-quantized value; A quantization step size is obtained according to a set decimal bit depth of the image feature value quantization, and a decimal part corresponding to the scaling value is divided by the quantization step size and rounded down to convert the result into a second binary number to obtain a second pre-quantization value; calculating a sum of the first prequantization value and the second prequantization value to obtain a third prequantization value; and A flag is set for the third pre-quantization value to obtain a quantization result of the image feature value.
2. The deep learning feature quantification method according to claim 1, in, Shifting the first binary number left by a predetermined number of bits comprises: The first binary number is shifted left by the fractional bit depth.
3. The deep learning feature quantification method according to claim 1 or 2, in, The third pre-quantization value setting flag is a binary number corresponding to the reciprocal of the scaling factor.
4. The deep learning feature quantification method according to claim 3, in, Obtaining the quantization result of the image characteristic value includes: The binary number corresponding to the reciprocal of the scaling factor is left-shifted by the sum of the integer bit depth and the decimal bit depth, and then added to the third pre-quantization value to obtain a quantization result of the image feature value.
5. The deep learning feature quantification method according to claim 4, further comprising: include: Perform bitwise AND calculation on the quantization result of the image feature value and the binary number corresponding to the reciprocal of the quantization step length minus 1 to obtain a quantization value of the decimal part of the scaling value; Multiplying the quantized value of the fractional part of the scaling value by the quantization step size to obtain an inverse quantized value of the fractional part of the scaling value; After the quantization result of the image feature value is right-shifted by the predetermined number of bits, the binary number corresponding to 2 raised to the first predetermined value power minus 1 is subjected to bitwise AND calculation to obtain a quantization value of the integer part of the scaling value; Calculate a decimal number corresponding to the sum of an inverse quantized value of a fractional part of the scaling value and a quantized value of an integer part of the scaling value to obtain the scaling value; Shifting the quantization result of the image feature value to the right by the sum of the integer bit depth and the decimal bit depth to obtain the flag bit; as well as The product of the decimal number corresponding to the flag bit and the scaling value is calculated to obtain the inverse quantized image feature value.
6. The deep learning feature quantification method according to claim 3, further comprising: include: After removing the flag bit of the quantization result of the image feature value, the binary number corresponding to the reciprocal of the quantization step size minus 1 is bitwise ANDed to obtain the quantization value of the decimal part of the scaling value; Multiplying the quantized value of the fractional part of the scaling value by the quantization step size to obtain an inverse quantized value of the fractional part of the scaling value; After removing the flag bit of the quantization result of the image feature value, right-shifting the predetermined number of bits, and performing bitwise AND calculation with the binary number corresponding to 2 raised to the first predetermined value power minus 1, to obtain a quantized value of the integer part of the scaling value; Calculate a decimal number corresponding to the sum of an inverse quantized value of a fractional part of the scaling value and a quantized value of an integer part of the scaling value to obtain the scaling value; as well as The scaling value is multiplied by the decimal number corresponding to the flag bit to obtain an inverse quantized image feature value.
7. A deep learning feature quantization device, include: The eigenvalue scaling unit is configured to determine whether the image eigenvalue is less than or equal to a first predetermined value power of 2, and if so, the scaling factor is set to 1, otherwise, the scaling factor is set to the reciprocal of a second predetermined value power of 2, wherein the first predetermined value is an integer bit depth of quantization of the set image eigenvalue, and the second predetermined value is a difference between the logarithm of the image eigenvalue with 2 as the base, rounded up, and the integer bit depth, and determine the scaling value corresponding to the image eigenvalue according to the scaling factor of the image eigenvalue, wherein the image eigenvalue is a floating point number; an integer part quantization unit, configured to convert the integer part corresponding to the scaling value into a first binary number, and shift the first binary number left by a predetermined number of bits to obtain a first pre-quantized value; A fractional part quantization unit is configured to obtain a quantization step size according to a set fractional bit depth of the image feature value quantization, divide the fractional part corresponding to the scaling value by the quantization step size, round down and convert the fractional part into a second binary number to obtain a second pre-quantized value; and The quantization result determination unit is configured to calculate the sum of the first pre-quantization value and the second pre-quantization value to obtain a third pre-quantization value, set a flag bit for the third pre-quantization value, and obtain a quantization result of the image feature value.
8. The deep learning feature quantization device according to claim 7, in, The third pre-quantization value setting flag is a binary number corresponding to the reciprocal of the scaling factor.
9. The deep learning feature quantization device according to claim 8, in, The quantization result determination unit is further configured to left-shift the binary number corresponding to the reciprocal of the scaling factor by the sum of the set integer bit depth and the decimal bit depth, and then add the binary number to the third pre-quantization value to obtain the quantization result of the image feature value.
10. The deep learning feature quantization device according to claim 9, in, A decimal part inverse quantization unit is configured to perform bitwise AND calculation on the quantization result of the image feature value and the binary number corresponding to the reciprocal of the quantization step length minus 1 to obtain a quantized value of the decimal part of the scaling value, and multiply the quantized value of the decimal part of the scaling value by the quantization step length to obtain an inverse quantized value of the decimal part of the scaling value; an integer part dequantization unit, configured to right-shift the quantization result of the image feature value by the predetermined number of bits, and then perform bitwise AND calculation with the binary number corresponding to 2 raised to the first predetermined value power minus 1, to obtain a quantized value of the integer part of the scaling value; a scaling value calculation unit, configured to calculate a decimal number corresponding to the sum of an inverse quantized value of a fractional part of the scaling value and a quantized value of an integer part of the scaling value, to obtain the scaling value; as well as The inverse quantization eigenvalue determination unit is configured to shift the quantization result of the image eigenvalue to the right by the sum of the integer bit depth and the decimal bit depth to obtain the flag bit, calculate the product of the decimal number corresponding to the flag bit and the scaling value to obtain the inverse quantization image eigenvalue.
11. The deep learning feature quantization device according to claim 8, further comprising: include: A decimal part inverse quantization unit is configured to remove the flag bit of the quantization result of the image feature value, perform bitwise AND calculation on the binary number corresponding to the reciprocal of the quantization step length minus 1 to obtain a quantized value of the decimal part of the scaling value, and multiply the quantized value of the decimal part of the scaling value by the quantization step length to obtain an inverse quantized value of the decimal part of the scaling value; an integer part dequantization unit, configured to remove the flag bit of the quantization result of the image feature value, right-shift the predetermined number of bits, and perform bitwise AND calculation on the binary number corresponding to 2 raised to the first predetermined value power minus 1, to obtain a quantized value of the integer part of the scaling value; a scaling value calculation unit, configured to calculate a decimal number corresponding to the sum of an inverse quantized value of a fractional part of the scaling value and a quantized value of an integer part of the scaling value, to obtain the scaling value; as well as The inverse quantization characteristic value determining unit is configured to multiply the scaling value by a decimal number corresponding to the flag bit to obtain an inverse quantization image characteristic value.
12. A deep learning feature quantization device, include: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the deep learning feature quantization method according to any one of claims 1 to 6 based on instructions stored in the memory.
13. A non-transitory computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the deep learning feature quantization method as described in any one of claims 1 to 6.
