A storage and analysis method for 4-bit feature maps in convolution calculation

Abstract

The application provides a storage and analysis method of 4-bit feature maps in convolution calculation, comprising the following steps: S1, preprocessing before storage of feature map data; S2, analysis of feature map data: 1) using right shift to obtain high 4-bit signed data; 2) using bit AND operation to obtain low 4-bit unsigned data, and then subtracting 8 from the unsigned data to obtain low 4-bit original data; S3, overall storage of data: in the use of instructions of a chip, one register loads 16 8-bit data at a time, and in the processing, every 32 data are grouped, and the 32 4-bit data are converted and stored into 16 8-bit stored data; S4, overall analysis of data: every 16 8-bit data actually stores 32 4-bit data; and the original sequence is analyzed according to the actual sequence. The method can reduce the data storage space by half, greatly reduces the loading time in the data loading process, and the equivalent speed is doubled, and the calculation amount is not increased.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a storage and analysis method of 4bit feature map in convolution calculation. BACKGROUND

[0002] Integrated circuit technology is now increasingly becoming the focus of technological development, and many chip manufacturers have also developed their own chips. In chip applications, there are also respective demand problems in their respective chip designs. For example, the chips produced by Beijing Junzheng Integrated Circuit Co., Ltd. currently on the market mostly support 8-bit data operation, and the corresponding minimum storage unit is 8-bit storage, which cannot directly support 4-bit storage. Storing 4-bit data as 8-bit storage data, that is, a byte, the minimum unit of using data is also a byte, which occupies more storage space, increases the loading data time, and increases the time of the entire convolution calculation.

[0003] In addition, the common terms in the prior art are as follows:

[0004] 1. Feature map: the result obtained after input data is subjected to convolution calculation is called feature map, and the result generated after the data is subjected to full connection is also called feature map. The size of the feature map is generally represented as length x width x depth, or 1 x depth. The depth is also referred to as the number of channels. SUMMARY

[0005] In order to solve the above problems in the prior art, the purpose of the present application is to reduce the occupied storage space. More feature map data is loaded into the secondary cache, which alleviates the waiting problem caused by data transmission. In particular, it can adapt to the needs of chip instructions such as Beijing Junzheng T30 and T31 models, and design the storage mode according to the efficiency of running convolution on the chip suitable for T30 and T31 models.

[0006] Specifically, the present application provides a storage and analysis method of 4bit feature map in convolution calculation, which comprises:

[0007] S1, preprocessing of feature map data before storage:

[0008] Since only 8-bit data can be stored at a time, preprocessing of the data is required before storage. In the 4-bit convolution calculation quantization processing, the high 4 bits of the 8-bit data are used to store 4-bit data with a sign, and the low 4 bits are used to store 4-bit data without a sign.

[0009] S2, analysis of feature map data:

[0010] 1) using right shift to obtain high 4-bit signed data;

[0011] 2) using bit and operation, that is, corresponding original unsigned 4-bit data & 00001111, obtaining low 4-bit unsigned data, and then unsigned minus 8 to obtain low 4-bit original data;

[0012] S3, overall storage of data:

[0013] In the instruction use of the chip, one register loads 16 8-bit data at a time, and in the processing, every 32 data is a group, and the 32 4-bit data is converted and stored into 16 8-bit stored data;

[0014] S4, overall analysis of data:

[0015] Every 16 8-bit data actually stores 32 4-bit data; the 32 4-bit data are arranged in the order of 0th, 16th, 1st, 17th, 2nd, 18th, …, 15th, and 31st relative to the order before conversion; and the original order is analyzed according to the actual order:

[0016] First, the data is loaded into the register vrs, and then the high 4-bit signed data is obtained by right shifting, and the result is stored in the register vrd0;

[0017] Then, the low 4-bit unsigned data is obtained by bit and operation on vrs, and then the low 4-bit original data is obtained by unsigned minus 8, and the result is stored in the register vrd1;

[0018] The storage order of vrd0 is 0th, 1st, 2nd, …, 15th; and the storage order of vrd1 is 16th, 17th, 18th, …, 31st.

[0019] In the 4-bit convolution calculation quantization processing in step S1, the convolution result is quantized to unsigned 4-bit data, and then further processed into signed 4-bit data according to actual needs; 4-bit signed data is used for each layer input, and the signed data can be directly used, or the unsigned data can be used, that is, the signed data is obtained by subtracting 8 from the unsigned data.

[0020] In step S1, the storage mode of the one 8-bit data includes: high 4-bit, with sign, that is, unsigned 4-bit minus 8 data, and then left shift 4 bits; low 4-bit, 8-bit unsigned storage, that is, without minus 8; then the two numbers are used for bit or operation, and the obtained data is the 8-bit data to be stored; let the high 4-bit in the 8-bit data be a, the corresponding original unsigned 4-bit data be x, the low 4-bit be b, the corresponding original unsigned 4-bit data be y, and the data after bit or operation be z, and the formula is:

[0021]

[0022] z = [(x - 8) « 4] | y (0≤x≤15, 0≤y≤15, x∈N, y∈N).

[0023] Suppose there are two unsigned 4-bit numbers x = 6, y = 11; the signed data a is processed as 6 - 8 = -2, and then left shifted by 4 bits to be -32; z = -32 | 11, z = -21.

[0024] In the analysis of data, the bitwise AND operation is used, that is, x & 00001111,

[0025] The formula is:

[0026]

[0027] Suppose z = -21, x = -21 >> 4, x = -2, y = (-21) & (0x0F), y = 11, the corresponding unsigned result is 6, 11, which is completely consistent with the original data a = 6, b = 11, z = -21.

[0028] In the step S3, every 32 4-bit data is a group, that is, an 8-bit data, and two 4-bit data are stored in each 8-bit data, the first 16 data are signed 4-bit data, and the last 16 data are unsigned 4-bit data.

[0029] Suppose the first 16 8-bit data are stored in the register vrs0, and the last 16 8-bit data are stored in the register vrs1, and the effective data of the 8-bit data in the register is only 4-bit data, the processed data is represented by the register vrd, and finally the data in the register vrd is saved to the ddr hardware.

[0030] Convert the unsigned 4-bit data to signed 4-bit data, and store the result in vrs0:

[0031] Use the subtraction instruction, input variable vrs0, 8, and output variable is vrs0, vrs0 stores 16 8-bit data; which can be represented as vrd1 = ingenic_sub(vrd1, 8);

[0032] Shift operation, result stored in vrs0:

[0033] Use the left shift instruction to left shift the data in the variable vrs0 by 4 bits; which can be represented as vrs0 = ingenic_shift_left(vrs0, 4);

[0034] Add the high 4-bit signed data and the low 4-bit unsigned data:

[0035] The input variables vrs0 and vrs1 are 16 8-bit data, and the output variable vrd is 16 8-bit data.

[0036] The step S4 further comprises:

[0037] The vrs shift operation is performed, and the result is stored in vrd0.

[0038] The data in the variable vrs0 is right shifted by 4 bits using the right shift instruction, and the result is vrd0 = ingenic_shift_right(vrs0, 4).

[0039] The vrs bit operation is performed, and the result is stored in vrd1.

[0040] The input variable vrs0 and 15 are 16 8-bit data, and the output variable vrd1 is 16 8-bit data, and the input variable vrs0 is 16 8-bit data; and the operation can be represented as vrd1 = ingenic_and(vrs0, 15).

[0041] The unsigned 4-bit data is converted into signed 4-bit data.

[0042] The input variable vrd1 and 8 are 16 8-bit data, and the output variable vrd1 is 16 8-bit data; and the operation can be represented as vrd1 = ingenic_sub(vrd1, 8).

[0043] The chip used in the instruction use of the chip is a chip with a simd instruction set, and is suitable for Beijing Junzheng T30 and T31 models, wherein there is such an instruction that the 8-bit sign bit does not participate in the left and right shift operation, but the 8-bit sign bit participates in the and or operation, so only the high 4 bits store the signed 4-bit data, and the low 4 bits store the unsigned 4-bit data; the conventional instruction 8-bit addition and subtraction operation. Any chip with such instruction function can use the method.

[0044] The step S4 further comprises: using vrd0 and vrd1 to perform convolution calculation.

[0045] Therefore, the application has the advantages that a storage and analysis method of 4-bit feature map in convolution calculation is provided, the data storage space is reduced by half, the loading time is greatly reduced in the data loading process, the equivalent speed is doubled, and the calculation amount is not increased. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application.

[0047] Figure 1 is a flow chart of the method of the present application. DETAILED DESCRIPTION

[0048] In order to enable persons skilled in the art to more clearly understand the technical content and advantages of the present application, the present application will be further described in detail in conjunction with the drawings.

[0049] As shown in Figure 1 The present application relates to a method for storing and analyzing 4-bit feature maps in convolution calculation, which comprises the following steps:

[0050] S1, preprocessing before storing feature map data:

[0051] Since only 8-bit data can be stored at a time, preprocessing of the data is required before storage. In the 4-bit convolution calculation quantization processing, the high 4 bits of the 8-bit data are used to store 4-bit data with a sign, and the low 4 bits are used to store 4-bit data without a sign.

[0052] S2, analysis of feature map data:

[0053] 1) using right shift to obtain high 4-bit signed data;

[0054] 2) using bitwise AND operation, i.e. corresponding original unsigned 4-bit data & 00001111, to obtain low 4-bit unsigned data, and then subtracting 8 from the unsigned data to obtain the original low 4-bit data;

[0055] S3, overall storage of data:

[0056] In the instruction use of the chip, one register loads 16 8-bit data at a time. In the processing, every 32 data form a group, and the 32 4-bit data are converted and stored into 16 8-bit stored data.

[0057] S4, overall analysis of data:

[0058] Every 16 8-bit data actually stores 32 4-bit data. The arrangement of the 32 4-bit data is as follows: 0th, 16th, 1st, 17th, 2nd, 18th, …, 15th, 31st, compared with the order before conversion. According to the actual order, the original order is analyzed:

[0059] First, the data is loaded into the register vrs, and then the high 4-bit signed data is obtained by right shift, and the result is stored in the register vrd0.

[0060] Then the vrs is processed by bit and operation to obtain the low 4-bit unsigned data, and then the unsigned data is reduced by 8 to obtain the low 4-bit original data, and the result is stored in the register vrd1.

[0061] The storage order of vrd0 is: the 0th, the 1st, the 2nd, …, the 15th; and the storage order of vrd1 is: the 16th, the 17th, the 18th, …, the 31st.

[0062] Specifically, the specific embodiments of the application can also be described as follows:

[0063] 1. Use requirements. The input feature map channel needs to be a multiple of 32. The instructions involved are the shift instruction, the subtraction instruction, the bit and instruction, and the data saving instruction corresponding to the Beijing Junzheng T30 and T31 model chips. The instructions are as follows:

[0064] a) Shift instruction:

[0065] Move each element in the variable i bits.

[0066] Left shift instruction

[0067] vrd = ingenic_shift_left(vrs, i)

[0068] Right shift simd instruction

[0069] vrd = ingenic_shift_right(vrs, i)

[0070] b) Subtraction instruction:

[0071] vrd = ingenic_sub(vrs, i);

[0072] The input variable vrs, i, and the output variable are vrd. vrd stores 16 8-bit data, and vrs stores 16 8-bit data. i is a conventional variable.

[0073] Equivalent operation:

[0074] vrd0 := vrs0 - i;

[0075] vrd1 := vrs1 - i;

[0076] …

[0077] vrd7 := vrs7 - i;

[0078] c) Bit and instruction:

[0079] vrd = ingenic_and(vrs, i);

[0080] The input variable is vrs, i, and the output variable is vrd. vrd stores 16 8-bit data, and vrs stores 16 8-bit data. i is a regular variable.

[0081] Equivalent operation:

[0082] vrd0: = vrs0 & i;

[0083] vrd1: = vrs1 & i;

[0084] ...

[0085] vrd7: = vrs7 & i;

[0086] d) Bit or instruction:

[0087] vrd = ingenic_or (vrs, i);

[0088] The input variable is vrs, i, and the output variable is vrd. vrd stores 16 8-bit data, and vrs stores 16 8-bit data. i is a regular variable.

[0089] Equivalent operation:

[0090] vrd0: = vrs0 | i;

[0091] vrd1: = vrs1 | i;

[0092] ...

[0093] vrd7: = vrs7 | i;

[0094] 1. Preprocessing before feature map data storage.

[0095] In the 4-bit convolution calculation quantization processing, the convolution result is quantized to unsigned 4-bit data, and whether to further process it into signed 4-bit data according to actual needs. Each layer input uses 4-bit signed data, so it can directly use signed data, or use unsigned data, and the unsigned data is subtracted by 8 to get signed data. According to this situation, the high 4 bits of 8 bits are stored with signed 4-bit data, and the low 4 bits are stored with unsigned 4-bit data. In the instructions of T30 and T31 type chips, the 8-bit sign bit does not participate in the left and right shift operation, but the 8-bit sign bit participates in the "and or" operation, and there is such instruction, so only the high 4 bits can be stored with signed 4-bit data, and the low 4 bits can be stored with unsigned 4-bit data. Since 8 bits can be stored at a time, the data needs to be processed before storage.

[0096] One 8-bit data storage mode: high 4 bits, with sign, that is, the data after 4-bit unsigned data is reduced by 8, and then left shift 4 bits; low 4 bits of 8-bit unsigned storage, that is, not reduced by 8; then the two numbers are obtained by using bit or operation, and the data obtained is the 8-bit data to be stored. Let the high 4 bits of the 8-bit data be a, the corresponding original 4-bit unsigned data be x, the low 4 bits be b, and the corresponding original 4-bit unsigned data be y, and the data after the bit or operation be z. The formula is:

[0097]

[0098] z = [(x-8) << 4] | y (0≤x≤15, 0≤y≤15, x∈N, y∈N).

[0099] For example: assuming that there are two 4-bit unsigned numbers x = 6 and y = 11. The a processing signed data is 6-8 = -2, and then left shift 4 bits is -32. z = -32 | 11, z = -21.

[0100] 3. Analysis of feature map data.

[0101] Data analysis:

[0102] 1) Use right shift to get high 4-bit signed data;

[0103] 2) Use bit and operation (x & 00001111) to get low 4-bit unsigned data, and then reduce 8 to get the original low 4-bit data.

[0104] The formula is:

[0105]

[0106] For example: assuming that z = -21. x = (-21 >> 4), x = -2, y = (-21) & (0x0F), y = 11, and the corresponding unsigned result is 6, 11. This result is exactly the same as the example in 1, "a = 6, b = 11, z = -21".

[0107] 4. Overall data storage

[0108] In the instruction use of Beijing Junzheng T30 and T31 models, one register loads 16 8-bit data at a time, so in the processing, every 32 4-bit data (only one 4-bit data is stored in an 8-bit data) is a group, the first 16 data are signed 4-bit data, and the last 16 data are unsigned 4-bit data. The 32 4-bit data are converted and stored into 16 8-bit stored data. The T30 and T31 instructions are processed as follows.

[0109] The first 16 8-bit data are stored in register vrs0, and the last 16 8-bit data are stored in register vrs1. The effective data of the 8-bit data is only 4-bit data. The processed data is represented by register vrd. Finally, the data in vrd is saved to ddr hardware.

[0110] The unsigned 4-bit data is converted into signed 4-bit data, and the result is stored in vrs0,

[0111] vrs0 = ingenic_sub (vrs0, 8) ;

[0112] The shift operation is performed, and the result is stored in vrs0,

[0113] vrs0 = ingenic_shift_left (vrs0, 4) ;

[0114] The high 4-bit signed data and the low 4-bit unsigned data are subjected to or operation,

[0115] vrd = ingenic_or (vrs0, vrs1) ;

[0116] 5. Overall analysis of data

[0117] Analysis of data:

[0118] Every 16 8-bit data actually stores 32 4-bit data. The 32 4-bit data are arranged in the following order: 0th, 16th, 1st, 17th, 2nd, 18th,..., 15th, 31st. According to the actual order, the original order is analyzed.

[0119] First, the data is loaded into register vrs, and then the high 4-bit signed data is obtained by right shifting, and the result is stored in register vrd0. Then, the low 4-bit unsigned data is obtained by performing bit and operation on vrs, and then the low 4-bit original data is obtained by subtracting 8 from the unsigned data, and the result is stored in register vrd1. The order stored in vrd0 is: 0th, 1st, 2nd,..., 15th. The order stored in vrd1 is: 16th, 17th, 18th,..., 31st. vrd0 and vrd1 are used for convolution calculation.

[0120] The shift operation is performed on vrs, and the result is stored in vrd0,

[0121] vrd0 = ingenic_shift_right (vrs0, 4) ;

[0122] The bit and operation is performed on vrs, and the result is stored in vrd1,

[0123] vrd1 = ingenic_and (vrs0, 15);

[0124] convert the unsigned 4-bit data into signed 4-bit data,

[0125] vrd1 = ingenic_sub (vrd1, 8) ;

[0126] The method can reduce the data storage space by half, greatly reduce the loading time in the data loading process, and equivalent speed is doubled without increasing the calculation amount.

[0127] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the embodiments of the present application can be variously changed and altered. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for storing and parsing 4-bit feature maps in convolution computation, characterized in that, The method includes: S1, Preprocessing before feature map data storage: Since only 8 bits of data can be stored at a time, data preprocessing is required before storage. In the 4-bit convolution calculation quantization process, the high 4 bits of the 8-bit data are stored using signed data, and the low 4 bits are stored using unsigned data. S2, Analysis of Feature Map Data: 1) Shift the 8 bits of data to be parsed to the right to obtain the high 4 bits of signed data; 2) Use bitwise AND operation, i.e., the original unsigned 4-bit data & 00001111, to get the lower 4 bits of unsigned data, and then subtract 8 from the unsigned data to get the lower 4 bits of the original data. S3, overall data storage: In the use of chip instructions, a register loads 16 8-bit data at a time. During processing, every 32 consecutive data are grouped together, and these 32 4-bit data are converted and stored as 16 8-bit data. S4, Overall Data Analysis: Every 16 8-bit data entries actually store 32 4-bit data entries; the order of these 32 4-bit data entries relative to their original order before conversion is: 0th, 16th, 1st, 17th, 2nd, 18th, ..., 15th, 31st; the original order is then parsed based on the actual order. First, the data is loaded into register vrs, then the high 4 bits of signed data are obtained by right shifting, and the result is stored in register vrd0; Next, perform a bitwise AND operation on vrs to obtain the lower 4 bits of unsigned data. Then, subtract 8 from the unsigned data to obtain the lower 4 bits of the original data. The result is stored in register vrd1. The storage order of vrd0 is: 0th, 1st, 2nd, ..., 15th; the storage order of vrd1 is: 16th, 17th, 18th, ..., 31st.

2. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 1, characterized in that, In step S1, during the quantization process of 4-bit convolution calculation, the convolution result is quantized to unsigned 4-bit data, and then further processed into signed 4-bit data according to actual needs. Each layer input uses 4 bits of signed data, which can be used directly or unsigned data, i.e., unsigned data is subtracted by 8 to obtain signed data.

3. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 2, characterized in that, In step S1, the storage method for an 8-bit data includes: the high 4 bits are signed, i.e., the data obtained by subtracting 8 from the unsigned 4 bits, and then left-shifting by 4 bits; the low 4 bits are stored as an unsigned 8-bit data, i.e., without subtracting 8; then, the two numbers are ORed using a bitwise operation, and the resulting data is the 8-bit data to be stored; let the high 4 bits of the 8-bit data be 'a', the corresponding original unsigned 4-bit data be 'x', the low 4 bits be 'b', the corresponding original unsigned 4-bit data be 'y', and the data obtained after the bitwise OR operation be 'z', as shown in the formula: z=[(x-8)<<4]|y(0≤x≤15,0≤y≤15,x∈N,y∈N).

4. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 3, characterized in that, Given two unsigned 4-digit numbers x = 6 and y = 11; process 'a' as signed data to get 6 - 8 = -2, then shift left by 4 bits to get -32; z = -32|11, so z = -21.

5. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 3, characterized in that, Use bitwise AND operation, i.e., z&00001111. The formula is:

6. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 5, characterized in that, Let z = -21, x = -21 >> 4, then x = -2, y = (-21) & (0x0F), then y = 11. The corresponding unsigned result is 6, 11. This result perfectly matches the original data a = 6, b = 11, z = -21.

7. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 5, characterized in that, In step S3, every 32 consecutive 4-bit data are grouped together, i.e., an 8-bit data set stores two 4-bit data sets. The first 16 data sets are signed 4-bit data sets, and the last 16 data sets are unsigned 4-bit data sets.

8. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 7, characterized in that, Suppose that register vrs0 stores the first 16 8-bit data and register vrs1 stores the last 16 8-bit data. Only one 4-bit data is valid in each of the 8-bit data. The processed data is represented by register vrd. Finally, the data in vrd is saved to the ddr hardware. Convert unsigned 4-bit data into signed 4-bit data and store the result in vrs0: Using the subtraction instruction, the input variables are vrs0 and 8, and the output variable is vrs0, which stores 16 8-bit data. The shift operation results in vrs0: Use the left shift instruction to shift the data in variable vrs0 to the left by 4 bits; Perform a bitwise OR operation between the high 4 bits of signed data and the low 4 bits of unsigned data: Using the bitwise OR instruction, the input variables vrs0 and vrs1, and the output variable vrd, store 16 8-bit data. vrs0 and vrs1 store 16 8-bit data.

9. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 8, characterized in that, Step S4 further includes: The result of the vrs shift operation is stored in vrd0: Use the right shift instruction to shift the data in variable vrs0 to the right by 4 bits; Perform a bitwise AND operation on vrs, and store the result in vrd1: Using the bitwise AND instruction, the input variable vrs0 is 15, and the output variable vrd1 is 16 8-bit data. vrd1 stores 16 8-bit data. Convert unsigned 4 bits to signed 4 bits of data: Using the subtraction instruction, the input variables are vrd1 and 8, and the output variable is vrd1; vrd1 stores 16 8-bit data.

10. The method for storing and parsing 4-bit feature maps in convolution calculation according to claim 1, characterized in that, The chip used in the instruction usage described above is a chip with a SIMD instruction set, which includes instructions where the 8-bit sign bit does not participate in left or right shift operations, but does participate in AND or OR operations. Therefore, only the high 4 bits are used to store the signed 4-bit data, and the low 4 bits are used to store the unsigned 4-bit data. The conventional instructions are 8-bit addition and subtraction instructions. Any chip with such instruction functionality can use this method.

Images