SIMD (Single Instruction Multiple Data)-based 4-bit operator accelerated calculation method

Abstract

The invention provides an SIMD (Single Instruction Multiple Data)-based 4-bit operator accelerated calculation method, which is characterized in that the 4-bit calculation logic is as follows: in the calculation process of a computer, the minimum calculation unit is 1 byte = 8 bits, and two pieces of 4-bit data are stored in 1 byte and are divided into high 4-bit data and low 4-bit data; an address skips to a unit byte, and a correct address and data stored in the address need to be calculated according to the index; an additive operator: vall = (a + b) * scale; inputting a shape (ibatch, ih, iw, ic); the method comprises the following steps: S1, index calculation: 32 channels are stored in a register each time; the method comprises the following steps: S2, reading input data; s3, performing addition calculation, using an instruction to complete valve = (a + b) * scale, and re-storing an output result as 4-bit data; s4, data storage: calculating an output bit index and byte jump according to an input calculation mode; s5, when the number of the remaining channels is smaller than 32chn, c + + calculation is used, and a result obtained through calculation is stored in a corresponding output address according to the bit width.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a method for accelerating calculation of a 4-bit operator based on SIMD. Background Art

[0002] In the prior art, low-bit computing means that the input and output are limited to 4 bits to reduce memory. SIMD: single instruction multiple data, a single instruction stream multiple data stream, a computing instruction can execute multiple data streams, which can improve the computing speed of the program. (Each register has a 512-bit bit width)

[0003] Simd parallel computing: parallel computing of multiple data streams to improve computing efficiency.

[0004] However, the main defect of the prior art is that when the number of channels is odd, the address jump cannot be offset by half a byte.

[0005] In addition, common terms in the prior art include:

[0006] feature_shape: the shape of the input data, which is abstractly understood as multi-dimensional data. From left to right, it is from high dimension to low dimension (NHWC) N: the number of FeatureMaps processed at a time, set to 1; H: the height of the input data, W: the width of the input data, C: the number of input channels. Summary of the invention

[0007] In order to solve the above problems, the purpose of this application is to solve the address jump problem based on SIMD when the channel is an odd number.

[0008] Specifically, the present invention provides a method for accelerating calculation based on SIMD 4-bit operators, wherein:

[0009] The 4-bit calculation logic is: the smallest calculation unit in the computer calculation process is 1 byte = 8 bits, and 1 byte stores two 4-bit data, which are divided into high and low 4-bit data; the address jump unit byte needs to calculate the correct address and the data stored in the address according to the index;

[0010] The method is applicable to various algorithms, and the core lies in the reading and storage of 4-bit data;

[0011] Among them, the addition operator: val = (a + b) * scale; input shape (ibatch, ih, iw, ic); S1: index calculation, 32 channels each time, stored in the register, that is, the nth data, and then the address offset is calculated according to the index; the data index idx is calculated according to h, w, c, that is, according to the hth row, wth column, cth channel point formula calculation: Idx = h*iw*ic+w*ic+c; the data index is calculated;

[0012] The pixel index gets the data index, expressed as: idx = h*iw*ic + w*ic + c;

[0013] Address offset gets the address offset, idx*4 can calculate the total number of offset bits, divided by 8, to get the number of bytes of the address jump, expressed as: Ptr_offset = idx*4bit / 8;

[0014] The bit index is calculated to obtain the bit position in the byte, the high and low 4 bits, expressed as: Bit_idx = idx% 2; S2: input data read:

[0015] Store the bit index into register BSHR_VR, read 33 channel data, 32 of which are valid and the remaining one is used for register offset;

[0016] Use SIMD instruction BSHR, expressed as BSHR (VR0, VR1, VR0) According to the 0-5 bits of VR1, if all are 0, do not right shift, if not 0, right shift one byte: the register is right shifted bit_idx bytes as a whole;

[0017] When bit_idx=0, it means that the index is an even number, there is no half-byte or 4-bit jump, and the register data does not need to be offset;

[0018] When bit_idx=1, it means that the index is an odd number, and the address jump is offset by half a byte less. The simd instruction reads the data and uses LA(o, VR0, 0, ptr, 0) to read the data from the input address ptr and store it in the VR0 register. There is a situation where one more number is read. At this time, the entire register is shifted right and BSHR(VR0, VR1, VR0) is used to shift it right by one byte to remove invalid data. This is why 33 channels are read:

[0019] 32 31 30 29 ...... 3 2 1 -1 32 31 30 ..... 4 3 2 1

[0020] The number 32 in the first row is the 33rd number. For ease of understanding, the register subscript index is understood as a specific value, and -1 represents the half-byte data that needs to be removed;

[0021] After the shift, the extra -1 data is removed and 32 is filled into the register for expansion;

[0022] After shifting, all data in the input register are valid data;

[0023] S3: Addition calculation, using the instruction ADD(VR0, VR1, VR0), adds the data of registers VR0 and VR1 and updates them to the VR0 register, completing val=(a+b)*scale, and re-saving the output result as 4-bit data;

[0024] S4: Data storage, calculate the output bit index and byte jump according to the input calculation method: shift_idx=bit_idx%2;

[0025] If it is 0, no jump is required. If it is 1, it needs to be shifted right by 1 byte using LA(w,BSHR_VR,0,&shift_idx,0). The identifier of whether the shift is required is stored in the BSHR_VR register.

[0026] Use BSHR (VR0, BSHR_VR, VR0) to shift according to the BSHR_VR register identifier;

[0027] Obit_idx: output bit index, used to calculate the output data storage location;

[0028] Set the output mask to OBIT_MASK.

[0029] When obit_idx = 0, it means the index is an even number and there is no problem with address offset; OBIT_MASK = 0;

[0030] When obit_idx = 1, it means the index is an odd number, and there is a missing half byte in the output address jump; use the instruction to read a number and store the data in the register; OBIT_MASK = 0xff;

[0031] Store the missing half byte data -1 into the register

[0032]

[0033]

[0034] Finally, -1 to 31, a 32-bit number, is stored in the address through the SIMD SA instruction, that is, SA(o, VR0, 0, out_ptr, 0) is used to store the VR0 register data in the output address out_ptr. The last number is stored in the specified bit of the specified address according to the high and low 4 bits. The numbers here are only used to represent the register data. For ease of understanding, 32 numbers from 1 to 32 are used to represent the original register data. -1 represents the data of the register after shifting. This data is invalid data and is represented by -1.

[0035] Use SIMD instruction ANDV, and operation, expressed as: ANDV(VR0, OBT_MASK, VR0), and determine whether to retain the original data based on OBIT_MASK;

[0036] Use SIMD instruction BSHL, expressed as: BSHL (VR1, BSHR_VR, VR1) Shift according to BSHR_VR register identifier: shift the register left by obit_idx bytes;

[0037] Use the SIMD instruction BSELV, expressed as: BSELV (OBT_MASK, VR0, VR1), to determine whether the first byte is read from the original address space or from the register based on whether a shift is required: merge the two output results; store them at the output address;

[0038] S5: When the number of remaining channels is less than 32chn, use C++ to calculate and store the calculated result into the corresponding output address according to the bit width, expressed as:

[0039]

[0040] When obit_idx=0, the value of val remains unchanged and is stored in the output address;

[0041] When obit_idx=1;val=val<<4, store in output address;

[0042] Out_val[idx / 2]+=val; the high and low 4 bits are stored twice to form a byte; the low 4 bits are passed through &0xf to ensure that the data threshold is [0,15], and the high 4 bits are stored in a byte by shifting 4 bits left, completing the two 4-bit data to form 8-bit data.

[0043] In the method:

[0044] Reading data:

[0045] One byte of data: val_8bit = 1000 1001

[0046] Lower 4 bits = val_8bit & 0xf = 1001 Take the lower 4 bits of a byte

[0047] High 4 bits = val_8it>>4 = 1000. After taking the high 4 bits of a byte, Simd uses LA (o, VR0, 0, ptr, 0) to read the data from the address and store it in the register, expanding the 4 bits of the 8-bit data to the required bit width for subsequent necessary calculations; storing data:

[0048] Low 4 bits of data: 9 = binary 1001

[0049] High 4 bits of data: 8 = 1000 << 4, 1000 0000

[0050] Lower 4 bits + upper 4 bits = 1000 1001, forming an 8-bit data and stored in the address.

[0051] Therefore, the advantage of the present application is that the address jump problem is solved by a simple method. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0053] Figure 1 It is a flow chart of the present application method. DETAILED DESCRIPTION

[0054] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the embodiment of the present application relates to a method for accelerating calculation based on SIMD 4-bit operators, in which:

[0056] 4-bit calculation logic means that in the computer calculation process, the minimum calculation unit is 1 byte = 8 bits. Two 4-bit data are stored in 1 byte, which are divided into high and low 4-bit data. The address jump unit is byte, and the correct address and the data stored in the address need to be calculated according to the index.

[0057] This method is applicable to other algorithms, and the core lies in the reading and saving of 4-bit data;

[0058] Take the addition operator as an example: val = (a + b) * scale; input shape (ibatch, ih, iw, ic); S1: index calculation, read 32 channels each time and store in the register; calculate the data index idx according to h, w, c, that is, the nth data, and then calculate the address offset according to the index; calculate according to the formula of the hth row, wth column, cth channel point: Idx = h*iw*ic+w*ic+c; calculate the data index;

[0059] Idx=h*iw*ic+w*ic+c;(pixel index) get data index;

[0060] Ptr_offset=idx*4bit / 8; (address offset) gets the address offset; idx*4 can calculate the total number of offset bits, divided by 8, to get the number of bytes of the address jump.

[0061] Bit_idx = idx% 2; (bit index) calculates the bit position in the byte, high and low 4 bits; S2: input data read:

[0062] Store the bit index into register BSHR_VR and read 33 channel data (32 of which are valid and the remaining one is used for register offset)

[0063] Use SIMD instruction BSHR: shift the register right by bit_idx bytes; expressed as: BSHR (VR0, VR1, VR0) According to bits 0 to 5 of VR1, if all bits are 0, do not shift right; if not 0, shift right by one byte;

[0064] When bit_idx=0, it means that the index is an even number, there is no half-byte jump (4bit), and the register data does not need to be offset;

[0065] When bit_idx=1, it means that the index is an odd number, the address jump is offset by half a byte less, the simd instruction reads the data, uses LA(o,VR0,0,ptr,0), reads the data from the input address ptr, and stores it in the VR0 register. If there is one more number read, the entire register is shifted right, and BSHR(VR0, VR1, VR0) is used to shift right one byte to remove invalid data. This is why 33 channels are read.

[0066] 32 31 30 29 ...... 3 2 1 -1 32 31 30 ..... 4 3 2 1

[0067] The number 9 is the lower 33 digits, and -1 represents the half byte of data that needs to be removed.

[0068] After the shift, the extra -1 data is removed and 9 is filled into the register for expansion.

[0069] After shifting, all data in the input register are valid data;

[0070] S3: Addition calculation, using the instruction ADD(VR0, VR1, VR0), adds the data of registers VR0 and VR1 and updates them to the VR0 register, completing val=(a+b)*scale, and re-saving the output result as 4-bit data;

[0071] S4: Data storage, based on the input calculation method, calculate the output bit index and byte jump, expressed as:

[0072] shift_idx = bit_idx % 2;

[0073] If it is 0, no jump is required. If it is 1, it needs to be shifted right by 1 byte.

[0074] LA(w,BSHR_VR,0,&shift_idx,0); stores the identifier of whether a shift is required in the BSHR_VR register.

[0075] BSHR(VR0, BSHR_VR, VR0) is shifted according to the BSHR_VR register identifier.

[0076] Obit_idx: output bit index, used to calculate the output data storage location;

[0077] Set the output mask, OBIT_MASK

[0078] When obit_idx = 0, it means the index is an even number and there is no problem with address offset; OBIT_MASK = 0

[0079] When obit_idx = 1, it means the index is an odd number, and there is a missing half byte in the output address jump; use the instruction to read a number and store the data in the register; OBIT_MASK = 0xff

[0080] Store the missing half byte data -1 into the register

[0081]

[0082]

[0083] Finally, -1 to 31, a 32-bit number, is stored in the address through the SIMD SA instruction. The numbers here are only used to represent register data. For ease of understanding, 1 to 32 are used to represent the original register data. -1 represents the data of the register after shifting. This data is invalid data and is represented by -1. The last number is stored in the specified bit of the specified address according to the high and low 4 bits. Use the SIMD instruction ANDV, and the operation is expressed as: ANDV(VR0, OBT_MASK, VR0), and judge whether the original data needs to be retained according to OBIT_MASK;

[0084] Use SIMD instruction BSHL, expressed as: BSHL (VR1, BSHR_VR, VR1) Shift according to BSHR_VR register identifier: shift the register left by obit_idx bytes;

[0085] Use the SIMD instruction BSELV, expressed as: BSELV (OBT_MASK, VR0, VR1), to determine whether the first byte is read from the original address space or from the register based on whether a shift is required: merge the two output results; store them at the output address;

[0086] S5: When the number of remaining channels is less than 32chn, use C++ to calculate and store the calculated result into the corresponding output address according to the bit width, expressed as:

[0087] if (obit_idx = 0) {

[0088] Out_addr[idx] = val;

[0089] }else{

[0090] Out_addr[idx]=Out_addr[idx]&0xf;clear the high 4 bits data

[0091] Out_addr[idx] = val << 4;

[0092] }.

[0093] When obit_idx=0;val value remains unchanged, store in output address When obit_idx=1;val=val<<4, store in output address

[0094] Out_val[idx / 2]+=val; The high and low 4 bits are stored twice to form a byte. The low 4 bits are passed through &0xf to ensure that the data threshold is [0,15]. The high 4 bits are shifted left by 4 bits and stored in a byte. The two 4-bit data form 8-bit data.

[0095] example:

[0096] Reading data:

[0097] One byte of data: val_8bit = 1000 1001

[0098] Lower 4 bits = val_8bit & 0xf = 1001 Take the lower 4 bits of a byte

[0099] The upper 4 bits = val_8it>>4 = 1000. After taking the upper 4 bits of a byte, SIMD reads the data and uses LA (o, VR0, 0, ptr, 0) to read the data from the address and store it in the register. This can expand the 4 bits of the 8-bit data to the required bit width for subsequent necessary calculations.

[0100] Storing data:

[0101] Low 4 bits of data: 9 = 1001 (binary)

[0102] High 4 bits of data: 8 = 1000 << 4, 1000 0000

[0103] Lower 4 bits + upper 4 bits = 1000 1001, forming an 8-bit data and stored in the address.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for accelerating calculation based on SIMD 4-bit operators, characterized in that: In the method: the 4-bit calculation logic is: the minimum calculation unit in the computer calculation process is 1 byte = 8 bits, and 1 byte stores two 4-bit data, which are divided into high and low 4-bit data; the address jump unit byte needs to calculate the correct address and the data stored in the address according to the index; The method is applicable to various algorithms, and the core lies in the reading and storage of 4-bit data; Among them, the addition operator: val = (a + b) * scale; input shape (ibatch, ih, iw, ic); S1: index calculation, 32 channels each time, stored in the register, that is, the nth data, and then the address offset is calculated according to the index; the data index idx is calculated according to h, w, c, that is, according to the hth row, wth column, cth channel point formula calculation: Idx = h*iw*ic+w*ic+c; the data index is calculated; The pixel index gets the data index, expressed as: idx = h*iw*ic + w*ic + c; Address offset gets the address offset, idx*4 can calculate the total number of offset bits, divided by 8, to get the number of bytes of the address jump, expressed as: Ptr_offset = idx*4bit / 8; The bit index is calculated to obtain the bit position in the byte, the high and low 4 bits, expressed as: Bit_idx = idx% 2; S2: input data read: Store the bit index into register BSHR_VR, read 33 channel data, 32 of which are valid and the remaining one is used for register offset; Use SIMD instruction BSHR, expressed as BSHR (VR0, VR1, VR0) According to the 0-5 bits of VR1, if all are 0, do not right shift, if not 0, right shift one byte: the register is right shifted bit_idx bytes as a whole; When bit_idx=0, it means that the index is an even number, there is no half-byte or 4-bit jump, and the register data does not need to be offset; When bit_idx=1, it means that the index is an odd number, and the address jump is offset by half a byte less. The simd instruction reads the data and uses LA(o, VR0, 0, ptr, 0) to read the data from the input address ptr and store it in the VR0 register. There is a situation where one more number is read. At this time, the entire register is shifted right and BSHR(VR0, VR1, VR0) is used to shift it right by one byte to remove invalid data. This is why 33 channels are read: The number 32 in the first row is the 33rd number. For ease of understanding, the register subscript index is understood as a specific value, and -1 represents the half-byte data that needs to be removed; After the shift, the extra -1 data is removed and 32 is filled into the register for expansion; After shifting, all data in the input register are valid data; S3: Addition calculation, using the instruction ADD(VR0, VR1, VR0), adds the data of registers VR0 and VR1 and updates them to the VR0 register, completing val=(a+b)*scale, and re-saving the output result as 4-bit data; S4: Data storage, calculate the output bit index and byte jump according to the input calculation method: shift_idx=bit_idx%2; If it is 0, no jump is required. If it is 1, it needs to be shifted right by 1 byte. Use LA(w,BSHR_VR,0,&shift_idx,0); store the identifier of whether the shift is needed in the BSHR_VR register; Use BSHR (VR0, BSHR_VR, VR0) to shift according to the BSHR_VR register identifier; Obit_idx: output bit index, used to calculate the output data storage location; Set the output mask to OBIT_MASK. When obit_idx = 0, it means the index is an even number and there is no problem with address offset; OBIT_MASK = 0; When obit_idx = 1, it means the index is an odd number, and there is a missing half byte in the output address jump; use the instruction to read a number and store the data in the register; OBIT_MASK = 0xff; Store the missing half byte data -1 into the register Finally, -1 to 31, a 32-bit number, is stored in the address through the SIMD SA instruction, that is, SA(o, VR0, 0, out_ptr, 0) is used to store the VR0 register data in the output address out_ptr. The last number is stored in the specified bit of the specified address according to the high and low 4 bits. The numbers here are only used to represent the register data. For ease of understanding, 32 numbers from 1 to 32 are used to represent the original register data. -1 represents the data of the register after shifting. This data is invalid data and is represented by -1. Use SIMD instruction ANDV, and operation, expressed as: ANDV(VR0, OBT_MASK, VR0), and determine whether to retain the original data based on OBIT_MASK; Use SIMD instruction BSHL, expressed as: BSHL (VR1, BSHR_VR, VR1) Shift according to BSHR_VR register identifier: shift the register left by obit_idx bytes; Use the SIMD instruction BSELV, expressed as: BSELV (OBT_MASK, VR0, VR1), to determine whether the first byte is read from the original address space or from the register based on whether a shift is required: merge the two output results; store them at the output address; S5: When the number of remaining channels is less than 32chn, use C++ to calculate and store the calculated result into the corresponding output address according to the bit width, expressed as: When obit_idx=0, the value of val remains unchanged and is stored in the output address; When obit_idx=1;val=val<<4, store in output address; Out_val[idx / 2]+=val; the high and low 4 bits are stored twice to form a byte; the low 4 bits are passed through &0xf to ensure that the data threshold is [0,15], and the high 4 bits are stored in a byte by shifting 4 bits left, completing the two 4-bit data to form 8-bit data.

2. A method for accelerating calculation based on SIMD 4-bit operators according to claim 1, characterized in that: In the method: Reading data: One byte of data: val_8bit = 1000 1001 Lower 4 bits = val_8bit & 0xf = 1001 Take the lower 4 bits of a byte High 4 bits = val_8it>>4 = 1000 Take the high 4 bits of a byte After Simd reads the data, it uses LA(o,VR0,0,ptr,0) to read the data from the address and store it in the register, expanding the 4 bits of the 8-bit data to the required bit width for subsequent necessary calculations; store data: Low 4 bits of data: 9 = binary 1001 High 4 bits of data: 8 = 1000 << 4, 1000 0000 Lower 4 bits + upper 4 bits = 1000 1001, forming an 8-bit data and stored in the address.

Images