An activation function-based single-table 128 entry optimization method

Abstract

This invention provides an optimization method for a single 128-entry table based on activation functions. The method accelerates the table lookup logic using SIMD instructions, merges indexes, sets corresponding masks, obtains the correct table entries, and then performs calculations. The method includes: Single-table lookup: Input data is determined to be positive, using only positive entries; Pos_dy: Positive entry 0, a total of 128 entries, each entry is 8 bits, totaling 512 bits, which can be stored in a register; Pos_y0: Positive entry 1, a total of 128 entries, each entry is 8 bits, totaling 512 bits, which can be stored in a register; S1: Input data is cilpped to a threshold range, making it a number between 0 and 127; S2: Original index calculation; S3: Determine the table entry index and use instructions to look up the table; S4: Separate the high and low 4 bits to obtain effective bit width data; S5: Multiply the data obtained from the table entries to obtain the final result. By using SIMD instructions and designing corresponding masks, the table lookup function based on a single 128-entry table based on activation functions is completed.

Description

Technical Field

[0001] This invention belongs to the field of computer computing and processing technology, and specifically relates to an optimization method for a single table with 128 entries based on an activation function. Background Technology

[0002] In existing technologies, neural networks can be trained to solve various tasks, such as image classification, speech recognition, and natural language processing. After training, a neural network can perform reasoning activities such as guessing and inferring from new data, and applying knowledge to new situations. Neural networks are the core of deep learning. They consist of multiple layers of neurons, each performing a specific computational task and passing the results to the next layer.

[0003] Single Instruction Multiple Data (SIMD) is a digital parallel computing method that allows a single instruction to execute multiple data streams, thus improving program speed (each register is 512 bits wide). Instructions can be used to split the bit width in parallel.

[0004] In using neural network methods, various function computations are often required, including activation functions, which provide the network with non-linear modeling capabilities. Common activation functions include... The activation function's implementation logic involves calculating the activation function result using a fixed formula.

[0005] The lookup table method is a common program acceleration technique that trades space (memory space) for time (running time). The basic process involves storing the results of frequently accessed databases or complex floating-point calculations in a data structure (arrays are most common). Later, the relevant data can be retrieved based on given conditions. For example, storing the tangent values ​​from 1 to 360 degrees in an array means that when 45 degrees is input, only the 45th element needs to be retrieved. This saves unnecessary computation time.

[0006] The lookup table logic approximates the functionality of complex activation functions by looking up tables.

[0007] Y_new=coef*dy+y0*pow(2,11)

[0008] The formula is derived from bilinear interpolation:

[0009] Result=y0*(1-ceof)+y1*ceof;

[0010] = y0 - y0*ceof + y1*ceof

[0011] =y0+(y1-y0)*ceof

[0012] =y0+dy*ceof

[0013] In addition, Single Table 128: Data is positive, fast speed, low precision, supports 4-bit quantization.

[0014] However, the existing technology has the following problems:

[0015] Currently, there is no specific optimization for single-table 128, where the data consists only of integers, requiring high speed, low precision, and support for 4-bit quantization.

[0016] Activation functions involve exponential and division calculations, which are slow to compute. Furthermore, although complex activation functions can be replaced by lookup table methods through approximate fitting, existing methods are computationally complex and consume a large amount of memory.

[0017] In addition, the technical terms in this field also include:

[0018] Commonly used character descriptions: word: C++ character, representing 32 bits; short: C++ character, representing 16 bits; byte: C++ character, representing 8 bits. Characters can be converted to each other based on their bit width; 1 word = 2 shorts = 4 bytes. Summary of the Invention

[0019] To address the aforementioned issues, the purpose of this application is to simplify complex activation logic using SIMD instructions and a single table 128 for activation functions, and to approximate the activation function result using a lookup table.

[0020] Specifically, this invention provides a single-table 128-entry optimization method based on activation functions. This method accelerates the table lookup logic through SIMD instructions, merges indexes, sets corresponding masks, obtains the correct table entries, and then performs calculations, including:

[0021] Single table query: If the input data is confirmed to be positive, only positive entries are used;

[0022] Pos_dy: Positive entry 0, a total of 128 entries, each entry is 8 bits, a total of 512 bits, which can be stored in a register;

[0023] Pos_y0: Positive entry 1, with a total of 128 entries, each entry is 8 bits, for a total of 512 bits, which can be stored in a register;

[0024] Once the calculation logic is determined, and the table entry values ​​and bit widths are specified, a single table with 128 entries is adopted: a total of 128 entries, each entry is 4 bits, for a total of 128 * 4 = 512 bits. SIMD is used to optimize the lookup method of the activation function; here, the activation function refers to any complex function, and the value of each entry is different for different complex functions.

[0025]

[0026]

[0027] Random data: any number from 0 to 127; assuming x = 59; 0011 1011

[0028] S1: Clip the input data to the threshold range so that it belongs to the number between 0 and 127;

[0029] S2: Original index calculation: Original table entry index 0~127, each table entry 4 bits, a total of 128 numbers; b) Word index word: Index shifting to obtain the index of the corresponding word;

[0030] SRAI(w,VR0,VR0,3); This function right-shifts the register data by 3 bits, converting the original index into a word index.

[0031] 0~127 is converted to 0~15; 0011 1011(59)>>3=0000 0111(7)b)Byte index: The lower two bits of the original index are valid and are used to select the byte data corresponding to the word;

[0032] Composite index: Combines the Word index and the Byte index, and uses the simd command to read table data;

[0033] ANDV(VR1,VR0,MASK);

[0034] Shift right by 1 bit, select the 8-bit data corresponding to the 4 bits, that is, two 4-bits, to form an 8-bit; 0011 1011(59)>>1=0001 1101&0x3=0000 0001(byte index)

[0035] MASK: Stores a pre-designed mask 0x3 to hold the required byte data; ANDV: Performs a bitwise AND operation between the VR0 register data and the MASK register data, and saves the result to the VR1 register;

[0036] 0011 1011(59)&0000 0011(3)=0000 0011

[0037] c) Merge the word index and the byte index, placing the word index in the lower 0-7 bits and the byte index in the higher 8s-9 bits;

[0038] SLLI(w, VR2, VR1, 8); Left-shifts each number by 8 bits, word by word.

[0039] 0000 0000 0000 0011<<8=0000 0011 0000 0000ORV(VR3,VR0,VR2): Performs an OR operation on the VR0 register and the VR2 register, merging the indices;

[0040] 0000 0000 0000 1110||0000 0011 0000 0000=0000 0011 0000 1110

[0041] Bits 0-7 represent word indexes, and bits 8-9 represent byte indexes;

[0042] S3: Determine the index of the table entry and use the command to look up the table;

[0043] GSHUFWB(VR0,VR3,POS): The VR3 register stores the index calculated in step 2, and the POS register stores the data required for table lookup;

[0044]

[0045] VR3 register value description:

[0046] 0-7 bits: Query the corresponding word index 0-15 in the POS register;

[0047] 8-9 bits: Query the byte index (0-3) in the word corresponding to bits 0-7, and store byte 0 of the word data corresponding to VR0;

[0048] 10-11: Locate bytes 0-3 corresponding to POS and store them into byte 1 of VR0;

[0049] 12-13: Locate bytes 0-3 corresponding to POS and store them into byte 2 of VR0;

[0050] 14-15: Locate bytes 0-3 corresponding to POS and store them in VR0 byte 3;

[0051] ORV(VR0, VR0, MASK1);

[0052] MASK1: 0000 0000 0000 0000 0000 0000 0000 1111

[0053] After the AND operation, the VR0 register retains only the corresponding byte of valid data;

[0054] S4: Separate the high and low 4 bits to obtain valid bit width data;

[0055] SRAI(w, VR1, VR0, 4): Reserve the high 4 bits of data and store the data in the VR1 register; where VR0: each word stores 1 byte of valid data; VR0, each word is right-shifted by 4 bits;

[0056] ORV(VR2, VR0, MASK2): Reserves the lower 4 bits of data, and stores the data in register VR2; where MASK2: 00000000 0000 0000 0000 0000 0000 0011,

[0057] This is a 128-bit fixed mask used to retain the lower 4 bits of data;

[0058] CLTE(w, VR3, MASK3, 0): If the MASK3 data is less than 0, all bits of word in register VR3 are set to 1, otherwise they are set to 0; the mask obtained in this way is used to select the high and low 4 bits of data;

[0059] Among them, MASK3: 0011 1011(59)&0000 0001=0000 0001,

[0060] Perform AND operations to obtain the parity flag and determine the parity sign;

[0061] BSELV(VR3, VR2, VR1): Selects based on the data bit of VR3. If the bit is 0, selects the corresponding bit data of VR2; otherwise, selects the corresponding bit data of VR1. In this way, each word in the 512 bits of the register can be configured with a corresponding positive or negative entry value.

[0062] S5: Perform multiplication on the data obtained from the table entries to get the final result.

[0063] The process of determining the calculation logic and the value and bit width of each entry (i.e., using a single table with 128 entries) further includes:

[0064] Input x is a 16-bit number based on a single table, which is guaranteed to be positive; the table can be set freely according to different entries.

[0065] 1): x_idx = x >> 9, right shift by 9 bits to get a value from 0 to 127. This value is used to look up table entries. Different indexes can retrieve different table entry values ​​for subsequent calculations.

[0066] 2): Look up the table, obtain the index of the table entry by shifting bits, and determine the required parameters based on the index.

[0067] _dy = dy[x_idx]; This returns the parameter of entry 0.

[0068] _y0 = y0[x_idx]; This retrieves the parameters for entry 1.

[0069] Coef = x & (pow(2,11) - 1); This returns the fractional part of the input data.

[0070] 3): Calculation results

[0071] Y_new=coef*dy+y0*pow(2,11)

[0072] The formula is derived from bilinear interpolation:

[0073] Result=y0*(1-ceof)+y1*ceof;

[0074] = y0 - y0*ceof + y1*ceof

[0075] =y0+(y1-y0)*ceof

[0076] =y0+dy*ceof

[0077] Rslt=y_new / pow(2,15+4-11);

[0078] in,

[0079] 15: Entry fusion bit, set to 15;

[0080] 4: The quantized data is 4 bits, indicating that the final output result is 4 bits wide;

[0081] 11: Calculating the fixed-point number of bits requires a restoration operation;

[0082] The 64 entries in a single table are fixed as Pow(2,8).

[0083] The method is implemented using SIMD instructions, specifically SRAI(w,VR0,VR0,3); shifting the register data right by 3 bits in units of words converts the original index into a word index.

[0084] MASK: Stores a pre-designed mask 0x3 to retain the required byte data; ANDV(VR1,VR0,MASK); Shift right by 1 bit to select the 8-bit data corresponding to 4 bits, and combine two 4-bit data into one 8-bit data;

[0085] SLLI(w, VR2, VR1, 8); Shifts each number 8 bits to the left, word by word.

[0086] ORV(VR3,VR0,VR2): Performs an OR operation between the VR0 register and the VR2 register, merging the indices.

[0087] GSHUFWB(VR0,VR3,POS): The VR3 register stores the index calculated in step 2, and the POS register stores the data required for table lookup;

[0088] ORV(VR0, VR0, MASK1);

[0089] MASK1: 0000 0000 0000 0000 0000 0000 0000 1111

[0090] After the AND operation, the VR0 register retains only the corresponding byte of valid data;

[0091] SRAI(w, VR1, VR0, 4): Retain the high 4 bits of data and store the data in register VR1; VR0: Store 1 byte of valid data per word; VR1: Shift each word right by 4 bits; ORV(VR2, VR0, MASK2): Retain the low 4 bits of data and store the data in register VR2; MASK2: 0000 0000 0000 0000 0000 0000 0011CLTE(w, VR3, MASK3, 0): If the MASK3 data is less than 0, then all bits of the word in register VR3 are set to 1, otherwise they are set to 0; the mask obtained in this way is used to select the high and low 4 bits of data;

[0092] MASK3: 0011 1011(59)&0000 0001=0000 0001, determine the odd or even sign; BSELV(VR3, VR2, VR1): select according to the data bit of VR3. If the bit is 0, select the corresponding bit data of VR2. Otherwise, select the corresponding bit data of VR1. In this way, according to the 512 bits in the register, each word can be configured with the corresponding positive or negative table entry value.

[0093] The method is used for 4-bit quantization models, 4-bit or 48-bit hybrid models.

[0094] The method is applicable to the activation processing of operators used in models such as image classification, speech recognition, and natural language processing.

[0095] Therefore, the advantage of this application is that by designing a corresponding mask through SIMD instructions, it is possible to complete the lookup function of a single table 128 entries based on the activation function. The single table 128 means that the data is only integers, requiring high speed, low precision, and supporting 4-bit quantization optimization. Attached Figure Description

[0096] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0097] Figure 1 This is a schematic diagram of this application.

[0098] Figure 2 This is a schematic diagram of the pseudocode implementation of the method in this application. Detailed Implementation

[0099] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0100] like Figure 1 As shown, this invention obtains the final activation function result by fixing table entries and performing table lookup. Common activation functions include slow operations such as exponential and division operations. Complex activations such as sigmoid / swish can be approximated using this method through table lookup. Specifically: single table lookup: if the input data is determined to be positive, only positive table entries are used.

[0101] Pos_dy: Positive entry 0, a total of 128 entries, each entry is 8 bits, a total of 512 bits, which can be stored in a register;

[0102] Pos_y0: Positive entry 1, with a total of 128 entries, each entry is 8 bits, for a total of 512 bits, which can be stored in a register;

[0103] For example, input x, a 16-bit number (the value can be freely set according to different table entries), based on a single table, and this number is guaranteed to be positive:

[0104] 1): x_idx = x >> 9, right shift by 9 bits to get a value from 0 to 127. This value is used to look up table entries. Different indexes can retrieve different table entry values ​​for subsequent calculations.

[0105] 2): Look up the table, obtain the index of the table entry by shifting bits, and determine the required parameters based on the index.

[0106] _dy = dy[x_idx]; This returns the parameter of entry 0.

[0107] _y0 = y0[x_idx]; This retrieves the parameters for entry 1.

[0108] Coef = x & (pow(2,11) - 1); This returns the fractional part of the input data.

[0109] 3): Calculation results

[0110] Y_new=coef*dy+y0*pow(2,11)

[0111] The formula is derived from bilinear interpolation:

[0112] Result=y0*(1-ceof)+y1*ceof;

[0113] = y0 - y0*ceof + y1*ceof

[0114] =y0+(y1-y0)*ceof

[0115] =y0+dy*ceof

[0116] Rslt=y_new / pow(2,15+4-11);

[0117] Wherein, 15: entry fusion bits, generally set to 15; 4: 4 bits of quantized data, indicating that the final output result is 4 bits wide; 11: calculation of fixed-point bits, which requires a restoration operation;

[0118] The 128th table entry is fixed as Pow(2,8).

[0119] Once the calculation logic is determined, along with the table entry values ​​and bit width, a single table with 128 entries is adopted: a total of 128 entries, each entry being 4 bits. SIMD is used to optimize the lookup method of this activation function.

[0120]

[0121]

[0122] For example, any number between 0 and 127: x = 59; 0011 1011

[0123] S1: Clip the input data to a threshold range so that it is a number between 0 and 127; for example, if the input data is 129, after the clip operation, the value will be 127.

[0124] S2: Original index calculation: Original table entry index 0~127, each table entry 4 bits, a total of 128 numbers; c) Word index (word): Index shifting to obtain the index of the corresponding word;

[0125] SRAI(w,VR0,VR0,3); This function shifts the register data 3 bits to the right, converting the original index into a word index.

[0126] 0~127 is converted to 0~15; 0011 1011(59)>>3=0000 0111(7)b)Byte index: The lower two bits of the original index are valid and are used to select the byte data corresponding to the word;

[0127] Composite index: Combines the Word index and the Byte index, and uses the simd command to read table data;

[0128] ANDV(VR1,VR0,MASK);

[0129] Shift right by 1 bit to select the 8-bit data corresponding to the 4 bits (two 4-bit values ​​are combined to form one 8-bit value).

[0130] 0011 1011(59)>>1=0001 1101&0x3=0000 0001(byte index)

[0131] MASK: Stores a pre-designed mask 0x3 to hold the required byte data. ANDV: Performs a bitwise AND operation between the VR0 register data and the MASK register data, and saves the result to the VR1 register. 0011 1011(59)&0000 0011(3)=00000011

[0132] c) Merge: Merge the word index and the byte index, placing the word index in the lower 0-7 bits and the byte index in the higher 8s-9 bits.

[0133] SLLI(w, VR2, VR1, 8); Left-shifts each number by 8 bits, in units of words: 0000 0000 0000 0011 << 8 = 0000 0011 0000 0000

[0134] ORV(VR3,VR0,VR2): Performs an OR operation between the VR0 register and the VR2 register, merging the indices;

[0135] 0000 0000 0000 1110||0000 0011 0000 0000=0000 0011 0000 1110

[0136] Bits 0-7 represent word indexes, and bits 8-9 represent byte indexes.

[0137] S3: Determine the index of the table entry and use the command to look up the table;

[0138] GSHUFWB(VR0,VR3,POS): The VR3 register stores the index calculated in step 2, and POS stores the data required for table lookup.

[0139]

[0140] VR3 register value description:

[0141] 0-7 bits: Query the corresponding word index 0-15 in the POS register;

[0142] 8-9 bits: Query the byte index (0-3) in the word corresponding to bits 0-7, and store byte 0 of the word data corresponding to VR0;

[0143] 10-11: Locate bytes 0-3 corresponding to POS and store them into byte 1 of VR0;

[0144] 12-13: Locate bytes 0-3 corresponding to POS and store them into byte 2 of VR0;

[0145] 14-15: Locate bytes 0-3 corresponding to POS and store them in VR0 byte 3;

[0146] ORV(VR0, VR0, MASK1); MASK1: 0000 0000 0000 0000 0000 00000000 1111

[0147] After the AND operation, the VR0 register retains only the corresponding byte of valid data. S4: Separate the high and low 4 bits to obtain the valid bit width data;

[0148] VR0: Each word stores 1 byte of valid data. VR0, each word is right-shifted by 4 bits: SRAI(w, VR1, VR0, 4); (Retaining the high 4 bits, the data is stored in the VR1 register) MASK2: 0000 0000 0000 0000 0000 0000 0000011ORV(VR2, VR0, MASK2); (Retaining the low 4 bits, the data is stored in the VR2 register)

[0149] MASK3: 0011 1011(59)&0000 0001=0000 0001, determine the parity symbol CLTE(w, VR3, MASK3, 0): if the MASK3 data is less than 0, then all bits of word in register VR3 are set to 1, otherwise they are set to 0; the mask obtained in this way is used to select the high and low 4 bits of data.

[0150] BSELV(VR3, VR2, VR1): Selects the corresponding bit data based on the VR3 data bit. If the bit is 0, selects the corresponding bit data of VR2; otherwise, selects the corresponding bit data of VR1. This allows each word in the 512 bits of the register to be configured with a corresponding positive or negative entry value. S5: Performs a multiplication operation on the data obtained from the entries to obtain the final result.

[0151] The following is the pseudocode implementation of the method, such as Figure 2 As shown:

[0152] SRAI(w,VR0,VR0,3); This function right-shifts the register data by 3 bits, converting the original index into a word index.

[0153] MASK: Stores a pre-designed mask 0x3 to retain the required byte data; ANDV(VR1,VR0,MASK); Shift right by 1 bit to select the 8-bit data corresponding to 4 bits, and combine two 4-bit data into one 8-bit data;

[0154] SLLI(w, VR2, VR1, 8); Shifts each number 8 bits to the left, word by word.

[0155] ORV(VR3,VR0,VR2): Performs an OR operation between the VR0 register and the VR2 register, merging the indices.

[0156] GSHUFWB(VR0,VR3,POS): The VR3 register stores the calculated index, and POS stores the data required for table lookup;

[0157] ORV(VR0, VR0, MASK1);

[0158] MASK1: 0000 0000 0000 0000 0000 0000 0000 1111

[0159] After the AND operation, the VR0 register retains only the corresponding byte of valid data;

[0160] SRAI(w, VR1, VR0, 4): Retain the high 4 bits of data and store the data in register VR1; VR0: Store 1 byte of valid data per word; VR1: Shift each word right by 4 bits; ORV(VR2, VR0, MASK2): Retain the low 4 bits of data and store the data in register VR2; MASK2: 0000 0000 0000 0000 0000 0000 0011CLTE(w, VR3, MASK3, 0): If the MASK3 data is less than 0, then all bits of the word in register VR3 are set to 1, otherwise they are set to 0; the mask obtained in this way is used to select the high and low 4 bits of data;

[0161] MASK3: 0011 1011(59)&0000 0001=0000 0001, determine the odd or even sign; BSELV(VR3, VR2, VR1): select according to the data bit of VR3. If the bit is 0, select the corresponding bit data of VR2. Otherwise, select the corresponding bit data of VR1. In this way, according to the 512 bits in the register, each word can be configured with the corresponding positive or negative table entry value.

[0162] This invention accelerates table lookup logic through SIMD instructions, improves computational efficiency by merging indexes, setting corresponding masks, obtaining the correct table entries, and then performing calculations.

[0163] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for optimizing a single 128-item table based on an activation function, characterized in that, The method accelerates the table lookup logic through SIMD instructions, merges indexes, sets corresponding masks, retrieves the correct table entries, and then performs calculations, including: Single table query: If the input data is confirmed to be positive, only positive entries are used; Pos_dy: Positive entry 0, with 128 entries, each entry is 8 bits, for a total of 512 bits, which can be stored in a register; Pos_y0: Positive entry 1, with 128 entries, each entry is 8 bits, for a total of 512 bits, which can be stored in a register; Once the calculation logic is determined, and the table entry values ​​and bit widths are specified, a single table with 128 entries is adopted: a total of 128 entries, each entry is 4 bits, for a total of 128 * 4 = 512 bits. SIMD is used to optimize the lookup method of the activation function; here, the activation function refers to any complex function, and the value of each entry is different for different complex functions. Random data: any number from 0 to 127; assuming x = 59; 0011 1011 S1: Clip the input data to the threshold range so that it belongs to the number between 0 and 127; S2: Original index calculation: Original table entry index 0~127, each table entry 4 bits, a total of 128 numbers; a) Word index: Shift the index to obtain the index of the corresponding word; SRAI(w,VR0,VR0,3); This function right-shifts the register data by 3 bits, converting the original index into a word index. 0~127 is converted to 0~15; 0011 1011(59)>>3=0000 0111(7) b) Byte Index: The lower two bits of the original index are valid and are used to select the byte data corresponding to the word; Composite index: Combines the Word index and the Byte index into a single index, and uses the simd command to retrieve table data; ANDV(VR1,VR0,MASK); Shift right by 1 bit, select the 8-bit data corresponding to the 4-bit, that is, two 4-bit groups, to form an 8-bit; 0011 1011(59)>>1=0001 1101&0x3=0000 0001(byte index) MASK: Stores a pre-designed mask 0x3 to hold the required byte data; ANDV: Performs a bitwise AND operation between the VR0 register data and the MASK register data, and saves the result to the VR1 register; 0011 1011(59)&0000 0011(3)＝0000 0011 c) Merge the word index and the byte index, placing the word index in the lower 0-7 bits and the byte index in the higher 8s-9 bits; SLLI(w, VR2, VR1, 8); Left-shifts each number by 8 bits, word by word. 0000 0000 0000 0011<<8＝0000 0011 0000 0000 ORV(VR3,VR0,VR2): Performs an OR operation between the VR0 register and the VR2 register, merging the indices; 0000 0000 0000 1110||0000 0011 0000 0000=0000 0011 0000 11100~7 bits represent word index, 8~9 represent byte index; S3: Determine the index of the table entry and use the command to look up the table; GSHUFWB(VR0,VR3,POS): The VR3 register stores the index calculated in step 2, and the POS register stores the data required for table lookup; VR3 register value description: 0-7 bits: Query the corresponding word index 0-15 in the POS register; 8-9 bits: Query the byte index (0-3) in the word corresponding to bits 0-7, and store byte 0 of the word data corresponding to VR0; 10-11: Locate bytes 0-3 corresponding to POS and store them into byte 1 of VR0; 12-13: Locate bytes 0-3 corresponding to POS and store them into byte 2 of VR0; 14-15: Locate bytes 0-3 corresponding to POS and store them in VR0 byte 3; ORV(VR0, VR0, MASK1); MASK1: 0000 0000 0000 0000 0000 0000 0000 1111 After the AND operation, the VR0 register retains only the corresponding byte of valid data; S4: Separate the high and low 4 bits to obtain valid bit width data; SRAI(w, VR1, VR0, 4): Reserve the high 4 bits of data and store the data in the VR1 register; where VR0: each word stores 1 byte of valid data; VR0, each word is right-shifted by 4 bits; ORV(VR2, VR0, MASK2): Reserve the lower 4 bits of data, and store the data in register VR2; where MASK2: 0000 00000000 0000 0000 0000 0000 0011, This is a 128-bit fixed mask used to retain the lower 4 bits of data; CLTE(w, VR3, MASK3, 0): If the MASK3 data is less than 0, all bits of word in register VR3 are set to 1, otherwise they are set to 0; the mask obtained in this way is used to select the high and low 4 bits of data; Among them, MASK3: 0011 1011(59)&0000 0001=0000 0001, Perform AND operations to obtain the parity flag and determine the parity sign; BSELV(VR3, VR2, VR1): Selects based on the data bit of VR3. If the bit is 0, selects the corresponding bit data of VR2; otherwise, selects the corresponding bit data of VR1. In this way, each word in the 512 bits of the register can be configured with a corresponding positive or negative entry value. S5: Perform multiplication on the data obtained from the table entries to get the final result.

2. The method for optimizing a single 128-entry table based on an activation function according to claim 1, characterized in that, The process of determining the calculation logic and the value and bit width of each entry (i.e., using a single table with 128 entries) further includes: Input x is a 16-bit number based on a single table, which is guaranteed to be positive; the table can be set freely according to different entries. 1): x_idx = x >> 9, right shift by 9 bits to get a value from 0 to 127. This value is used to look up table entries. Different indexes can retrieve different table entry values ​​for subsequent calculations. 2): Look up the table, obtain the index of the table entry by shifting bits, and determine the required parameters based on the index. _dy = dy[x_idx]; This returns the parameter of entry 0. _y0 = y0[x_idx]; This retrieves the parameters for entry 1. Coef = x & (pow(2,11) - 1); This returns the fractional part of the input data. 3): Calculation results Y_new=coef*dy+y0*pow(2,11) The formula is derived from bilinear interpolation: Result=y0*(1-ceof)+y1*ceof; = y0 - y0*ceof + y1*ceof =y0+(y1-y0)*ceof =y0+dy*ceof Rslt=y_new / pow(2,15+4-11); in, 15: Entry fusion bit, set to 15; 4: The quantized data is 4 bits, indicating that the final output result is 4 bits wide; 11: Calculating the fixed-point number of bits requires a restoration operation; The 64 entries in a single table are fixed as Pow(2,8).

3. The method for optimizing a single 128-item table in a neural network based on an activation function according to claim 1, characterized in that, The method is implemented using SIMD instructions, and the specific instruction code includes: SRAI(w,VR0,VR0,3); This function right-shifts the register data by 3 bits, converting the original index into a word index. MASK: Stores a pre-designed mask 0x3 to retain the required byte data; ANDV(VR1,VR0,MASK); Shift right by 1 bit to select the 8-bit data corresponding to 4 bits, and combine two 4-bit data into one 8-bit data; SLLI(w, VR2, VR1, 8); Shifts each number 8 bits to the left, word by word. ORV(VR3,VR0,VR2): Performs an OR operation between the VR0 register and the VR2 register, merging the indices. GSHUFWB(VR0,VR3,POS): The VR3 register stores the index calculated in step 2, and the POS register stores the data required for table lookup; ORV(VR0, VR0, MASK1); MASK1: 0000 0000 0000 0000 0000 0000 0000 1111 After the AND operation, the VR0 register retains only the corresponding byte of valid data; SRAI(w, VR1, VR0, 4): Reserves the high 4 bits of data, storing the data in register VR1; VR0: Stores 1 byte of valid data per word; VR1: Shifts each word 4 bits to the right; ORV(VR2, VR0, MASK2): Reserves the low 4 bits of data, storing the data in register VR2; MASK2: 0000 0000 0000 0000 0000 0000 0000 0011 CLTE(w, VR3, MASK3, 0): If the MASK3 data is less than 0, all bits of the word in register VR3 are set to 1, otherwise they are set to 0; the mask obtained in this way is used to select the high and low 4 bits of data; MASK3: 0011 1011(59)&0000 0001=0000 0001, determine the odd / even sign; BSELV(VR3, VR2, VR1): Selects the corresponding bit data based on the VR3 data bit. If the bit is 0, selects the corresponding bit data of VR2; otherwise, selects the corresponding bit data of VR1. In this way, each word can be configured with a corresponding positive or negative entry value based on the 512 bits in the register.

4. The method for optimizing a single 128-entry table based on an activation function according to claim 1, characterized in that, The method is used for 4-bit quantization models, 4-bit or 48-bit hybrid models.

5. The optimization method for a single 128-entry table based on an activation function according to claim 1, characterized in that, The method is applicable to the activation processing of operators used in models such as image classification, speech recognition, and natural language processing.

Images