3 results about "16-bit" patented technology

This invention provides a lookup table method for half-precision floating-point number FP16, including: S1, 1) NNA quantization input; 2) Int16 input: 16 bits output to Lbit, S=0 indicates a positive number, V=0 indicates no overflow; Int16 input, FP16 input: converted to FPi; lookup table value FPi; obtain offset(FPPi), scale_n(FPPi), scale_p(FPPi); add offset, according to the result: positive multiply by scale_p, negative multiply by scale_n; obtain: absolute value converted to integer, take the lower 16 bits and output to Lbit, overflow 16 bits V=1, otherwise V=0, output overflow to V, output sign bit to S; S2, obtain SVLbit 16-bit positive integer; S3, according to LU_Table, SAT_T0, IDX0_T, get: V=1: take the value SAT_T0 (S=1) or SAT_T1 (S=0), V=0: take the high 5 bits of Lbit as idx, S=1: if idx≥IDX0_T1 take the value SAT_T0, S=0: idx+=IDX0_T1, if idx>31 take the value SAT_T1, take the low 11 bits of Lbit as dx, use idx and dx to look up the table and calculate y; S4, lookup result y (FPin); true value output: convert to FP16; quantization output: get ofbit, convert to integer and saturate to ofbit unsigned number; S5, NNA quantization output or XOR RAM port.

The application discloses a 64-bit fixed-point trigonometric function operation method of an ARM processor, which comprises the following steps: firstly, defining a 64-bit fixed-point number and a data type conversion mechanism; defining a sin trigonometric function table based on a 32-bit fixed-point number and storing the table in the form of integer data; secondly, converting a floating-point number to be operated into a 64-bit fixed-point number, performing a modulus operation on the 64-bit fixed-point number and a 64-bit fixed-point number form of π / 2, obtaining a trigonometric function variable, representing the variable in the form of integer data, and then incrementally searching a table to obtain a sin value of the converted 32-bit fixed-point number in the sin trigonometric function table; using the 64-bit fixed-point number to represent and using linear interpolation to compensate the value of the lower 16 bits of the decimal point, so as to obtain a sin value corresponding to the 64-bit fixed-point number; and finally, according to the sin value, using a trigonometric function transformation formula to calculate the values of trigonometric functions cos and tan. The operation method can realize high-precision decimal trigonometric function operation with high operation speed in the ARM processor.

The application provides an optimization method of a low-bit arbitrary-size independent convolution simd, comprising the following steps: S1, processing of convolution kernel data, the data storage mode adopts a convolution calculation mode, which can greatly reduce the loading time, and the data needs to be converted; converting the data; S2, design of convolution calculation and accumulation: since the accumulation sum cannot exceed 16 bits after multiplication and adjacent addition, the data must be processed; first, using the 16-bit accumulation sum to calculate, calculating 14 times each time, since the instruction is multiplied first and then adjacent addition in the calculation, the actual instruction accumulation number is 7 times; then, converting the data in the register into 32 bits and accumulating into a 32-bit register. The application can realize a speed increase of several times, and can increase by about 35 times relative to a C program. The 16-bit accumulation can be used, and the speed can be improved.