ARMv8 Microkernel for Binary Ternary Matrix Multiplication

The patent segments the matrix multiplication process into microkernels that operate on small blocks of data (e.g., 4x4 or 8x8 blocks). This segmentation allows processing data in chunks that fit within cache memory, eliminating the need for large intermediate matrices required by im2col transformation while maintaining computational efficiency through block-wise operations.

The patent transforms the traditional GeMM approach by introducing a new dimension of processing through SIMD vectorization. Instead of processing scalar values sequentially, the invention processes multiple data elements simultaneously in parallel lanes, effectively adding a dimension of parallelism that reduces memory access requirements while maintaining high throughput.

The patent applies different quantization strategies to different parts of the neural network based on their importance and sensitivity to quantization error. Critical layers maintain higher precision while less sensitive layers use aggressive binarization, allowing the system to achieve high speed with minimal quality loss by optimizing local quality requirements for different network components.

The patent creates a composite quantization scheme that combines binary weights with higher-precision activations or vice versa. This composite approach allows the network to benefit from the speed of binary operations while maintaining sufficient precision through the higher-bit components, achieving a balance between inference speed and recognition quality.

The patent extracts the computationally intensive parts of ternary operations and replaces them with optimized sequences of binary operations. By taking out the complex ternary multiplication and replacement logic, the invention uses a series of simpler binary XOR and population count operations that achieve the same result with lower computational complexity and better hardware utilization.

The patent substitutes traditional ternary arithmetic operations with bitwise logical operations and population count instructions native to ARM processors. This substitution replaces complex mechanical arithmetic with simpler bitwise operations that leverage the processor's existing instruction set, reducing computational complexity while maintaining ternary functionality.

The patent implements dynamic instruction selection that adapts to the available processor features. The system detects whether SIMD extensions or specialized instructions are available and dynamically chooses the appropriate instruction set, allowing the same software to achieve high performance on capable processors while maintaining compatibility with standard ARM processors through fallback to basic instructions.

The patent creates a universal matrix multiplication implementation that works across different ARM processor variants. By designing microkernels that can operate with both standard ARM instructions and optimized SIMD instructions, the invention achieves multi-functionality that maintains broad compatibility while delivering high performance when advanced instructions are available.