1results about How to "Increase resource consumption" patented technology

This invention discloses a method for processing matrix multiplication weight data in model inference, adapted to large-scale matrix multiplication in BFP format for large models in the Transformer architecture. Addressing the pain points of existing architectures regarding BFP exponent and mantissa splitting, bandwidth, and resource consumption, this method employs (64,64) weight blocks, dual address generator 1:4 scheduling, waterline flow control, and double buffering techniques to achieve full-bandwidth read / write between DDR, BRAM, and computational units. Through multi-core parallelism and multi-round multiplication and accumulation, performance is improved by more than 3 times compared to FP16, reducing memory usage and design complexity while ensuring inference accuracy, providing an efficient hardware acceleration solution for LLM inference in resource-constrained environments.