Near Data Processing for Deep Neural Network Quantization

The system separates quantization operations from general processing by dedicating specific hardware components (quantization units) within the memory controller to handle quantization tasks, while general processors focus on other operations. This segmentation resolves the contradiction by improving quantization efficiency without increasing overall resource consumption.

The patent introduces a memory controller with integrated quantization units as an intermediary between memory and general processors. This intermediary handles quantization operations locally, eliminating the need for general processors to assist with quantization, thereby improving efficiency and reducing computing resource occupation.

The system performs quantization operations preliminarily within the memory controller before data is transferred to processing units. By preparing quantized data in advance during memory access operations, the system eliminates subsequent quantization delays and unnecessary data access, directly improving processing speed and reducing time loss.

The memory controller performs quantization operations on its own without requiring external processor assistance. This self-service capability allows the system to quantize data during normal memory operations, eliminating additional data access cycles and improving overall processing efficiency.

The system dynamically changes the precision parameter of data representations by performing quantization operations that convert high-precision floating-point numbers to lower-precision formats. This parameter change reduces memory bandwidth and storage requirements while maintaining sufficient model accuracy through controlled quantization processes.

The patent applies different precision levels to different parts of the neural network processing pipeline. High-precision operations are performed only where necessary for model accuracy, while quantized lower-precision operations are used for other computations. This local quality approach optimizes the balance between accuracy and resource usage.