ANN Crossbar Processor With Multi-Mode Weights for Energy-Area Balance

The weight storage is segmented into two distinct types: volatile memory (SRAM/DRAM) for flexible, trainable weights that can be updated, and non-volatile memory (Flash, FRAM, MRAM) for fixed, pre-trained weights that remain static. This segmentation allows the system to selectively apply programmability only where needed while using fixed storage for the majority of weights, thereby reducing overall energy consumption and chip area while maintaining necessary flexibility for training and adaptation.

Different regions of the neural network are assigned different weight storage types based on their specific requirements. Trainable layers use volatile memory for weight storage to enable gradient updates during backpropagation, while fixed layers use non-volatile memory for weight storage to eliminate repeated loading and reduce energy consumption. This local quality differentiation optimizes the balance between flexibility and energy efficiency for each specific functional region.

The parameter storage is divided between on-chip volatile memory (SRAM/DRAM) and on-chip non-volatile memory (Flash/FRAM/MRAM). Volatile memory stores only the essential trainable parameters that require frequent updates, while non-volatile memory stores the larger set of fixed pre-trained parameters. This segmentation enables the system to process large neural networks with billions of parameters without proportionally increasing the area of expensive volatile memory, thus improving processing capability while controlling chip area.

Pre-trained weights are copied into non-volatile memory during manufacturing or initial setup, creating a permanent on-chip copy that eliminates the need to repeatedly load these weights from off-chip memory during inference. This copying strategy reduces both chip area requirements and energy consumption by making frequently used weights immediately accessible without requiring large volatile memory buffers or repeated data transfers.

The patent extracts and removes the need for repeated off-chip memory accesses by implementing on-chip non-volatile memory storage for fixed weights. This extraction eliminates the energy-intensive data transfer path between off-chip DRAM and the processing core for weights that do not change during inference, thereby reducing energy consumption while maintaining the cost benefits of using standard off-chip DRAM for volatile data storage.

Weights are preliminarily loaded and stored in on-chip non-volatile memory during chip manufacturing or initial system setup, before the actual inference operations begin. This preliminary action ensures that all fixed weights are already present in fast, energy-efficient on-chip storage, eliminating the need for subsequent energy-hungry data transfers from off-chip memory during the inference process, thus reducing operational energy consumption.