Crossbar Binary Mask for Neural Network Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face challenges in multitask adaptation due to catastrophic forgetting, where updating parameters to learn new tasks leads to forgetting old knowledge, and existing hardware solutions like ReRAM crossbars require significant energy and memory overhead for mask-based methods.
Innovation Solution
The Crossbar Binary Mask (XBM) method learns column-wise binary masks for multitask adaptation, reducing hardware overhead and energy consumption by controlling entire columns instead of individual cells, using a Gumbel-Sigmoid trick for gradient estimation and minimizing reprogramming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If mask-based methods (Piggyback, KSM) are used to alleviate catastrophic forgetting, then performance on old tasks is preserved, but mask memory size and hardware overhead increase significantly
Solution Approach 1:
The patent segments the mask memory into column-wise groups where each group shares common parameters. Instead of storing individual mask values for each weight parameter, the system divides weights into groups and stores shared control signals at the group level, dramatically reducing mask memory size while maintaining the ability to selectively activate/deactivate weight groups for different tasks.
Solution Approach 2:
The patent creates a universal mask control mechanism that can serve multiple tasks simultaneously through column-wise grouping. The same set of control signals can be used across different task adaptations by selectively activating different column groups, making the mask system multi-functional and reducing overall memory requirements.
2Use of energy by moving object
If ReRAM crossbar architecture is used for DNN acceleration, then energy consumption is reduced, but hardware complexity and mask buffer requirements increase
Solution Approach 1:
The patent segments the ReRAM crossbar architecture into column-wise groups with shared control mechanisms. By organizing the crossbar array into groups that share control signals and buffer resources, the system reduces the overall hardware complexity while maintaining energy-efficient in-memory computing capabilities for multitask adaptation.
Solution Approach 2:
The patent merges multiple mask control functions into a unified column-wise buffer system. Instead of having separate buffers for each weight parameter, the system combines control signals at the column group level, reducing hardware complexity while maintaining the ability to perform task-specific adaptations in the ReRAM crossbar architecture.
3Productivity
If fine-tuning is used to adapt DNN to new tasks, then learning efficiency is improved, but catastrophic forgetting of old tasks occurs
Solution Approach 1:
The patent introduces dynamic mask control that can be adjusted during task adaptation. The system dynamically activates or deactivates specific column groups based on task requirements, allowing the model to adapt to new tasks through fine-tuning while preserving important parameters from old tasks through selective masking, thus preventing catastrophic forgetting.
Solution Approach 2:
The patent introduces column-wise masks as an intermediary mechanism between the backbone model parameters and the task-specific adaptations. These masks act as a mediator that allows fine-tuning for new tasks while protecting critical parameters from being overwritten, enabling learning efficiency improvement without catastrophic forgetting.
Data Source
AI summary
A neural network accelerator architecture for multiple task adaptation comprises a volatile memory comprising a plurality of subarrays, each subarray comprising M rows and N columns of volatile memory cells; a source line driver connected to a plurality of N source lines, each source line corresponding to a column in the subarray; a binary mask buffer memory having size at least N bits, each bit corresponding to a column in the subarray, where a 0 corresponds to turning off the column for a convolution operation and a 1 corresponds to turning on the column for the convolution operation; and a controller configured to selectively drive each of the N source lines with a corresponding value from the mask buffer; wherein each column in the subarray is configured to store a convolution kernel.