Buffer Memory Architecture for CNN Processing Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN-based processing units face challenges with slow computational speed, high power consumption, and impractical integration of memory technologies for storing imagery data and filter coefficients, along with security concerns for mission-critical applications.
Innovation Solution
A buffer memory architecture for CNN-based processing units is developed, incorporating STT-RAM or OST-MRAM memories for both filter coefficients and imagery data, with a one-time-programming (OTP) memory for security, all fabricated on the same silicon chip, using techniques such as breaking down the oxide barrier of MTJ elements to create OTP memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If different memory technologies (SRAM for imagery data, Flash for filter coefficients) are integrated on a single silicon chip, then data storage requirements are met, but manufacturing complexity increases and power consumption becomes too high
Solution Approach 1:
The patent merges multiple memory types (SRAM, OTP, and coefficient storage) into a unified memory subsystem architecture. All memory elements use the same MTJ-based cell structure, allowing them to be fabricated using identical processes on the same silicon chip. This consolidation eliminates the complexity of integrating dissimilar memory technologies while maintaining the ability to store both imagery data and filter coefficients efficiently.
Solution Approach 2:
The memory subsystem employs a universal MTJ-based cell design that can serve multiple functions: storing imagery data in SRAM mode, storing filter coefficients in OTP mode, and providing fast access for both data types. The same basic cell structure can be configured differently through fabrication parameters to achieve various memory characteristics, reducing overall device complexity.
2Quantity of substance
If multiple memory types are integrated on a single chip, then storage requirements are satisfied, but power consumption increases excessively
Solution Approach 1:
The patent applies local quality by assigning different operational characteristics to different regions of the memory subsystem. SRAM cells use standard read/write operations for frequently accessed imagery data, while OTP cells use one-time programming for filter coefficients. Each memory type is optimized for its specific function, minimizing unnecessary power consumption from operations not required for that particular memory region.
Solution Approach 2:
By combining multiple memory types into a unified architecture using the same MTJ cell structure, the patent reduces redundant control logic and interface circuits that would otherwise be needed for separate memory implementations. This consolidation lowers overall power consumption while maintaining the ability to store both imagery data and filter coefficients.
3Adaptability or versatility
If CNN processing is performed using software solutions or general-purpose hardware, then flexibility is maintained, but computational speed becomes too slow for practical AI processing
Solution Approach 1:
The patent segments the CNN processing function into dedicated hardware components: CNN logic circuits are separated from general-purpose processing units and integrated directly with the memory subsystem. This segmentation allows convolution operations to be performed in parallel with memory access, dramatically increasing computational speed while maintaining the ability to process different types of input data through configurable filter coefficients.
Solution Approach 2:
The memory subsystem acts as an intermediary between input data and CNN processing logic, providing on-chip storage for both imagery data and filter coefficients. This eliminates the need for external memory access during processing, reducing latency and enabling faster computation while maintaining flexibility through programmable filter coefficient storage in OTP memory.
4Ease of manufacture
If standard MTJ elements are used in memory cells, then manufacturing is simplified, but security requirements for mission-critical applications cannot be met
Solution Approach 1:
The patent applies local quality by implementing OTP functionality in specific regions of the memory subsystem where security is required, such as filter coefficient storage. The OTP cells use modified MTJ structures with broken-down oxide barriers that allow one-time programming but prevent subsequent modification. This localized security enhancement maintains ease of manufacture for the majority of the memory subsystem while providing cryptographic security where needed.
Solution Approach 2:
The patent uses composite MTJ cell structures that combine standard MTJ elements with additional security features. The OTP cells incorporate broken-down oxide barriers and specialized write circuitry that enable one-time programming. This composite approach maintains compatibility with standard fabrication processes while adding security functionality through material and structural modifications in specific cell regions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables efficient processing-in-memory, low power consumption, and high read/write speed for large amounts of imagery data, while ensuring security through the use of OTP memory, making CNN-based ICs more practical for AI applications.
Implementation Method 1
MTJ element which has a top ferromagnetic layer, a bottom ferromagnetic layer and an oxide barrier layer in between
Implementation Method 2
breaking down oxide barrier of the MTJ element in either STT-RAM or OST-MRAM cell
Data Source
AI summary
CNN based digital IC for AI contains a number of CNN processing units. A first CNN processing unit contains CNN logic circuits operatively coupling to a memory subsystem, which includes a first one-time-programming (OTP) memory for filter coefficients and a second memory for imagery data. A second CNN processing unit contains CNN logic circuits operatively coupling to a memory subsystem that includes a first memory for filter coefficients, a second memory for imagery data and a third OTP memory for unique data pattern (e.g., security purpose). Either STT-RAM or OST-MRAM can be configured as different memories of the memory subsystem.