In-Memory Compute Structure for Convolutional Neural Network Energy Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hardware and methods are inadequate for performing computationally intensive arithmetic operations like convolutions in machine learning, particularly due to inefficiencies in data reuse, redundant data access, and high energy consumption.
Innovation Solution
A cache and compute structure that efficiently processes data by rotating and shifting it within a memory system, allowing for the reuse of loaded data without re-fetching, and integrating compute hardware directly into memory to reduce data movement and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional general-purpose processors are used to perform convolution operations, then the system can handle diverse computing tasks, but the computational throughput is limited and energy consumption is high
Solution Approach 1:
The patent merges memory and compute functions into a single integrated structure where memory cells directly perform arithmetic operations. This eliminates the separation between storage and processing units, allowing data to be computed in-place without being moved to separate processing units, thereby increasing throughput while reducing energy consumption associated with data movement.
Solution Approach 2:
The patent introduces sense amplifiers as intermediary components that serve dual functions: reading data from memory cells and performing arithmetic operations on that data. These sense amplifiers act as mediators between storage and computation, enabling in-memory computing without requiring traditional separate processing units.
2Productivity
If data is repeatedly accessed from standard memory during arithmetic operations, then data can be reused for multiple calculations, but the number of read and write operations increases significantly
Solution Approach 1:
The patent performs preliminary actions by loading data into the compute structure once and then reusing it for multiple arithmetic operations through rotation and shifting. This eliminates the need to repeatedly read from and write to standard memory, reducing access time and improving data reuse efficiency.
Solution Approach 2:
The patent implements a nested structure where a compute structure is integrated within or directly coupled to the memory system. This nested arrangement allows the compute structure to access and process memory data in-place, minimizing the need for repeated data access operations and reducing the time overhead associated with data movement.
3Reliability
If intermediate results are stored and retrieved from memory locations, then arithmetic operations can be completed, but the computational time increases due to added storage and retrieval steps
Solution Approach 1:
The patent merges the functions of intermediate result storage and computation into the same in-place structure. Intermediate results are maintained within the compute structure during arithmetic operations, eliminating the need to repeatedly write to and read from separate memory locations, thereby reducing computational time while maintaining computation accuracy.
4Adaptability or versatility
If software functions are embedded in microprocessors to convert convolution operations into matrix-multiply operations, then convolution can be performed, but redundant data generation and access occurs
Solution Approach 1:
The patent applies local quality by enabling the memory system to perform specific arithmetic operations (such as convolutions and pooling) directly at the location where data is stored. This localized computation eliminates the need for global data reformatting and redundant data generation required by traditional software-based conversion methods, while maintaining operation flexibility through configurable compute structures.
Data Source
AI summary
Described herein are systems and methods that increase the utilization and performance of computational resources, such as storage space and computation time, thereby, reducing computational cost. Various embodiments of the invention provide for a hardware structure that allows both streaming of source data that eliminates redundant data transfer and allows for in-memory computations that eliminate requirements for data transfer to and from intermediate storage. In certain embodiments, computational cost is reduced by using a hardware structure that enables mathematical operations, such as element-wise matrix multiplications employed by convolutional neural networks, to be performed automatically and efficiently.