CIM Memory Macro with Integrated Weight Buffers for Concurrent Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices face issues such as lowered performance, increased processing time, and higher power consumption due to the need to stop CIM operations whenever weight data updating is performed, as they require external weight buffers and access the whole memory array for both tasks.
Innovation Solution
Incorporating weight buffers within the same memory macro as the memory array, allowing simultaneous performance of weight data updating and CIM operations, thereby eliminating or reducing the need for external weight buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If weight data updating is performed in existing memory devices, then weight data can be updated, but CIM operations must be stopped and processing time increases
Solution Approach 1:
The memory array is divided into multiple banks, and weight buffers are integrated within each bank. This segmentation allows different banks to operate independently, enabling CIM operations in one bank while weight data updating occurs in another bank simultaneously, thus reducing processing time without compromising updating accuracy.
Solution Approach 2:
Weight buffers are merged with the memory array structure, with each memory bank containing its own integrated weight buffer. This combining of weight storage and data storage functions within the same bank structure enables concurrent operations, eliminating the time loss associated with stopping CIM operations for weight updates.
2Quantity of substance
If external weight buffers are used, then weight data can be stored, but chip area increases
Solution Approach 1:
The weight buffers are merged with the memory array to form an integrated structure where each memory bank contains its own weight buffer. This integration eliminates the need for separate external weight buffer memory devices, reducing overall chip area while maintaining full weight data storage capacity through in-place buffering within the memory banks.
Solution Approach 2:
The memory banks are designed to serve multiple functions: they store both weight data and perform CIM operations using the same physical structure. The integrated weight buffers within each bank provide multi-functionality, allowing the same memory infrastructure to handle both weight storage and computational operations, thereby reducing the need for additional dedicated weight buffer memory and reducing chip area.
3Reliability
If weight data updating is performed, then weight data can be refreshed, but power consumption increases
Solution Approach 1:
The memory system is segmented into multiple independent banks, each with its own integrated weight buffer. This segmentation allows weight data updating to be performed locally within individual banks without affecting other banks, enabling selective and efficient power management where only the necessary banks consume power during updating operations, thus maintaining data freshness while reducing overall power consumption.
Solution Approach 2:
By merging weight buffers with memory banks, the patent enables combined weight storage and computation functionality within the same structure. This integration allows for more efficient power management where the same physical infrastructure supports both weight data storage and CIM operations, reducing the need for separate power-intensive external buffer operations and enabling concurrent operations without proportional power increases.
Data Source
AI summary
A memory device has a memory array including a memory segment to store weight data, a weight buffer coupled to the memory segment and configured to hold new weight data to be updated in the memory segment, a logic circuit, and a computation circuit coupled to an output of the logic circuit. The logic circuit further has a first input coupled to the memory segment by a bit line, and a second input configured to receive input data. The logic circuit is configured to generate, at the output, intermediate data corresponding to the input data and the weight data read from the memory segment through the bit line. The computation circuit is configured to, based on the intermediate data, generate output data corresponding to a computation performed on the input data and the weight data read from the at least one memory segment.