Composite Memory Units for In-Memory Computing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In-memory computing devices face challenges with volatile memory cells in terms of high power consumption and inference accuracy, while non-volatile memory cells suffer from device variability and fluctuations in stored weights, leading to less accurate output data.

Innovation Solution

An integrated circuit with an in-memory computing device implementing a neural network, utilizing an array of composite memory units comprising volatile and non-volatile memory cells, where data transfer between cells is enabled by intra-unit data paths and control switches, allowing for fast and accurate sum-of-products operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If volatile memory cells are used for in-memory computing, then fast sum-of-products operations can be performed, but power consumption increases and data transfer time increases

Engineering Contradiction:

Improvesum-of-products operation speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The patent divides the memory system into two distinct segments: volatile memory cells (first memory cells) dedicated to performing sum-of-products operations, and non-volatile memory cells (second memory cells) dedicated to storing weight data. This segmentation allows each type of memory cell to be optimized for its specific function, enabling fast computations while reducing overall power consumption by keeping weight data in low-power non-volatile storage.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a data transfer mechanism with control switches that acts as an intermediary between non-volatile weight storage and volatile computation units. Weight data is transferred only when needed for computation, allowing the system to maintain weight data in low-power non-volatile memory while enabling fast access to volatile memory for actual computations, thus reducing overall power consumption.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Use of energy by moving object

If non-volatile memory cells are used for storing weights, then power consumption is reduced, but device variability causes fluctuations in stored weights leading to lower inference accuracy

Engineering Contradiction:

Improvepower consumptionVSAvoidinference accuracy

Core Design Contradiction:

Use of energy by moving objectVSMeasurement precision

Solution Approach 1:

The patent separates the functions of weight storage and weight usage into different memory types. Non-volatile memory cells store weight data with reduced power consumption, while volatile memory cells perform the actual computations. This segmentation allows the system to tolerate some variability in non-volatile weight storage because the volatile memory provides a stable, refreshable environment for actual computation operations.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The volatile memory cells act as an intermediary buffer between the non-volatile weight storage and the computation process. Weight data is transferred from non-volatile to volatile memory before use, allowing for potential recalibration or refreshing of the weight values in the volatile memory, thereby compensating for variability and drift that occur in non-volatile storage over time.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Speed

If weight data is transferred from non-volatile to volatile memory, then computation speed improves, but data transfer time is required

Engineering Contradiction:

Improvecomputation speedVSAvoiddata transfer time

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent implements preliminary action by pre-transferring weight data from non-volatile to volatile memory before computation is needed, or maintaining weight data in volatile memory during active computation periods. Control switches enable selective data transfer, allowing the system to prepare computation data in advance, thereby reducing the impact of transfer time on overall computation speed.

Inventive Principle:

Principle #10Preliminary action

4Measurement precision

If composite memory units with both volatile and non-volatile cells are used, then inference accuracy and power efficiency improve, but device complexity increases

Engineering Contradiction:

Improveinference accuracyVSAvoidmemory unit structure

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent merges volatile and non-volatile memory cells into composite memory units that function together as a unified system. Each composite unit contains both types of memory cells with controlled data transfer paths, allowing the system to achieve both low power consumption and high inference accuracy while managing complexity through integrated design rather than separate systems.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11119674B2Memory devices and methods for operating the same

Publication Date: 2021.09.14 MACRONIX INTERNATIONAL CO LTD

AI summary

A memory device includes an array of composite memory units. At least one of the composite memory units comprises a first memory cell of a first type, a second memory cell of a second type, a first intra-unit data path connecting the first memory cell to the second memory cell, and a first data path control switch. The first data path control switch is responsive to a data transfer enable signal which enables data transfer between the first memory cell and the second memory cell through the first intra-unit data path.