In-Memory Neural Network Accelerator for On-Chip Training

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The limited computation ability of hardware has hindered the development and deployment of artificial neural networks, particularly in edge devices where on-chip training is necessary due to time delays and security concerns related to data communication with the cloud.

Innovation Solution

A computing in memory accelerator is designed for neural networks, comprising a memory, data buffer unit, pooling unit, loss computing unit, and three macro circuits, which facilitate forward propagation, weighting update, and gradient operations, reducing data communication latency and enabling efficient on-chip training.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data communication between edge devices and cloud is performed for neural network training, then neural network performance can be optimized through cloud computing resources, but time delay occurs and security risks increase

Engineering Contradiction:

Improveneural network training capabilityVSAvoiddata communication time delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments the neural network training function into on-chip components (macro circuits for matrix multiplication, accumulation, and activation) that can operate independently on edge devices, eliminating the need for continuous cloud communication while maintaining training capability

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The edge device is equipped with self-contained training capabilities through integrated macro circuits that perform forward propagation, backpropagation, and weight updates locally, allowing the system to serve itself without external cloud assistance

Inventive Principle:

Principle #25Self-service

2Productivity

Engineering Contradiction:

Improveneural network training capabilityVSAvoiddata security

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent extracts the training data and processing operations from the cloud environment and keeps them locally on the edge device, eliminating the security vulnerability of transmitting sensitive data over networks while maintaining full training functionality

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The on-chip macro circuits act as intermediaries that process data locally without requiring external cloud communication, thereby eliminating the security risk vector while maintaining training capabilities

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If traditional processors are used for neural network computation, then flexible programming is possible, but computation ability is limited

Engineering Contradiction:

Improveprogramming flexibilityVSAvoidcomputation ability

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent replaces traditional general-purpose processor mechanics with specialized hardware macro circuits that perform matrix operations directly in memory, achieving both high computation speed and maintained programmability through configurable circuit operations

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The macro circuits are designed with multi-functional capabilities to handle various neural network operations (matrix multiplication, accumulation, activation functions) within a unified hardware structure, providing both speed and flexibility

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Reliability

If on-chip training is implemented in edge devices, then data security and response time are improved, but device complexity increases

Engineering Contradiction:

Improvedata securityVSAvoidon-chip training structure

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent merges the memory and computation functions into a unified on-chip structure where macro circuits are integrated with memory arrays, eliminating the need for separate processing units and reducing overall system complexity despite adding training capabilities

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The macro circuits are nested within the memory structure, with computation units embedded alongside storage elements, creating a compact hierarchical organization that minimizes external connections and simplifies the overall device architecture

Inventive Principle:

Principle #7Nested doll (Nesting)

Data Source

PatentUS20240220573A1Computing in Memory Accelerator for Applying to a Neural Network

Publication Date: 2024.07.04 NAT CHENG KUNG UNIV

AI summary

A computing in memory accelerator for applying to a neural network includes a memory, a data buffer unit, a pooling unit, a loss computing unit, a first macro circuit, a second macro unit, a third macro unit, and a multiplexer. The memory is used for saving data. The data buffer unit is coupled to the memory and used to buffer data outputted from the memory. The pooling unit is coupled to the memory and used to pool data for acquiring a maximum pooling value. The loss computing unit is coupled to the memory and used to compute output loss. The first macro circuit, the second macro unit, and the third macro unit are coupled to the data buffer unit. The multiplexer is coupled to the pooling unit, the first macro circuit, the second macro unit, and the third macro unit and used to generate output data.