Compute-in-Memory Neural Networks With Bitwise Activation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional compute-in-memory (CIM) accelerators require additional circuitry and increased power consumption for implementing activation functions in artificial neural networks, leading to higher circuit area and processing delays.

Innovation Solution

Implement a bitwise modified rectified linear unit (bm-ReLU) activation function within the CIM accelerator, which generates an output proportional to the input below a critical threshold and independent of the input above it, allowing for parameter adjustments to reduce power consumption and processing time without additional circuitry.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If additional circuitry is used to implement activation functions in conventional CIM accelerators, then the activation function can be processed, but the circuit area increases

Engineering Contradiction:

Improveactivation function processing capabilityVSAvoidcircuit area

Core Design Contradiction:

Adaptability or versatilityVSArea of stationary object

Solution Approach 1:

The patent merges the activation function processing capability directly into the CIM memory array by implementing bitwise operations at the memory level. The activation function is applied during the MAC operation itself through bitwise AND operations between weight bits and input signals, eliminating the need for separate activation function circuitry while maintaining the required adaptability.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces bitwise intermediate representations (breaking weights into bits) as a mediator between the MAC operation and the activation function. This intermediate bit-level representation enables the activation function to be applied through simple bitwise operations within the memory array, avoiding the need for complex additional circuitry.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If additional circuitry is used to implement activation functions in conventional CIM accelerators, then the activation function can be processed, but power consumption increases

Engineering Contradiction:

Improveactivation function processing capabilityVSAvoidpower consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by stationary object

Solution Approach 1:

The patent combines activation function processing with the existing MAC operation in the CIM memory array. By applying the activation function through bitwise operations during the same time the MAC operation occurs, no additional power-consuming circuitry is required, thus maintaining adaptability while reducing power consumption.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent ensures continuous useful action by applying the activation function simultaneously with the MAC operation rather than sequentially. This eliminates idle time and additional power consumption that would result from separate processing stages, maintaining full utilization of the memory array throughout the computation.

Inventive Principle:

Principle #20Continuity of useful action

3Adaptability or versatility

If additional circuitry is used to implement activation functions in conventional CIM accelerators, then the activation function can be processed, but processing delays increase

Engineering Contradiction:

Improveactivation function processing capabilityVSAvoidprocessing delay

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent merges the activation function application with the MAC operation execution in the CIM memory array. Both operations occur simultaneously during the same time window, eliminating the sequential processing delay that would occur if the activation function were applied after the MAC operation completed.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent performs preliminary action by preparing the bit-level weight representations and configuring the memory array before the MAC operation begins. This preliminary setup enables the activation function to be applied immediately during the MAC operation without requiring additional processing time afterward.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12373686B2Methods of operating an artificial neural network using a compute-in-memory accelerator and a bitwise activation function

Publication Date: 2025.07.29 SAMSUNG ELECTRONICS CO LTD

AI summary

A method includes providing an artificial neural network comprising a compute-in-memory accelerator, the artificial neural network further comprising a hidden layer including a first plurality of artificial neurons; and training the artificial neural network using a bitwise modified rectified linear unit activation function for ones of the first plurality of artificial neurons, the bitwise modified rectified linear unit activation function comprising a bit activation function, which is configured to generate an output that is proportional to an input when the input is less than a critical threshold and configured to generate an output that is independent of the input when the input is greater than the critical threshold, wherein the input comprises a sum, across a second plurality of artificial neurons of a preceding layer of the artificial neural network having a plurality of weights associated therewith, respectively, of a product of an output from a respective one of the second plurality of artificial neurons and one bit of a respective one of the plurality of weights.