Multiplier-less Neural Network Architecture for CNN Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current neural network architectures, particularly convolutional neural networks (CNNs), are resource-intensive due to the need for multiplication operations, which consume significant processing resources, circuit area, and power, making them inefficient for machine learning applications that require lower power and cost while maintaining accuracy.

Innovation Solution

The implementation of a multiplier-less neural network architecture that uses logarithmic weight representation and XOR operations instead of multiplication, along with sorting circuitry to eliminate the need for multipliers, reducing the requirement for costly hardware and optimizing resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional neural network architectures use multiplication operations for matrix multiplication, then computational accuracy is maintained, but processing resources, circuit area, and power consumption increase significantly

Engineering Contradiction:

Improvecomputational accuracyVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent replaces the traditional multiplication operation (mechanical/computational system) with a combination of logarithmic transformation and addition operations. By converting the multiplication problem into an addition problem through logarithmic domain transformation, the system achieves the same computational function with significantly lower power consumption and reduced circuit complexity

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the mathematical parameters from direct multiplicative values to logarithmic representations. This parameter transformation allows the system to perform matrix multiplication through addition operations in the logarithmic domain, thereby reducing power consumption and hardware requirements while maintaining computational accuracy

Inventive Principle:

Principle #35Parameter changes

2Reliability

If traditional neural network architectures use multiplication operations, then computational accuracy is maintained, but circuit area and processing resources increase

Engineering Contradiction:

Improvecomputational accuracyVSAvoidcircuit area

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent substitutes the complex multiplication circuitry with simpler addition circuits and logarithmic transformation blocks. This replacement dramatically reduces the circuit area required for matrix multiplication operations while preserving computational accuracy through the mathematical equivalence of the transformation

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

By transforming weights and inputs into logarithmic representations, the patent changes the computational parameters to enable addition-based processing instead of multiplication. This parameter change allows the use of simpler, less resource-intensive adder circuits, thereby reducing overall circuit area

Inventive Principle:

Principle #35Parameter changes

3Reliability

If traditional neural network architectures use multiplication operations, then computational accuracy is maintained, but processing time and resource consumption increase

Engineering Contradiction:

Improvecomputational accuracyVSAvoidprocessing efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent replaces time-consuming multiplication operations with faster addition operations in the logarithmic domain. Since addition is computationally less intensive and can be performed more quickly with fewer hardware resources, this substitution improves processing efficiency and throughput while maintaining accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Adaptability or versatility

If multiplier hardware is used for neural network computations, then computational capability is achieved, but cost and power envelope increase

Engineering Contradiction:

Improvecomputational capabilityVSAvoidpower envelope

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent eliminates the need for expensive and power-intensive multiplier hardware by substituting it with addition hardware and logarithmic transformation blocks. This substitution maintains the necessary computational capability for neural network matrix multiplication while operating within a significantly reduced power envelope

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11593619B2Computer architecture for multiplier-less machine learning

Publication Date: 2023.02.28 RAYTHEON CO

AI summary

A computer architecture for multiplier-less machine learning is disclosed. According to some aspects, a neural network apparatus include processing circuitry and memory. The processing circuitry accesses a plurality of weights for a neural network layer and an input vector for the neural network layer, the input vector comprising a plurality of data values. The processing circuitry provides the plurality of weights and the input vector to an addition layer. The addition layer generates data value-weight pairs and, for each data value-weight pair, creates an input block comprising a sum of the data value and the weight. The processing circuitry sorts the input blocks generated by the addition layer. The processing circuitry cancels any opposite signed input blocks from the sorted input blocks to generate a set of blocks. The processing circuitry outputs a Kth largest value from the set of blocks. K is a positive integer.