Partial Sum Pre-computation for Quantized Neural Network Resource Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Convolutional neural networks (CNNs) are computationally intensive, and existing methods for implementing quantized neural networks (QNNs) on programmable devices, such as FPGAs, face challenges in resource efficiency due to the increase in resource count for higher quantization levels, particularly with XNOR operations.

Innovation Solution

The implementation of partial sum pre-computation techniques that convert multiply-accumulate operations into memory lookup operations, utilizing a pre-compute circuit to store pre-computed outputs for each memory lookup operation, thereby reducing the computational burden and resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If quantization level is increased beyond binary, then network accuracy is improved, but resource count increases significantly

Engineering Contradiction:

Improvenetwork accuracyVSAvoidresource count

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent pre-computes and stores partial sums in lookup tables before the main neural network computation. This preliminary action allows the actual inference to use simpler memory lookup operations instead of complex multiply-accumulate operations, enabling higher quantization levels without proportionally increasing resource requirements.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces partial sum pre-computation as an intermediary step between weight storage and final output generation. By pre-computing partial sums and storing them in lookup tables, the system creates an intermediate representation that simplifies the main computation, allowing higher quantization to be achieved with fewer resources.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If multiply-accumulate operations are used for neural network computation, then computational accuracy is maintained, but device resource consumption increases

Engineering Contradiction:

Improvecomputational accuracyVSAvoidresource consumption

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent replaces the mechanical computation system (multiply-accumulate operations requiring arithmetic logic units) with a memory-based system (lookup tables requiring only address decoding and data retrieval). This substitution maintains computational accuracy while significantly reducing the complexity of the required hardware resources.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the fundamental parameter of computation from arithmetic operations to memory access operations. By transforming the computation model from calculate-then-store to lookup-then-use, the system maintains accuracy while reducing resource consumption, as memory access is simpler than arithmetic computation in FPGA architectures.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11663490B1Partial sum pre-computation to implement quantized neural networks on programmable devices

Publication Date: 2023.05.30 XILINX INC

AI summary

An example method of implementing a quantized neural network (QNN) for a programmable device includes: identifying multiply-accumulate operations of neurons in the QNN; converting the multiply-accumulate operations to memory lookup operations; and implementing the memory lookup operations using a pre-compute circuit for the programmable device, the pre-compute circuit storing a pre-computed output of a neuron in the QNN for each of the memory lookup operations.