Target-NPU Model Adaptation for Low-Power Neural Inference

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing electronic devices face challenges in efficiently performing artificial neural network inference operations due to high power consumption, heat generation, memory requirements, and cost, particularly when utilizing CPUs and GPUs, and neuromorphic integrated circuits are prone to noise and require large analog-to-digital conversion areas.

Innovation Solution

A standalone, low-power, low-cost neural network processing unit (NPU) with a digital processing element array, SRAM memory, and an NPU scheduler that optimizes resource allocation and reduces data size by adjusting bit operations based on predefined operation orders and model structure.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If CPUs or GPUs are used for neural network inference operations, then computational capability is improved, but power consumption increases

Engineering Contradiction:

Improvecomputational capabilityVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent segments the neural network inference task into specialized processing units within the NPU architecture, including MAC units for multiply-accumulate operations, activation units for non-linear transformations, and pooling units for feature downsampling. This segmentation allows each unit to be optimized for specific operations, improving computational efficiency while reducing overall power consumption compared to general-purpose CPUs or GPUs.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the operational parameters by using fixed-point arithmetic instead of floating-point arithmetic, and by implementing configurable precision modes (e.g., 8-bit, 16-bit integers). This parameter change significantly reduces power consumption while maintaining acceptable inference accuracy, as the NPU can adjust precision based on the specific neural network model and application requirements.

Inventive Principle:

Principle #35Parameter changes

2Use of energy by moving object

If neuromorphic integrated circuits are used, then power consumption is reduced, but noise susceptibility increases

Engineering Contradiction:

Improvepower consumptionVSAvoidnoise susceptibility

Core Design Contradiction:

Use of energy by moving objectVSReliability

Solution Approach 1:

The patent replaces the analog/neuromorphic computing approach with a digital implementation. The NPU uses digital logic circuits for neural network operations, including digital MAC units, digital activation functions, and digital data paths. This substitution eliminates the noise susceptibility inherent in analog circuits while maintaining low power consumption through optimized digital architecture and fixed-point arithmetic.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Use of energy by moving object

If neuromorphic integrated circuits are used, then power consumption is reduced, but device area increases due to analog-to-digital conversion

Engineering Contradiction:

Improvepower consumptionVSAvoiddevice area

Core Design Contradiction:

Use of energy by moving objectVSArea of stationary object

Solution Approach 1:

The patent extracts and eliminates the analog-to-digital conversion stage entirely by implementing a fully digital NPU architecture. Neural network weights and activations are represented and processed directly in digital form from input to output. This extraction removes the large area requirement for ADC circuits that would be needed in hybrid analog-digital systems, while maintaining the low power consumption benefits of specialized neural processing.

Inventive Principle:

Principle #2Taking out (Extraction)

4Measurement precision

If high precision floating-point operations are used, then inference accuracy is improved, but memory requirements increase

Engineering Contradiction:

Improveinference accuracyVSAvoidmemory requirements

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent implements dynamic precision adjustment where the NPU can configure its operational precision (e.g., 8-bit, 16-bit integers or floating-point) based on the specific neural network model being executed and the required accuracy level. This dynamic adaptation allows the system to use lower precision formats for models where high precision is not critical, reducing memory requirements for weights and activations, while switching to higher precision when accuracy demands it.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20250284536A1System and method for generating target-NPU-adapted neural network models

Publication Date: 2025.09.11 DEEPX CO LTD

AI summary

A neural network processing unit (NPU) includes a plurality of processing elements configured to execute operations of an artificial neural network (ANN) model; an NPU memory system coupled to the plurality of processing elements; and an NPU controller configured to manage the plurality of processing elements and the NPU memory system based on control information derived from an modified ANN model