Neural Processing Element With SIMD Compute Lanes

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing DNN processing solutions, such as general-purpose processors like GPUs, face challenges in reducing power consumption, making them unsuitable for long-duration operations in battery-powered devices like smartphones and AR/VR devices, where low power consumption is critical.

Innovation Solution

A neural processing element with SIMD compute lanes is introduced, featuring hardware binary multipliers and adder trees that reduce the number of accumulator bits, thereby minimizing power consumption, and includes a storage element for storing binary outputs, optimizing DNN processing efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If general-purpose processors like GPUs are used for DNN processing, then computational performance is improved, but power consumption increases

Engineering Contradiction:

Improvecomputational performanceVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The processor is divided into multiple independent compute lanes (first compute lane, second compute lane, etc.), each capable of performing DNN operations simultaneously. This segmentation allows parallel processing to improve computational performance while each lane can operate independently to manage power consumption efficiently

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the architectural parameters by reducing the number of accumulator bits per multiplier and implementing SIMD compute lanes with specific bit-width operations (e.g., 8-bit multipliers producing 16-bit results). These parameter changes optimize the balance between computational performance and power consumption for DNN workloads

Inventive Principle:

Principle #35Parameter changes

2Use of energy by moving object

If ASICs are used for DNN processing, then power consumption is reduced, but performance and power efficiency need continued improvement

Engineering Contradiction:

Improvepower consumptionVSAvoidprocessing performance

Core Design Contradiction:

Use of energy by moving objectVSProductivity

Solution Approach 1:

The compute lanes are designed with dynamic characteristics where each lane can process data independently with configurable bit-widths. The architecture allows flexible operation modes that can adapt to different DNN computational patterns, improving both performance and power efficiency compared to static ASIC designs

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The processor architecture provides multi-functionality through multiple compute lanes that can handle various DNN operations (convolution, multiplication, addition) with different data precisions. This universal design enables the same hardware to optimize for both performance and power efficiency across different workloads

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentEP3991024B1Neural processing element with single instruction multiple data (SIMD) compute lanes

Publication Date: 2024.11.13 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2
Figure 3

AI summary

An architecture is disclosed for an neural processing element having single instruction, multiple data ("SIMD") compute lanes. The neural processing element includes compute lanes having multipliers configured to multiply a binary operand with another binary operand to generate a binary output. The neural processing element also includes a single adder tree for summing the binary outputs of the hardware binary multipliers. The neural processing element also includes a storage element for storing a binary output of the single hardware binary adder tree.