Streaming Convolution Subtask Storage for Lower CPU Bandwidth

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing machine learning systems, such as CNNs, rely heavily on CPUs for operations, leading to significant bandwidth consumption and increased power usage.

Innovation Solution

A neural processor circuit is used to perform convolution operations in a streaming manner by dividing them into multiple subtasks, utilizing a neural engine circuit and a neural task manager to execute these subtasks efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If CPU is used to execute machine learning operations, then ease of operation is improved, but power consumption increases

Engineering Contradiction:

Improveease of operationVSAvoidpower consumption

Core Design Contradiction:

Ease of operationVSUse of energy by stationary object

Solution Approach 1:

The patent replaces the CPU-based mechanical computing system with a specialized neural processing unit that uses vector processing architecture. This substitution maintains ease of operation through programmable interfaces while dramatically reducing power consumption by optimizing the computing architecture for specific neural network operations.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the operational parameters by implementing a vector processing architecture with specialized functional units for neural network operations. This includes using multiple processing elements that can simultaneously perform multiply-accumulate operations, thereby reducing the overall power consumption while maintaining operational ease through high-level programming interfaces.

Inventive Principle:

Principle #35Parameter changes

2Ease of operation

If CPU is used to execute machine learning operations, then ease of operation is improved, but bandwidth consumption increases

Engineering Contradiction:

Improveease of operationVSAvoidbandwidth consumption

Core Design Contradiction:

Ease of operationVSQuantity of substance

Solution Approach 1:

The patent replaces the general-purpose CPU system with a dedicated neural processing unit that has integrated memory and processing capabilities. This substitution reduces bandwidth consumption by eliminating the need for constant data transfer between CPU and main memory, while maintaining ease of operation through programmable task management.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent implements a universal neural processing architecture that can handle various machine learning operations including convolutions, activations, and pooling operations. This multi-functional design reduces bandwidth consumption by processing diverse operations within the same hardware system, eliminating the need for separate processing units and their associated data transfer requirements.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If convolution operations are performed in streaming manner with multiple subtasks, then productivity is improved, but device complexity increases

Engineering Contradiction:

Improveefficiency of convolution operationsVSAvoiddevice complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments convolution operations into multiple independent subtasks that can be executed in parallel by different processing elements. Each subtask handles a specific portion of the convolution computation, allowing simultaneous execution and thereby improving productivity. The segmentation is managed through a task scheduler that coordinates the subtasks without requiring complex inter-subtask communication.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements preliminary action by pre-configuring the neural processing unit with task descriptor templates and subtask structures before execution. The task scheduler prepares the execution plan in advance, organizing convolution operations into predetermined subtask sequences. This preliminary preparation reduces runtime complexity and enables efficient parallel execution, improving productivity without proportionally increasing device complexity.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20260073182A1Subtask storage for streaming convolutions in neural network processor

Publication Date: 2026.03.12 APPLE INC

AI summary

Embodiments relate to streaming convolution operations in a neural processor circuit that includes a neural engine circuit and a neural task manager. The neural task manager obtains multiple task descriptors and multiple subtask descriptors. Each task descriptor identifies a respective set of the convolution operations of a respective layer of a set of layers. Each subtask descriptor identifies a corresponding task descriptor and a subset of the convolution operations on a portion of a layer of the set of layers identified by the corresponding task descriptor. The neural processor circuit configures the neural engine circuit for execution of the subset of the convolution operations using the corresponding task descriptor. The neural engine circuit performs the subset of the convolution operations to generate output data that correspond to input data of another subset of the convolution operations identified by another subtask descriptor from the list of subtask descriptors.