Neural Processor Circuit Parallel Task Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neural network processing systems rely solely on CPUs, leading to significant bandwidth consumption and increased power usage due to the inefficiencies in performing convolution and non-convolution operations.

Innovation Solution

A neural processor circuit that includes both neural engine circuits for convolution operations and planar engine circuits for non-convolution operations, with a data processor circuit managing data dependencies and enabling parallel execution of tasks to optimize resource utilization and reduce power consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a CPU is used to perform both convolution and non-convolution operations, then the system is simple and flexible, but bandwidth consumption and power usage increase significantly

Engineering Contradiction:

Improvesystem simplicityVSAvoidpower consumption

Core Design Contradiction:

Device complexityVSUse of energy by moving object

Solution Approach 1:

The processor is divided into two specialized functional units: a neural engine for convolution operations and a planar engine for non-convolution operations. This segmentation allows each unit to be optimized for its specific task, improving energy efficiency while maintaining system functionality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The neural engine and planar engine share common resources including an operand cache, data memory interface, and control logic. This multi-functionality allows the system to handle both convolution and non-convolution operations efficiently without requiring completely separate processing paths.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Device complexity

If a CPU is used to perform both convolution and non-convolution operations, then the system is simple and flexible, but bandwidth consumption increases significantly

Engineering Contradiction:

Improvesystem simplicityVSAvoidbandwidth consumption

Core Design Contradiction:

Device complexityVSQuantity of substance

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The neural engine and planar engine share common resources including an operand cache, data memory interface, and control logic. By merging these resources, the system reduces overall bandwidth consumption compared to having completely separate processing paths for each operation type.

Inventive Principle:

Principle #5Merging (Combining)

3Device complexity

If neural engine tasks and planar engine tasks are executed sequentially, then data dependencies are easier to manage, but processing efficiency decreases

Engineering Contradiction:

Improvedata dependency managementVSAvoidprocessing efficiency

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The system dynamically schedules and executes neural engine tasks and planar engine tasks based on data availability and task dependencies. This dynamic execution approach allows the system to maintain simple data dependency management while achieving high processing efficiency through parallel task execution when conditions permit.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The control logic monitors the execution status of both neural engine and planar engine tasks, using this feedback information to determine when data dependencies are satisfied and when parallel execution can begin. This feedback mechanism enables efficient parallel processing while maintaining proper dependency management.

Inventive Principle:

Principle #23Feedback

4Productivity

If separate processing paths are used for convolution and non-convolution operations, then processing efficiency improves, but device complexity increases

Engineering Contradiction:

Improveprocessing efficiencyVSAvoiddevice complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11934941B2Asynchronous task execution for neural processor circuit

Publication Date: 2024.03.19 APPLE INC

AI summary

A neural processor circuit includes one or more planar engine circuits that perform non-convolution operations in parallel with convolution operations performed by one or more neural engine circuits. The neural engine circuits perform the convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data. The planar engine circuits perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data. A data processor circuit in the neural processor circuit addresses data dependency between the one or more neural engine tasks and the one or more planar engine tasks by controlling reading of the neural output data as the planar input data by the planar engine circuits or reading of the planar output data as the neural input data by the neural engine circuits.