Parallel Multicast Instruction Generation for Hierarchical AI Processors

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing deep learning processors, such as GPUs, face inefficiencies due to architectural differences from general-purpose processors, requiring specialized instruction generators to optimize performance.

Innovation Solution

An arithmetic processing device with hierarchical blocks and memory layers, utilizing a compiler to generate parallel data transfer instructions, including unicast and multicast instructions, to efficiently move data between blocks in the hierarchy, optimizing data transfer paths.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a general-purpose processor architecture is used for deep learning, then hardware versatility is maintained, but calculation speed and efficiency deteriorate

Engineering Contradiction:

Improvecalculation speedVSAvoidhardware versatility

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The processor is divided into multiple blocks (first blocks and second blocks) with arithmetic units and memories, allowing specialized deep learning operations while maintaining modular architecture that can be adapted to different workloads

Inventive Principle:

Principle #1Segmentation

2Productivity

If specialized deep learning processor architecture is used, then calculation speed is improved, but instruction generation complexity increases

Engineering Contradiction:

Improvecalculation speedVSAvoidinstruction generation complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The instruction generating device is designed to handle multiple types of data transfers (first, second, third, and fourth data transfers) through a unified approach, reducing the complexity of instruction generation while maintaining high performance

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If traditional data transfer instructions are used, then instruction simplicity is maintained, but data transfer efficiency deteriorates

Engineering Contradiction:

Improvedata transfer efficiencyVSAvoidinstruction sequence complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

Multiple data transfer operations are combined into parallel executions, where first data transfers occur within first blocks and second data transfers occur between first and second blocks simultaneously, reducing the total number of instructions while improving efficiency

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The instruction generating device pre-determines the optimal combination and execution order of multicast instructions based on data transfer requirements, preparing the instruction sequence in advance to minimize complexity during execution

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20260056738A1Instruction generating method, arithmetic processing device, and instruction generating device

Publication Date: 2026.02.26 PREFERRED NETWORKS INC

AI summary

An arithmetic processing device includes second blocks, each including first blocks and one second memory, and each of the first blocks including one arithmetic unit and one first memory. The arithmetic processing device performs, in parallel, at least one of first, second, third, or fourth data transfers, by executing an instruction sequence. Sources and destinations of the first data transfers are one or more first blocks, sources of the second data transfers are one or more first blocks, destinations thereof are one or more second blocks, sources of the third data transfers are one or more second blocks, destinations thereof are one or more first blocks, and sources and destinations of the fourth data transfers are one or more second blocks. The instruction sequence includes a combination and execution order of at least one multicast instruction selected from more than one type of multicast instructions.