Neural Network Processor Instruction Generation for Memory Layout Alignment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Neural network processors face inefficiencies due to the need for memory layout conversion instructions when the memory layout of operands does not match the supported layout, leading to increased processing time and reduced efficiency.

Innovation Solution

An instruction generation method that determines whether the first memory layout of input feature data meets a rearrangement condition, allowing for the generation of executable instructions that align with the second memory layout supported by the neural network processor, thereby eliminating the need for memory layout conversion instructions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If memory layout conversion instructions are used to convert the memory layout of operands to match the neural network processor's supported layout, then the memory layout compatibility is improved, but the processing time increases and processing efficiency decreases

Engineering Contradiction:

Improvememory layout compatibilityVSAvoidprocessing efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The instruction generation device performs preliminary action by determining whether the first memory layout meets the rearrangement condition before generating the executable instruction. This pre-check allows the system to prepare the memory layout in advance according to the neural network processor's requirements, avoiding the need for time-consuming conversion instructions during actual processing.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes the parameter of memory layout by determining whether the first memory layout meets the rearrangement condition and generating executable instructions that align with the second memory layout supported by the hardware unit. This parameter change eliminates the need for memory layout conversion instructions, thereby improving processing efficiency.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If memory layout conversion instructions are used to convert the memory layout of operands, then the memory layout compatibility is improved, but the processing time increases

Engineering Contradiction:

Improvememory layout compatibilityVSAvoidprocessing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If memory layout conversion instructions are used to convert the memory layout of operands, then the memory layout compatibility is improved, but the overhead increases

Engineering Contradiction:

Improvememory layout compatibilityVSAvoidprocessing overhead

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system extracts and eliminates the memory layout conversion instruction from the processing pipeline by determining whether the first memory layout meets the rearrangement condition. When the condition is met, the executable instruction can be generated directly without requiring the separate memory layout conversion step, thereby reducing processing overhead.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentEP4550213A1Instruction generation method, apparatus, storage medium and electronic device

Publication Date: 2025.05.07 BEIJING HORIZON INFORMATION TECH CO LTD

Figure 1~2
Figure 3~4
Figure 5

AI summary

Disclosed are an instruction generation method and apparatus and an electronic device. The method includes: determining first parameter information of a memory transfer operator and a first memory layout corresponding to input feature data; determining a second memory layout supported by a hardware unit executing the memory transfer operator; in response to that the first memory layout is different from the second memory layout, determining whether the first memory layout meets a rearrangement condition; and in response to that the first memory layout meets the rearrangement condition, generating an executable instruction for the neural network processor based on the first parameter information, the first memory layout, and the second memory layout. According to technical solutions of this disclosure, the input feature data can be accessed according to a memory layout supported by the neural network processor, without designing an additional memory layout conversion instruction, thereby reducing instruction overhead.