Processor Swizzle Operation Using Internal Storage Control Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing input data by convolutional neural networks (CNNs) is inefficient due to the need for extensive CPU instructions to rearrange data into suitable formats for subsequent processing stages, particularly when performing parallel operations, leading to high memory bandwidth usage and inefficiencies in data transfer.
Innovation Solution
A data processing system with a processor, internal storage, and a register array that allows for swizzle operations to efficiently transfer and rearrange data within the register array using control data stored in internal storage, reducing the number of instructions needed and enhancing vector processing capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is transferred between stages using larger memories (SRAM/SDRAM), then data storage capacity is improved, but memory bandwidth usage increases and processing efficiency deteriorates
Solution Approach 1:
The patent embeds register arrays directly within the neural network accelerator core, creating a nested structure where small, fast registers are contained within the processing unit. This eliminates the need for external memory transfers by nesting the storage solution directly where it is needed, providing both sufficient capacity for intermediate results and the speed required for efficient processing.
Solution Approach 2:
The patent introduces register arrays as intermediary storage between the convolution stage and subsequent processing stages. These registers act as a buffer that holds convolved data in an optimized format, eliminating the need to transfer data to larger but slower memories. The registers serve as a mediator that provides both storage and fast access for subsequent operations.
2Adaptability or versatility
If CPU instructions are used to rearrange data into suitable formats, then data formatting flexibility is improved, but the number of instructions increases and processing time deteriorates
Solution Approach 1:
The patent performs data rearrangement and formatting operations in advance, during the convolution stage, so that data is output in the correct format for subsequent processing. By preliminarily organizing data into the required format before it leaves the convolution stage, the system eliminates the need for additional formatting instructions later, reducing overall processing time while maintaining flexibility.
Solution Approach 2:
The patent replaces software-based data rearrangement (CPU instructions) with hardware-based reconfiguration capabilities within the neural network accelerator. The system uses configurable data paths and register assignments that can be programmed to perform different formatting operations in hardware, which is significantly faster than executing multiple CPU instructions for the same task.
3Adaptability or versatility
If control data is transferred with each swizzle instruction, then instruction flexibility is improved, but memory bandwidth usage increases and instruction overhead deteriorates
Solution Approach 1:
The patent creates a copy of the register array configuration within the neural network accelerator's control logic. Instead of transferring control data repeatedly with each instruction, the system copies the essential configuration information into hardware structures that can be reused. This allows the same control data to govern multiple operations without requiring repeated transfers, reducing memory bandwidth usage while maintaining flexibility.
Solution Approach 2:
The patent performs control data setup in advance, configuring the register arrays and data paths before execution begins. By preliminarily establishing the control configuration, the system eliminates the need to transfer control data with each subsequent swizzle instruction. The control data is loaded once and then reused throughout execution, significantly reducing memory bandwidth requirements while preserving full instruction flexibility.
Data Source
AI summary
A data processing system includes: a processor; a data interface for communication with a control unit, the processor being on one side of the data interface; internal storage accessible by the processor, the internal storage being on the same side of the data interface as the processor; and a register array accessible by the processor and comprising a plurality of registers, each register having a plurality of vector lanes. The storage is arranged to store control data indicating an ordered selection of vector lanes of one or more of the registers. The processor is arranged to, in response to receiving instruction data from a control unit, perform a swizzle operation in which data is selected from one or more source registers in the register array, and transferred to a destination register. The data is selected from vector lanes in accordance with control data stored in the internal storage.