Hardware Data Formatting for Non-Consecutive Array Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional microprocessor systems face performance bottlenecks in machine learning and artificial intelligence processing due to latency incurred from reading individual data elements from memory, especially during wide convolution operations involving large input matrices, limiting throughput.
Innovation Solution
A microprocessor system with a computational array and hardware data formatter that loads consecutive elements in parallel, minimizing memory reads and cache checks by dividing input elements into subsets where only start and end elements require memory address calculations, allowing entire subsets to be loaded together, reducing latency and increasing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional microprocessor systems load each data element from unique memory addresses during array computational operations, then data can be accessed individually, but latency is incurred for each read operation reducing throughput
Solution Approach 1:
The patent segments the computational array into multiple independent processing elements that can be configured to process data in different patterns. This segmentation allows the system to switch between consecutive and non-consecutive data loading modes, optimizing for throughput by processing multiple elements simultaneously rather than sequentially accessing memory for each element.
Solution Approach 2:
The patent implements preliminary action by pre-loading data into on-chip memory structures before computational operations begin. This allows data to be staged and organized in advance, eliminating the need for repeated memory access during computation and reducing latency while maintaining high throughput.
2Productivity
If multiple GPU cores are used for parallel processing to decrease computation time, then computational throughput increases, but memory read latency still limits overall performance
Solution Approach 1:
The patent merges multiple processing elements into a unified computational array structure that shares common data paths and memory interfaces. This consolidation reduces the overhead of multiple independent memory access operations while maintaining parallel processing capabilities, thereby decreasing memory read latency without sacrificing computational throughput.
Solution Approach 2:
The patent introduces on-chip memory structures as intermediary buffers between main memory and processing elements. This intermediary layer allows data to be cached and reused across multiple processing operations, reducing the frequency and latency of main memory reads while sustaining high computational throughput through efficient data reuse.
3Productivity
If consecutive elements are loaded together for parallel processing, then loading efficiency increases, but flexibility to process non-consecutive elements decreases
Solution Approach 1:
The patent implements dynamic configurability in the computational array, allowing processing elements to be reconfigured on-the-fly between consecutive and non-consecutive data loading modes. This dynamic adaptation enables the system to optimize for loading efficiency when processing consecutive elements while maintaining the flexibility to handle non-consecutive element patterns when required by the computational task.
Data Source
AI summary
A microprocessor system comprises a computational array and a hardware data formatter. The computational array includes a plurality of computation units that each operates on a corresponding value addressed from memory. The values operated by the computation units are synchronously provided together to the computational array as a group of values to be processed in parallel. The hardware data formatter is configured to gather the group of values, wherein the group of values includes a first subset of values located consecutively in memory and a second subset of values located consecutively in memory. The first subset of values is not required to be located consecutively in the memory from the second subset of values.