Hardware Data Formatting for Non-Consecutive Array Memory Loads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional microprocessor systems face performance bottlenecks in machine learning and artificial intelligence processing due to latency incurred from reading individual data elements from memory, especially during wide convolution operations involving large input matrices, limiting throughput.
Innovation Solution
A microprocessor system with a computational array and hardware data formatter that loads consecutive elements in parallel, minimizing memory reads and cache checks by dividing input elements into subsets where only start and end elements require memory address calculations, allowing entire subsets to be loaded together, reducing latency and increasing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional microprocessor systems load individual data elements from memory for computational operations, then each element can be processed accurately, but latency increases and throughput decreases due to the need to calculate individual memory addresses and perform separate memory reads for each element
Solution Approach 1:
The patent divides the input data into multiple subsets, where each subset contains multiple consecutive elements. Instead of loading individual elements separately, the system loads entire subsets in parallel from memory, reducing the number of memory read operations and address calculations required. This segmentation approach maintains processing accuracy while significantly improving throughput and reducing latency.
Solution Approach 2:
The patent combines multiple consecutive data elements into single subset structures that can be loaded together in parallel. By merging elements into subsets and loading them simultaneously rather than sequentially, the system reduces memory access latency and increases overall throughput without sacrificing the ability to process individual elements correctly.
2Productivity
If multiple GPU cores are used for parallel processing to decrease computation time, then computational throughput increases, but memory read latency remains a bottleneck that limits overall system performance
Solution Approach 1:
The patent segments input data into subsets that can be efficiently processed by multiple GPU cores in parallel. Each subset contains multiple consecutive elements that can be loaded together, allowing GPU cores to work simultaneously on different subsets while minimizing memory read operations. This segmentation enables effective parallel processing without being bottlenecked by sequential memory access.
Solution Approach 2:
The patent performs preliminary organization of data into subsets before the computational processing begins. By pre-grouping consecutive elements into subsets that can be loaded in parallel, the system prepares the data structure in advance to minimize memory read latency during the actual computational phase, allowing GPU cores to immediately begin processing without waiting for sequential data loading.
3Productivity
If consecutive elements are loaded in parallel by dividing input elements into subsets, then memory reads and cache checks are minimized, but additional hardware structure (hardware data formatter) is required
Solution Approach 1:
The patent introduces a hardware data formatter as an intermediary component between memory and the computational array. This formatter automatically divides input elements into subsets and manages the parallel loading process, reducing the complexity burden from the main processing system. The intermediary handles the subset organization and parallel loading operations, allowing the core computational system to focus on processing while still achieving improved throughput.
Data Source
AI summary
A microprocessor system comprises a computational array and a hardware data formatter. The computational array includes a plurality of computation units that each operates on a corresponding value addressed from memory. The values operated by the computation units are synchronously provided together to the computational array as a group of values to be processed in parallel. The hardware data formatter is configured to gather the group of values, wherein the group of values includes a first subset of values located consecutively in memory and a second subset of values located consecutively in memory. The first subset of values is not required to be located consecutively in the memory from the second subset of values.