Hardware Data Formatting for Non-Consecutive Array Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional microprocessor systems face performance bottlenecks in machine learning and artificial intelligence processing due to latency incurred from reading individual data elements from memory, especially during wide convolution operations involving large input matrices, limiting throughput.
Innovation Solution
A microprocessor system with a computational array and hardware data formatter that loads consecutive elements in parallel, minimizing memory reads by dividing input elements into subsets where each subset is consecutive, reducing the need for per-element memory address calculations and cache checks, and allowing non-consecutive element access with reduced latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional microprocessor systems load each data element from unique memory addresses sequentially, then memory access can be performed, but throughput is limited due to latency incurred from reading individual data elements
Solution Approach 1:
The patent divides input data into multiple subsets, where each subset contains consecutive data elements. The computational array processes one subset per clock cycle in parallel, eliminating the need to load individual elements sequentially from memory. This segmentation approach allows the system to load fewer subsets concurrently rather than individual elements, significantly reducing memory access latency and improving throughput.
2Speed
If parallel processing is used to decrease computation time, then processing speed increases, but throughput remains limited due to latency from reading input data from memory
Solution Approach 1:
The hardware data formatter pre-processes and organizes input data into consecutive subsets before they reach the computational array. By preparing the data in advance in a format optimized for parallel processing, the system eliminates the need for per-element address calculations and memory reads during the actual computation phase, allowing parallel processing to operate at full speed without memory latency bottlenecks.
3Measurement precision
If individual memory address calculations are performed for each data element, then precise data access is achieved, but computational overhead and latency increase
Solution Approach 1:
The patent combines multiple data elements into subsets with consecutive memory addresses, where each subset can be accessed using a single base address and offset calculations. This merging approach maintains precise data access within each subset while eliminating the need for individual address calculations for every element, significantly improving computational efficiency and reducing overhead.
Data Source
AI summary
A microprocessor system comprises a computational array and a hardware data formatter. The computational array includes a plurality of computation units that each operates on a corresponding value addressed from memory. The values operated by the computation units are synchronously provided together to the computational array as a group of values to be processed in parallel. The hardware data formatter is configured to gather the group of values, wherein the group of values includes a first subset of values located consecutively in memory and a second subset of values located consecutively in memory. The first subset of values is not required to be located consecutively in the memory from the second subset of values.