Matrix Reformatting Networks for Cache-Miss Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing architectures face inefficiencies in handling large datasets lacking spatial locality, leading to cache misses and reduced computational performance in applications like large social graphs and neural network training, where cache systems are ineffective due to non-local data access patterns.
Innovation Solution
Implementing a switching functional unit within a processor system that reorders memory content by integrating a data reordering network, allowing data to be rearranged across cache lines to improve cache utilization and reduce cache misses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in traditional memory architecture without reordering, then memory access follows sequential patterns, but cache hit ratios deteriorate due to non-local data access patterns in applications like social graphs and neural networks
Solution Approach 1:
The patent applies preliminary action by reordering data in memory before it is accessed by the processor. A data reordering network intercepts data as it leaves memory, extracts indexing information, and rearranges the data into an order that matches the processor's access patterns. This preliminary reordering ensures that when the processor accesses the data, it can achieve higher cache hit ratios and reduced memory access latency, directly addressing the contradiction between computational efficiency and memory access time.
2Productivity
If data is reordered across cache lines to match processor access patterns, then cache utilization improves, but device complexity increases due to the data reordering network
Solution Approach 1:
The patent introduces an intermediary data reordering network that sits between the memory system and the processor. This intermediary component performs the complex reordering operation without requiring changes to the core processor architecture. The data reordering network extracts indexing information from outgoing data, uses this information to determine optimal cache line arrangements, and reorders data accordingly. This approach improves cache utilization while isolating the complexity to a dedicated intermediary component rather than increasing overall processor complexity.
3Manufacturing precision
If multiple sorting stages are used to sort all n records, then sorting precision improves, but processing time increases due to iterative operations
Solution Approach 1:
The patent applies segmentation by dividing the sorting process into multiple stages, where each stage handles a portion of the sorting task. Instead of attempting to sort all n records in a single operation, the data reordering network performs iterative sorting operations across multiple passes. Each stage sorts a subset of records based on extracted control strings, and subsequent stages refine the ordering. This segmented approach achieves high sorting accuracy while managing processing time by distributing the sorting workload across multiple specialized stages rather than one monolithic operation.
Data Source
AI summary
A data ordering device includes a plurality of inputs N and a plurality of outputs M. There is a sorting network coupled between the plurality of inputs N and the plurality of outputs M. There are one or more latches comprising a buffer coupled between each input of the plurality of inputs N and a corresponding input of the sorting network. There are one or more latches comprising a buffer coupled between each output of the plurality of outputs M and a corresponding output of the sorting network. There is an input for a control signal operative to initiate a sorting of data between the plurality of inputs N and the plurality of outputs M. The data ordering device is coupled to a core of a central processing unit.