Programmable Device Interleaving Memory Access Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inefficient memory access in computationally intensive data systems, particularly in big data applications, leads to performance degradation due to limited DDR channels and mismatched read sizes, causing bottlenecks in operations like gather and join operations.
Innovation Solution
A programmable device, such as a field programmable gate array (FPGA), receives memory access requests from a host and interleaves them across multiple communication channels, using a subset of bits in the address to determine the appropriate channel, thereby improving memory access efficiency and reducing data replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory access requests are handled through limited DDR channels, then system structure is simple, but memory access efficiency deteriorates causing performance bottleneck
Solution Approach 1:
The patent segments memory access requests into multiple independent communication channels (e.g., 16 channels) within the programmable device. Each channel handles specific address ranges or request types, allowing parallel processing of memory operations. This segmentation transforms a single bottleneck channel into multiple concurrent pathways, dramatically improving memory access throughput while maintaining manageable channel complexity through systematic organization.
Solution Approach 2:
The patent introduces a new dimension of parallelism by implementing multiple communication channels that operate simultaneously. Instead of increasing the bandwidth of a single channel, the system adds channel multiplicity as a new dimension for handling memory requests. This dimensional expansion allows the system to process multiple memory operations in parallel, overcoming the limitations of traditional single-channel DDR architecture.
2Productivity
If minimum read size is set to 64 Bytes for DDR access, then channel utilization is improved, but data retrieval efficiency worsens when smaller data sizes are needed
Solution Approach 1:
The patent implements local quality by allowing different communication channels to handle different data size requirements. Specific channels are optimized or configured for specific read sizes (e.g., 4 Bytes, 8 Bytes, 16 Bytes, 32 Bytes, 64 Bytes). This enables the system to match the data retrieval operation to the appropriate channel based on the actual data size needed, avoiding unnecessary data transfer while maintaining efficient channel utilization.
Solution Approach 2:
The patent changes the parameter of read size from a fixed minimum of 64 Bytes to a variable parameter that can be adjusted based on actual data requirements. By implementing multiple channels with different read size capabilities and dynamically selecting the appropriate channel for each memory access request, the system optimizes the balance between channel utilization and data retrieval efficiency, transferring only the necessary amount of data.
3Productivity
If multiple communication channels are implemented for memory access, then memory access efficiency is improved, but channel determination complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-configuring and pre-organizing multiple communication channels with specific functions, address ranges, or optimization characteristics before memory access operations begin. Each channel is pre-established with its handling rules and routing logic, so that when memory requests arrive, the system can quickly determine the appropriate channel based on pre-defined criteria (such as address bits or data size requirements) without complex real-time decision-making.
4Speed
If data is stored and accessed through host CPU, then system architecture is simple, but processing speed deteriorates due to memory bottleneck
Solution Approach 1:
The patent introduces a programmable device as an intermediary component between the host CPU and the memory system. This intermediary contains multiple communication channels and intelligent routing logic that handles memory access requests, performs data preprocessing, and manages channel allocation. By offloading memory management tasks from the host CPU to this dedicated intermediary device, the system achieves faster processing speed while the increased architectural complexity is contained within the intermediary component rather than the overall system.
Data Source
AI summary
Examples described herein relate to efficient memory access. An example is a system includes a programmable device, and a memory. The programmable device is coupled to the host and receives the read/write requests and the addresses associated therewith. The programmable device interleaves the read/write requests across multiple communication channels based on a subset of bits within each address. The memory receives the read/write requests from the programmable device. The memory stores contents associated with the addresses for write requests and returns contents associated with the addresses for a read request to the programmable device. The programmable device returns the received contents to the host for processing.