Memory Request Channels for Ordered Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In memory systems, the out-of-order processing of requests due to parallel execution of operations leads to increased latency and reduced resource utilization, deteriorating efficiency and performance.
Innovation Solution
A memory system design that includes a physical channel for in-order processing and multiple virtual channels for parallel processing, with an arbiter selecting the appropriate channel based on the required degree of parallelism to maintain request order and optimize resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multiple FIFO memories are provided for parallel processing, then processing speed is improved, but request order information is lost
Solution Approach 1:
The patent segments the request processing into multiple independent streams by introducing multiple virtual channels corresponding to different processes. Each virtual channel maintains its own request queue and order information, allowing parallel processing while preserving per-process request ordering. This resolves the contradiction by enabling simultaneous processing (speed improvement) while maintaining organized request handling through process-specific channels (information preservation).
Solution Approach 2:
The patent introduces virtual channels as intermediary structures between the host and memory chips. These virtual channels act as mediators that receive requests, maintain order information for each process, and forward them to appropriate memory chips. This intermediary layer enables parallel processing while preserving request order information through the virtual channel structure.
2Productivity
If out-of-order processing is enabled for parallel execution, then productivity is improved, but latency increases
Solution Approach 1:
The patent implements dynamic process management where the system can flexibly switch between in-order and out-of-order processing modes based on workload characteristics. When processes are independent, out-of-order processing enables parallel execution and improves productivity. When processes depend on each other, the system switches to in-order processing to minimize latency. This dynamic adaptability resolves the contradiction between productivity and latency.
Solution Approach 2:
The patent changes the processing mode parameter dynamically based on process characteristics. By detecting whether processes can be executed in parallel or must follow strict order, the system adjusts the processing parameter (in-order vs out-of-order mode) to optimize performance. This parameter change approach allows the system to achieve high productivity when appropriate while maintaining low latency when needed.
3Productivity
If in-order processing is maintained, then resource utilization efficiency is improved, but processing speed decreases
Solution Approach 1:
The patent segments requests into different virtual channels based on process identification. This segmentation allows the system to apply in-order processing to each individual process (maintaining resource utilization efficiency) while enabling parallel processing across different processes (improving overall processing speed). The segmented structure resolves the contradiction by allowing both efficient resource usage and high throughput simultaneously.
Solution Approach 2:
The patent adds a virtual channel dimension to the request processing architecture. Instead of processing requests in a single dimension (either in-order or out-of-order), the system creates multiple virtual channel dimensions, each maintaining its own order. This dimensional expansion allows parallel processing across dimensions while maintaining order within each dimension, achieving both high speed and efficient resource utilization.
Data Source
AI summary
A memory system includes a plurality of memory chips, a memory, and a controller. The memory chips are capable of operating in parallel. The memory includes a physical channel region and a plurality of virtual channel regions, each corresponding to one of a plurality of processes executed on the memory chips according to the requests. The controller stores the requests issued from the host in the physical channel region in order of acquisition from the host, and an entry for each of the requests in one of the virtual channel regions. When a required degree of parallelism of the processes is less than a threshold, the controller selects a next request to be executed using the physical channel region. When the required degree of parallelism is greater than or equal to the threshold, the controller selects a next request to be executed using one of the virtual channel regions.