Fine-Grained Memory Controller for Non-Contiguous Matrix Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory controllers are inefficient in accessing non-contiguous memory locations, particularly in neural network computations where matrices are stored as rows and columns, leading to performance degradation due to the need for multiple small memory accesses, which can result in accessing garbage data and are not effectively managed by scatter-gather techniques.
Innovation Solution
A memory controller is configured to perform fine-grained memory accesses using a start address, a count indicator, and an address offset, allowing it to skip zero or more elements across multiple dimensions of a matrix, enabling efficient access to non-contiguous locations by calculating the address offset based on element size and step count.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory controllers access consecutive memory locations for performance optimization, then memory access efficiency is improved, but the ability to access non-contiguous memory locations (such as matrix elements stored in row-column format) deteriorates
Solution Approach 1:
The memory access operation is segmented into multiple independent access commands. Instead of attempting to access non-contiguous elements in a single operation, the controller divides the access into sequential commands, each targeting a specific memory location or small contiguous block, thereby maintaining access efficiency while achieving non-contiguous access patterns.
Solution Approach 2:
The patent introduces a new dimension of control by adding the stride parameter to the traditional start address and length parameters. This additional parameter enables the memory controller to jump between non-contiguous memory locations while maintaining efficient access patterns, effectively adding a dimensional degree of freedom to the access control mechanism.
2Ease of operation
If multiple small memory accesses are performed to access non-contiguous locations, then access to specific elements is achieved, but system performance deteriorates due to increased access overhead and potential garbage data retrieval
Solution Approach 1:
The memory controller performs slightly more access operations than strictly necessary by using a length parameter that covers the entire range, then uses the stride parameter to selectively retrieve only the needed elements. This approach trades a small amount of excess data transfer for significantly reduced access overhead and eliminated garbage data retrieval issues.
3Adaptability or versatility
If scatter-gather techniques are used to manage non-contiguous memory accesses, then some flexibility is provided, but the techniques are not effective for fine-grained accesses within matrices
Solution Approach 1:
The patent changes the fundamental parameters of memory access control by introducing the stride parameter, which allows precise control over the spacing between accessed elements. This parameter change enables fine-grained access patterns specifically tailored for matrix operations, where elements need to be accessed with regular intervals corresponding to row or column strides, something that traditional scatter-gather techniques cannot efficiently handle.
Data Source
AI summary
Systems and methods are provided to perform fine-grained memory accesses using a memory controller. The memory controller can access elements stored in memory across multiple dimensions of a matrix. The memory controller can perform accesses to non-contiguous memory locations by skipping zero or more elements across any dimension of the matrix.