Logical Memory Banks for Rearrangement-Free Matrix Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix operations in machine learning models require significant processing time and power due to the need for data rearrangement and packing/unpacking to access data samples in memory, which are not organized in a conducive manner.
Innovation Solution
Organize physical memory banks into logical memory banks with a base size, accessible via a shared memory crossbar, allowing for arbitrary sample widths and stride lengths to access data without the need for rearrangement, using a common memory crossbar to facilitate efficient memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in conventional memory formats (row major or column major order), then data can be stored efficiently, but data rearrangement and packing/unpacking operations are required to access samples in the specific order needed for matrix operations, increasing processing time and power consumption
Solution Approach 1:
The memory system is segmented into multiple memory banks, each capable of independent access. This segmentation allows parallel access to different data samples without requiring sequential rearrangement operations, thereby improving processing speed while maintaining manageable complexity through modular architecture
Solution Approach 2:
The patent introduces a new dimension to memory organization by mapping multi-dimensional matrix data (rows, columns, and sample dimensions) directly to memory bank addresses. This dimensional mapping allows samples to be accessed in any order without rearrangement, as the memory structure itself supports the required access patterns through its addressing scheme
2Reliability
If data rearrangement and packing/unpacking operations are performed to access samples in the required order, then correct data access is achieved, but processing overhead increases and power consumption rises
Solution Approach 1:
Data is pre-organized in memory banks according to the access patterns required by matrix operations. By performing the organization action in advance during data loading, the system eliminates the need for rearrangement operations during actual computation, ensuring correct data access while minimizing energy consumption during processing
Solution Approach 2:
The memory bank structure is designed to automatically provide data in the correct order for matrix operations through its addressing mechanism. The system serves itself by having the memory architecture inherently support the required access patterns without needing additional processing steps for data rearrangement, thereby reducing power consumption
3Ease of operation
If data rearrangement operations are performed to obtain relevant samples from memory, then data can be accessed in the specific order needed, but the amount of time involved in performing mathematical operations increases
Solution Approach 1:
The memory bank system is designed with universal access capabilities that can handle any sample access pattern required by different matrix operations. The same memory structure supports various operation types (matrix-to-vector, matrix-to-matrix multiplication, convolution) without requiring operation-specific rearrangement, making data access equally easy for all operations while eliminating time overhead
Data Source
AI summary
Certain aspects of the present disclosure provide techniques and apparatus for efficiently accessing memory in a computing system. An example method includes organizing a plurality of physical memory banks having a base size into a plurality of logical memory banks. A request to execute operations on the plurality of physical memory banks is received. The request to execute the operations comprises a request to interact with data having a sample width based on the base size. Responsive to receiving the request to execute the operations, the operations are executed on one or more logical memory banks of the plurality of logical memory banks via a memory crossbar shared across the plurality of logical memory banks. An amount of the data on which the operations are executed is a multiple of the sample width, and each logical memory bank has a size based on the base size and a multiplier value.