Virtual Padding Unit for Matrix Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for changing the size or shape of matrices by adding padding elements are inefficient as they require reading and writing the entire matrix to memory, leading to increased memory usage, performance costs, and cache miss issues due to excessive memory traffic.
Innovation Solution
A virtual padding unit is implemented in computing devices to generate and provide output values for a padded data structure without storing all padding elements in memory, maintaining records of memory addresses for both the initial and padded structures, and determining whether a location is a padding element or not to output the appropriate values or addresses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If padding elements are added to change the size or shape of a matrix, then the matrix can be optimized for arithmetic operations, but the entire matrix must be read from and written to memory, increasing memory usage and memory traffic
Solution Approach 1:
The patent creates a virtual copy of the padding elements in the memory address space without physically storing them. The virtual padding unit intercepts memory access requests and synthesizes padding values on-demand, allowing the system to treat the padded matrix as if it were fully stored while avoiding the actual memory allocation and traffic associated with physical padding storage.
Solution Approach 2:
The virtual padding unit acts as an intermediary between the processing unit and the physical memory. It intercepts memory access requests, determines whether accessed addresses correspond to padding elements or actual data, and either synthesizes padding values or forwards requests to physical memory. This mediator layer eliminates the need to physically store padding elements while maintaining the illusion of a fully padded matrix structure.
2Productivity
If the entire matrix is read from and written to memory to add padding, then the padded matrix can be used for operations, but this process takes time and increases memory traffic
Solution Approach 1:
The system performs preliminary configuration by establishing the virtual address space mapping between the original matrix and the desired padded matrix structure. Instead of performing the time-consuming operation of reading, padding, and writing back the entire matrix, the virtual padding unit is pre-configured with the mapping information, enabling on-demand synthesis of padding values during subsequent operations without repeated memory traffic.
Solution Approach 2:
The virtual address space creates a logical copy of the padded matrix structure without physical duplication. The virtual padding unit synthesizes padding values on-demand based on the pre-established mapping, eliminating the time required to physically generate and store padding elements while maintaining full operational capability.
3Adaptability or versatility
If padding elements are stored in memory, then the padded data structure is available for operations, but memory capacity is consumed and cache miss rates increase due to excessive memory traffic
Solution Approach 1:
The virtual address space creates a logical representation of the padded data structure without consuming physical memory capacity. The virtual padding unit synthesizes padding values on-demand, allowing the system to work with flexible padded data structures of various sizes without actually allocating the corresponding memory capacity, thereby preventing cache miss issues associated with large padded matrices.
Solution Approach 2:
The patent introduces a virtual dimension to the memory hierarchy by creating a virtual address space that extends beyond the physical memory boundaries. This additional virtual dimension allows the system to represent larger padded data structures than physically available, with the virtual padding unit translating virtual addresses to physical addresses and synthesizing values for addresses that would exceed physical memory capacity.
Data Source
AI summary
A virtual padding unit provides a virtual padded data structure (e.g., virtually padded matrix) that provides output values for a padded data structure without storing all of the padding elements in memory. When the virtual padding unit receives a virtual memory address of a location in the virtual padded data structure, the virtual padding unit checks whether the location is a non-padded location in the virtual padded data structure or a padded location in the virtual padded data structure. If the location is a padded location in the virtual padded data structure, the virtual padding unit outputs a padding value rather than a value stored in the virtual padded data structure. If the location is a non-padded location in the virtual padded data structure, a value stored at the location is output.