Masking Row Column Positions for Matrix Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing methods for 2D convolution operations in machine learning and image processing are inefficient due to the need for remapping data structures, leading to increased latency and memory usage, particularly in neural networks where calculation of sums of products across non-adjacent memory addresses is complex.
Innovation Solution
The implementation of matrix processing circuitry with masking capabilities allows for the skipping of certain rows or columns in input data, enabling 2D convolution operations to be split into 1×1 convolutions without remapping, using masking state data to treat masked positions as representing a masking value, thereby reducing memory access and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data remapping is performed to enable 2D convolution operations, then the operations can be executed, but latency increases and memory usage increases
Solution Approach 1:
The patent extracts and eliminates the remapping step from the 2D convolution process by applying masking directly to the input data at its original layout. The masking circuitry applies row and column masks to select only the relevant elements needed for convolution, removing the need for intermediate data restructuring and directly reducing processing latency.
Solution Approach 2:
The patent segments the 2D convolution operation into masked row and column selections that can be processed independently. By dividing the input data into masked rows and columns that correspond to specific convolution kernel positions, the system can process these segments in parallel without requiring full data remapping, thereby reducing both time and memory overhead.
2Productivity
If data remapping is performed to enable 2D convolution operations, then the operations can be executed, but memory usage increases
Solution Approach 1:
The patent removes the intermediate remapped data structure from memory by applying masking operations directly to the original input data layout. The masking circuitry selectively accesses only the required elements from the original data structure, eliminating the need to store separate remapped versions and reducing overall memory consumption.
Solution Approach 2:
The patent segments memory access patterns to load only the specific rows and columns needed for each convolution operation rather than loading entire data structures. This selective memory access through masking reduces the quantity of data that needs to be stored in memory during processing.
3Loss of time
If masking operation is applied to skip certain rows or columns, then remapping is eliminated and latency is reduced, but additional masking circuitry is required
Solution Approach 1:
The patent implements masking circuitry that can be integrated into existing matrix processing units to perform multiple functions: selecting rows, selecting columns, and enabling/disabling specific convolution operations. This multi-functional masking approach reduces the need for separate dedicated circuits for each function, thereby limiting the increase in device complexity while achieving latency reduction.
4Productivity
If masking is applied to treat certain positions as masking values, then direct operation application is enabled and performance improves, but complexity in managing masking state data increases
Solution Approach 1:
The patent applies masking state data in advance to mark which rows and columns should be treated as masking values before the actual convolution operation begins. This preliminary masking configuration allows the processing circuitry to directly skip over irrelevant elements during computation, improving neural network performance while managing complexity through pre-computed masking patterns that can be reused across multiple operations.
Data Source
AI summary
An apparatus comprises matrix processing circuitry to perform a matrix processing operation on first and second input operands to generate a result matrix, where the result matrix is a two-dimensional matrix; operand storage circuitry to store information for forming the first and second input operands for the matrix processing circuitry; and masking circuitry to perform a masking operation to mask at least part of the matrix processing operation or the information stored to the operand storage circuitry based on masking state data indicative of one or more masked row or column positions to be treated as representing a masking value. This is useful for improving performance of two-dimensional convolution operations, as the masking can be used to mask out selected rows or columns when performing the 2D convolution as a series of 1×1 convolution operations applied to different kernel positions.