Widening Arithmetic via Intra-Register Lanes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data processing apparatuses face challenges in performing arithmetic operations, particularly multiplication, due to widening issues when dealing with compact number formats like integers, leading to reduced throughput and increased complexity in accumulation operations.
Innovation Solution
The apparatus performs dot product operations by extracting multiple data elements from each source register, treating them as vectors for multiplication and summing the results independently across intra-register lanes, which enhances throughput and simplifies the accumulation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional multiplication operations are performed on compact number formats (e.g., 8-bit integers), then storage and memory bandwidth costs are reduced, but the result requires more bits (e.g., 16-bit) leading to widening which increases accumulation register width requirements and reduces throughput
Solution Approach 1:
The 128-bit register is segmented into multiple 32-bit lanes (four lanes), allowing independent processing of multiple multiplication operations simultaneously. Each lane can handle one multiplication and accumulation operation, enabling parallel execution that increases throughput while maintaining support for compact number formats
Solution Approach 2:
The patent introduces an intra-register lane dimension, transforming the single-wide multiplication operation into multiple narrow operations executed in parallel across different lanes. This dimensional transformation allows the system to perform widening operations on compact numbers while maintaining high throughput by utilizing the lane-based parallel architecture
2Measurement precision
If the accumulator width is increased to accommodate widening results, then accuracy is maintained, but the number of operations per cycle is limited by the register width
Solution Approach 1:
The accumulation register is divided into multiple 32-bit lanes, each capable of independent accumulation operations. This segmentation allows eight operations per cycle (4 multiplies and 4 accumulates) by processing multiple data elements in parallel across lanes, rather than being limited to a single wide accumulation operation
Solution Approach 2:
The patent uses 32-bit lanes for accumulation, which is wider than the minimum required for some operations but provides sufficient headroom for multiple operations. This partial widening approach balances accuracy requirements with throughput capabilities, allowing multiple operations to be performed simultaneously without requiring full 128-bit width for each operation
3Productivity
If truncation or saturation is used to limit accuracy, then throughput is improved, but calculation accuracy is reduced
Solution Approach 1:
By segmenting operations into lanes with independent accumulation registers, the system can maintain full precision in each lane without requiring truncation or saturation. The parallel architecture ensures that each operation completes fully before moving to the next, maintaining accuracy while achieving high throughput through parallel execution
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data processing apparatus, a method of operating a data processing apparatus, a non-transitory computer readable storage medium, and an instruction are provided. The instruction specifies a first source register and a second source register. In response to the instruction control signals are generated, causing processing circuitry to perform a dot product operation. For this operation at least a first data element and a second data element are extracted from each of the first source register and the second source register, such that then at least first data element pairs and second data element pairs are multiplied together. The dot product operation is performed independently in each of multiple intra-register lanes across each of the first source register and the second source register. A widening operation with a large density of operations per instruction is thus provided.