FPGA Data Masking Circuit for Low-Overhead Transformer Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for data masking in machine learning models, particularly in transformer models, face challenges such as high memory storage and bandwidth overhead, arithmetic overhead, and inefficiencies in computing masked data.
Innovation Solution
The implementation of a hardware acceleration method using a field programmable gate array (FPGA) that includes a data masking circuit capable of generating masks on-device, applying masks without floating-point arithmetic, and selectively using external or internally generated masks to optimize performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data masking is performed using software implementations on general purpose processors, then flexibility and ease of implementation are maintained, but computing performance and energy efficiency deteriorate
Solution Approach 1:
The patent replaces software-based data masking operations on general purpose processors with hardware-based masking circuits implemented on FPGAs. This substitution of mechanical/software systems with dedicated hardware circuits achieves higher computing performance and energy efficiency while maintaining reconfigurability through FPGA's inherent flexibility.
2Productivity
If data masking is performed using ASIC implementations, then computing performance and energy efficiency are improved, but device complexity and fabrication cost increase
Solution Approach 1:
The patent employs FPGAs which provide dynamic reconfigurability between fabrication and operation. Unlike fixed ASIC implementations, FPGAs allow the masking circuit logic to be modified and reconfigured in the field through configuration files, achieving ASIC-level performance while maintaining software-like flexibility for different masking patterns and model architectures.
3Adaptability or versatility
If masks are stored externally and loaded into memory, then mask generation flexibility is maintained, but memory storage and bandwidth overhead increase
Solution Approach 1:
The patent implements self-service mask generation within the FPGA masking circuit, where the hardware logic autonomously generates masking patterns based on input data characteristics and configuration parameters. This eliminates the need to store and load external mask data, reducing memory overhead while maintaining flexibility through programmable logic that can adapt to different masking requirements.
4Measurement precision
If floating-point arithmetic is used for data masking operations, then precision is maintained, but arithmetic overhead and computing time increase
Solution Approach 1:
The patent changes the operational parameters of the masking circuit by implementing fixed-point or integer arithmetic instead of floating-point operations in the FPGA hardware. This parameter change in numerical representation maintains sufficient precision for masking operations while dramatically reducing arithmetic overhead and computing time, as hardware circuits for fixed-point arithmetic are simpler and faster to implement.
Data Source
AI summary
A field programmable gate array (FPGA) including a configurable interconnect fabric connecting a plurality of logic blocks, the configurable interconnect fabric and the logic blocks being configured to implement a data masking circuit configured to: receive input data including data values at a plurality of indices of the input data; select between a data value of the data values and an alternative value using a masking multiplexer to generate masked data, the masking multiplexer being controlled by a mask value of a plurality of mask values at indices corresponding to the indices of the input data; and output the masked data. In some examples, the configurable interconnect fabric and the logic blocks are further configured to implement a mask generation circuit configured to generate the mask values. In some examples, the mask values are received from external memory.