Mixed Precision Memory Device for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices struggle to efficiently support machine learning tasks, particularly in mixed precision training, where they face challenges in balancing precision, memory usage, and performance.
Innovation Solution
A memory device with a first cell array for storing weight data with first or second precision, a second cell array for storing loss data with first precision, a third cell array for storing gradient data with first precision, a computation circuit for performing operations like multiplying, dividing, or rounding based on a scaling factor during mixed precision training, and a scaling circuit for outputting the scaling factor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If mixed precision training is implemented using conventional memory devices, then memory usage is reduced, but training performance and speed deteriorate due to lack of integrated computation capabilities
Solution Approach 1:
The patent merges computation circuits (multiplier, adder, scaler, rounder) directly into the memory device structure, combining storage and processing functions in a single integrated system. This allows weight data to be stored and processed without external memory accesses, simultaneously reducing memory usage and improving training speed through in-memory computation.
Solution Approach 2:
The memory device is designed with multi-functional capabilities: it can store weight data in different precisions (first and second precision), perform multiplication operations, addition operations, scaling operations, and rounding operations. This universal design enables the same hardware to handle various mixed precision training requirements efficiently.
2Measurement precision
If computation operations are performed externally to memory devices, then precision is maintained, but memory bandwidth consumption increases and bottlenecks occur
Solution Approach 1:
Weight data is pre-stored in the memory device in both first precision and second precision formats. This preliminary preparation allows the computation circuit to directly access and process the appropriate precision data without requiring external memory accesses during training operations, maintaining precision while reducing bandwidth consumption.
Solution Approach 2:
The computation circuit acts as an intermediary between stored weight data and processing requirements. It performs multiplication, addition, scaling, and rounding operations within the memory device, eliminating the need for data to travel to external processors and back, thus maintaining precision while improving bandwidth efficiency.
3Measurement precision
If single precision is used for all data, then computation accuracy is maintained, but memory usage and power consumption increase
Solution Approach 1:
The patent applies different precision levels to different data elements locally: weight data can be stored in second precision (lower precision) while the computation circuit maintains first precision (higher precision) for critical operations. This local quality differentiation allows reduced memory usage while maintaining computation accuracy where it matters most.
Solution Approach 2:
The system dynamically changes precision parameters based on operational requirements. Weight data is stored in second precision to reduce memory usage, but the computation circuit can switch to first precision for multiplication and addition operations, and use scaling/rounding operations to maintain overall computation accuracy. This parameter flexibility resolves the contradiction between memory capacity and computation accuracy.
4Ease of operation
If multiple cell arrays are used for different data types, then data organization is improved, but device complexity increases
Solution Approach 1:
The memory device is segmented into multiple cell arrays, each dedicated to storing specific data types (weight data, bias data, etc.). This segmentation improves data organization and access efficiency by allowing targeted operations on specific data types. The complexity is managed through systematic design where each cell array follows the same structural pattern.
Solution Approach 2:
Despite having multiple cell arrays, the patent uses a universal computation circuit design that can handle different data types from different cell arrays. The same multiplier, adder, scaler, and rounder circuits process various data types, reducing the overall system complexity compared to having dedicated processing units for each data type.
Data Source
AI summary
A memory device for supporting machine learning includes a first cell array configured to store weight data at a first precision or a second precision, a second cell array configured to store loss data at the first precision, a third cell array configured to store gradient data at the first precision, and a computation circuit configured to perform at least one of a multiplying operation, a dividing operation, or a rounding operation corresponding to a scaling factor during mixed-precision training that uses the first precision and the second precision.