Dual Exponent Bounding Box Floating-Point for Neural Matrix Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning and artificial intelligence techniques face challenges in performing real-time feature extraction due to the computational expense of training and using neural networks, which is not efficiently supported by general-purpose processors.
Innovation Solution
The use of dual exponent formats, such as single exponent bounding box floating-point (SE-BBFP) and dual exponent bounding box floating-point (DE-BBFP), for matrix operations in neural networks, which allows for improved system performance by reducing memory and network bandwidth usage, and enables faster hardware, reduced memory overhead, and simpler hardware design.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used to execute machine learning models, then system versatility is maintained, but computational speed and real-time processing capability deteriorate
Solution Approach 1:
The patent segments the processing system into two distinct parts: a general-purpose processor that handles model execution and a specialized coprocessor that handles computationally intensive matrix operations. This segmentation allows the general-purpose processor to maintain versatility while the coprocessor provides high-speed specialized computation, resolving the contradiction between system versatility and computational speed.
2Measurement precision
If standard floating-point formats are used for matrix operations, then computational precision is maintained, but memory bandwidth usage and storage requirements increase
Solution Approach 1:
The patent changes the parameter representation by using a modified floating-point format where the exponent is stored separately from the significand. This parameter change allows the system to maintain computational precision when needed while using less memory bandwidth during operations, as the separate exponent storage enables more efficient data movement and processing.
3Measurement precision
If high-precision floating-point formats are used for neural network operations, then computational accuracy is improved, but hardware complexity and energy consumption increase
Solution Approach 1:
The patent segments the floating-point representation into separate components (exponent and significand) that can be processed independently. This segmentation allows the hardware to maintain high computational accuracy while reducing overall hardware complexity, as the separate exponent storage enables simpler arithmetic operations and more efficient resource utilization in the coprocessor.
Data Source
AI summary
Apparatus and methods are disclosed for performing matrix operations, including operations suited to neural network and other machine learning accelerators and applications, using dual exponent formats. Disclosed matrix formats include single exponent bounding box floating-point (SE-BBFP) and dual exponent bounding box floating-point (DE-BBFP) formats. Shared exponents for each element are determined for each element based on whether the element is used as a row of matrix tile or a column of a matrix file, for example, for a dot product operation. Computing systems suitable for employing such neural networks include computers having general-purpose processors, neural network accelerators, or reconfigure both logic devices, such as Field programmable gate arrays (FPGA). Certain techniques disclosed herein can provide improved system performance while reducing memory and network bandwidth used.