Logarithmic Neural Network Inference With 32-Bit BMLNS Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods face challenges in training and optimizing neural networks in the logarithmic domain due to complexity, accuracy issues, and limitations in applying post-training techniques like pruning and quantization, especially for large-scale models, and conventional techniques require separate storage for magnitude, sign, and iszero flags, leading to increased memory overhead.
Innovation Solution
A method for neural network inferencing in the logarithmic domain using a Bit Manipulation based Logarithmic Number System (BMLNS) technique, which converts pre-trained models to logarithmic domain by storing magnitude, sign, and iszero in a single 32-bit integer, optimizing memory usage and enabling efficient deployment on edge devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If training is performed in logarithmic domain, then inference efficiency is improved, but training complexity and accuracy requirements increase
Solution Approach 1:
The patent segments the neural network processing into two distinct phases: training in real domain and inference in logarithmic domain. This separation allows each phase to utilize the most suitable computational domain, avoiding the complexity of logarithmic training while retaining the efficiency benefits for inference.
Solution Approach 2:
The patent performs preliminary conversion of the trained model from real domain to logarithmic domain before inference. The model is trained in real domain first, then converted to logarithmic domain representations (weights, biases, activation functions) in advance, enabling efficient logarithmic inference without the complexity of logarithmic training.
2Productivity
If model is converted to logarithmic domain, then multiplication operations are reduced to addition, but conversion complexity increases
Solution Approach 1:
The patent changes the numerical representation parameters by converting model weights, biases, and activation functions from real domain values to logarithmic domain values. This parameter transformation enables multiplication operations to be performed as addition operations in the logarithmic domain, improving computational efficiency.
3Measurement precision
If separate storage is used for magnitude, sign, and iszero flags, then logarithmic number accuracy is improved, but memory overhead increases
Solution Approach 1:
The patent merges the separate storage of magnitude, sign, and iszero flags into a unified 32-bit integer structure. Instead of allocating separate memory locations for each component, the patent combines them into a single compact representation, reducing memory overhead while maintaining the ability to accurately represent logarithmic numbers.
Solution Approach 2:
The patent implements a nested structure where the sign and iszero flags are embedded within the 32-bit integer representation alongside the magnitude. This nesting allows efficient packing of multiple pieces of information into a single data structure, optimizing memory usage.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
This disclosure relates generally to neural network inferencing, and more particularly, to a method and system for neural network inferencing in logarithmic domain. The conventional techniques include training a neural network in logarithmic domain and performing inferencing. This leads to less accuracy, challenge in converting large models and unable to perform optimization. The present disclosure converts a pre-trained neural network into logarithmic domain using a bit manipulation based logarithm number system technique wherein the neural network is pre-trained in real time or in logarithmic domain. The method converts the weights, neural network layers and activation function into logarithmic domain. The method uses a 32-bit integer variable to store a logarithm number which leads to memory efficiency. The disclosed method is used for inferencing of convolutional neural network for natural language processing, image recognition and so on.