Neural Network Layer Contraction via Reference Sample Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing apparatuses inefficiently handle large amounts of input data in real-time, leading to slow operation speeds and high computational costs.
Innovation Solution
The method involves determining a reference sample from sequential input data, performing an inference process, and calculating layer contraction parameters to approximate the inference process. These parameters include a weight matrix, bias vector, and binary mask, allowing for efficient affine transformation-based inference on subsequent input samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a typical neural network processes large amounts of input data in real time, then the inference accuracy is maintained, but the operation speed decreases and computational cost increases
Solution Approach 1:
The patent segments the neural network processing into two distinct parts: (1) a reference sample processed through the full multi-layer neural network to obtain accurate results, and (2) subsequent sequential input samples processed through a simplified affine transformation model. This segmentation allows the system to maintain high accuracy on the reference sample while achieving fast processing speeds on subsequent samples that exploit spatio-temporal redundancies.
Solution Approach 2:
The patent changes the computational parameters from complex multi-layer neural network operations (with multiple weight matrices and activation functions) to a simplified affine transformation using a single weight matrix W, bias vector b, and binary mask M. This parameter change dramatically reduces computational complexity while maintaining acceptable accuracy for sequential data with temporal correlations.
2Measurement precision
If a typical neural network processes large amounts of input data in real time, then the inference accuracy is maintained, but the computational cost increases
Solution Approach 1:
The patent segments the neural network processing into two distinct parts: (1) a reference sample processed through the full multi-layer neural network to obtain accurate results, and (2) subsequent sequential input samples processed through a simplified affine transformation model. This segmentation allows the system to maintain high accuracy on the reference sample while achieving fast processing speeds on subsequent samples that exploit spatio-temporal redundancies.
Solution Approach 2:
The patent changes the computational parameters from complex multi-layer neural network operations (with multiple weight matrices and activation functions) to a simplified affine transformation using a single weight matrix W, bias vector b, and binary mask M. This parameter change dramatically reduces computational complexity while maintaining acceptable accuracy for sequential data with temporal correlations.
3Measurement precision
If layer contraction parameters are determined for each sequential input sample, then the inference accuracy is improved, but the operation speed decreases
Solution Approach 1:
The patent performs preliminary action by determining the layer contraction parameters (weight matrix W, bias vector b, and binary mask M) once for a reference sample before processing subsequent sequential input samples. These pre-determined parameters are then reused for multiple subsequent samples, eliminating the need to recalculate them for each sample and thereby significantly reducing processing time while maintaining inference accuracy through the affine transformation model.
4Measurement precision
If the full neural network architecture is used for all input samples, then the inference accuracy is maintained, but the device complexity increases
Solution Approach 1:
The patent applies local quality by using different processing approaches for different parts of the input data sequence: the reference sample undergoes full multi-layer neural network processing to ensure accuracy and generate contraction parameters, while subsequent samples use the simplified affine transformation model. This localized approach reduces overall device complexity while maintaining necessary accuracy for the reference sample and achieving efficiency for sequential processing.
Data Source
AI summary
A processor-implemented neural network method includes: determining a reference sample among sequential input samples to be processed by a neural network, the neural network comprising an input layer, one or more hidden layers, and an output layer; performing an inference process of obtaining an output activation of the output layer based on operations in the hidden layers corresponding to the reference sample input to the input layer; determining layer contraction parameters for determining an affine transformation relationship between the input layer and the output layer, for approximation of the inference process; and performing inference on one or more other sequential input samples among the sequential input samples using affine transformation based on the layer contraction parameters determined with respect to the reference sample.