Neural Network Presorter for Input Data Sorting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face inefficiencies in processing similar instances of input data, leading to increased computational efforts and energy consumption due to repeated calculations and memory accesses.
Innovation Solution
Implementing a technique to store and reuse previous result values from nodes in the neural network, combined with a presorter to determine a sorted order for processing instances of input data, which enhances the similarity between consecutive inputs, thereby reducing the number of computations and memory accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If neural networks process instances of input data through normal computation of internal values and activation functions, then processing accuracy is maintained, but electrical power consumption and processing time increase due to repeated computations
Solution Approach 1:
The patent applies preliminary action by sorting instances of input data before processing them through the neural network. The presorter arranges input data in an order that maximizes similarity between consecutive inputs, enabling the system to reuse previously computed node results for similar inputs. This preliminary sorting action sets up the conditions for reduced computation in subsequent processing steps.
Solution Approach 2:
The patent implements discarding and recovering by storing results from previously processed input instances and reusing them for similar inputs. When consecutive inputs after sorting are sufficiently similar, the system recovers previously computed node results instead of performing full recomputation, thereby discarding redundant calculation efforts while maintaining processing accuracy.
2Loss of time
If neural networks reuse stored result values from previous computations, then electrical power consumption and latency are reduced, but computational accuracy may be compromised due to approximation
Solution Approach 1:
The patent applies local quality by selectively reusing stored result values only for specific nodes in the neural network where input similarity justifies approximation. The system evaluates similarity between consecutive inputs and applies result reuse locally to individual nodes rather than uniformly across the entire network, maintaining accuracy where needed while optimizing performance where approximation is acceptable.
Solution Approach 2:
The patent implements parameter changes by dynamically adjusting the similarity threshold parameter that controls when stored results are reused. The system can modify this threshold parameter based on input characteristics and performance requirements, allowing flexible control over the trade-off between latency reduction and accuracy maintenance without requiring structural changes to the neural network.
3Productivity
If instances of input data are sorted to enhance similarity between consecutive inputs, then result reuse efficiency is improved, but additional processing overhead is introduced by the presorter
Solution Approach 1:
The patent introduces an intermediary component, the presorter, that sits between the input data source and the neural network processor. This intermediary performs the sorting function to arrange inputs for optimal similarity, acting as a mediator that enables result reuse without modifying the core neural network architecture or computation logic.
Solution Approach 2:
The presorter is designed with multi-functionality to handle various types of input data (images, audio, text) and can operate with different similarity metrics and thresholds. This universal design allows the same sorting mechanism to serve multiple processing scenarios, reducing the need for specialized components for each data type or application.
Data Source
AI summary
An electronic device including a neural network processor and a presorter is described. The presorter determines a sorted order to be used by the neural network processor for processing a set of instances of input data through the neural network, the determining including rearranging an initial order of some or all of the instances of input data so that instances of input data having specified similarities among the some or all of the instances of input data are located nearer to one another in the sorted order. The presorter provides, to the neural network processor, the sorted order to be used for controlling an order in which instances of input data from among the set of instances of input data are processed through the neural network. A controller in the electronic device adjusts operation of the presorter based on efficiencies of the presorter and the neural network processor.