Neural Network Branching Based on Feature Map Entropy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processing systems lack efficiency in handling varying complexity of input data, leading to unnecessary resource utilization and processing burdens, as they often employ uniform processing sequences for all data without considering the inherent properties of the input features.
Innovation Solution
Implementing a neural network architecture with branching points that allow selective use of different sequences of layers based on the relative information entropy of the output feature maps, enabling simpler processing for less complex data and more complex processing for data requiring higher accuracy, thereby optimizing resource usage and reducing processing burdens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If uniform processing sequences are used for all input data, then the neural network architecture is simple and easy to implement, but processing efficiency decreases and resource utilization becomes unnecessary for less complex data
Solution Approach 1:
The neural network architecture is segmented into multiple processing paths (first path and second path) with different levels of complexity. The system divides the processing sequence into distinct segments that can be selectively executed based on data characteristics, allowing simpler data to take shorter paths and complex data to take longer paths, thereby improving overall processing efficiency without requiring every possible processing step to be executed for all data.
Solution Approach 2:
The architecture introduces dynamic path selection where the processing route is not fixed but adapts based on the input data properties. By evaluating the complexity of output feature maps from previous layers, the system dynamically determines which processing path to execute, making the network's processing behavior flexible and adaptive rather than static and uniform, thus optimizing resource utilization.
2Reliability
If all data is processed using the most complex processing sequence, then processing accuracy is maintained for all data types, but power consumption increases and processing time increases unnecessarily
Solution Approach 1:
Different processing paths are provided with different levels of complexity tailored to local needs. The first processing path uses simpler operations for less complex data while the second path uses more complex operations for data requiring higher accuracy. This local differentiation ensures that processing resources are allocated appropriately to each specific data processing task, avoiding unnecessary energy consumption for simple data while maintaining accuracy for complex data.
Solution Approach 2:
The system changes the processing parameters (complexity level, number of processing steps) based on the characteristics of the input data and intermediate feature maps. By adjusting these parameters dynamically - selecting simpler paths when data complexity is low and more complex paths when data complexity is high - the system optimizes the trade-off between processing accuracy and energy consumption for each individual data processing instance.
3Reliability
If all data is processed using the most complex processing sequence, then processing accuracy is maintained for all data types, but processing time increases unnecessarily
Solution Approach 1:
The processing sequence is segmented into multiple paths with different lengths and complexities. By segmenting the processing workflow, the system can execute simpler, shorter paths for less complex data, reducing processing time, while reserving more complex, longer paths for data that requires higher accuracy, thus optimizing the time-accuracy trade-off across all data types.
Solution Approach 2:
Instead of always executing the full complex processing sequence, the system applies partial processing (first path) when sufficient for the data complexity, and only uses excessive processing (second path) when necessary for accuracy. This partial action approach reduces unnecessary processing time while maintaining accuracy for complex data that truly requires it.
4Use of energy by moving object
If simpler processing sequences are used for less complex data, then power consumption and processing time are reduced, but processing accuracy may be compromised
Solution Approach 1:
The system incorporates feedback mechanisms where the complexity of output feature maps from previous processing layers is evaluated and used to determine the appropriate processing path for subsequent layers. This feedback loop ensures that simpler paths are selected only when the data characteristics justify them, and that accuracy requirements are met by switching to more complex paths when necessary, thus balancing energy consumption with processing accuracy.
Solution Approach 2:
The patent replaces rigid mechanical processing sequences with a more adaptive system that uses property-based selection. Instead of fixed processing steps, the system uses evaluations of data properties (information entropy of feature maps) to substitute and select appropriate processing paths, allowing flexible adaptation between simplicity and complexity based on actual data requirements.
Data Source
AI summary
When executing a neural network comprising a sequence of plural layers of neural network processing in which at least one of the layers of the sequence of plural layers of the neural network is followed by two or more branches of neural network processing, each branch comprising a different sequence of one or more layers of neural network processing, the branch or branches to use for the neural network processing following the layer of the neural network that is followed by the two or more branches of neural network processing is selected based on a property or properties of the output feature map from the layer that is followed by the two or more branches.