Neural Network Max Pooling Restricted Connectivity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural networks are not robust enough to handle corrupt, invalid, or erroneous data during machine learning processing, which can lead to inaccurate results, as they continue to process and pass such data through layers without filtering.
Innovation Solution
A neural network configuration that selectively excludes or suppresses data from feature streams likely to be corrupted by using max pooling and restricted connectivity between nodes, comparing values from combined feature streams to those from streams that exclude the potentially corrupted data, and passing only the maximum values to subsequent layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional neural networks process all input data through all layers, then complete data processing is achieved, but corrupted data propagates through the network reducing reliability
Solution Approach 1:
The neural network is segmented into multiple feature streams (e.g., first feature stream and second feature stream) that can be independently processed. This segmentation allows the network to identify and isolate corrupted data in one stream while continuing to process valid data in other streams, thereby improving reliability without requiring a complete redesign of the network architecture.
Solution Approach 2:
A gating mechanism or selector layer is introduced as an intermediary between feature streams and subsequent processing layers. This intermediary compares values from different feature streams and selectively passes only the most reliable data forward, preventing corrupted data from propagating through the network while maintaining the overall network structure.
2Reliability
If max pooling is used to select maximum values from multiple feature streams, then corrupted data is suppressed, but additional computational steps are required
Solution Approach 1:
Multiple feature streams are merged into a combined feature stream through element-wise addition or concatenation. This merging allows the network to leverage information from multiple sources while the subsequent max pooling operation selects the most reliable values, achieving robustness against corruption without requiring separate processing paths for each stream.
Solution Approach 2:
The max pooling operation is applied preliminarily to compare and select values from multiple feature streams before passing data to subsequent processing layers. This preliminary selection of maximum values ensures that corrupted data (which typically has lower or more negative values) is filtered out early in the processing pipeline, improving robustness without adding significant computational overhead later.
3Measurement precision
If feature streams are combined without filtering, then comprehensive data processing is achieved, but corrupted data from one stream can corrupt the combined output
Solution Approach 1:
The corruption in one feature stream is converted into a benefit by using it as a signal for the gating mechanism or max pooling operation to identify and suppress problematic data. The corrupted values (typically lower or more negative) serve as indicators that trigger the selective filtering mechanism, transforming the harmful effect of corruption into a useful signal for data quality assessment.
Solution Approach 2:
Different quality control mechanisms are applied to different feature streams based on their individual characteristics. Instead of uniformly processing all streams, the network applies selective gating or max pooling operations locally to each stream or combination of streams, allowing high-quality data to pass through while blocking corrupted data in specific localized regions of the feature space.
Data Source
AI summary
Features are disclosed for improving the robustness of a neural network by using multiple (e.g., two or more) feature streams, combing data from the feature streams, and comparing the combined data to data from a subset of the feature streams (e.g., comparing values from the combined feature stream to values from one of the component feature streams of the combined feature stream). The neural network can include a component or layer that selects the data with the highest value, which can suppress or exclude some or all corrupted data from the combined feature stream. Subsequent layers of the neural network can restrict connections from the combined feature stream to a component feature stream to reduce the possibility that a corrupted combined feature stream will corrupt the component feature stream.