Neural Network Drop Layer for Compressed Feature Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face reduced inference performance due to task-irrelevant content in input data, which existing methods struggle to effectively remove, leading to suboptimal recognition and classification outcomes.
Innovation Solution
A processor-implemented method using a drop layer in neural networks that generates compressed representation data by dropping feature values based on assigned drop probabilities, with probabilities determined by relevance to the target task, employing binomial distribution functions for stochastic or deterministic drop operations to preserve relevant information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing methods are used to process input data, then the neural network can process all input features, but task-irrelevant content reduces inference performance and recognition accuracy
Solution Approach 1:
The patent applies the extraction principle by introducing a drop layer that selectively removes (drops) task-irrelevant feature values from the input data. The drop layer identifies and extracts only the relevant features needed for the target task, discarding harmful task-irrelevant content. This is achieved through drop probability assignments that determine which feature values to retain and which to remove, thereby improving inference performance by eliminating harmful factors.
Solution Approach 2:
The patent implements local quality by assigning different drop probabilities to different feature values based on their relevance to the target task. Instead of uniformly processing all features, the system applies localized quality control where each feature value is evaluated individually. Features with high relevance receive low drop probabilities (preserved), while task-irrelevant features receive high drop probabilities (removed), creating a non-uniform, quality-based filtering mechanism.
2Productivity
If all feature values are preserved in the neural network, then complete information is available for processing, but computation time and resource consumption increase
Solution Approach 1:
The drop layer extracts and removes redundant or less important feature values from the input data before processing. By selectively dropping feature values based on their relevance to the target task, the system reduces the volume of data that needs to be processed by subsequent neural network layers, thereby improving computation efficiency without losing critical information needed for accurate inference.
Solution Approach 2:
The patent applies partial action by not preserving all feature values, but only the necessary subset required for the target task. The drop layer performs partial processing by retaining only relevant features and discarding others, achieving a balance between maintaining sufficient information for accurate results and reducing computational burden through selective feature preservation.
3Measurement precision
If a drop layer is introduced to remove task-irrelevant content, then recognition accuracy improves, but the neural network structure becomes more complex
Solution Approach 1:
The drop layer serves as an intermediary component between the input data and the main neural network processing layers. It acts as a mediator that pre-processes the input by removing task-irrelevant content before the data enters the subsequent neural network layers. This intermediary structure improves recognition accuracy by filtering harmful content while maintaining a relatively simple implementation through a single additional layer with drop probability parameters.
Data Source
AI summary
A neural network includes a drop layer configured to drops feature values. A method of computation using the neural network includes extracting feature data from input data using a first portion of a neural network, generating compressed representation data of the extracted feature data by dropping a feature value from the extracted feature data at a drop layer of the neural network based on a drop probability corresponding to the feature value, and indicating an inference result from the compressed representation data using a second portion of the neural network.


