Neural Network Pooling via Class Probability Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional pooling methods in neural network models, such as convolutional neural networks (CNNs), result in a loss of input data, which degrades the performance of the neural network.
Innovation Solution
The method involves training a neural network model by receiving input data and target data, extracting a feature map, performing pooling based on a probability for each class of the feature map, and determining a loss that includes an auxiliary loss from the pooling process. This approach helps in minimizing data loss during the pooling operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional pooling methods (mean value or representative value) are used, then the pooling operation can be performed simply, but input data is lost and neural network performance degrades
Solution Approach 1:
The patent changes the pooling operation from using fixed statistical parameters (mean, representative value) to using dynamically calculated probability distributions based on class-specific pixel ratios. This transforms the pooling process into a probability-weighted aggregation that preserves class information, thereby reducing data loss while maintaining operational simplicity.
Solution Approach 2:
The patent introduces auxiliary loss that provides feedback during training to ensure the pooling operation preserves class information. The auxiliary loss compares the pooled feature map with the original input, creating a feedback mechanism that guides the network to learn pooling operations that minimize information loss while maintaining the simplicity of the operation.
2Productivity
If conventional pooling methods are used, then the pooling process is computationally efficient, but neural network model performance is degraded due to data loss
Solution Approach 1:
The patent changes the computational approach by replacing complex per-pixel processing with class-level probability calculations. Instead of processing each pixel individually, the system calculates the ratio of pixels belonging to each class within pooling regions, then uses these probabilities as weights for aggregation. This parameter change maintains computational efficiency while significantly improving model performance by preserving class information.
Solution Approach 2:
The patent applies partial action by focusing the pooling operation on calculating class probabilities rather than preserving every individual pixel detail. This selective approach maintains sufficient information for accurate classification while reducing computational complexity, achieving a balance between efficiency and performance.
3Quantity of substance
If pooling is performed to reduce data size, then the model can process data more efficiently, but the original input data is lost
Solution Approach 1:
The patent creates a probabilistic copy of the input data during pooling by calculating class probability distributions for each pooling region. Instead of discarding the original data, the system generates a probability map that copies and preserves class information in a compressed representation. This probability map serves as an informative summary that maintains the essence of the original data while reducing its size.
Solution Approach 2:
The patent changes the representation parameter from raw pixel values to class probability distributions. This transformation allows the data to be compressed more effectively while preserving meaningful information. The probability parameters capture the essential class characteristics of each region, enabling efficient data reduction without significant information loss.
Data Source
AI summary
Disclosed are a method and apparatus for training a neural network model to increase performance of the neural network model, the method including receiving input data and target data, pooling, by a neural network model, on a feature map extracted from the input data based on a probability for each of classes of the feature map, generating output data by inputting the input data to a neural network model, determining a loss based on comparing the output data and the target data and an auxiliary loss of the pooling, and training the neural network model based on the loss.