Feature Map Normalization Using Region-Selective Neural Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data normalization methods, such as batch normalization, are inefficient in handling outliers and require calculating mean and standard deviation for the entire batch, leading to inefficiencies and sensitivity to data set variations.
Innovation Solution
A neural network operation method that selectively normalizes a predetermined area of a feature map based on position information, using a parameter vector to determine normalization parameters for the selected area and skip normalization for the rest, thereby improving computational efficiency and reducing gradient vanishing/exploding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If batch normalization is performed on the entire batch data, then data normalization is achieved, but computational efficiency deteriorates and sensitivity to outliers increases
Solution Approach 1:
The patent divides the feature map into multiple regions (e.g., foreground region containing objects and background region) and performs normalization separately on each region. This segmentation allows the system to compute normalization parameters only for relevant regions rather than the entire batch, improving computational efficiency while maintaining normalization effectiveness for object detection tasks.
Solution Approach 2:
The patent applies different normalization strategies to different regions of the feature map. Specifically, normalization is applied to the foreground region where objects are located, while the background region may be excluded or handled differently. This local quality approach ensures that computational resources are focused on the most important areas, improving both efficiency and effectiveness.
2Reliability
If normalization is performed on the entire feature map, then comprehensive data normalization is achieved, but computational complexity increases
Solution Approach 1:
The patent extracts and identifies the foreground region (containing objects of interest) from the entire feature map using object detection results. By taking out only the relevant foreground region for normalization processing, the system reduces computational complexity while maintaining normalization quality where it matters most for detection accuracy.
Solution Approach 2:
Instead of normalizing the entire feature map (excessive action), the patent applies normalization only to the foreground region (partial action). This partial normalization approach is sufficient for achieving good detection performance while significantly reducing computational complexity compared to full-map normalization.
3Measurement precision
If mean and variance are calculated for the entire batch, then accurate normalization parameters are obtained, but processing time increases
Solution Approach 1:
The patent segments the batch data processing into region-specific calculations. Mean and variance are calculated only for the foreground region rather than the entire batch, reducing processing time while maintaining parameter accuracy for the relevant detection areas.
Solution Approach 2:
The patent changes the scope of parameter calculation from the entire batch to only the foreground region. This parameter change in the calculation scope reduces processing time while the parameters remain accurate for the regions that matter most for object detection performance.
Data Source
AI summary
A processor-implemented neural network operation method includes: receiving a feature map on which a neural network operation is to be performed; selecting a predetermined area from the feature map; generating a normalization parameter based on the predetermined area; and performing the neural network operation based on the normalization parameter.