Channel Attention Aggregation for Local Feature Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image classification models primarily rely on global features, leading to lower classification accuracy due to the inability of attention layers to effectively learn the importance of local features when local features are normalized to the same value.
Innovation Solution
A data processing method and apparatus that includes a data processing network, an attention network, and a data aggregation network, where the attention network outputs fixed attention features with varying attention weights, allowing the model to learn the importance of each local feature, and a data aggregation network processes intermediate data based on these fixed attention features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing classification models rely on global features, then the model structure is simple, but classification accuracy is low
Solution Approach 1:
The patent divides the image processing into local feature extraction and global feature processing. The attention network segments the feature map into multiple channels, each representing different local features, and processes them independently before aggregation. This segmentation allows the model to capture both local and global information, resolving the contradiction between simple structure and high accuracy.
Solution Approach 2:
The patent introduces a channel dimension for attention weights that is independent of the spatial dimensions. By adding this new dimension, the model can apply attention mechanisms without increasing spatial complexity, enabling local feature processing while maintaining overall model efficiency.
2Measurement precision
If attention network processes each local feature independently, then local feature importance is captured, but attention weights become uniform when local features are normalized to the same value
Solution Approach 1:
The patent applies preliminary normalization to local features before attention processing, which standardizes the input distribution. This preliminary action ensures that subsequent attention mechanisms can effectively differentiate feature importance without being biased by scale differences, while the channel-wise attention then restores the differentiation capability.
Solution Approach 2:
The patent changes the parameterization of attention weights from spatial-based to channel-based. By computing attention weights at the channel level rather than spatial level, the model maintains adaptability in distinguishing feature importance even when local features are normalized, as channel statistics preserve relative importance information.
3Productivity
If channel pruning is applied to reduce redundant parameters, then model efficiency is improved, but classification accuracy may be compromised
Solution Approach 1:
The attention network automatically identifies and prioritizes important channels through learned attention weights, enabling the model to self-select which features to retain. This self-service mechanism guides channel pruning by providing data-driven importance scores, ensuring that pruning removes only redundant channels while preserving accuracy-critical features.
Solution Approach 2:
The patent replaces manual or uniform channel selection with a learned attention mechanism. Instead of using fixed rules or random pruning, the attention network dynamically determines channel importance based on the input data, allowing intelligent substitution of mechanical pruning approaches with adaptive, data-driven selection that maintains accuracy while improving efficiency.
Data Source
AI summary
Data processing method and apparatus are disclosed. The method includes: processing input data based on the data processing network, to obtain intermediate data; acquiring a fixed attention feature output by the attention network, which is obtained after training the model with at least an initialized attention feature, respective attention weights included in the initialized attention feature are not all the same; processing the intermediate data based on the data aggregation network and according to the fixed attention feature, to obtain output data. The fixed attention network does not receive any input previously, that is, attention feature output by the attention network is irrelevant to input data. The fixed attention feature can not only allow a more important parameter of the model to play a greater role to improve accuracy of the model and determine importance of the parameter, but also be conducive to further compression of the model.