Video Classification Model Using Feature Contribution Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Object classification based on video images faces challenges due to poor image quality and significant changes in object attitude, leading to poor classification performance, especially when compared to still image classification.
Innovation Solution
A device and method utilizing a pre-trained classification model with a feature extraction unit, contribution calculation unit, and feature aggregation unit to extract and aggregate features of multiple images in a video group, considering the contribution of each image to the classification result, thereby improving classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If object classification is performed based on video images, then more temporal information is available for classification, but image quality deteriorates and object attitude changes significantly leading to poor classification performance
Solution Approach 1:
The video sequence is segmented into multiple image groups, where each group contains multiple images. The classification model processes each image group independently to generate intermediate classification results, which are then aggregated to produce the final classification result. This segmentation allows the system to handle poor image quality at the individual image level while leveraging temporal information across multiple images within each group.
Solution Approach 2:
The system merges multiple intermediate classification results from different image groups through a result aggregation module. By combining the classification outcomes from multiple image groups, the system compensates for the poor quality of individual images and achieves more reliable overall classification performance, resolving the contradiction between available temporal information and image quality deterioration.
2Loss of information
If multiple images are processed individually through feature extraction, then more information is available for classification, but the complexity of processing increases
Solution Approach 1:
The system performs preliminary feature extraction on individual images within each image group before aggregation. The feature extraction layer processes each image to extract relevant features, which are then passed to the aggregation layer. This preliminary action preserves information from multiple images while organizing the processing pipeline to manage complexity systematically through modular layers.
Solution Approach 2:
The aggregation layer acts as an intermediary between individual image feature extraction and final classification. It receives features from multiple images, performs weighted aggregation based on contribution calculations, and produces a consolidated feature representation. This intermediary structure manages processing complexity by providing a systematic way to combine multiple image features without requiring complex individual processing for each image.
3Device complexity
If conventional classification methods are used on video images, then processing is simpler, but classification accuracy deteriorates due to poor image quality and object attitude changes
Solution Approach 1:
The system dynamically adjusts the processing approach by dividing the video sequence into multiple image groups and processing each group through the classification model. The model dynamically handles variations in object attitude and image quality by generating intermediate results for each group and aggregating them. This dynamic multi-group processing improves classification accuracy while maintaining reasonable processing complexity through systematic organization.
Solution Approach 2:
The system changes the parameter of processing granularity by transitioning from processing individual images to processing groups of images. Each image group is processed to generate intermediate classification results, which are then aggregated. This parameter change from individual image processing to group-based processing improves classification accuracy by leveraging temporal information while managing complexity through structured aggregation.
Data Source
AI summary
A device and a method for classification using a pre-trained classification model and a computer readable storage medium are provided. The device is configured to extract, for each of multiple images in a target image group to be classified, a feature of the image using a feature extraction layer of the pre-trained classification model; calculate, for each of the multiple images, a contribution of the image to a classification result of the target image group using a contribution calculation layer of the pre-trained classification model; aggregate extracted features of the multiple images based on calculated contributions of the multiple images, to obtain an aggregated feature as a feature of the target image group; and classify the target image group based on the feature of the target image group.