Fine-Grained Retail Product Classification With Object-Part Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing product classification systems in retail environments face challenges due to variations in image quality, illumination, and viewing angles, making it difficult to accurately classify similar-looking products.
Innovation Solution
A system utilizing a reconstruction-classification network (RC-Net) and a stacked convolutional Long Short-Term Memory (conv-LSTM) network for fine-grained classification, which includes preprocessing, object-level and part-level information extraction, and clustering to enhance classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a single template image is used for product classification, then the system is simple to implement, but classification accuracy deteriorates due to environmental variations
Solution Approach 1:
The patent segments the product classification task into multiple components: capturing multiple template images from different angles and conditions, extracting multiple features (color, text, shape), and combining multiple classification scores. This segmentation allows the system to handle environmental variations by analyzing products from multiple perspectives rather than relying on a single template.
Solution Approach 2:
The patent changes multiple parameters simultaneously: using multiple template images with different illumination conditions, capturing images at different scales and poses, and extracting features across different color spaces (RGB, HSV, LAB). This multi-parameter approach enables the system to maintain classification accuracy despite variations in the retail environment.
2Measurement precision
If multiple features and networks are used for fine-grained classification, then classification accuracy is improved, but system complexity increases
Solution Approach 1:
The patent divides the complex classification task into separate modules: a color feature extraction module, a text feature extraction module, a shape feature extraction module, and a combination module that integrates multiple classification scores. Each module focuses on a specific aspect, making the overall complex system manageable and maintainable while achieving high accuracy.
Solution Approach 2:
The patent creates a multi-functional system where a single product classification system handles multiple types of features (color, text, shape) and multiple template images simultaneously. The framework is designed to be universal, accepting various input types and processing them through standardized pipelines, which reduces the need for separate systems for different classification tasks.
3Adaptability or versatility
If product images are captured in uncontrolled retail environments, then the system is adaptable to real-world usage, but image quality and consistency deteriorate
Solution Approach 1:
The patent performs preliminary actions by capturing multiple template images under controlled conditions before deployment, storing them as reference patterns. During real-world operation, the system compares captured product images against these pre-captured templates, allowing it to adapt to retail environments while maintaining reliability through reference-based matching rather than requiring consistent imaging conditions.
Solution Approach 2:
The patent converts the harmful effect of environmental variations (different illumination, angles, and backgrounds in retail stores) into a benefit by training the system to recognize products across these variations. The system uses multiple template images captured under different conditions as training data, so that the very variations that could harm classification become part of the learning process, improving robustness to real-world conditions.
Data Source
AI summary
The fine-grained variations in product images are usually due to slight variations in text, size, and color of the package. Both marginal variations in image content and illumination poses an important challenge in product classification. This disclosure relates to a system and method for fine-grained classification of similar-looking products utilizing object-level and part-level information. The system simultaneously captures an object-level and part-level information of the product. The object-level classification score of the product is estimated with the trained RC-Net, a deep supervised convolutional autoencoder. For annotation-free modelling of part-level information of the product the discriminative part-proposal of the product is identified around the BRISK key points. An ordered sequence of the discriminative part-proposals and the product image, encoded using stacked convolutional LSTM network, estimates the part-level classification score. Finally, the trained RC-Net and stacked conv-LSTM network jointly classifies the product image based on the final classification score.