Image Classification Confidence Recalibration Using Shelf Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural network (CNN) models struggle to accurately classify products in retail environments due to variations in size and text visibility, leading to inaccurate confidence levels, especially when products have similar appearances and are captured at irregular distances or angles.
Innovation Solution
Integrate contextual information, specifically volume co-occurrence probabilities, into the confidence levels predicted by CNN models to recalibrate and enhance the accuracy of product classifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN models classify products based on image data alone, then classification speed is maintained, but classification accuracy deteriorates due to size and text visibility variations
Solution Approach 1:
The patent combines CNN-based image classification with rule-based contextual validation (volume co-occurrence checks) into a unified classification system. The system merges visual recognition results with logical constraints about product packaging relationships to achieve higher accuracy without completely redesigning the classification architecture.
Solution Approach 2:
The patent introduces an intermediary validation layer that acts as a mediator between the CNN classification output and the final classification result. This intermediary layer checks volume co-occurrence relationships and corrects misclassifications without requiring changes to the core CNN model, thus improving accuracy while maintaining relatively simple system architecture.
2Productivity
If images are captured at irregular distances and angles to cover more products, then productivity increases, but measurement precision deteriorates due to size and text visibility issues
Solution Approach 1:
The patent performs preliminary classification using the CNN model on all detected products before applying contextual validation. This preliminary action allows the system to quickly process images at irregular distances and angles, then subsequently correct any misclassifications using volume co-occurrence rules, thereby maintaining both high productivity and accuracy.
Solution Approach 2:
The patent implements a feedback mechanism where classification results are validated against known volume co-occurrence relationships. When inconsistencies are detected (e.g., impossible volume combinations), the system provides feedback to correct the classification, ensuring high accuracy even when images are captured under challenging conditions that prioritize throughput.
3Productivity
If confidence levels are used to filter classification results, then processing efficiency improves, but reliability deteriorates due to inaccurate confidence predictions
Solution Approach 1:
The patent introduces an intermediary confidence recalibration step that adjusts the raw CNN confidence levels based on volume co-occurrence validation results. This intermediary layer preserves the efficiency benefit of confidence-based filtering while correcting the reliability issue by recalibrating confidences to reflect actual classification accuracy more accurately.
Solution Approach 2:
The patent changes the confidence level parameter from raw CNN output to recalibrated confidence that incorporates contextual validation results. This parameter transformation allows the system to maintain efficient confidence-based filtering while improving the reliability of confidence predictions through adjustment based on volume co-occurrence relationships.
Data Source
AI summary
Methods, systems, articles of manufacture, and apparatus to recalibrate confidences for image classification are disclosed. An example apparatus to classify an image includes an image crop detector to detect a first image crop from the image, the first image crop corresponding to a first object, a grouping controller to select a second image crop corresponding to a second object at a location of the first object, a prediction generator to, in response to executing a trained model, determine a label corresponding to the first object and a confidence level associated with the label, and a confidence recalibrator to recalibrate the confidence level based on a probability of the first object having a first attribute based on the second object having a second attribute, the confidence level recalibrated to increase an accuracy of the image classification.