Self-Checkout Item Recognition Using Augmented Training Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional tracking systems are limited in their ability to track a wide range of merchandise items with varying characteristics and environmental factors, particularly in retail environments with frequent inventory changes and visual similarities, hindering seamless self-checkout experiences.
Innovation Solution
A centralized computing device communicates with sensors and mechanical structures in a self-checkout vehicle, utilizing neural networks trained with normalized and augmented images from multiple angles and backgrounds to identify merchandise items in real-time, and updates the training data dynamically to adapt to inventory changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional tracking systems are used to monitor merchandise items, then the system structure remains simple, but the system cannot track a wide range of merchandise items with varying characteristics and environmental factors
Solution Approach 1:
The patent replaces conventional mechanical tracking systems with a neural network-based computer vision system. The neural network processes images from cameras to identify and track merchandise items, replacing physical sensors and mechanical tracking components. This substitution enables the system to handle diverse merchandise characteristics while maintaining manageable complexity through software-based solutions.
Solution Approach 2:
The patent employs parameter changes by training the neural network with images captured under varying parameters including different angles, lighting conditions, backgrounds, and merchandise orientations. This training approach enables the system to adapt to diverse environmental factors and merchandise characteristics, significantly improving tracking versatility.
2Measurement precision
If neural networks are trained with comprehensive training data covering multiple angles and backgrounds, then merchandise identification accuracy improves, but data processing time and computational resources increase
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network with comprehensive training data covering multiple angles, lighting conditions, and backgrounds before deployment. This offline training phase prepares the model in advance, enabling rapid real-time identification during actual operation without requiring extensive processing time during checkout.
Solution Approach 2:
The system implements dynamics by continuously updating and retraining the neural network with new training data as inventory changes. This dynamic adaptation allows the system to maintain high identification accuracy for new products while optimizing processing efficiency through iterative improvements.
3Reliability
If the training dataset is extended with augmented images, then the neural network becomes more robust to environmental variations, but the training process becomes more complex and time-consuming
Solution Approach 1:
The patent uses copying by creating augmented training images through transformations of existing training data. The neural network is trained with copied and transformed versions of original images including rotations, flips, color adjustments, and background replacements. This approach builds robustness without requiring proportional increases in physical training data collection.
4Productivity
If real-time merchandise identification is implemented, then customer service efficiency improves, but the system must continuously adapt to inventory changes and new products
Solution Approach 1:
The patent implements feedback mechanisms where the system continuously receives input from image capture devices during checkout, processes this data through the neural network, and provides real-time identification results. The system also incorporates feedback loops for continuous learning and adaptation to new inventory items, maintaining both high productivity and adaptability.
Solution Approach 2:
The system applies dynamics by enabling continuous updates to the neural network training data as inventory changes. New merchandise items can be added to the training set, and the model can be retrained or fine-tuned to maintain real-time identification capability across evolving inventory compositions.
Data Source
AI summary
Disclosed are technologies for generating training data for identification neural networks. Series of images are captured of a plurality of merchandise items from different angles and with different background assortments of other merchandise items. A labeled training dataset is generated for the plurality of merchandise items. The series of captured images is normalized, where the merchandise occupies a threshold percentage of pixels in the normalized image. The training dataset is extended by applying augmentation operations to the normalized images to generate a plurality of augmented images. Each image is stored in the training dataset as a unique training data point for the given merchandise item it depicts. Labels are generated mapping each training data point to attributes associated with the depicted merchandise item. Input neural networks are trained on the labeled training dataset to perform real-time identification of selected merchandise items placed into a self-checkout apparatus by a user.