Commodity Recognition Using Image Similarity and PLU Reference Matching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing generic object recognition technologies face challenges in accurately and efficiently recognizing commodities in a retail setting, particularly in determining the category and sales registration of fruits and vegetables, due to limitations in image similarity comparison and user confirmation processes.

Innovation Solution

An information processing apparatus and method that uses an image capturing unit to compare captured images with reference images stored in a PLU file, determining commodities based on similarity thresholds and accepting user confirmation for accurate sales registration, while displaying candidate commodities with lower similarity scores.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If generic object recognition technology is used to recognize commodities by comparing characteristic quantities, then commodity category recognition is enabled, but recognition accuracy and efficiency are insufficient for retail settings

Engineering Contradiction:

Improvecommodity recognition accuracyVSAvoidrecognition efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent uses image copying by capturing images of commodities and storing them as reference images in a PLU file. The system compares captured images with stored reference images to identify commodities, replacing complex characteristic quantity extraction with direct image comparison, thereby improving both accuracy and efficiency

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical/process-based characteristic quantity extraction and comparison method with an image-based recognition system. By using image capturing units and image comparison algorithms, the system achieves faster and more accurate commodity recognition without manual intervention

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If strict similarity threshold is used to determine commodity matches, then recognition accuracy is improved, but processing time increases due to multiple comparisons

Engineering Contradiction:

Improvecommodity identification accuracyVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-capturing and storing reference images of commodities in a PLU file before actual recognition tasks. This allows the system to quickly compare incoming images against pre-prepared references, reducing processing time while maintaining high similarity threshold requirements for accurate identification

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements a two-stage recognition process: first comparing with reference images at a high similarity threshold for quick identification, and only when that fails, performing additional comparisons or manual verification. This partial application of strict thresholding reduces overall processing time while maintaining accuracy for clear matches

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If automated commodity recognition is implemented, then sales registration speed is improved, but user confirmation capability is reduced

Engineering Contradiction:

Improvesales registration speedVSAvoiduser confirmation capability

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent implements feedback by displaying recognition results to operators and allowing them to confirm or correct the identified commodity. The system provides visual feedback showing the matched reference image and commodity information, enabling operators to verify accuracy while maintaining fast automated processing for clear matches

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent enables self-service by allowing the automated system to handle clear commodity recognitions without operator intervention. When the similarity threshold is met, the system automatically completes the recognition, and only ambiguous cases require manual confirmation, thus maintaining both speed and user control

Inventive Principle:

Principle #25Self-service

4Measurement precision

If multiple candidate commodities are displayed for user selection, then recognition accuracy is improved, but device complexity increases

Engineering Contradiction:

Improvecommodity recognition accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the commodity recognition process into automated high-confidence matches and manual confirmation cases. By dividing recognition into these segments, the system displays multiple candidates only when necessary, reducing overall system complexity while maintaining high accuracy through selective use of candidate display

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS9042660B2Information processing apparatus and information processing method

Publication Date: 2015.05.26 TOSHIBA TEC KK

AI summary

According to one embodiment, an information processing apparatus includes an acquirement unit and a reporting unit. The acquirement unit is configured to acquire an image captured by a image capturing section. In a situation that a similarity representing a degree with which the image of an object captured by the image capturing section is similar to the reference image of each commodity meets a condition of determining a captured commodity as one commodity in the commodities corresponding to the reference image, the reporting unit is configured to report a situation that the captured commodity is determined as the commodity meeting the condition and corresponding to the reference image.