CNN-Based Produce Recognition with Bayesian PLU Verification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for produce recognition and verification at retail self-checkout terminals are inefficient, require significant manual maintenance, and are susceptible to fraud, with customers often entering incorrect PLU codes, leading to significant losses for retailers.

Innovation Solution

A system utilizing a Convolutional Neural Network (CNN) to generate a feature vector from an item image, combined with Bayesian inference engines trained on sales data, provides a pick list of potential PLU codes and verifies the entered code, ensuring accurate and efficient produce identification and fraud prevention across multiple retailers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If manual lookup methods (books, pamphlets, on-screen guides) are used to find PLU codes, then customers can identify produce items, but the process is time-consuming and difficult for untrained customers

Engineering Contradiction:

Improveease of produce identificationVSAvoidtime for produce lookup

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The patent replaces manual mechanical lookup methods (flipping through books, navigating on-screen guides) with an automated computer vision system using digital cameras and machine learning algorithms to instantly recognize produce items and return PLU codes, eliminating the need for customers to manually search through databases

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service produce identification by automatically capturing images of produce items and generating PLU code suggestions without requiring customer intervention or prior knowledge, allowing any customer to quickly identify produce items

Inventive Principle:

Principle #25Self-service

2Ease of operation

If existing computer vision approaches are used for produce recognition, then the lookup process is simplified, but the systems require significant manual maintenance and training work from retailers

Engineering Contradiction:

Improveease of produce lookupVSAvoidcomplexity of system maintenance

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The system incorporates feedback mechanisms where customers can correct misidentifications by selecting from suggested PLU codes, and this feedback is used to continuously improve and retrain the machine learning models, reducing the need for manual maintenance while improving accuracy over time

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent uses dynamic parameter adjustment where the system adapts to different produce items, lighting conditions, and camera perspectives by continuously refining its recognition parameters through customer feedback and automated learning, reducing the need for manual system reconfiguration

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If existing produce recognition systems are implemented individually per SCO/SST or store, then local customization is possible, but the ability to share imagery and models between systems is limited

Engineering Contradiction:

Improvelocal customization capabilityVSAvoidefficiency of model sharing

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent creates a universal produce recognition system that operates across multiple SCO/SST terminals and stores while maintaining local customization capabilities, allowing models and imagery to be shared and updated centrally across the entire retailer network, improving efficiency and reducing redundancy

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Ease of operation

If customers can enter their own PLU codes, then they have flexibility in input, but customers can intentionally enter wrong PLU codes causing significant loss for retailers

Engineering Contradiction:

Improveflexibility in PLU entryVSAvoidaccuracy of PLU code entry

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The system replaces manual PLU code entry with automated computer vision-based identification, where the system captures images of produce items and automatically determines the correct PLU code, eliminating the possibility of intentional incorrect entry while maintaining ease of use

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system provides real-time feedback by displaying suggested PLU codes based on the captured image, allowing customers to verify and confirm the identification, which prevents incorrect entries while maintaining user flexibility and control

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12406266B2Image-based produce recognition and verification

Publication Date: 2025.09.02 NCR VOYIX CORP

AI summary

An image of a candidate produce item is received during a transaction at a transaction terminal. A Feature Vector (FV) for the image is produced. Sales data associated with Produce Look Up (PLU) codes is obtained. Bayesian produce recognition engines are provided the FV and the corresponding sales data. Probabilities returned by the engines are evaluated and a pick list of produce items are produced and/or an entered PLU code provided by an operator of the terminal during the transaction for the candidate produce item is verified or identified as counterfeit.