Product Label Recognition With Fused Image And Text Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Customers are unable to analyze unique identification codes such as barcodes or QR codes without a dedicated reader device, and there is growing interest in using smartphones for product search or identification.
Innovation Solution
A method and electronic device that utilize a feature information encoder model to obtain and fuse image and text features from a product's label, matching the fused features against a database to provide product information, with the ability to update the model and database as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a dedicated reader device is used to analyze unique identification codes, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent applies universality by enabling smartphones to perform product identification functions that previously required dedicated reader devices. The system uses existing smartphone capabilities (camera, processor, display) to capture images of products, process them through AI models, and display identification results, thereby eliminating the need for separate dedicated reading devices while maintaining identification accuracy
Solution Approach 2:
The patent replaces traditional mechanical/optical barcode scanning systems with an AI-based image processing system. Instead of using dedicated optical readers to scan barcodes, the system captures product images with a smartphone camera and uses trained AI models to extract and analyze features, substituting mechanical scanning with intelligent image analysis
2Ease of operation
If smartphone-based product recognition is implemented, then ease of operation is improved, but measurement precision deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-training AI models with large datasets of product images and features before deployment. The system performs offline training to extract meaningful features from diverse product images, store them in databases, and prepare classification models in advance. This preliminary preparation ensures that when the smartphone actually performs product recognition, it can achieve high accuracy despite using a general-purpose device
Solution Approach 2:
The patent changes parameters by transforming product images into extracted feature representations that capture essential characteristics. The system converts raw pixel data into meaningful feature vectors through AI processing, adjusting the representation parameters to optimize both the simplicity of operation and the precision of identification by working with transformed rather than raw image data
Data Source
AI summary
A method and electronic device for recognizing a product are provided. The method includes obtaining first feature information and second feature information from an image related to a product, obtaining fusion feature information based on the first feature information and the second feature information by using a main encoder model that reflects a correlation between feature information of different modalities, matching the fusion feature information against a database of the product, and providing information about the product, based on a result of the matching.