Recipe OCR and LLM Ingredient Mapping for Shopping Cart Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face challenges in converting handwritten physical recipes into shopping carts due to the lack of a straightforward method for translating generic ingredients into specific products, leading to ambiguity and inconsistency across different recipes and stores, and the manual process is time-consuming and prone to errors.
Innovation Solution
Utilizing optical character recognition (OCR) and a large language model (LLM) to automatically extract ingredients from a physical recipe and map them to specific items available at retailers, populating a shopping cart with a user interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual entry of ingredients from physical recipes is used, then users can convert recipes to shopping lists, but the process is time-consuming and prone to errors
Solution Approach 1:
The patent replaces the mechanical manual entry process with an optical recognition system. The OCR technology captures images of physical recipes and automatically extracts ingredient text, eliminating the need for manual typing and significantly reducing conversion time while improving accuracy.
Solution Approach 2:
The system enables the recipe conversion process to serve itself by automatically extracting ingredients from captured images and mapping them to shopping list items without requiring user intervention in the data entry phase, thus improving productivity and reducing time loss.
2Ease of operation
If generic ingredient terms are mapped to specific products, then shopping lists can be created, but ambiguity and inconsistency arise due to variability in ingredient descriptions
Solution Approach 1:
The patent introduces an intermediary processing layer that includes text normalization and disambiguation algorithms. This intermediary system standardizes generic ingredient terms by comparing them against a database of known ingredients, resolving ambiguities and ensuring consistent mapping to specific products, thereby improving both ease of operation and reliability.
3Measurement precision
If automated OCR and language model processing is implemented, then recipe conversion accuracy improves, but system complexity increases
Solution Approach 1:
The patent segments the complex automated processing system into distinct functional modules: OCR text extraction module, text normalization module, language model processing module, and ingredient mapping module. This segmentation allows each component to be optimized independently while maintaining overall system accuracy, managing complexity through modular design.
Data Source
AI summary
Embodiments relate to utilizing an optical character recognition extraction and a large language model (LLM) to automatically populate a shopping cart of a user of an online system with items from a physical recipe. The online system receives an image capturing the physical recipe and extracts a raw text from the received image. The online system generates a prompt for input into the LLM, the prompt including a task request for the LLM to generate a list of ingredients using the raw text. The online system inputs the prompt into the LLM to generate the list of ingredients. The online system maps the list of ingredients to a list of items available by one or more retailers associated with the online system. The online system causes a device of the user to display a user interface with the list of items.