OCR Recipe Extraction With LLM Ingredient-to-Cart Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face challenges in converting handwritten physical recipes into shopping lists due to the lack of a seamless method for translating generic ingredients into specific products, exacerbated by variability in ingredient descriptions and the absence of standardized product information across recipes and stores, leading to ambiguity and manual, error-prone entry processes.
Innovation Solution
An optical character recognition (OCR) system coupled with a large language model (LLM) is used to extract and translate ingredients from physical recipes into a shopping cart, mapping them to specific items available at retailers, thereby automating the conversion process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual entry of ingredients from physical recipes is used, then users can convert recipes to shopping lists, but the process is time-consuming and error-prone
Solution Approach 1:
The patent replaces manual mechanical entry with automated optical character recognition (OCR) technology to extract text from physical recipes, and further replaces manual interpretation with a language model to map ingredients to products. This substitution of manual operations with automated systems directly resolves the contradiction by dramatically improving conversion speed while eliminating manual time investment.
2Ease of operation
If generic ingredient terms are mapped to specific products, then shopping lists can be created, but variability in ingredient descriptions causes ambiguity and inconsistency
Solution Approach 1:
The patent introduces a language model as an intermediary between the generic ingredient terms extracted from recipes and the specific products in the online store catalog. This intermediary layer processes the ambiguous ingredient descriptions, resolves variations in terminology, and maps them to standardized product representations, thereby maintaining both ease of operation and mapping reliability.
3Ease of operation
If automated conversion from physical recipes to shopping carts is implemented, then manual effort is reduced, but infrastructure complexity increases due to need for OCR and language model integration
Solution Approach 1:
The patent merges multiple previously separate functions into a unified automated conversion system. The OCR component, language model processing, and product mapping are integrated into a single workflow that users can access through a simple interface. This merging allows the system to handle the complexity internally while presenting ease of operation to users, resolving the contradiction between automation benefits and infrastructure complexity.
Data Source
AI summary
Embodiments relate to utilizing an optical character recognition extraction and a large language model (LLM) to automatically populate a shopping cart of a user of an online system with items from a physical recipe. The online system receives an image capturing the physical recipe and extracts a raw text from the received image. The online system generates a prompt for input into the LLM, the prompt including a task request for the LLM to generate a list of ingredients using the raw text. The online system inputs the prompt into the LLM to generate the list of ingredients. The online system maps the list of ingredients to a list of items available by one or more retailers associated with the online system. The online system causes a device of the user to display a user interface with the list of items.