Augmented Menu Overlay via Mobile Image Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Restaurant patrons face difficulties in obtaining real-time information about menu items, such as descriptions, reviews, and critiques, while at the establishment, as existing methods rely on printed menus or biased staff descriptions, and lack immediate access to dynamic content like special deals or user-generated reviews.
Innovation Solution
A method and apparatus using a mobile device to capture and parse menu images, transmitting parsed text and location information to a server, which returns annotations such as reviews, ratings, and images to be overlaid on the menu items in real-time, enabling augmented reality display of menu information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If printed menus or staff descriptions are used to provide menu information, then the system is simple and easy to operate, but real-time information access and dynamic content delivery are limited
Solution Approach 1:
The patent introduces a server as an intermediary between the mobile device and the menu information source. The server receives images captured by the mobile device, processes them to extract menu item information, and returns augmented information including reviews, ratings, and descriptions. This mediator enables real-time information access while keeping the mobile device relatively simple.
Solution Approach 2:
The patent replaces the mechanical system of printed menus and manual staff descriptions with an automated digital system. The mobile device captures images, and optical character recognition (OCR) technology automatically extracts text and menu item information, eliminating the need for physical menus and reducing reliance on staff for information delivery.
2Productivity
If real-time menu information access is implemented using mobile devices and image processing, then dynamic content delivery is improved, but the complexity of the system increases
Solution Approach 1:
The server performs preliminary processing of menu information by maintaining a database of menu items, descriptions, reviews, and ratings beforehand. When a mobile device captures a menu image, the server quickly matches the extracted text with pre-processed data, enabling fast information delivery without requiring complex real-time processing for each query.
Solution Approach 2:
The system uses image copying and pattern matching techniques. The mobile device captures an image of the menu, extracts text through OCR, and the server compares this extracted information with stored templates or databases of known menu items. This copying approach enables rapid identification and information retrieval without complex analysis.
3Loss of information
If augmented reality display with overlays is used to show menu information, then information accessibility is improved, but the ease of operation is reduced
Solution Approach 1:
The patent applies local quality by overlaying specific information (reviews, ratings, descriptions) only on or near the relevant menu items in the captured image. Each menu item receives targeted augmentation based on user selection or automatic detection, rather than displaying all information uniformly across the entire screen, making the interface more intuitive and easier to navigate.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
The subject matter disclosed herein relates to a method comprising displaying in a camera view of a mobile device a captured image of one or more items listed in a menu of items available for selection at a point of interest identifiable, at least in part, by a location. A method may further include transmitting a message comprising parsed text of the one or more items and information representative of the location, and receiving, in response to the transmission of the message, annotations to be displayed in the camera view.