AR Virtual Order Assistant for Fast Casual Food Prep

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Fast casual restaurants face challenges in accurately preparing customized meals due to employee errors during peak hours, leading to incorrect orders and potential food safety issues, especially with customers having allergies.

Innovation Solution

A system utilizing natural language processing (NLP) and augmented reality (AR) to analyze customer orders, identify ingredient locations, and provide an optimized sequence for employees to select ingredients quickly and accurately, reducing human errors and ensuring order accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If employees manually prepare customized meals during peak hours, then customer service continues, but human errors increase leading to incorrect orders and food safety issues

Engineering Contradiction:
Improveorder accuracyVSAvoidmeal preparation speed
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

An AR-based virtual assistant system acts as an intermediary between the customer order and the employee preparation process. The system receives order data, processes it through NLP to identify ingredients and their locations, then provides augmented reality guidance to the employee, eliminating direct human error while maintaining rapid preparation.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical human decision-making and ingredient selection process with an automated AR-guided system. The system uses computer vision to locate ingredients, calculates optimal pickup sequences, and displays step-by-step instructions in the employee's field of view, substituting human cognitive processing with automated computational processes.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If employees quickly select ingredients during peak hours, then meal preparation speed increases, but errors in ingredient selection and cross-contamination risk increase

Engineering Contradiction:
Improvemeal preparation speedVSAvoidingredient selection accuracy
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The AR system provides real-time feedback to the employee during ingredient selection. It displays the current location, highlights target ingredients, shows the remaining pickup sequence, and confirms correct ingredient selection, allowing the employee to quickly select ingredients while receiving continuous verification to prevent errors and cross-contamination.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent uses color-coded visual indicators in the AR overlay to distinguish different ingredient types and their required sequence. Ingredients are highlighted with specific colors or visual markers that change based on their position in the pickup sequence, enabling rapid identification and selection while preventing cross-contamination through clear visual differentiation.

Inventive Principle:
Principle #32Color changes

3Adaptability or versatility

If the preparation area contains multiple ingredients, then meal customization options increase, but difficulty in identifying and locating specific ingredients increases

Engineering Contradiction:
Improvemeal customization optionsVSAvoidingredient location identification
Core Design Contradiction:
Adaptability or versatilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent adds a digital overlay dimension to the physical preparation area. The AR system superimposes virtual information (ingredient locations, pickup sequences, visual markers) onto the real-world view, creating a layered interface that provides precise location identification without adding physical clutter to the preparation space.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system creates a digital copy or representation of the preparation area layout and ingredient positions. This virtual map is displayed through the AR headset, showing the employee exactly where each ingredient is located and in what sequence to pick them up, eliminating the need to visually search through the physical arrangement of multiple ingredients.

Inventive Principle:
Principle #26Copying

4Measurement precision

If AR overlay with visual indicators is displayed to employee, then ingredient identification accuracy improves, but interface complexity increases

Engineering Contradiction:
Improveingredient location accuracyVSAvoidAR interface complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The AR interface is segmented into distinct functional elements: location markers for ingredients, sequence indicators, current position display, and confirmation signals. Each element serves a specific purpose and is spatially separated in the field of view, reducing cognitive load while providing comprehensive guidance for accurate ingredient identification and selection.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS11308432B2Augmented reality virtual order assistant
Publication Date: 2022.04.19 INTERNATIONAL BUSINESS MACHINE CORPORATION
  • US11308432B2 patent drawing
  • US11308432B2 patent drawing
  • US11308432B2 patent drawing

AI summary

An approach is provided for generating an interface. Using natural language processing and natural language understanding, ingredients are derived from audio data from a customer's spoken order of ingredients of a food item. Locations of the ingredients in a preparation area are identified. A location of an employee is determined. Distances of the ingredient locations to the employee location are calculated. A sequence of selecting the ingredients is determined so that the employee selecting the ingredients in the sequence optimizes a speed at which the food item is prepared. An overlay to an interface for viewing in an augmented reality (AR) headset worn by the employee is generated and displayed. The overlay includes indicators overlaying an image of the preparation area. The indicators mark the ingredients, mark the sequence of selecting the ingredients, and distinguish the ingredients from other ingredients in the preparation area.