Real-Time Food Recognition via Multi-Modal Video Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current nutrition tracking tools are difficult to use, requiring manual entry of food names and quantities, leading to user dissatisfaction and low adherence, as they lack real-time nutritional information capabilities.
Innovation Solution
A mobile device-based system that uses a food-recognition engine to identify foods in real-time through video processing, displaying nutritional information without the need for typing, by employing multiple modalities like visual recognition, barcodes, and Nutrition Facts, leveraging machine-learning algorithms on edge devices for fast and accurate food logging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual entry methods are used for food logging, then users can track nutrition, but user effort and time consumption increase significantly
Solution Approach 1:
The system performs automatic food recognition and nutrition calculation without requiring user input. The camera captures images of foods, the system automatically identifies them using image recognition algorithms, retrieves nutritional information from databases, and logs everything automatically, allowing the tracking system to serve itself rather than requiring manual user entry
Solution Approach 2:
The patent replaces the manual mechanical process of typing and selecting foods with an automated optical recognition system. Image recognition technology and computer vision algorithms substitute for manual data entry, automatically identifying foods from photographs and extracting nutritional information without user interaction
2Reliability
If manual entry methods are used for food logging, then nutrition tracking is possible, but time consumption increases
Solution Approach 1:
The system pre-loads a comprehensive database of food items and nutritional information before use. During actual logging, the pre-prepared database enables instant matching and identification of foods from images, eliminating the time required for manual searching and data entry during meal logging moments
Solution Approach 2:
Automated image recognition and computer vision algorithms replace manual food logging processes. The system automatically captures food images, identifies items through pattern recognition, and logs nutritional data instantly, reducing the time-consuming manual processes of typing, searching, and selecting foods
3Ease of operation
If real-time food recognition is implemented, then user engagement improves, but device complexity increases
Solution Approach 1:
The patent introduces a camera as an intermediary device between the user and the food logging system. Instead of requiring direct user input, the camera captures images that serve as intermediaries for automatic food identification, simplifying user interaction while managing system complexity through standardized image processing pipelines
4Productivity
If automated food recognition is used, then logging speed increases, but measurement precision requirements increase
Solution Approach 1:
The system performs more comprehensive image analysis than strictly necessary for basic identification. It processes multiple image features, considers various food attributes, and performs redundant verification steps to ensure high accuracy in food identification and portion size estimation, exceeding minimum requirements to maintain precision at high speeds
Data Source
AI summary
A food-recognition engine can be used with a mobile device to identify, in real-time, foods present in a video stream. To capture the video stream, a user points a camera of the mobile device at foods they are about to consume. The video stream is displayed, in real-time, on a screen of the mobile device. The food-recognition engine uses several neural networks to recognize, in the video stream, food features, text printed on packaging, bar codes, logos, and “Nutrition Facts” panels. The neural-network outputs are combined to identify foods with high probabilities. The foods may be packaged or unpackaged, branded or unbranded, and labeled or unlabeled, and may appear simultaneously within the view of the mobile device. Information about recognized foods is displayed on the screen while the video stream is captured. The user may log identified foods with a gesture and without typing.