Joint Model for Food Recognition with Multi-Label Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current food recognition systems fail to accurately identify food items when users input aliases or images, as they cannot calculate total calorie or provide nutritional ingredients without knowing the exact food name, and existing products only provide unit calorie without context.
Innovation Solution
A joint model using a convolutional neural network with multiple sub-networks processes images to output category and size labels, allowing for the acquisition of food information such as name, calorie, and nutritional ingredients by querying a database based on these labels, reducing complexity and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single neural network is used for food recognition, then the network structure is simple, but it cannot output multiple labels (category and size) simultaneously, requiring separate models and increasing system complexity
Solution Approach 1:
The joint model is designed to perform multiple functions simultaneously - it outputs both category labels and size labels for food items in a single processing pass. The shared feature extraction layers serve both classification tasks, making the model universal for multi-label food recognition rather than requiring separate specialized models
Solution Approach 2:
The joint model is segmented into distinct functional components: shared feature extraction layers that process input images, a category classification branch, and a size classification branch. This segmentation allows each component to specialize in its function while maintaining overall system efficiency through shared representations
2Measurement precision
If multiple separate models are used for different food attributes, then each model can be optimized for its specific task, but the training data requirements increase significantly
Solution Approach 1:
The joint model merges the category classification and size classification tasks into a single unified model. By combining these tasks, the model can leverage the correlation between category and size information, allowing it to learn from a smaller dataset while maintaining or improving prediction accuracy for both attributes
3Loss of information
If existing food recognition systems are used, then they can provide basic food identification, but they cannot calculate total calorie or provide nutritional ingredients without knowing the exact food name
Solution Approach 1:
The joint model performs preliminary classification by predicting both category and size labels simultaneously before database lookup. This preliminary action provides sufficient information (category + size) to query the database for detailed nutritional data, enabling calorie calculation and nutritional ingredient provision without requiring exact food name matching
Data Source
AI summary
An object information processing method includes: acquiring an image of an object to be detected; inputting the image into a joint model to obtain a plurality of labels of the object to be detected; and obtaining object information of the object to be detected based on the plurality of labels. The joint model includes a convolutional neural network, the convolutional neural network includes a plurality of sub-networks. The plurality of sub-networks include an underlying processing network output of the underlying processing network and perform operation processing on the output of the underlying processing network to output a label, and labels output by different output networks are different.