Food inversion method based on multi-task learning and attention mechanism

A multi-task learning and attention technology, applied in the field of image recognition, can solve the problems of low calorie prediction accuracy, dependence on ingredients and cooking methods, etc.

Pending Publication Date: 2021-03-12
孙成林 +4
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The present invention designs and develops a food inversion method based on multi-task learning and attention mechanism. The purpose of the present invention is to solve the...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Food inversion method based on multi-task learning and attention mechanism
  • Food inversion method based on multi-task learning and attention mechanism
  • Food inversion method based on multi-task learning and attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0144] This system directly predicts the calorie value of a food image, without the need for the user to manually input other information, does not require the image shooting angle, etc., does not need to use special equipment such as depth cameras, and the user operation is more convenient.

[0145] Such as Figure 9 , Figure 10 As shown, the model adopts a multi-task convolutional neural network, and simultaneously learns four tasks of calorie prediction, food classification, ingredient prediction and recipe prediction during training, which effectively improves the accuracy of food classification and calorie prediction; the experimental results show that the present invention Compared with the single-task convolutional neural network calorie prediction model (correlation coefficient = 0.7217), the correlation coefficient of the proposed model (correlation coefficient = 0.7679) was improved by 0.0462.

[0146] The invention constructs a food data set, which contains 281 fi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a food inversion method based on multi-task learning and an attention mechanism, and the method comprises the following steps: 1, collecting food data, and constructing a recipe data set; 2, establishing and training a food material text model based on an attention mechanism, and obtaining a corresponding food material text by inputting a food picture; 3, establishing and training a menu generation model, and obtaining a menu text corresponding to the food picture by inputting the food picture and the food material text; 4, converting the food material text and the recipe text into corresponding food material vectors and recipe vectors, and establishing and training a multi-task convolutional neural network model; and inputting a to-be-detected food picture into themulti-task convolutional neural network model to obtain a food category, a calorie value, a food material vector and a recipe vector corresponding to the to-be-detected food picture.

Description

technical field [0001] The invention relates to the technical field of image recognition, in particular to a food inversion method based on multi-task learning and attention mechanism. Background technique [0002] In recent years we have witnessed many remarkable achievements in research on visual recognition tasks, including image classification, entity recognition, and image semantic segmentation. However, compared with general image recognition tasks, food image understanding faces a more difficult challenge, because food and its constituent ingredients undergo various cutting and cooking operations, resulting in changes in shape, morphology, texture, and color. There are various changes, and there are often mutual occlusions between different ingredients in the dishes. Thus, the challenges of food image analysis go beyond purely computer vision tasks. [0003] An early ingredient recognition model is PFD (Pairwise Partial Feature Distribution), which uses the results ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04G06N3/08G06K9/62
CPCG06N3/08G06N3/045G06F18/285
Inventor 孙成林白洪涛蔡芷薇何丽莉曹英晖
Owner 孙成林
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap