Food volume estimation method based on single depth map deep learning view synthesis

Abstract

The invention discloses a food volume estimation method based on single depth map deep learning view synthesis, and relates to the technical field of diet evaluation, and the method is characterized in that the method comprises the following steps: 1) placing each object item at an original point, and capturing object item depth images of different visual angles; 2) rendering the captured depth image into a training data set; 3) using the invisible view angle and the invisible object item to predict a result of using the input image by using the invisible view angle and the invisible object item based on a view synthesis method of the deep neural network; 4) obtaining a complete three-dimensional point cloud of the target object item; 5) preprocessing the depth image of the object item; 6) further optimizing the preprocessed point cloud by adopting an ICP (Inductively Coupled Plasma) algorithm; and 7) carrying out grid division on the object item by adopting an Alpha shape method and forming a three-dimensional grid so as to obtain the volume of the object item. Based on an integration method of a depth sensing technology and deep learning view synthesis, a single depth image can be obtained at any convenient angle, so that accurate food volume estimation is realized.

Description

technical field [0001] The present invention relates to the technical field of diet evaluation, and more specifically, it relates to a food volume estimation method based on deep learning view synthesis of a single depth map. Background technique [0002] An objective dietary assessment system can help users understand their eating behavior and enable targeted interventions to address underlying health problems. To accurately quantify dietary intake, the weight or volume of food needs to be measured. [0003] For volume estimation, previous studies mainly focus on using model-based or stereo-based methods, which rely on manual intervention, or require users to capture multiple frames from different viewpoints, and the operation method is relatively complicated; in addition, various methods based on Computer vision technology is used to solve the problem of quantifying food portions. Food volume measurement technology can be divided into two categories: model-based and stere...

AI Technical Summary

Problems solved by technology

[0003] For volume estimation, previous studies mainly focus on using model-based or stereo-based methods, which rely on manual intervention, or require users to capture multiple frames from different viewpoints, and the operation method is relatively complicated; in addition, various methods based on Computer vision technology is used to solve the problem of quantifying food portions. Food volume measurement technology can be divided into two categories: model-based and stereo-based. Although these technologies have shown good performance, they still have the following problems:

[0005] (2) Stereo-based methods require participants to take multiple food images from different perspectives, and the operation method is complicated;

[0006] (3) Other methods require feature point extraction and matching

For food objects with smooth surfaces or inconspicuous textures, feature points cannot be effectively extracted, resulting in failure of 3D reconstruction;

[0007] (4) When images are taken from different viewing angles, the reflected light of the object will change, which affects the accuracy of feature point matching and 3D reconstruction;

Images