Non-contact computer vision food nutrient identification method, system and device

Abstract

The present invention provides a non-contact computer vision food nutrient identification method, system and equipment, including: capturing and capturing meal images; identifying the meal images; determining the type of the meal in the first level classification based on the identified meal, recorded as the first type; determining the type of the meal in the second level classification based on the identified first type of the meal, recorded as the second type; and obtaining nutrient information of the meal based on the first type and the second type. The present invention can make detailed distinctions in food, and the identification of nutrients is more accurate. The present invention classifies and identifies food by main ingredients and side dishes, and can identify different cooking methods for the same dish. The present invention is particularly suitable for identifying different nutrients resulting from different cooking methods in different parts of my country.

Description

Technical Field

[0001] The present invention relates to the technical field of dietary nutrient recognition, and in particular to a non-contact computer vision food nutrient recognition method, system and device. Background Art

[0002] Nowadays, a balanced diet is gaining more and more attention in society. It is closely related to people's dining nutrition and life health. The proper nutrition provided by a balanced diet can meet the needs of human growth, development and various physiological and physical activities. In other words, a balanced diet is the foundation for ensuring health.

[0003] Due to my country's vast territory, different regions often cook the same dish differently. For example, some regions use more cooking oil, while others rely on local ingredients, which often contain higher levels of certain nutrients, such as sugar, than the same ingredients in other regions. Even porridge, such as white porridge, vegetable porridge, and beef porridge, has different nutrient contents, necessitating further refinement of food classification.

[0004] Patent document CN110517752A discloses a method for collecting dietary intake information in real time: a grid background image is laid out on a horizontal table in a well-lit area; the subject's usual intake of food is placed in standard tableware, which is then placed on the grid background image; before and after the meal, the subject's meal is photographed using a smart camera from four directions: directly above, from the side, from the front and from the top, and from the back and from the top; the subject's basic information and meal information are transmitted to professional backstage evaluators via the "Dietary Assistant" WeChat applet; after receiving the meal images, the backstage professionals estimate the subject's intake of various foods in the diet based on the meal images, food ingredients, and cooking methods, combined with pre-established food assessment reference maps, and use this information for subsequent dietary evaluation and guidance. A shortcoming of this patent document is that it cannot make detailed distinctions between foods. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the present invention aims to provide a non-contact computer vision food nutrient identification method, system and device.

[0006] According to the present invention, a non-contact computer vision food nutrient identification method is provided, comprising:

[0007] Step S1: photographing and collecting meal images;

[0008] Step S2: identifying the meal image;

[0009] Step S3: Determine the type of the meal in the first level classification based on the identified meal, and record it as the first type;

[0010] Step S4: determining the type of the meal in the second level classification based on the identified first type of the meal, which is recorded as the second type;

[0011] Step S5: Obtaining nutrient information of the meal according to the first type and the second type;

[0012] The first level classification distinguishes meals by main ingredients, the second level classification distinguishes meals by side ingredients, and the second level classification is a sub-classification of the first level classification.

[0013] Preferably, in step S1, a meal image is captured by a depth camera, wherein the meal image contains the meal and a container containing the meal;

[0014] In step S2, the meal image is recognized to obtain a container, and the meal is further recognized based on the container;

[0015] In step 3, the type of meal in the first level classification is identified by using a trained first neural network; wherein the training samples of the first neural network include meal image samples that distinguish meals by different main ingredients;

[0016] In step 4, the type of meal in the second level classification is identified by the trained second neural network; wherein each first type of meal has multiple different second type meal sub-categories.

[0017] Preferably, in step S2, the meal image is identified to obtain the volume of the meal;

[0018] In step S5, based on the volume of the meal, nutrient information of the meal is obtained according to the first type and the second type.

[0019] Preferably, the volume of the meal is obtained, comprising:

[0020] For the meal image, removing the point cloud of the container to obtain a point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the volume of the meal based on the grid;

[0021] For solid dishes, the overall volume of the dish is calculated based on the grid surface corresponding to the surface of the dish, and the volume of the dish is obtained by excluding the spaces between the food materials from the overall volume.

[0022] Among them, for dishes with solid-liquid mixed properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the solid in the meal and the volume of the liquid in the meal are obtained according to the set solid-liquid ratio as the volume of the meal.

[0023] A non-contact computer vision food nutrient recognition system provided by the present invention includes:

[0024] Module M1: capture and collect meal images;

[0025] Module M2: identifying the meal image;

[0026] Module M3: Based on the identified meal, determine the type of the meal in the first level classification, recorded as the first type;

[0027] Module M4: determining the type of the meal in the second level classification based on the identified first type of the meal, recorded as the second type;

[0028] Module M5: obtaining nutrient information of the meal according to the first type and the second type;

[0029] The first level classification distinguishes meals by main ingredients, the second level classification distinguishes meals by side ingredients, and the second level classification is a sub-classification of the first level classification.

[0030] Preferably, in the module M1, a meal image is captured by a depth camera, wherein the meal image contains the meal and a container containing the meal;

[0031] In the module M2, the meal image is recognized to obtain a container, and the meal is further recognized based on the container;

[0032] In step 3, the type of meal in the first level classification is identified by using a trained first neural network; wherein the training samples of the first neural network include meal image samples that distinguish meals by different main ingredients;

[0033] In step 4, the type of meal in the second level classification is identified by the trained second neural network; wherein each first type of meal has multiple different second type meal sub-categories.

[0034] Preferably, in the module M2, the meal image is recognized to obtain the volume of the meal;

[0035] In the module M5 , nutrient information of the meal is obtained based on the volume of the meal and the first type and the second type.

[0036] Preferably, the volume of the meal is obtained, comprising:

[0037] For the meal image, removing the point cloud of the container to obtain a point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the volume of the meal based on the grid;

[0038] For solid dishes, the overall volume of the dish is calculated based on the grid surface corresponding to the surface of the dish, and the volume of the dish is obtained by excluding the spaces between the food materials from the overall volume.

[0039] Among them, for dishes with solid-liquid mixed properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the solid in the meal and the volume of the liquid in the meal are obtained according to the set solid-liquid ratio as the volume of the meal.

[0040] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the non-contact computer vision food nutrient identification method are implemented.

[0041] According to the present invention, a dietary nutrition recognition terminal intelligent device includes a controller, and also includes a camera and a depth camera for collecting dietary images under the control of the controller;

[0042] The controller includes the non-contact computer vision food nutrient recognition system, or includes the computer-readable storage medium storing a computer program.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention can distinguish food in detail and identify nutrients more accurately.

[0045] 2. The present invention classifies and identifies food by main ingredients and side ingredients, and can identify different cooking methods of the same dish.

[0046] 3. The present invention is particularly suitable for identifying different nutrients resulting from different cooking methods in different parts of my country. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0048] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0049] Figure 2 Schematic diagram of the training process of the neural network of the present invention.

[0050] Figure 3 This is a schematic diagram of a two-dimensional information image of a meal and a container from a top-down perspective captured by the camera of the present invention.

[0051] Figure 4This is a schematic diagram of a three-dimensional information image of food and containers captured by the depth camera of the present invention.

[0052] Figure 5 Schematic diagram of the principle of meal volume calculation of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0054] According to the present invention, a non-contact computer vision food nutrient identification method is provided, comprising:

[0055] Step S1: photographing and collecting meal images;

[0056] Step S2: identifying the meal image;

[0057] Step S3: Determine the type of the meal in the first level classification based on the identified meal, and record it as the first type;

[0058] Step S4: determining the type of the meal in the second level classification based on the identified first type of the meal, which is recorded as the second type;

[0059] Step S5: Obtaining nutrient information of the meal according to the first type and the second type;

[0060] The first level classification distinguishes meals by main ingredients, the second level classification distinguishes meals by side ingredients, and the second level classification is a sub-classification of the first level classification.

[0061] Specifically, the main ingredients of different regional cuisines are usually the same, but the ingredients are usually different. For example, the main ingredients of scrambled eggs with tomatoes are tomatoes and eggs, but the ingredients of Xinjiang scrambled eggs with tomatoes are diced onions, and more cooking oil is used, while the ingredients of Shanghai scrambled eggs with tomatoes are chopped green onions, and the cooking oil is less and lighter. Another example is fried noodles. The main ingredients are noodles, but the ingredients of Xinjiang fried noodles are onions and shredded beef, a small amount of light soy sauce, while the ingredients of Shanghai fried noodles are mung bean sprouts and shredded pork, a large amount of light soy sauce and dark soy sauce. Taking porridge as an example, porridge is the first-level classification, and its subcategories include vegetable porridge, preserved egg porridge, fish porridge, beef porridge, etc., and the nutrient content is also different. Therefore, it is necessary to subdivide food.

[0062] The present invention can thus distinguish foods in detail and more accurately identify nutrients. It can also categorize and identify foods by main ingredients and side dishes, allowing it to identify different preparations of the same dish. This invention is particularly suitable for identifying the different nutrients resulting from the varying preparation methods of dishes from different regions of my country.

[0063] In step S1, a meal image is captured by a depth camera, wherein the meal image contains the meal and the container containing the meal; the meal image includes: a two-dimensional information image of a top-down perspective captured by a camera, and a three-dimensional information image captured by a depth camera, respectively. Figure 3 The two-dimensional information image shown, Figure 4 The three-dimensional information image shown. According to the two-dimensional information image, the dishes and containers of the meal can be identified by the trained neural network. During the training process, images of different combinations of dishes and containers are prepared as samples so that the neural network can identify the corresponding dishes and containers. The dishes of the meal are the names of the meals, that is, the names of the dishes, to distinguish different meals. For example, under the same volume, the protein content of stir-fried green peppers is less than that of roasted shredded pork, so different meals need to be distinguished to use the known information to obtain the nutrient content of the meal according to the volume of the meal. The attributes of the dishes of the meal include solid, solid-liquid mixed, and liquid, so that the calculation of the volume of the meal is more realistic.

[0064] For example, dishes can be scrambled eggs with tomatoes, stir-fried pork with green peppers, stewed beef with potatoes, stewed short ribs with beans, tofu and mushroom soup, and preserved egg and lean meat porridge. Among these dishes, the attributes of scrambled eggs with tomatoes and stir-fried pork with green peppers are classified as solid. Solid meals are mainly solid ingredients, with less soup. The surface of the soup is not visible from a top-down perspective because it is blocked by the solid ingredients. The attributes of stewed beef with potatoes and stewed short ribs with beans are classified as mixed solid-liquid meals. Mixed solid-liquid meals are mainly solid ingredients, with more soup. The surface of the soup is visible from a top-down perspective, but the surface area of ​​the soup is smaller than the surface area of ​​the solid ingredients. The attributes of tofu and mushroom soup and preserved egg and lean meat porridge are classified as liquid. Liquid meals are mainly liquid with solid as a supplement. The surface of the liquid is visible from a top-down perspective, and the surface area of ​​the liquid is larger than the surface area of ​​the solid ingredients.

[0065] For solid-type meals such as scrambled eggs with tomatoes and stir-fried pork with green peppers, there will be gaps between different ingredients and between the same ingredients. Eliminating these gaps can improve the accuracy of the calculation. For solid-liquid mixed-type meals such as stewed beef with potatoes and stewed pork ribs with beans, part of the beef, potatoes, beans, and pork ribs are submerged under the surface of the soup and are not visible, so it is necessary to estimate the volume of the solid ingredients under the surface of the soup. For liquid-type meals such as tofu and mushroom soup and preserved egg and lean meat porridge, the main component is soup, so the overall volume of the meal is calculated based on the surface area of ​​the soup.

[0066] A container containing food is placed on a support surface. The distance between the support surface and the camera is known, as is the distance between the support surface and the depth camera. Dimensional information such as the container's diameter and 3D model information such as its surface shape are also known. Therefore, after obtaining a 2D information image, the ratio between the container's diameter in the 2D information image and its actual diameter can be used to calculate the scale ratio for the actual size of the food in the 2D information image. In a preferred embodiment, the distance between the support surface and the camera is fixed, as is the distance between the support surface and the depth camera. The support surface, camera, and depth camera are assembled according to these fixed values. The 2D information image is a color image, showing the food, container, and support surface. The 3D information image also shows the food, container, and support surface. Accordingly, the learning samples for the deep learning neural network for recognizing the food and container also include the food, container, and support surface. The 3D information image contains point cloud data, and point clouds of the container and food are obtained based on the 3D information image. Specifically, it is the point cloud corresponding to the area of ​​the container surface not blocked by the meal from a top-down perspective, and the point cloud corresponding to the meal surface. Since the shape of the bottom of the meal is limited by the inner surface of the container used to hold the meal, the shape outline of the inner surface can be used as the shape outline of the bottom of the meal, while the shape of the top of the meal will change due to the operation of the person who serves the food. For example, even if it is the same dish, if the amount of food is different, the height of the top of the meal will be different. For example, with the same amount of food, if the person who serves the food is accustomed to spreading the food evenly or stacking it to form a pyramid, the shape of the top of the meal will also be different. Therefore, using a depth camera to collect the point cloud corresponding to the top of the meal to reflect its shape will help to accurately evaluate the subsequent nutrients.

[0067] In step S2, the meal image is identified to obtain a container, and the meal is further identified based on the container; the meal image is identified to obtain the volume of the meal; obtaining the volume of the meal includes: for the meal image, removing the point cloud of the container to obtain the point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the volume of the meal based on the grid; wherein, for dishes with solid attributes, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the meal is obtained by excluding the gaps between the foods from the overall volume; wherein, for dishes with solid-liquid mixed attributes, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the overall volume is obtained according to the set solid-liquid ratio to obtain the volume of the solid in the meal and the volume of the liquid in the meal as the volume of the meal.

[0068] Based on the meal image and known information, the point cloud of the container is removed to obtain a point cloud of the meal; the known information includes a 3D model of the container. To determine the shape of the top of the meal, the point cloud of the meal must be distinguished from the point cloud of the container. The container information is known, and since the distance between the depth camera and the container and its supporting surface is also known, the dimensions of the container in the 3D information image and the known 3D model of the container can be adjusted to a uniform scale. At this uniform scale, the point cloud in the 3D information image is overlaid with the 3D model of the container, where the outer edge of the container, such as the rim of a plate, is above the supporting surface. Based on this, the outer edge of the container in the 3D information image is identified. The outer edge of the 3D model of the container is aligned with the outer edge of the container in the 3D information image. The point cloud overlapping with the 3D model of the container is identified as the point cloud of the container, and the point cloud within the container point cloud is identified as the point cloud of the meal. In a variation, the point cloud of the container and the point cloud of the meal can also be distinguished by color, based on the known color of the container. To facilitate container identification, identification information, such as a QR code, can also be provided on the surface of the container.

[0069] For dishes with solid properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the meal is obtained by excluding the gaps between the food materials from the overall volume; for example, in the dish salt and pepper strips, the strips are long strips, and the strips are usually not arranged neatly in sequence, but overlapped and stacked with each other, with large gaps between the strips. If the gaps are also included in the volume of the meal and involved in the calculation and evaluation of nutrients, the evaluation results will be inaccurate, so the gaps need to be excluded.

[0070] For dishes with mixed solid-liquid properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the solid in the meal and the volume of the liquid in the meal are obtained according to the set solid-liquid ratio as the volume of the meal; for example, potatoes and beef brisket are stews, and the soup is obviously more than that of stir-fried dishes. For dishes with mixed solid-liquid properties, in addition to the potato cubes and beef brisket cubes visible on the surface of the soup, another part of the potato cubes and beef brisket cubes are immersed in the soup. Therefore, through the set solid-liquid ratio, such as the typical value, the volumes of solids and liquids in the meal can be obtained, thereby evaluating the nutrients in the solids and liquids respectively, and the weighted summation can be used to obtain the total nutrients.

[0071] The calculating the entire volume of the meal according to the grid surface corresponding to the surface of the meal comprises:

[0072] The mesh surface corresponding to the surface of the meal is enlarged to the actual size; based on the known distance between the supporting plane and the depth camera, the known height, diameter and other dimensions of the container, as described above, the scaling ratio can be obtained, and the mesh surface corresponding to the surface of the meal is enlarged to the actual size. According to the boundary line, the portion of the mesh surface of the inner surface of the container located below the boundary line is obtained. Specifically, in order to obtain the boundary line, the characteristic information of the container is obtained based on the meal image recognition, wherein the characteristic information can be, for example, a QR code, which is used to indicate a unique container or a unique container category of the same specification; then, based on the characteristic information of the container, the point cloud of the container indicated by the characteristic information is removed, and the mesh surface of the inner surface of the container of the actual size of the meal in the container indicated by the characteristic information is obtained; based on the point cloud of the container and the point cloud of the meal, the boundary line between the inner surface of the container and the meal is obtained, that is, the boundary line between the mesh surface of the inner surface of the container and the mesh surface of the meal surface. In a preferred embodiment, based on the distinction between the container point cloud and the meal point cloud, multiple coils coaxial with the circular container are set in the three-dimensional information image. The coils are in contact with the container point cloud and the meal point cloud. The number of container points and meal points on each coil is counted, and coils with equal numbers of both are used as the intersection line. The mesh surface of the actual-sized meal surface and the mesh surface of the container inner surface located below the intersection line are synthesized into a closed spatial curved surface; as shown in FIG. Figure 5 As shown, Figure 5 The diagram shows a mesh surface 100 of the meal surface, a mesh surface 200 of the container inner surface, a portion 300 of the mesh surface of the container inner surface below the boundary line, a boundary line 400, and a container 500. The volume of the inner space of the closed curved surface is taken as the volume of the meal.

[0073] In step 3, the type of meal in the first-level classification is identified using a trained first neural network, wherein the training samples of the first neural network include meal image samples that distinguish meals by different main ingredients. In step 4, the type of meal in the second-level classification is identified using a trained second neural network, wherein each first-type meal has multiple different second-type meal subcategories. Specifically, the present invention prepares two types of samples to train the first neural network and the second neural network, respectively, so that when identifying the first-level classification, the interference of ingredients on the recognition is reduced, and when identifying the second-level classification, the ingredients are distinguished more specifically.

[0074] In step S5, based on the volume of the meal, the nutrient information of the meal is obtained according to the first type and the second type. According to the volume of the meal, the weight of the meal is obtained, wherein the relationship between the unit volume of the meal and the nutrient can be expressed by the relationship between the unit weight of the meal and the nutrient, for example, the volume of the meal is multiplied by the nutrient content per unit volume to obtain the nutrient content of the meal, or the weight of the meal is multiplied by the nutrient content per unit mass to obtain the nutrient content of the meal. Then, the nutrient content is corrected according to the first type and the second type. For example, the first type is scrambled eggs with tomatoes, and the second type is scrambled eggs with tomatoes of Xinjiang cuisine. The lycopene content of Xinjiang tomatoes is higher, so the nutrient content needs to be corrected.

[0075] A non-contact computer vision food nutrient recognition system provided by the present invention includes:

[0076] Module M1: capture and collect meal images;

[0077] Module M2: identifying the meal image;

[0078] Module M3: Based on the identified meal, determine the type of the meal in the first level classification, recorded as the first type;

[0079] Module M4: determining the type of the meal in the second level classification based on the identified first type of the meal, recorded as the second type;

[0080] Module M5: obtaining nutrient information of the meal according to the first type and the second type;

[0081] The first level classification distinguishes meals by main ingredients, the second level classification distinguishes meals by side ingredients, and the second level classification is a sub-classification of the first level classification.

[0082] In the module M1, a meal image is captured by a depth camera, wherein the meal image contains the meal and a container containing the meal;

[0083] In the module M2, the meal image is recognized to obtain a container, and the meal is further recognized based on the container;

[0084] In step 3, the type of meal in the first level classification is identified by using a trained first neural network; wherein the training samples of the first neural network include meal image samples that distinguish meals by different main ingredients;

[0085] In step 4, the type of meal in the second level classification is identified by the trained second neural network; wherein each first type of meal has multiple different second type meal sub-categories.

[0086] In the module M2, the meal image is recognized to obtain the volume of the meal;

[0087] In the module M5 , nutrient information of the meal is obtained based on the volume of the meal and the first type and the second type.

[0088] The volume of the meal obtained includes:

[0089] For the meal image, removing the point cloud of the container to obtain a point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the volume of the meal based on the grid;

[0090] For solid dishes, the overall volume of the dish is calculated based on the grid surface corresponding to the surface of the dish, and the volume of the dish is obtained by excluding the spaces between the food materials from the overall volume.

[0091] Among them, for dishes with solid-liquid mixed properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the solid in the meal and the volume of the liquid in the meal are obtained according to the set solid-liquid ratio as the volume of the meal.

[0092] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the non-contact computer vision food nutrient identification method are implemented.

[0093] According to the present invention, a dietary nutrition recognition terminal intelligent device includes a controller, and also includes a camera and a depth camera for collecting dietary images under the control of the controller;

[0094] The controller includes the non-contact computer vision food nutrient recognition system, or includes the computer-readable storage medium storing a computer program.

[0095] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0096] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A non-contact computer vision food nutrient identification method, characterized in that: include: Step S1: photographing and collecting meal images; Step S2: identifying the meal image; Step S3: Determine the type of the meal in the first level classification based on the identified meal, and record it as the first type; Step S4: determining the type of the meal in the second level classification based on the identified first type of the meal, which is recorded as the second type; Step S5: Obtaining nutrient information of the meal according to the first type and the second type; The first level classification distinguishes meals by main ingredients, and the second level classification distinguishes meals by side ingredients. The second level classification is a sub-classification of the first level classification. By classifying and identifying food by main ingredients and side ingredients, different methods of cooking the same dish can be refined. In step S1, a meal image is captured by a depth camera, wherein the meal image contains the meal and a container containing the meal; In step S2, the meal image is recognized to obtain a container, and the meal is further recognized based on the container; In step S3, the type of meal in the first level classification is identified by using a trained first neural network; wherein the training samples of the first neural network include meal image samples that distinguish meals by different main ingredients; In step S4, the type of meal in the second level classification is identified by the trained second neural network; wherein each first type of meal has multiple different second type of meal sub-categories; In the step S2, the meal image is recognized to obtain the volume of the meal; In step S5, based on the volume of the meal, nutrient information of the meal is obtained according to the first type and the second type.

2. The non-contact computer vision food nutrient identification method according to claim 1, characterized in that: The volume of the meal obtained includes: For the meal image, removing the point cloud of the container to obtain a point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the volume of the meal based on the grid; For solid dishes, the overall volume of the dish is calculated based on the grid surface corresponding to the surface of the dish, and the volume of the dish is obtained by excluding the spaces between the food materials from the overall volume. Among them, for dishes with solid-liquid mixed properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the solid in the meal and the volume of the liquid in the meal are obtained according to the set solid-liquid ratio as the volume of the meal.

3. A non-contact computer vision food nutrient identification system, characterized in that: include: Module M1: capture and collect meal images; Module M2: identifying the meal image; Module M3: Based on the identified meal, determine the type of the meal in the first level classification, recorded as the first type; Module M4: determining the type of the meal in the second level classification based on the identified first type of the meal, recorded as the second type; Module M5: obtaining nutrient information of the meal according to the first type and the second type; The first level classification distinguishes meals by main ingredients, and the second level classification distinguishes meals by side ingredients. The second level classification is a sub-classification of the first level classification, classifying and identifying food by main ingredients and side ingredients, and refining it to different ways of cooking the same dish. In the module M1, a meal image is captured by a depth camera, wherein the meal image contains the meal and a container containing the meal; In the module M2, the meal image is recognized to obtain a container, and the meal is further recognized based on the container; In the module M3, the type of meal in the first level classification is identified by using a trained first neural network; wherein the training samples of the first neural network include meal image samples that distinguish meals by different main ingredients; In the module M4, the type of meal in the second level classification is identified by the trained second neural network; wherein each first type of meal has multiple different second type of meal sub-categories; In the module M2, the meal image is recognized to obtain the volume of the meal; In the module M5 , nutrient information of the meal is obtained based on the volume of the meal and the first type and the second type.

4. The non-contact computer vision food nutrient identification system according to claim 3, characterized in that: The volume of the meal obtained includes: For the meal image, removing the point cloud of the container to obtain a point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the volume of the meal based on the grid; For solid dishes, the overall volume of the dish is calculated based on the grid surface corresponding to the surface of the dish, and the volume of the dish is obtained by excluding the spaces between the food materials from the overall volume. Among them, for dishes with solid-liquid mixed properties, the overall volume of the meal is calculated based on the grid surface corresponding to the surface of the meal, and the volume of the solid in the meal and the volume of the liquid in the meal are obtained according to the set solid-liquid ratio as the volume of the meal.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the non-contact computer vision food nutrient identification method according to any one of claims 1 to 2 are implemented.

6. A dietary nutrition identification terminal intelligent device, characterized in that: It includes a controller, and also includes a camera and a depth camera for collecting meal images under the control of the controller; The controller includes a non-contact computer vision food nutrient identification system according to any one of claims 3 to 4, or includes a computer-readable storage medium storing a computer program according to claim 5.

Images