Health management method based on image recognition

Abstract

Provided in the present invention are a health management method and apparatus based on image recognition. The method comprises: S1, acquiring a food picture; S2, if there is no food labeling in the picture, extracting contour images of objects from the picture, matching the contour image of each object with pre-stored information of each food type, and determining a first target contour image and a food type therein; S3, binarizing target contour images; S4, inputting the binarized target contour images into a pre-trained three-dimensional reconstruction model, and outputting a three-dimensional point cloud; S5, calculating the volume of the three-dimensional point cloud on the basis of the three-dimensional point cloud under a world coordinate system, and determining the food volumes in the target contour images on the basis of the volume of the three-dimensional point cloud; and S6, on the basis of the volume-to-weight conversion ratio of the food volumes to food types corresponding thereto, obtaining the calories and nutritional ingredients of food in the food picture. The use by a user is more convenient, and the intake of the calories and nutritional ingredients in human bodies can be accurately recorded.

Description

A health management method based on image recognition Technical Field

[0001] The present invention relates to the technical field of food image data processing, and in particular to a health management method based on image recognition. Background Art

[0002] At present, with the improvement of living standards, more and more people are beginning to pay attention to healthy eating and healthy living, such as how to effectively control the calorie intake of food, and how to effectively control the nutritional intake of food to balance it with the human body's calorie consumption and nutritional needs in daily life, so that people can maintain a healthy weight without malnutrition.

[0003] However, most people currently have no specific concept of the calories and nutrients the body consumes daily. They generally rely on the guidance of professionals such as nutritionists and fitness coaches to roughly understand the calories and nutritional content of certain specific foods. However, for my country's rich catering food, various portions and types of food, people often cannot accurately and conveniently calculate their calories and nutritional content.

[0004] Currently, there are applications related to weight loss and health management that require users to manually input the types and weight of food consumed daily in order to analyze and obtain the body's daily calorie and nutrient intake. Multiple manual inputs cause inconvenience to users, and sometimes users are unable to accurately estimate the weight of the food, resulting in deviations in the recorded calories and nutrients, making it impossible for users to accurately grasp the body's daily calorie and nutrient intake.

[0005] To this end, a health management method based on image recognition is urgently needed.

[0006] Summary of the Invention

[0007] In view of the problems existing in the above technologies, the present invention solves them to at least a certain extent. To this end, the present invention provides a health management method based on image recognition, which makes it more convenient for users to use and can accurately record the calories and nutrients consumed by the human body.

[0008] In order to achieve the above objectives, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, the present invention provides a health management method based on image recognition, comprising:

[0010] Step S1: Obtain a food picture taken by a user and determine whether there is a food logo in the food picture;

[0011] Step S2: If no food identifier exists in the food image, extract the outline image of the objects in the food image, match the outline image of each object with the pre-stored food type information, and determine the first target outline image and the food type within the first target outline image;

[0012] Step S3, binarize the target contour image, set the food part to 1, and set the background part to 0;

[0013] Step S4: input the binarized target contour image into a pre-trained 3D reconstruction model to output a 3D point cloud; the 3D reconstruction model includes a feature encoder and a point cloud predictor connected in sequence, the feature encoder is composed of a convolutional layer and a ReLU layer, and the point cloud predictor is a fully connected network;

[0014] Step S5: calculating the volume of the three-dimensional point cloud according to the world coordinate system, and determining the volume of the food in the target contour image according to the volume of the three-dimensional point cloud;

[0015] Step S6: Obtain the weight of the food type in the food picture based on the volume of the food and the volume-to-weight conversion ratio of the corresponding food type; and obtain the calories and nutritional components of the food in the food picture based on the weight of the food type in the food picture.

[0016] Optionally, in step S2, a semantic segmentation network is used to extract contour images of objects in the food image.

[0017] Optionally, in step S2, the foreground contour and the background contour in the food picture are separated, and each contour image belonging to the foreground contour is matched with each pre-stored food type information to determine the first target contour image and the food type within the first target contour image.

[0018] Optionally, the contour image of each object is matched with each pre-stored food type information to determine the first target contour image and the food type within the first target contour image, including: matching the contour image of each object with each pre-stored food type information, and matching the object contour image and food type information with the highest matching rate to determine the first target contour image and the food type within the first target contour image.

[0019] Optionally, before step S1, the method further includes: receiving food images for training to generate training samples; inputting the training samples into the three-dimensional reconstruction model for model training based on the loss function;

[0020] The expression of the loss function is:

[0021] Where L is the loss function; i is the training sample; is the predicted 3D point cloud output by the 3D reconstruction model S under training sample i; is the real three-dimensional information of training sample i; is an unordered set of points; for 's mapping.

[0022] Optionally, the health management method based on image recognition also includes: step S7, if there is a food label in the food picture, reading the basic information of the food according to the food label, and obtaining the calories and nutritional components of the food in the food picture based on the basic information of the food.

[0023] Optionally, the basic information of the food includes weight information, calorie information and nutritional information of the food.

[0024] Optionally, the food label includes a food weight label, a food calorie label, a food nutrient label, a food additive content label and a food taste and texture label; the food weight label is displayed as a collection of weight attributes, numbers and measurement units; the food calorie label is displayed as comparison information between the food calories and the standard calories required by adults per day, and / or, comparison information between the food calories and the standard calories required by minors per day; the food nutrient label is displayed as comparison information between each nutrient component of the food and the corresponding standard nutrient component required by adults per day, and / or, comparison information between each nutrient component of the food and the corresponding standard nutrient component required by minors per day; the food additive content label is displayed as a collection of food additive type, numbers and measurement units; the food taste and texture label is displayed as comparison information between the taste of the food and the taste of the representative food, and comparison information between the taste of the food and the taste of the representative food.

[0025] Optionally, the food identification also includes a food code; by reading the food code, the origin of the food, the food growth process, environment, storage and sales process can be traced.

[0026] In a second aspect, the present invention provides a health management device based on image recognition, comprising:

[0027] The acquisition module is used to obtain the food pictures taken by the user and determine whether there is a food logo in the food picture;

[0028] a target contour image selection module, configured to extract contour images of objects in the food image if no food logo exists in the food image, match the contour image of each object with pre-stored food type information, and determine the target contour image and the food type within the target contour image;

[0029] The 3D reconstruction module is used to binarize the target outline image, setting the food portion to 1 and the background portion to 0. The binarized target outline image is input into a pre-trained 3D reconstruction model, which outputs a 3D point cloud. The 3D reconstruction model consists of a feature encoder and a point cloud predictor connected in sequence. The feature encoder consists of a convolutional layer and a ReLU layer, and the point cloud predictor is a fully connected network.

[0030] The first calorie and nutrient content determination module is used to calculate the volume of the three-dimensional point cloud based on the three-dimensional point cloud in the world coordinate system, and determine the volume of the food in the target contour image based on the volume of the three-dimensional point cloud; obtain the weight of the food type in the food image based on the volume-to-weight conversion ratio of the food volume and its corresponding food type; and obtain the calories and nutrient content of the food in the food image based on the weight of the food type in the food image.

[0031] The beneficial effects of the present invention are:

[0032] The image recognition-based health management method provided by the present invention can analyze the calorie content and nutritional content of food in food images using different methods depending on whether the food image contains food labels. In particular, when food labels are absent, the method accurately predicts the three-dimensional shape of the food through food contour extraction and three-dimensional modeling, and thus accurately calculates the calorie content and nutritional content of the food. Users no longer need to manually input the food type and weight; they only need to take a photo of the food, and the photo will be automatically analyzed to determine the food type, calorie content, and nutritional content, making it more convenient for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:

[0034] FIG1 is a flow chart of a health management method based on image recognition according to a specific embodiment of the present invention;

[0035] FIG2 is a schematic structural diagram of a health management device based on image recognition according to a specific embodiment of the present invention;

[0036] FIG3 is a schematic diagram of a food label according to a specific embodiment of the present invention.

[0037] DESCRIPTION OF REFERENCE NUMERALS 1 Acquisition module 2 Target contour image selection module 3 Three-dimensional reconstruction module 4 First calorie and nutrient component determination module 5 Second calorie and nutrient component determination module DETAILED DESCRIPTION

[0038] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0039] An embodiment of the present invention proposes a health management method based on image recognition. The method can analyze the calories and nutritional components of food in a food picture using different methods according to whether there is a food label in the food picture. The user does not need to manually enter the food type and food weight. The user only needs to take a photo of the food, and the food photo can be automatically analyzed to analyze the food type, food calories and food nutritional components, making it more convenient for users to use.

[0040] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0041] FIG1 is a flow chart of a health management method based on image recognition provided by the present invention. As shown in FIG1 , the health management method based on image recognition includes:

[0042] Step S1: Obtain a food picture taken by a user, and determine whether there is a food logo in the food picture.

[0043] Preferably, the food picture taken by the user contains only one type of food. This makes it easier for the subsequent image recognition method to identify the food in the picture and accurately analyze the type, calories and nutritional content of the food in the picture.

[0044] Step S2: If there is no food label in the food picture, extract the contour image of the object in the food picture, match the contour image of each object with each pre-stored food type information, and determine the first target contour image and the food type in the first target contour image.

[0045] Preferably, a semantic segmentation network is used to extract the outline images of objects in food images. Further preferably, the semantic segmentation network uses Deeplab v3+. In this way, the outline images of objects in food images can be accurately extracted.

[0046] Preferably, each pre-stored food type information is image information of a different food type, such as image information of bread (excluding container), image information of eggs (excluding container), and so on.

[0047] Preferably, in step S2, the foreground contour and the background contour in the food picture are separated, and each contour image belonging to the foreground contour is matched with each pre-stored food type information to determine the first target contour image and the food type within the first target contour image.

[0048] Preferably, the method of matching each object's outline image with each pre-stored piece of food type information to determine the first target outline image and the type of food within the first target outline image includes: matching each object's outline image with each pre-stored piece of food type information, matching the object outline image and food type information with the highest matching rate exceeding a preset threshold, and pairing the object outline image and food type information to determine the first target outline image and the type of food within the first target outline image. In this way, the food image in the food image can be extracted, facilitating subsequent 3D modeling based on the food image.

[0049] Further preferably, step S2 further includes performing shape matching on the contour image of each object with pre-stored shape information for each type of container to determine a second target contour image. This allows for the extraction of food containers from the image, and by combining the food image and the food container for analysis, more accurate caloric content and nutritional content of the food can be obtained.

[0050] Step S3: Binarize the first target contour image, set the food portion to 1, and set the background portion to 0.

[0051] Preferably, step S3 further comprises: binarizing the second target contour image, setting the container portion to 1 and the background portion to 0.

[0052] Step S4: input the binarized first target contour image into a pre-trained 3D reconstruction model and output a 3D point cloud; the 3D reconstruction model includes a feature encoder and a point cloud predictor connected in sequence, the feature encoder is composed of a convolutional layer and a ReLU layer, and the point cloud predictor is a fully connected network.

[0053] Through the feature encoder, each input image is encoded into a latent feature representation, and the point cloud predictor predicts the coordinate information of the 3D point cloud based on the input latent feature representation.

[0054] Preferably, the binarized first and second object contour images are input into a pre-trained 3D reconstruction model, which outputs a 3D point cloud of the first object contour image. In this way, the 3D modeling of the food is performed by combining the food image and the food container, resulting in a more accurate 3D point cloud.

[0055] Step S5: Calculate the volume of the three-dimensional point cloud according to the three-dimensional point cloud in the world coordinate system, and determine the volume of the food in the target contour image according to the volume of the three-dimensional point cloud.

[0056] Step S6: Obtain the weight of the food type in the food picture based on the volume of the food and the volume-to-weight conversion ratio of the corresponding food type; and obtain the calories and nutritional components of the food in the food picture based on the weight of the food type in the food picture.

[0057] Step S7: If there is a food identifier in the food picture, the basic information of the food is read according to the food identifier, and the calories and nutritional components of the food in the food picture are obtained according to the basic information of the food.

[0058] Specifically, the basic information of food includes the weight information, calorie information and nutritional information of the food.

[0059] Step S8: Based on the basic information of the food, obtain the additive content and taste information of the food in the food picture, and provide personalized suggestions to the user based on the calories, nutrients, additive content and taste information of the food in the food picture.

[0060] Preferably, before step S1, the method further includes: receiving food pictures for training to generate training samples; and inputting the training samples into a three-dimensional reconstruction model for model training based on a loss function.

[0061] The expression of the three-dimensional reconstruction model is: S = G(i, r, θ); wherein G is a neural network, r is a disturbance variable, and θ is a network parameter.

[0062] The expression of the loss function is:

[0063] Where L is the loss function; i is the training sample; is the predicted 3D point cloud output by the 3D reconstruction model S under training sample i; is the real three-dimensional information of training sample i; is an unordered set of points; for 's mapping.

[0064] The 3D reconstruction model trained under the above loss function has high stability and accuracy.

[0065] In summary, the image recognition-based health management method provided by the present invention can analyze the calories and nutritional content of food in food images using different methods depending on whether there are food labels in the food images. In particular, when there are no food labels in the food images, the three-dimensional shape of the food can be accurately predicted through food contour extraction and three-dimensional modeling, and the calories and nutritional content of the food can be accurately calculated. There is no need for the user to manually enter the food type and weight. The user only needs to take a photo of the food, and the food photo can be automatically analyzed to analyze the food type, food calories, and food nutritional content, making it more convenient for users to use.

[0066] The image recognition-based health management method provided by the present invention can accurately obtain the food type, calories and nutritional components in the picture, and recommend personalized and balanced meal plans based on personal needs, preferences and health status, helping people choose healthy food. It can also more comprehensively understand a person's eating habits and lifestyle choices over time, making the recommendations more precise and accurate.

[0067] Specifically, as shown in FIG3 , the food identification includes a food weight identification, a food calorie identification, a food nutrient component identification, a food additive content identification, and a food taste and texture identification.

[0068] The food weight label is displayed as a combination of weight attributes, numbers and measurement units to indicate the actual weight of the food. As an example, the food weight label is displayed as "net content 50 grams".

[0069] The food calorie label is displayed as a combination of calorie attributes, numbers and measurement units to represent the actual calorie content of the food. As an example, the food calorie label is displayed as "calories 1500 joules".

[0070] Preferably, the food calorie label is displayed as a comparison of the food calorie content with the standard calorie content required by an adult per day, and / or the food calorie content with the standard calorie content required by a minor per day. As an example, the standard calorie content required by an adult per day is 100. The calories contained in the food are compared with the standard calorie content required by an adult per day and are marked with percentages, numbers and graphics. This is concise and easy to understand, making it convenient for people to choose, so that people can accurately control their calorie intake and reduce the occurrence of obesity. In addition, the label is also easy for computers to recognize. Existing foods use digital tables to mark the calories contained in the food, which is not intuitive enough. Because there is no comparison object, consumers do not know whether they have consumed too many or too few calories, and cannot achieve the purpose of reasonable calorie intake. Through the label proposed by the present invention, consumers can easily know how much of the food they need to consume.

[0071] Nutrition Facts labels display a combination of nutrient type, number, and unit of measurement to indicate the actual amount of each nutrient in the food. For example, a nutrition fact label might display "Fat 8g, Protein 15g, Carbohydrate 20g, Vitamins 15mg, Sodium 10mg."

[0072] Preferably, the food nutrient label is displayed as comparison information of each nutrient component of the food with the corresponding standard nutrient components required by adults every day, and / or, comparison information of each nutrient component of the food with the corresponding standard nutrient components required by minors every day.

[0073] For example, the standard daily nutritional requirement for an adult is 100. Comparing the nutritional content of a food with this standard nutritional requirement and displaying it using percentages, numbers, and graphics is concise, easy to understand, and convenient for people to choose. This allows people to accurately control their nutritional intake and avoid overnutrition and malnutrition. Furthermore, the label is easily recognized by computers. Existing food labels use digital tables to display the nutritional content of the food, which is not intuitive. Without a comparison object, consumers cannot tell whether they are consuming too much or too little, thus failing to achieve the goal of proper nutritional intake. With the label proposed by the present invention, consumers can easily determine how much of a food they need to consume.

[0074] The content labeling of food additives displays the actual amount of each food additive in the food as a combination of the food additive type, a number, and a unit of measurement. Specifically, different graphic sizes and lengths are used to represent the relative value of the additive content, while the absolute value of the additive content is represented by a number. As an example, the content labeling of food additives may be displayed as "Antioxidant 150 mg, Food Coloring 135 mg, Preservative 100 mg, Leavening Agent 75 mg, Emulsifier 60 mg."

[0075] The food taste and texture label displays information comparing the taste of the food in question with that of a representative food, and also comparing the taste of the food in question with that of a representative food. Specifically, different sizes and lengths of graphics represent the relative value of the food's taste and texture, while numbers represent the absolute value of the food's taste and texture. For example, let's use the sweetness of food as an example. The representative food for sweetness is sucrose. The sweetness of a saturated aqueous solution of sucrose at 25°C is defined as 100 degrees, while the sweetness of pure water is defined as 0 degrees. The sweetness is then divided into 100 levels, each level being 1 degree. A unit of the food is extracted and dissolved in a certain amount of water, allowing all the sugar to dissolve. The sugar content of the food is then measured and compared to the sucrose sweetness level, resulting in a sweetness of 35 degrees. The greater the sweetness, the sweeter the food. This system can identify and indicate the taste and texture of food, providing consumers with an intuitive and concise basis for food selection.

[0076] Preferably, food labels also include an allergen label, which displays a combination of the allergen's properties and the allergen components in the food. For example, the allergen label might display "Allergen Peanuts." Food labeling provides allergy warnings and provides a basis for allergic reactions. For example, someone allergic to peanuts will see this label and know that the food is not suitable for them.

[0077] Preferably, food identification also includes a food code. By reading the food code, it is possible to trace the origin of the food, the food's growth process, the environment, storage, and sales process, forming a food chain. This allows tracking the entire process of food from farm to table. If a product recall is required, the manufacturer can use the food chain to accurately locate the affected batch, thereby reducing the waste and costs caused by large-scale recalls. Once the product is delivered, retailers and consumers can use the QR code to view key product information, making it easier for consumers to choose. Specifically, in this embodiment, the food code is displayed in the form of a QR code.

[0078] As shown in FIG2 , the present invention also provides a health management device based on image recognition, comprising: an acquisition module 1 for acquiring a food picture taken by a user and determining whether there is a food logo in the food picture; a target contour image selection module 2 for extracting the contour image of the object in the food picture if there is no food logo in the food picture, matching the contour image of each object with each pre-stored food type information, and determining the first target contour image and the food type in the first target contour image; a three-dimensional reconstruction module 3 for binarizing the target contour image, setting the food part to 1 and the background part to 0; and reconstructing the binarized target contour image. The contour image is input into a pre-trained three-dimensional reconstruction model, and a three-dimensional point cloud is output; the three-dimensional reconstruction model includes a feature encoder and a point cloud predictor connected in sequence, the feature encoder is composed of a convolutional layer and a ReLU layer, and the point cloud predictor is a fully connected network; the first calorie and nutrient content determination module 4 is used to calculate the volume of the three-dimensional point cloud based on the three-dimensional point cloud in the world coordinate system, and determine the volume of food in the target contour image based on the volume of the three-dimensional point cloud; according to the volume-to-weight conversion ratio of the food volume and its corresponding food type, the weight of the food type in the food image is obtained; according to the weight of the food type in the food image, the calories and nutritional content of the food in the food image are obtained.

[0079] Preferably, the health management device based on image recognition also includes: a second calorie and nutrient content determination module 5, which is used to read the basic information of the food according to the food label if there is a food label in the food picture, and obtain the calories and nutrient content of the food in the food picture based on the basic information of the food.

[0080] It should be noted that the specific functions of each module in the image recognition-based health management device provided by the present invention, as well as the image recognition-based health management process, can be referred to the detailed description of the image recognition-based health management method provided in the above embodiment, and will not be repeated here.

[0081] In summary, the image recognition-based health management device provided by the present invention can analyze the calorie content and nutritional content of food in food images using different methods depending on whether food labels are present. In particular, when food labels are absent, the device accurately predicts the three-dimensional shape of the food through food contour extraction and three-dimensional modeling, and thus accurately calculates the calorie content and nutritional content of the food. This eliminates the need for users to manually input food type and weight; users simply need to take a photo of the food, and the device automatically analyzes the food photo to determine the food type, calorie content, and nutritional content, making it more convenient for users.

[0082] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0083] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0084] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0085] In the description of this specification, the terms "one embodiment", "some embodiments", "embodiments", "examples", "specific examples" or "some examples" refer to the specific features, structures, materials or characteristics described in conjunction with the embodiment or example and included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0086] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A health management method based on image recognition, characterized in that: include: Step S1, obtaining a food picture taken by a user, and determining whether there is a food logo in the food picture; Step S2: if there is no food logo in the food picture, extract the contour image of the object in the food picture, match the contour image of each object with each pre-stored food type information, and determine the first target contour image and the food type in the first target contour image; Step S3, binarize the target contour image, set the food part to 1, and set the background part to 0; Step S4, inputting the binarized target contour image into a pre-trained 3D reconstruction model, and outputting a 3D point cloud; the 3D reconstruction model includes a feature encoder and a point cloud predictor connected in sequence, the feature encoder is composed of a convolutional layer and a ReLU layer, and the point cloud predictor is a fully connected network; Step S5, calculating the volume of the three-dimensional point cloud according to the three-dimensional point cloud in the world coordinate system, and determining the volume of the food in the target contour image according to the volume of the three-dimensional point cloud; Step S6, obtaining the weight of the food type in the food picture according to the volume of the food and the volume-to-weight conversion ratio of the corresponding food type; Get the calories and nutritional content of the food in the food picture based on the weight of the food type in the food picture.

2. The health management method based on image recognition according to claim 1, characterized in that: In step S2, a semantic segmentation network is used to extract contour images of objects in the food image.

3. The health management method based on image recognition according to claim 1 or 2, characterized in that: In step S2, The foreground contour and the background contour in the food picture are separated, and each contour image belonging to the foreground contour is matched with each pre-stored food type information to determine the first target contour image and the food type in the first target contour image.

4. The health management method based on image recognition according to claim 1, characterized in that: Matching the contour image of each object with each pre-stored food type information to determine the first target contour image and the food type in the first target contour image, including: The contour image of each object is matched with each pre-stored food type information. If there are object contour images and food type information whose matching rate exceeds a preset threshold, the object contour image and food type information with the highest matching rate are paired to determine the first target contour image and the food type within the first target contour image.

5. The health management method based on image recognition according to claim 1, characterized in that: Before step S1, the method further includes: receiving food images for training and generating training samples; and inputting the training samples into a three-dimensional reconstruction model for model training based on a loss function; The expression of the loss function is: Where L is the loss function; i is the training sample; is the predicted 3D point cloud output by the 3D reconstruction model S under training sample i; is the real three-dimensional information of training sample i; is an unordered set of points; for 's mapping.

6. The health management method based on image recognition according to claim 1, characterized in that: Also includes: Step S7: If there is a food label in the food picture, the basic information of the food is read according to the food label, and the calories and nutritional components of the food in the food picture are obtained according to the basic information of the food.

7. The health management method based on image recognition according to claim 6, characterized in that: The basic information of food includes the weight, calorie and nutritional information of the food.

8. The health management method based on image recognition according to claim 6, characterized in that: Food labels include food weight labels, food calorie labels, food nutrient content labels, food additive content labels and food taste and texture labels; Food weight labels are displayed as a collection of weight attributes, numbers, and units of measurement; The food calorie label displays information comparing the food calorie content with the standard daily calorie requirement for an adult, and / or, information comparing the food calorie content with the standard daily calorie requirement for a minor; The food nutrition label displays the comparison information of each nutrition component of the food with the corresponding standard nutrition component required by adults every day, and / or the comparison information of each nutrition component of the food with the corresponding standard nutrition component required by minors every day; The content of additives in food is displayed as a combination of food additive type, number and measurement unit; The food taste and texture identification is displayed as the comparison information between the taste of the food and the taste of the representative food, and the comparison information between the taste of the food and the taste of the representative food.

9. The health management method based on image recognition according to claim 8, characterized in that: Food identification also includes food codes; Read food codes and trace the origin of food, the growth process, environment, storage and sales of the food.

10. A health management device based on image recognition, characterized in that: include: An acquisition module (1) is used to acquire a food picture taken by a user and determine whether a food logo exists in the food picture; A target contour image selection module (2) is used to extract the contour image of the object in the food image if there is no food logo in the food image, match the contour image of each object with each pre-stored food type information, and determine the target contour image and the food type in the target contour image; The three-dimensional reconstruction module (3) is used to binarize the target contour image, set the food part to 1, and the background part to 0; input the binarized target contour image into the pre-trained In the trained 3D reconstruction model, a 3D point cloud is output; the 3D reconstruction model includes a feature encoder and a point cloud predictor connected in sequence, the feature encoder is composed of a convolutional layer and a ReLU layer, and the point cloud predictor is a fully connected network; The first calorie and nutrient content determination module (4) is used to calculate the volume of the three-dimensional point cloud according to the three-dimensional point cloud in the world coordinate system, determine the volume of the food in the target contour image according to the volume of the three-dimensional point cloud; and obtain the weight of the food type in the food image according to the volume-to-weight conversion ratio of the food volume and the corresponding food type; Get the calories and nutritional content of the food in the food picture based on the weight of the food type in the food picture.

Images