Multi-mode accurate quantification nutritional diet management system

Abstract

The invention discloses a multi-modal accurate quantification nutritional diet management system, and the system comprises an image processing module which collects image data of dishes, carries out the preprocessing of the image data, carries out the self-adaptive division of the image data according to the types and numbers of the dishes, and extracts contour features, the fuzzy reasoning module recognizes the dishes after contour feature division and outputs recognition results and corresponding confidence coefficients, the weight distribution module calculates a weight ratio distribution proportion based on an image region area proportion and a confidence coefficient proportion of each dish, dynamic compensation calibration is carried out in combination with total weight data to obtain weight, and the weight distribution proportion is calculated according to the weight distribution proportion. The nutrition calculation module performs nutrition data correction according to the weight and a preset nutrition database to obtain nutritional ingredient data, and the intelligent decision-making module generates diet suggestions based on the nutritional ingredient data, the health target of the user and the dining scene. Therefore, the recognition precision and the real-time performance can be improved, the recognition and weight calculation precision of the mixed dishes can be improved, and meanwhile operation is simplified to adapt to various requirement scenes.

Description

Technical Field

[0001] This invention relates to the field of nutrition and diet management, specifically to a multimodal, precise, quantitative nutrition and diet management system. Background Technology

[0002] With the rapid rise in overweight and obesity rates among young people in my country and the high incidence of chronic diseases such as hypertension and diabetes, precise dietary management has become a key requirement for health management.

[0003] Currently, mobile input software relies on users to input the names of dishes, which can easily lead to a decrease in data reliability due to input errors. Image recognition tools are generally based on two-dimensional image recognition, which can easily overlook the depth of the plate and the volume misjudgment caused by the stacking of ingredients, especially for mixed dishes or dishes that are not identical, their recognition rate and ingredient analysis accuracy are both relatively low.

[0004] In addition, existing solutions lack a linkage mechanism between weight and recognition results, which may result in significant weight errors and thus fail to meet the needs of precise nutrition management. Furthermore, they only generate static diet plans based on fixed goals and cannot dynamically adjust suggestions based on scenarios such as meal times and before or after exercise. Moreover, they require taking photos and uploading them to the cloud for processing, resulting in significant end-to-end delays that disrupt the dining experience.

[0005] Finally, existing technologies often rely on cloud computing power, which can easily trigger network fluctuations or server outages during mealtimes, leading to functional paralysis. Furthermore, the high cost of model training makes it difficult for small and medium-sized enterprises to afford. Summary of the Invention

[0006] The present invention aims to at least partially solve the technical problems in the above-mentioned technologies.

[0007] Therefore, this invention discloses a multimodal precision quantitative nutrition and diet management system, comprising:

[0008] The image processing module is used to acquire image data of the dishes and preprocess the image data, and adaptively divide the image data according to the number of types of dishes to extract the contour features of each dish.

[0009] The fuzzy inference module is used to identify the dishes after the contour features are divided, so as to output the identification result and corresponding confidence level of each dish;

[0010] The weight allocation module is used to calculate the weight allocation ratio of each dish based on the area ratio of the image region of each dish and the confidence ratio, and to perform dynamic compensation calibration in combination with the total weight data of the dishes to obtain the weight of each dish.

[0011] The nutrition calculation module is used to correct the nutritional data based on the weight of each dish and a preset nutrition database, combined with the cooking method of the dish, so as to obtain the nutritional composition data of each dish.

[0012] The intelligent decision-making module is used to generate dietary recommendations based on the nutritional data, the user's health goals, and the dining scenario.

[0013] The multimodal precision quantitative nutrition diet management system disclosed in this invention can improve recognition accuracy and real-time performance, as well as the accuracy of recognition and weight calculation of mixed dishes, while simplifying operation to adapt to various demand scenarios.

[0014] In addition, the multimodal precision quantitative nutrition diet management system disclosed in this invention may also have the following additional technical features:

[0015] In one embodiment of the present invention, the image processing module includes, but is not limited to, a multi-channel camera and an electronic scale;

[0016] The multi-channel camera captures image data of the dish at a resolution of 1280×720, and preprocesses the image data to generate image data at a resolution of 640×480.

[0017] The electronic scale collects the total weight data of the dishes.

[0018] In one embodiment of the present invention, the image processing module adaptively divides the image data according to the following steps:

[0019] When a type of dish is detected, a full-screen scan is used;

[0020] When two types of dishes are detected, the image data is horizontally divided into two regions, left and right.

[0021] When three types of dishes are detected, the image data is divided into three regions: top, left, right, and bottom.

[0022] When four types of dishes are detected, the image data is divided into four regions: top left, bottom left, top right, and bottom right.

[0023] When five or more types of dishes are detected, the image data is divided into a grid for simultaneous detection of multiple dishes.

[0024] The extraction of the contour features adopts a multi-level screening strategy, with the screening conditions being: the area is greater than or equal to 0.5% of the total area of ​​the image data and the contour coefficient is 0.005 × contour perimeter.

[0025] In one embodiment of the present invention, the fuzzy inference module employs a test-time enhancement strategy to perform horizontal flipping, vertical flipping, brightness adjustment, and contrast adjustment enhancement processing on the image data, and averages the results of each inference as the final recognition result.

[0026] The identification result includes the name of the dish and the corresponding confidence level, using a Top 3 probability output method, and combined with a dynamic confidence level threshold of 0.6~0.7 to prevent false detections.

[0027] In one embodiment of the present invention, the weight distribution module performs dynamic compensation calibration according to the following steps:

[0028] If the error between the initial weight allocation of each dish and the total weight data exceeds 1 gram, the ratio of the total weight data to the initial weight allocation is used as a correction factor, and the initial weight allocation of each dish is multiplied by the correction factor to obtain the calibrated weight.

[0029] In one embodiment of the present invention, the nutritional data includes, but is not limited to: food calories, protein, fat and carbohydrate indicators, as well as cooking methods such as frying and stir-frying.

[0030] In one embodiment of the present invention, the intelligent decision-making module is based on a pre-trained DeepSeek model of scenario-based nutrition advice samples, which include devaluation scenarios, hypertension management scenarios, and post-exercise supplementation scenarios. The dietary advice includes nutritional balance assessment, targeted improvement suggestions, and scenario-based health tips.

[0031] In one embodiment of the present invention, in the weighting algorithm of the weight allocation module, the weight of the image region area ratio is 80%, and the weight of the confidence ratio is 20%.

[0032] Additional features and advantages of this invention will be set forth in the description which follows, or may be learned by practicing the invention. Attached Figure Description

[0033] The technical solution and beneficial effects of the present invention will become apparent and readily understood from the following description in conjunction with the accompanying drawings, wherein:

[0034] Figure 1 This is a flowchart of the multimodal precision quantitative nutrition diet management system of the present invention;

[0035] Figure 2 This is a rendering of the multimodal precision quantitative nutrition diet management system of the present invention. Detailed Implementation

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0037] The multimodal precision quantitative nutrition and diet management system disclosed in this invention will now be described with reference to the accompanying drawings.

[0038] like Figure 1 and Figure 2 As shown, the multimodal precision quantitative nutrition and diet management system includes:

[0039] The image processing module is used to acquire image data of the dishes and preprocess the image data, and adaptively divide the image data according to the number of types of dishes in order to extract the contour features of each dish.

[0040] The fuzzy inference module is used to identify dishes after their contour features have been segmented, and to output the identification result and corresponding confidence score for each dish.

[0041] The weight allocation module is used to calculate the weight allocation ratio of each dish based on the area ratio and confidence ratio of the image region of each dish, and to perform dynamic compensation calibration by combining the total weight data of the dishes to obtain the weight of each dish.

[0042] The nutrition calculation module is used to correct the nutritional data based on the weight of each dish and a preset nutrition database, combined with the cooking method of the dish, in order to obtain the nutritional composition data of each dish.

[0043] The intelligent decision-making module generates dietary recommendations based on nutritional data, the user's health goals, and the dining scenario.

[0044] Image processing modules, including but not limited to: multi-channel cameras and electronic scales;

[0045] Multiple cameras capture 1280×720 resolution image data of the dishes, and preprocess the image data to generate 640×480 resolution image data;

[0046] The electronic scale collects the total weight data of the dishes.

[0047] The image processing module adaptively partitions the image data according to the following steps:

[0048] When a type of dish is detected, a full-screen scan is used;

[0049] When two types of dishes are detected, the image data is horizontally divided into two regions, left and right.

[0050] When three types of dishes are detected, the image data is divided into three regions: top, left, right, and bottom.

[0051] When four types of dishes are detected, the image data is divided into four regions: top left, bottom left, top right, and bottom right.

[0052] When five or more types of dishes are detected, the image data is divided into a grid for simultaneous detection of multiple dishes.

[0053] The extraction of contour features adopts a multi-level screening strategy. The screening conditions are: the area is greater than or equal to 0.5% of the total area of ​​the image data and the contour coefficient is 0.005 × contour perimeter.

[0054] The fuzzy inference module employs a test-time enhancement strategy, which performs horizontal flipping, vertical flipping, brightness adjustment, and contrast adjustment on the image data, and averages the results of each inference as the final recognition result.

[0055] The identification results include the name of the dish and its corresponding confidence level. The Top 3 probability output method is used, and a dynamic confidence level threshold of 0.6~0.7 is used to detect false positives.

[0056] The weight distribution module performs dynamic compensation calibration according to the following steps:

[0057] If the error between the initial weight allocation of each dish and the total weight data exceeds 1 gram, the ratio of the total weight data to the initial weight allocation is used as a correction factor, and the initial weight allocation of each dish is multiplied by the correction factor to obtain the calibrated weight.

[0058] Nutritional data, including but not limited to: food calories, protein, fat and carbohydrate levels, as well as cooking methods such as frying and stir-frying.

[0059] The intelligent decision-making module is based on a pre-trained DeepSeek model using scenario-based nutrition advice samples, which include scenarios such as value reduction, hypertension management, and post-exercise supplementation. Dietary advice includes nutritional balance assessment, targeted improvement suggestions, and scenario-based health tips.

[0060] In the weighting algorithm of the weight allocation module, the weight of the image region area ratio is 80%, and the weight of the confidence ratio is 20%.

[0061] In one embodiment of the present invention:

[0062] The image processing module includes multiple cameras and an electronic scale, both of which are connected to the core processing unit. After the system starts, the multiple cameras initialize according to preset parameters to ensure that they can acquire food image data at a resolution of 1280×720.

[0063] The electronic scale simultaneously completes zero-point calibration to eliminate environmental interference and ensure weight acquisition accuracy. Afterwards, both the scale and the electronic scale enter standby mode, waiting to receive relevant acquisition signals from the dishes.

[0064] When a user places a plate containing food on the electronic scale, the scale collects the total weight of the plate and food in real time and transmits the data to the core processing unit.

[0065] At the same time, multiple cameras start image acquisition to obtain food image data at a resolution of 1280×720;

[0066] The image processing module adjusts the image resolution from 1280×720 to 640×480 to reduce the computational load of subsequent data processing. Then, through conventional image denoising and edge optimization, it provides a clear image for subsequent contour feature extraction and region segmentation.

[0067] The image processing module performs vegetable type detection on the preprocessed 640×480 resolution image and performs adaptive segmentation based on the number of detected vegetable types.

[0068] If one type of dish is detected, it will be classified using a full-screen scan.

[0069] If two types of dishes are detected, the image is horizontally divided into two regions, left and right.

[0070] If three types of dishes are detected, the image will be divided into three regions: top, left, right, and bottom.

[0071] In the contour feature extraction stage, a multi-level filtering strategy is adopted to retain only contours with an area greater than or equal to 0.5% of the total area of ​​the image data and a contour coefficient of 0.005 × contour perimeter, while filtering out invalid contours such as plate edges and food scraps, and finally obtaining the effective contours and corresponding image area of ​​each type of dish.

[0072] The fuzzy inference module identifies food images with valid outlines after segmentation. At the same time, it adopts a test-time enhancement strategy to enhance the image data by horizontal flipping, vertical flipping, brightness adjustment, and contrast adjustment. The average of each inference result is taken as the final recognition result.

[0073] The recognition results include the dish name and corresponding confidence level. The Top 3 probability output method is used, and false detection results are filtered by a dynamic confidence level threshold of 0.6~0.7. If the confidence level of a dish is lower than the threshold, it is determined to be invalid recognition. Only the dish name and confidence level data that meet the threshold requirements are retained for subsequent weight distribution calculation.

[0074] The weight allocation module calculates the weight of each type of dish based on two types of data:

[0075] First, the weighting is calculated using a weighted algorithm that assigns 80% weight to the area of ​​the image region and 20% weight to the confidence level, to determine the weight distribution ratio for each type of dish.

[0076] Next is the initial weight allocation. Combining the total weight data collected by the electronic scale, the total weight is multiplied by the weight allocation ratio of each type of dish to obtain the initial allocated weight of each type of dish.

[0077] Finally, dynamic compensation calibration is performed. The sum of the initial allocated weights of all dishes is calculated. If the error between the sum and the total weight data exceeds 1 gram, the ratio of the total weight data to the sum of the initial allocated weights is used as a correction factor. The initial allocated weight of each type of dish is multiplied by the correction factor to obtain the final weight after calibration.

[0078] The nutrition calculation module calls upon a preset nutrition database (containing indicators such as food calories, protein, fat, and carbohydrates), and corrects the nutritional data by combining the calibrated weight of each type of dish with the corresponding cooking method (such as deep-frying or stir-frying).

[0079] For fried dishes, adjust the calorie and fat indicators based on the impact coefficient of frying method on nutritional components in the database.

[0080] For stir-fried dishes, adjust the nutritional data according to the correction rules corresponding to the stir-frying method;

[0081] The same applies to other cooking methods, all of which are executed based on the database's preset correction logic;

[0082] Finally, we obtained specific nutritional data for each type of dish (including specific values ​​for calories, protein, fat, and carbohydrates).

[0083] The intelligent decision-making module calls a DeepSeek model pre-trained based on scenario-based nutrition advice samples (including depreciation scenarios, hypertension management scenarios, and post-exercise supplementation scenarios), and inputs the following three types of data:

[0084] Nutritional data for each type of dish, user-set health goals (such as weight loss, hypertension management), and current dining scenario (such as dining after exercise).

[0085] The model generates dietary recommendations based on the above data, which include three parts:

[0086] The first step is a nutritional balance assessment, which analyzes whether the nutritional components of the current meal meet the user's health goals and scenario needs;

[0087] Then come targeted improvement suggestions, proposing adjustment plans for nutritional imbalances (such as insufficient protein or excessive fat);

[0088] Finally, there are scenario-based health tips, which provide appropriate dietary supplements or taboos based on the dining scenario (such as after exercise).

[0089] In summary, the multimodal precision quantitative nutrition diet management system disclosed in this invention improves recognition accuracy and real-time performance, as well as the accuracy of recognition and weight calculation of mixed dishes, while simplifying operation to adapt to various demand scenarios.

[0090] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A multimodal, precise, quantitative nutrition and diet management system, characterized in that, include: The image processing module is used to acquire image data of the dishes and preprocess the image data, and adaptively divide the image data according to the number of types of dishes to extract the contour features of each dish. The fuzzy inference module is used to identify the dishes after the contour features are divided, so as to output the identification result and corresponding confidence level of each dish; The weight allocation module is used to calculate the weight allocation ratio of each dish based on the area ratio of the image region of each dish and the confidence ratio, and to perform dynamic compensation calibration in combination with the total weight data of the dishes to obtain the weight of each dish. The nutrition calculation module is used to correct the nutritional data based on the weight of each dish and a preset nutrition database, combined with the cooking method of the dish, so as to obtain the nutritional composition data of each dish. The intelligent decision-making module is used to generate dietary recommendations based on the nutritional data, the user's health goals, and the dining scenario.

2. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, The image processing module includes, but is not limited to: a multi-channel camera and an electronic scale; The multi-channel camera captures image data of the dish at a resolution of 1280×720, and preprocesses the image data to generate image data at a resolution of 640×480. The electronic scale collects the total weight data of the dishes.

3. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, The image processing module adaptively divides the image data according to the following steps: When a type of dish is detected, a full-screen scan is used; When two types of dishes are detected, the image data is horizontally divided into two regions, left and right. When three types of dishes are detected, the image data is divided into three regions: top, left, right, and bottom. When four types of dishes are detected, the image data is divided into four regions: top left, bottom left, top right, and bottom right. When five or more types of dishes are detected, the image data is divided into a grid for simultaneous detection of multiple dishes. The extraction of the contour features adopts a multi-level screening strategy, with the screening conditions being: the area is greater than or equal to 0.5% of the total area of ​​the image data and the contour coefficient is 0.005 × contour perimeter.

4. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, The fuzzy inference module employs a test-time enhancement strategy to perform horizontal flipping, vertical flipping, brightness adjustment, and contrast adjustment enhancement processing on the image data, and averages the results of each inference as the final recognition result. The identification result includes the name of the dish and the corresponding confidence level, using a Top 3 probability output method, and combined with a dynamic confidence level threshold of 0.6~0.7 to prevent false detections.

5. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, The weight distribution module performs dynamic compensation calibration according to the following steps: If the error between the initial weight allocation of each dish and the total weight data exceeds 1 gram, the ratio of the total weight data to the initial weight allocation is used as a correction factor, and the initial weight allocation of each dish is multiplied by the correction factor to obtain the calibrated weight.

6. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, The nutritional data includes, but is not limited to: food calories, protein, fat and carbohydrate levels, as well as cooking methods such as frying and stir-frying.

7. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, The intelligent decision-making module is based on a pre-trained DeepSeek model of scenario-based nutrition advice samples, which include scenarios of value reduction, hypertension management, and post-exercise supplementation. The dietary advice includes nutritional balance assessment, targeted improvement suggestions, and scenario-based health tips.

8. The multimodal precision quantitative nutrition and diet management system as described in claim 1, characterized in that, In the weighting algorithm of the weight allocation module, the weight of the image region area ratio is 80%, and the weight of the confidence ratio is 20%.

Images