Dietary nutritional ingredient calculation method and system based on image recognition

Abstract

The invention discloses a dietary nutritional ingredient calculation method and system based on image recognition. The method comprises the following steps: acquiring a to-be-identified dish image; identifying the dish image by using a VGGNet model integrated with an ECA attention mechanism and a bottleneck layer to obtain a dish name; according to the names of the dishes, matching and obtaining corresponding food material information and standard quantities in a recipe library; receiving an adjustment instruction of a user on the food material information or the standard weight, and generating an adjusted food material weight; and querying unit nutritional ingredients of each food material from a nutritional ingredient table according to the adjusted food material quantity, and calculating the total nutritional ingredients of the dish based on a preset formula. According to the method, the optimized deep learning model is introduced to automatically identify the dishes, and a user personalized adjustment function is combined, so that the problems of tedious manual input, low identification precision and poor flexibility in the prior art are solved, and the automation degree, the accuracy and the user experience of dietary nutritional ingredient calculation are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of diet management and the field of image processing, and more particularly, to a diet nutrition component calculation method and system based on image recognition, an electronic device and a storage medium. BACKGROUND

[0002] At present, with the increasing popularity of healthy diet concepts, people pay more and more attention to the nutritional components in diet, and thus seek more accurate and convenient diet management methods. The diet nutrition component calculation tools on the market mostly rely on user manual input of food material names, quantities and cooking methods. This method is not only inefficient, but also prone to inaccurate calculation results due to user input errors.

[0003] Some applications introduce a bar code scanning function, which is convenient for processing packaged food, but cannot solve the problem of nutrition calculation of user-made dishes or complex dishes. In order to make up for these deficiencies, image recognition technology has been applied to diet management tools. Users capture dish images, and the system automatically identifies dishes and estimates nutritional components. However, the existing image recognition technology still has great challenges in recognition accuracy and precision when dealing with dishes with complex ingredients and high mixing degree, and the recognition effect is not satisfactory. In addition, existing tools also generally face the problems of incomplete food material database, untimely updates and lack of personalized adjustment flexibility. Users cannot accurately adjust the food material quantities according to their own needs, resulting in a deviation between the calculation results and the actual intake, which reduces the practicability and user experience of the tool.

[0004] Therefore, how to provide a more efficient, accurate and personalized diet nutrition component calculation method has become a technical problem to be solved in the field. SUMMARY

[0005] The present application aims to provide a diet nutrition component calculation method and system based on image recognition to solve the technical problems of low calculation efficiency, poor accuracy and insufficient personalization of diet nutrition component calculation in the prior art.

[0006] The first aspect of the present application discloses a diet nutrition component calculation method based on image recognition; the method comprises:

[0007] Step S1: obtaining a dish image to be recognized;

[0008] Step S2: using a pre-trained convolutional neural network model integrated with an efficient channel attention (ECA) mechanism and a bottleneck layer to recognize the dish image, and obtaining a dish name;

[0009] Step S3: according to the dish name, matching and obtaining corresponding food material information and standard quantities in a preset recipe library;

[0010] Step S4: receiving a user's adjustment instruction on the food material information or standard serving size, and generating an adjusted food material serving size;

[0011] Step S5: querying the unit nutritional components of each food material from a nutritional component table according to the adjusted food material serving size, and calculating the total nutritional components of the dish.

[0012] Preferably, the convolutional neural network model is a VGGNet model.

[0013] The ECA attention mechanism is integrated after the convolutional layer of the VGGNet model, and is used to adaptively adjust the weight of each channel through one-dimensional convolution.

[0014] The bottleneck layer performs dimension reduction and dimension increase processing on the feature map through a combination structure of depth separable convolution and point-wise convolution.

[0015] Preferably, the step of matching the dish name in the preset recipe library comprises:

[0016] An exact matching algorithm is used to compare the dish name with the dish names in the recipe library; if the exact matching fails, a fuzzy matching algorithm based on cosine similarity is used to determine the most similar dish.

[0017] Preferably, the method further comprises:

[0018] Receiving a user-uploaded new dish image and corresponding food material information, and updating the new dish image and corresponding food material information to the recipe library.

[0019] Preferably, before using the convolutional neural network model to identify the dish image, the method further comprises:

[0020] Preprocessing the dish image, the preprocessing including size adjustment, data enhancement, and pixel value normalization.

[0021] Preferably, the output layer of the VGGNet model uses a Softmax function for classification, and a cross-entropy loss function and an Adam optimization algorithm are used for model training.

[0022] Preferably, in step S5, the calculation formula of the total nutritional components of the dish is as follows:

[0023]

[0024] wherein Y represents the total nutritional components, Y represents the total nutritional components of the dish, and usually includes heat, protein, fat, carbohydrate, etc.; n represents the total number of food materials in the dish; C irepresents the nutritional component of the i-th food material, usually the content of the nutritional component in 100 grams of the food material; W i represents the serving size of the i-th food material, usually in grams (g), provided by the user; i represents the specific food material number, ranging from 1 to n;

[0025] After the calculation is completed, the system will generate a detailed nutritional component report showing the content and proportion of each component; the report will list the nutritional component information of each food material and summarize the final nutritional component data.

[0026] The second aspect of the present application discloses a dietary nutritional component calculation system based on image recognition; the system comprises:

[0027] An image acquisition module configured to acquire a dish image to be recognized;

[0028] An image recognition module configured to utilize a pre-trained convolutional neural network model integrating an efficient channel attention mechanism and a bottleneck layer to recognize the dish image and obtain a dish name;

[0029] A recipe matching module configured to match and acquire corresponding food material information and standard serving size in a preset recipe library according to the dish name;

[0030] A user interaction module configured to receive user adjustment instructions for the food material information or standard serving size and generate adjusted food material serving size;

[0031] A nutrition calculation module configured to query the unit nutritional components of each food material from a nutritional component table according to the adjusted food material serving size and calculate the total nutritional component of the dish, wherein Y is the total nutritional component.

[0032] The third aspect of the present application discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps of any one of the dietary nutritional component calculation methods based on image recognition in the first aspect of the present application.

[0033] The fourth aspect of the present application discloses a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of any one of the dietary nutritional component calculation methods based on image recognition in the first aspect of the present application.

[0034] According to the technical content disclosed in the present application, the following beneficial effects are achieved:

[0035] The application provides a meal nutrition component calculation method based on image recognition, which can automatically and accurately identify dish images by adopting an optimized convolutional neural network model integrated with an ECA attention mechanism and a bottleneck layer.

[0036] Other features and advantages of the present application will become apparent from the following detailed description of illustrative embodiments thereof, which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0038] Figure 1 A flow chart of a meal nutrition component calculation method based on image recognition according to an embodiment is provided;

[0039] Figure 2 A system structure and principle block diagram provided by the embodiment of the application;

[0040] Figure 3 A VGGNet network structure schematic diagram optimized in the embodiment of the application;

[0041] Figure 4 A structure diagram of a meal nutrition component calculation system based on image recognition according to the embodiment of the application;

[0042] Figure 5 A structure diagram of an electronic device according to the embodiment of the application. DETAILED DESCRIPTION

[0043] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0044] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting to the scope of the application or its applications or uses.

[0045] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0046] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0047] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0048] Example 1:

[0049] Reference Figure 1 and Figure 2 This invention provides a method for calculating dietary nutrients based on image recognition. This method aims to improve the accuracy and efficiency of dietary nutrient calculation through intelligent technology. The overall concept is as follows: the user provides an image of a dish, the system identifies the dish using an optimized deep learning model, automatically matches ingredients and standard portion sizes from a database, allows for personalized adjustments by the user, and finally accurately calculates the total nutritional components based on the adjusted data.

[0050] This invention discloses a method for calculating dietary nutrient components based on image recognition. Figure 1 This is a flowchart of a dietary nutrient composition calculation method based on image recognition according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes:

[0051] Step S101: Obtain the image of the dish to be identified.

[0052] Users can take photos of dishes using their mobile devices or computer cameras, or upload an existing image of a dish from their local photo album. Before recognition, the system preprocesses the input dish image to meet the model's input requirements and improve model performance. Preprocessing steps typically include:

[0053] (1) Size adjustment: Adjust the input food images of any size to a fixed size, such as 224×224×3, to match the input layer requirements of the convolutional neural network model.

[0054] (2) Data augmentation: Perform geometric transformations such as random cropping, rotation, and horizontal flipping on the image to increase the diversity of training data, thereby improving the generalization ability of the model and making it more robust to food images under different angles and lighting conditions.

[0055] (3) Standardization: Pixel values ​​in the image are normalized, for example, by scaling the RGB value of each pixel from the [0,255] interval to the [0,1] interval, or by subtracting the mean pixel value of the dataset and dividing by the standard deviation. This helps accelerate convergence during model training. In the output layer, the Softmax function is used to normalize the classification results, outputting the probability distribution of each dish category. The predicted probabilities of each category are given. Loss function: Since it is a multi-class classification problem, the cross-entropy loss function is used. Optimization algorithm: The Adam optimization algorithm is used to update the weight parameters. Adam combines momentum and adaptive learning rate, which can effectively accelerate the training process.

[0056] Step S102: Use an optimized convolutional neural network model to identify the dish image and obtain the dish name. Specifically, use a pre-trained convolutional neural network model that integrates an efficient channel attention ECA mechanism and a bottleneck layer to identify the dish image and obtain the dish name;

[0057] In this embodiment, the convolutional neural network model used is an optimized version of the VGGNet architecture. VGGNet is a classic deep convolutional neural network, characterized by stacking multiple 3×3 small convolutional kernels to deepen the network layers, thereby effectively extracting multi-level features of the image.

[0058] The existing VGGNet network structure consists of 19 hidden layers (16 convolutional layers and 3 fully connected layers). VGGNet's structure is simple; all convolutional layers use 3"×"3 small convolutional kernels, ensuring the receptive field remains unchanged while increasing network depth, and avoiding the local information loss problem caused by large convolutional kernels sliding with high overlap. A ReLU activation function is connected after the convolutional layers to improve the network's expressive power and generalization ability. The pooling layers use 2"×"2 small convolutional kernels, halving the feature map size and doubling the number of channels. In the network parameter and structure table, D and E represent the commonly used VGG-16 and VGG-19. The former has 13 convolutional layers with a kernel size of 3×3, 5 max-pooling layers, and 3 fully connected layers, while the latter has 16 convolutional layers with a kernel size of 3×3, 5 max-pooling layers, and 3 fully connected layers.

[0059] To improve the performance and efficiency of VGGNet in dietary image recognition tasks, this invention makes two key improvements to its architecture, as shown in the following figure. Figure 3 As shown:

[0060] (1) Introducing the ECA (Efficient Channel Attention) mechanism. The ECA mechanism is an efficient channel attention method. In this embodiment, the ECA module is integrated into...

[0061] The VGGNet network is located after some or all of the convolutional layers. It captures local cross-channel interaction information through a non-dimensionality-reduced one-dimensional convolution, adaptively learning and adjusting the weights of each feature channel. This allows the network to focus more on feature channels that are crucial to food classification (such as channels that reflect food color and texture) and suppress irrelevant feature channels, thereby improving sensitivity to complex food image details and classification accuracy without significantly increasing model parameters and computational cost.

[0062] (2) Introducing the Bottleneck Layer. The bottleneck layer is introduced into the middle part of the network, and its structure typically adopts a "dimensionality reduction-convolution-dimensionality increase" pattern. Specifically, a 1×1 convolution is first used to reduce the dimensionality of the input feature map, then a 3×3 depthwise separable convolution is used for feature extraction, and finally a 1×1 convolution is used to restore the feature map to its original dimension. In this way, the bottleneck layer can significantly reduce the number of network parameters and computational cost while maintaining or even improving the feature extraction capability, thereby accelerating the model's training and inference process, making the model more lightweight, and suitable for deployment on resource-constrained devices such as mobile devices.

[0063] During the training phase, the optimized VGGNet model uses the Softmax function in its output layer to convert the network's output logits into a probability distribution corresponding to each dish category. During training, the cross-entropy loss function is used to measure the difference between the model's predicted probabilities and the true labels, and the Adam optimization algorithm is used to update the network weights. The Adam algorithm combines the advantages of momentum and adaptive learning rate, efficiently guiding the model to convergence. After training, the model can be used to recognize input dish images and output the name of the dish with the highest confidence.

[0064] Step S103: Based on the dish name, match and retrieve the corresponding ingredient information and standard serving size from the recipe library. Specifically, based on the dish name, match and retrieve the corresponding ingredient information and standard serving size from a preset recipe library;

[0065] The recipe library is a pre-built database that stores standardized names of numerous dishes, lists of required ingredients, standard serving sizes for each ingredient, and cooking methods. After identifying the dish name in step S102, the recipe matching module uses that name as a search keyword to match it within the recipe library.

[0066] The matching process can be divided into two steps:

[0067] (1) Exact match: First, try to match the identified dish name with the name in the recipe library exactly.

[0068] (2) Fuzzy matching: If exact matching fails (e.g., due to slight deviations in the recognition results or the dish having an alias), the system will use a fuzzy matching algorithm. In this embodiment, a cosine similarity-based algorithm can be used to convert the dish name into a word vector, and the most relevant dish in the recipe library can be found by calculating the cosine similarity between the vectors.

[0069] Once a match is successful, the system extracts the ingredient list and the standard portion size of each ingredient from the corresponding recipe entry (e.g., braised pork belly - 500g pork belly, 50g rock sugar, 30ml soy sauce, etc.).

[0070] Step S104: Receive the user's instruction to adjust the ingredient information or standard portion size, and generate the adjusted ingredient portion size.

[0071] Considering that actual cooking and consumption portions may differ from standard recipes, this invention includes a user interaction module. This module displays the ingredient list and standard portions obtained in step S103 to the user. Users can modify the portion size of each ingredient through an intuitive interface (e.g., adjusting the portion of pork belly from 500 grams to 400 grams), and can even add or delete ingredients. The system receives the user's adjustment instructions and generates a revised ingredient portion list reflecting the actual situation.

[0072] In addition, this user interaction module also supports users uploading new dishes not yet in the recipe library. Users can upload images of new dishes and manually enter their name, ingredients, and portion sizes. This new data, after review, can be added to the recipe library, thus enabling dynamic updates and continuous expansion of the database.

[0073] Step S105: Calculate the total nutritional content of the dish based on the adjusted ingredient quantities. Specifically, based on the adjusted ingredient quantities, look up the unit nutritional content of each ingredient from the nutrition facts table and calculate the total nutritional content of the dish.

[0074] The nutrition facts calculation module calculates the nutritional components based on the food list and quantities finally confirmed by the user in step S104. The system queries a separate nutrition facts database, which stores detailed nutritional information (such as calories, protein, fat, carbohydrates, vitamins, minerals, etc.) for each 100 grams of various foods.

[0075] For each ingredient in the list, the system obtains its unit nutrient content and performs a weighted sum based on its adjusted serving size; the total nutrient content is calculated according to the following preset formula:

[0076]

[0077] Where Y represents total nutrient content, which indicates the total nutrient content of the dish, typically including calories, protein, fat, carbohydrates, etc.

[0078] n represents the total number of ingredients in the dish;

[0079] C i This represents the nutritional composition of the i-th food ingredient, usually the content of that nutrient per 100 grams of the ingredient (such as calories, protein, fat, etc.).

[0080] W i This indicates the quantity of the i-th ingredient, usually in grams (g), provided by the user; i represents the specific ingredient number, ranging from 1 to n, that is, each ingredient in the dish.

[0081] Step S106: The system will generate a detailed nutritional information report, showing the content and proportion of each component. The report will list the nutritional information of each food ingredient and summarize it into the final nutritional data. Specifically, the system will perform the above calculations on all major nutrients (calories, protein, fat, carbohydrates, etc.) and finally generate a detailed nutritional information report, which will be presented to the user.

[0082] This invention, through a unique technical solution and innovative structural design, significantly improves the efficiency and accuracy of dietary nutrient composition calculation methods, demonstrating several superior benefits compared to existing technologies. Firstly, this invention makes a significant breakthrough in automated recognition and information extraction. By employing convolutional neural networks for automatic food image recognition, users only need to upload food images, and the system can accurately identify the food and extract relevant information. In optimizing the VGGNet architecture, an ECA attention mechanism and a bottleneck layer were introduced, effectively improving the network's performance and efficiency. This innovation not only solves the problems of tedious and error-prone manual input in traditional methods but also greatly reduces the workload of manual operation, improving recognition efficiency and accuracy.

[0083] Compared to existing technologies, this invention significantly improves image recognition accuracy. By employing deep learning technology, especially the feature extraction and classification capabilities of convolutional neural networks, the system can more accurately identify complex dishes and diverse ingredients. Existing technologies often suffer from recognition errors or blurriness when processing complex images or dishes with highly mixed components. This invention solves this problem through optimization of the deep learning model, enabling more accurate identification of different types of dishes and their components, ensuring the accuracy of subsequent nutritional component calculations.

[0084] Furthermore, the dynamically updated recipe database and nutrition facts table employed in this invention are among its outstanding advantages. By combining with the recipe database and nutrition facts table, the system can automatically obtain the ingredients and standard serving sizes of dishes, and extract detailed nutritional information for each ingredient from the nutrition facts table. This approach ensures that users not only obtain comprehensive nutritional data when acquiring dietary nutritional information, but also guarantee the accuracy and timely updates of the data. Compared to traditional methods that rely on static databases and incomplete information, this invention effectively solves the problems of incomplete and unupdable ingredient information, greatly improving the breadth and depth of the data.

[0085] In terms of adding functionality, this invention also has unique advantages. Through flexible user interaction functions, users can adjust the amount and content of each ingredient recommended by the system according to their individual needs, meeting the personalized dietary requirements of different users. Users can modify the ingredients and portions of dishes according to actual conditions, and the system will recalculate the nutritional components in real time based on the user's adjustments. This flexibility not only provides users with greater control, but also enables the system to adapt to various dietary preferences and health needs, enhancing the system's personalized service capabilities.

[0086] In addition, this invention also features new recipe upload and database update functions, allowing users to upload custom recipe images and related data. The system automatically updates the recipe library based on user input, ensuring continuous expansion and up-to-date database content. This function ensures the system can keep up with the latest recipe types and nutritional data in real time, greatly improving the system's usability and timeliness, and providing users with more comprehensive and real-time data support when obtaining dietary nutrition information.

[0087] This invention also achieves significant results in improving user experience. By simplifying the operation process, the system automates image recognition and information extraction. Users no longer need tedious manual input; they can quickly obtain nutritional information about dishes simply by uploading images. The operation is simple and quick, saving considerable time and effort. Simultaneously, the system effectively reduces errors during manual input, ensuring the accuracy and reliability of the calculation results. Users no longer need to worry about manually entering ingredients and quantities, and the reduction of data deviations caused by human error greatly enhances the system's credibility and user satisfaction.

[0088] In summary, this invention, through its automated and intelligent technical solutions, not only overcomes various shortcomings of existing technologies, increases system functionality and flexibility, reduces error rates, and simplifies operation processes, but also improves system stability, accuracy, and user experience. Compared to existing technologies, this invention provides a more efficient, accurate, convenient, and personalized method for calculating dietary nutrient components, with significant advantages and broad application prospects.

[0089] Example 2:

[0090] According to the embodiment, a method for calculating dietary nutrients based on image recognition is provided. Specific embodiments for the recognition of dietary images are given below:

[0091] Step S101: Obtain the image of the dish to be identified.

[0092] (1) Data preprocessing

[0093] Image input: Adjust the input food image to a fixed size, such as 224×224×3, which is a commonly used input size for VGGNet.

[0094] Data augmentation: Performing operations such as random cropping, rotation, and flipping on images increases the diversity of data, thereby improving the model's generalization ability.

[0095] Standardization: Normalize the pixel values ​​of an image, scaling the value of each pixel to the range of [0,1], or subtracting the mean of the data, to facilitate model training.

[0096] (2) Softmax layer, loss function and optimization

[0097] In the output layer, the Softmax function is used to normalize the classification results, outputting the probability distribution for each dish category. The predicted probabilities for each category are calculated. Loss function: Due to the multi-class classification problem, the cross-entropy loss function is used. Optimization algorithm: The Adam optimization algorithm is used to update the weight parameters. Adam combines momentum and adaptive learning rate, which can effectively accelerate the training process.

[0098] (3) Model training and testing

[0099] During training, the training set is divided into multiple batches, with each batch using a mini-batch of data. The gradient is calculated using backpropagation, and the network parameters are updated using an optimization algorithm. This process is repeated multiple times (e.g., 200 epochs) until the loss function converges.

[0100] During testing, an independent test set is used to evaluate the model, checking its accuracy and generalization ability. The output shows the model's classification results for different dish images, determining the final dish category.

[0101] Step S102: Use an optimized convolutional neural network model to identify the dish image and obtain the dish name.

[0102] The system uses VGGNet to identify the food categories in input images and matches them with recipes from a recipe database to obtain the corresponding ingredients and standard serving sizes. Based on the ingredients and standard serving sizes, it extracts relevant nutritional information from a nutrition facts label. For user interaction, users can modify the ingredient quantities according to their needs, and the system recalculates the nutritional composition based on the modified data.

[0103] Step S103: Based on the dish name, match and obtain the corresponding ingredient information and standard serving size from the recipe library.

[0104] In the dietary image recognition and nutrient composition prediction method based on VGGNet network, its main function is to automatically match the corresponding dishes in the recipe library according to the dish category identified by the image classification module, thereby extracting the ingredients, standard serving size and related nutritional information of the dishes, and providing necessary data support for subsequent nutrient composition prediction.

[0105] (1) Name matching

[0106] The first step in recipe matching is dish name matching. The image classification module uses VGGNet to identify the dish name or category label, and this information is used as input to directly compare with dish names in the recipe database. The recipe database typically contains standardized names, ingredients, and cooking methods for various dishes. Therefore, the output of the image classification module will perform exact or fuzzy matching with the dish names in the recipe database.

[0107] To improve matching accuracy, the system can employ various name matching techniques. For example, natural language processing (NLP) can be used to preprocess dish names, removing redundancy or synonyms to ensure accurate matching even with slight differences in dish names. For cases where precise matching is not possible, the system can also utilize fuzzy matching algorithms such as cosine similarity to ensure the fault tolerance of the matching process. Cosine similarity is a measure of the similarity between two vectors, calculated by taking the cosine of the angle between them. The cosine value ranges from -1 to 1, where 1 represents completely identical directions, 0 represents orthogonal directions (no similarity), and -1 represents completely opposite directions.

[0108] (2) Recipe Extraction

[0109] Once a dish name is successfully matched, the system extracts relevant information from the matched recipe. Recipes typically contain detailed ingredient lists, including the types, quantities, and standard serving sizes of all ingredients. This data is crucial for calculating dietary components. Based on the identified dish category, the system retrieves an ingredient list from the recipe database and extracts the standard serving size for each ingredient. This data includes not only the types of ingredients themselves but also the specific proportions or quantities of each ingredient, such as 500 grams of chicken and 200 grams of carrots.

[0110] In addition to ingredients and portion sizes, the recipe matching module also retrieves relevant cooking methods and precautions from the recipe. While this information is not directly related to the calculation of nutritional components, it is crucial for the system to conduct comprehensive dietary management and personalized recommendations.

[0111] Step S104: Receive the user's instruction to adjust the ingredient information or standard portion size, and generate the adjusted ingredient portion size.

[0112] Considering that actual cooking and consumption portions may differ from standard recipes, this invention includes a user interaction module. This module displays the ingredient list and standard portions obtained in step S103 to the user. Users can modify the portion size of each ingredient through an intuitive interface (e.g., adjusting the portion of pork belly from 500 grams to 400 grams), and can even add or delete ingredients. The system receives the user's adjustment instructions and generates a revised ingredient portion list reflecting the actual situation.

[0113] In addition, this user interaction module also supports users uploading new dishes not yet in the recipe library. Users can upload images of new dishes and manually enter their name, ingredients, and portion sizes. This new data, after review, can be added to the recipe library, thus enabling dynamic updates and continuous expansion of the database.

[0114] Data transmission with user interaction module

[0115] The ingredient and standard portion information extracted from the recipe library will be passed to the user interaction module. At this stage, users can personalize the ingredient quantities and contents. For example, if a user wants to reduce or increase the quantity of certain ingredients, the interaction module will allow the user to make modifications and provide feedback to the system. The system will then recalculate the nutritional composition of the dish based on the modified data and generate new dietary recommendations.

[0116] The recipe matching module, through close cooperation with the user interaction module, can flexibly meet users' dietary needs, making dietary image recognition and nutrient prediction not only an automated process, but also highly personalized and customizable, further enhancing the system's usability and user experience.

[0117] In summary, the recipe matching module is a key component in realizing dietary image recognition and nutritional composition prediction. Through accurate dish name matching and effective information extraction, it ensures the system's efficiency and accuracy, while also providing flexible user interaction functions to help users better manage their diet and control their nutrition.

[0118] Specifically as follows:

[0119] In the VGGNet-based dietary image recognition and nutrient composition prediction method, the user interaction module plays a crucial role in connecting recipe matching and nutrient composition calculation. The core function of this module is to provide an intuitive and user-friendly interface, allowing users to adjust dietary content, portion sizes, and ingredient ratios according to their individual needs. It also allows users to upload new dishes and update the recipe library, ensuring the system's flexibility and usability.

[0120] (1) Portion adjustment function

[0121] Based on the recipes recommended by the recipe matching module, users may adjust the portion sizes and ingredient proportions according to their needs. To facilitate this, the user interaction module provides a simple and intuitive interface. Users can easily modify the portion sizes using sliders, input boxes, or up and down arrow buttons. The system updates the ingredient ratios for each dish in real time based on user adjustments, ensuring that the modified dishes still meet the basic cooking requirements.

[0122] During this process, the system provides real-time feedback, displaying the modified dish and its updated ingredient values. After the user adjusts the ingredient quantities, the interaction module promptly transmits this information to the nutrition calculation module for subsequent nutrition prediction.

[0123] (2) New dish upload function

[0124] For dishes not yet included in the database, users can expand the recipe library by uploading new dish images and related ingredient information. This feature allows users to take photos or select existing dish images and upload them. The system will use image recognition technology, processing the images through a VGGNet network to identify the dish's name, main ingredients, and cooking method. Users also need to add detailed information about the dish, including the type and weight of ingredients and the amount of seasonings used during cooking.

[0125] Uploaded new dish information will be automatically parsed by the system and added to the recipe library for future use by the recipe matching and nutritional component calculation modules. To ensure accuracy, the system will manually review the new dishes after uploading to ensure data integrity and correctness.

[0126] (3) Personalized recipe recommendations and dynamic adjustments

[0127] The user interaction module also offers a personalized recipe recommendation function. Based on the user's health goals (such as weight loss, muscle gain, blood sugar control, etc.), the system can intelligently recommend suitable dishes. Users can input their health preferences or special needs (such as low salt, sugar-free, etc.) in the system settings, and the system will dynamically adjust the recommended recipes based on these needs.

[0128] In addition, the system allows users to save frequently used adjustments for easy reuse in the future. User-adjusted recipes are automatically synchronized to the database, ensuring that every user receives personalized and real-time updated dietary recommendations.

[0129] (4) Integration with the nutrient calculation module

[0130] The user interaction module works closely with the nutrition facts calculation module. When a user adjusts the portion size of a dish or uploads a new dish, the system transmits the modified data to the nutrition facts calculation module in real time. This module calculates the nutritional components of the dish based on the new ingredient quantities and proportions, including information such as calories, protein, fat, and carbohydrates. These calculation results are then fed back to the user interface, helping users understand the nutritional value of each dish and make appropriate adjustments based on their individual nutritional needs.

[0131] In summary, the user interaction module not only provides flexible menu adjustment functions but also supports the uploading of new dishes and database updates, ensuring the system's personalization and timeliness. Through its organic integration with image recognition, recipe matching, and nutrient calculation modules, users can more accurately control their diet and achieve scientific and healthy dietary management goals.

[0132] Step S105: Calculate the total nutritional content of the dish based on the adjusted ingredient quantities.

[0133] The nutrition facts calculation module calculates the nutritional components based on the food list and quantities finally confirmed by the user in step S104. The system queries a separate nutrition facts database, which stores detailed nutritional information (such as calories, protein, fat, carbohydrates, vitamins, minerals, etc.) for each 100 grams of various foods.

[0134] For each ingredient in the list, the system obtains its unit nutrient content and performs a weighted sum based on its adjusted serving size; the total nutrient content is calculated according to the following preset formula:

[0135]

[0136] Where Y represents total nutrient content, which indicates the total nutrient content of the dish, typically including calories, protein, fat, carbohydrates, etc.

[0137] n represents the total number of ingredients in the dish;

[0138] C i This represents the nutritional composition of the i-th food ingredient, usually the content of that nutrient per 100 grams of the ingredient (such as calories, protein, fat, etc.).

[0139] W i This indicates the quantity of the i-th ingredient, usually in grams (g), provided by the user; i represents the specific ingredient number, ranging from 1 to n, that is, each ingredient in the dish.

[0140] The system will perform the above calculations on all major nutrients (calories, protein, fat, carbohydrates, etc.) and finally generate a detailed nutrition report, which will be presented to the user.

[0141] In the dietary image recognition and nutrient composition prediction method based on the VGGNet network, the core task of the nutrient composition calculation module is to calculate the dietary nutrient composition of the dish based on the ingredient information identified by the image classification module and the ingredient quantities provided by the user interaction module. The following is the implementation process and detailed calculation formulas for this module.

[0142] (1) Obtain nutritional information

[0143] The nutrition facts calculation module first needs to extract the nutritional data for each ingredient from a database containing comprehensive information on ingredients. This data includes calories (energy), protein, fat, carbohydrates, vitamins, and minerals, all expressed per 100 grams of ingredient. The system automatically searches for and extracts detailed information for each ingredient, summarizing it based on the type and quantity of the ingredient.

[0144] (2) Calculate nutrient composition

[0145] After obtaining the basic nutritional components of the ingredients, the system will calculate the total nutritional components of the dish based on the portion sizes entered by the user. The nutritional components of each ingredient will be weighted according to its actual portion size. The calculation formula is as follows:

[0146]

[0147] Where Y represents total nutrients, which indicates the total nutritional composition of the dish, typically including calories, protein, fat, carbohydrates, etc.

[0148] n represents the total number of ingredients in the dish.

[0149] Ci represents the nutritional composition of the i-th food ingredient, usually the content of that nutrient per 100 grams of food (such as calories, protein, fat, etc.).

[0150] Wi represents the quantity of the i-th ingredient, usually in grams (g), provided by the user.

[0151] i represents the specific ingredient number, ranging from 1 to n, which is each ingredient in the dish.

[0152] (3) Output results

[0153] After the calculations are complete, the system will generate a detailed nutritional report, showing the content and proportion of each component. The report will list the nutritional information for each food ingredient and summarize it into the final nutritional data.

[0154] 2.7 Summary

[0155] This invention combines VGGNet image recognition technology with a nutrient composition calculation module to achieve nutrient composition prediction based on dietary images. By automatically retrieving nutritional information of ingredients from a database and combining it with the ingredient quantities provided by the user, it calculates the accurate total nutrient composition of the dish, providing users with detailed and personalized dietary advice to help them better manage their dietary health.

[0156] In summary, by combining advanced image recognition technology and dynamic database management, this invention achieves efficient and accurate calculation of dietary nutrients, providing a user-friendly solution that significantly improves user experience and calculation accuracy.

[0157] Example 3:

[0158] This invention discloses a dietary nutrient composition calculation system based on image recognition. Figure 4 This is a structural diagram of a dietary nutrient composition calculation system based on image recognition according to an embodiment of the present invention; as shown. Figure 4 As shown, the system 100 includes:

[0159] The image acquisition module 101 is configured to acquire an image of the dish to be identified;

[0160] The image recognition module 102 is configured to use a pre-trained convolutional neural network model that integrates an efficient channel attention mechanism and a bottleneck layer to recognize the dish image and obtain the dish name;

[0161] The recipe matching module 103 is configured to match and obtain the corresponding ingredient information and standard serving size from a preset recipe library based on the dish name.

[0162] User interaction module 104 is configured to receive user instructions to adjust the ingredient information or standard portion size, and generate the adjusted ingredient portion size.

[0163] The nutrition calculation module 105 is configured to query the unit nutrient content of each ingredient from the nutrition facts table based on the adjusted ingredient quantity, and calculate the total nutrient content of the dish, where Y is the total nutrient content.

[0164] Example 4:

[0165] This invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the image recognition-based dietary nutrient calculation method according to any one of the embodiments of this invention.

[0166] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 5 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0167] Those skilled in the art will understand that Figure 5 The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0168] Example 5:

[0169] This invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image recognition-based dietary nutrient calculation method according to any one of Embodiment 1 of this invention.

[0170] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0171] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0172] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0173] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0174] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0175] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0176] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0177] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0178] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0179] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A method for calculating dietary nutrient components based on image recognition, characterized in that, The method includes: Step S1: Obtain the image of the dish to be identified; Step S2: Use a pre-trained convolutional neural network model that integrates an efficient channel attention ECA mechanism and a bottleneck layer to identify the dish image and obtain the dish name; Step S3: Based on the name of the dish, match and obtain the corresponding ingredient information and standard serving size from the preset recipe library; Step S4: Receive the user's instruction to adjust the ingredient information or standard portion size, and generate the adjusted ingredient portion size; Step S5: Based on the adjusted ingredient quantities, look up the unit nutrient content of each ingredient from the nutrition facts table, and calculate the total nutrient content of the dish.

2. The method according to claim 1, characterized in that, The convolutional neural network model is the VGGNet model; The ECA attention mechanism is integrated after the convolutional layer of the VGGNet model and is used to adaptively adjust the weights of each channel through one-dimensional convolution. The bottleneck layer uses a combination of depthwise separable convolution and pointwise convolution to perform dimensionality reduction and dimensionality increase on the feature map.

3. The method according to claim 1, characterized in that, The steps for matching the dish name in a preset recipe library include: An exact matching algorithm is used to compare the dish name with the dish names in the recipe library; if the exact matching fails, a fuzzy matching algorithm based on cosine similarity is used to determine the most similar dish.

4. The method according to claim 1, characterized in that, The method further includes: Receive images of new dishes and corresponding ingredient information uploaded by users, and update the images of new dishes and corresponding ingredient information to the recipe library.

5. The method according to claim 1, characterized in that, Before using the convolutional neural network model to recognize the food image, the method further includes: The food image is preprocessed, including resizing, data augmentation, and pixel value normalization.

6. The method according to claim 2, characterized in that, The output layer of the VGGNet model uses the Softmax function for classification and employs the cross-entropy loss function and the Adam optimization algorithm for model training.

7. The method according to claim 1, characterized in that, In step S5, the formula for calculating the total nutritional content of the dish is as follows: Where Y represents total nutrient content, indicating the total nutritional composition of the dish, typically including calories, protein, fat, and carbohydrates; n represents the total number of ingredients in the dish; C i This represents the nutritional composition of the i-th food ingredient, typically expressed as the content of that nutrient per 100 grams of the ingredient; W i This indicates the quantity of the i-th ingredient, usually in grams (g), provided by the user; i represents the specific ingredient number, ranging from 1 to n. Once the calculations are complete, the system will generate a detailed nutritional report, showing the content and proportion of each component. The report will list the nutritional information of each food ingredient and summarize it into the final nutritional data.

8. A dietary nutrient composition calculation system based on image recognition, characterized in that, The system includes: The image acquisition module is configured to acquire images of the dishes to be identified; The image recognition module is configured to use a pre-trained convolutional neural network model that integrates an efficient channel attention mechanism and a bottleneck layer to recognize the dish image and obtain the dish name; The recipe matching module is configured to match and obtain the corresponding ingredient information and standard serving size from a preset recipe library based on the dish name; The user interaction module is configured to receive user instructions to adjust the ingredient information or standard portion size, and generate the adjusted ingredient portion size. The nutrition calculation module is configured to query the unit nutrient content of each ingredient from the nutrition facts table based on the adjusted ingredient quantities, and calculate the total nutrient content of the dish, where Y is the total nutrient content.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the dietary nutrient composition calculation method based on image recognition as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image recognition-based dietary nutrient calculation method according to any one of claims 1 to 7.

Images