A food nutrition calculation method and system based on image recognition

Abstract

The application discloses a food nutrition calculation method and system based on image recognition, and the food nutrition calculation method comprises the following steps: collecting a target image by using a mobile terminal, wherein the target image comprises a display screen area of an electronic scale and a scale disc area of the electronic scale; performing image recognition on the target image, and extracting food weight information and food category information; calculating food nutrition component information based on the food weight information, the food category information and a preset nutrition database; and displaying the food weight information, the food category information and the food nutrition component information by using the mobile terminal. The application realizes intelligent calculation of nutrition components by using an ordinary electronic scale and a mobile terminal, reduces the hardware transformation cost and use threshold, and improves the user operation convenience and equipment compatibility.

Description

Technical Field

[0001] This invention relates to the field of food nutrition testing technology, specifically a food nutrition calculation method and system based on image recognition. Background Technology

[0002] With the increasing demand for healthy eating and precise nutrition management, rapid detection and real-time calculation of food nutrient components have become core requirements for scenarios such as daily dietary management, fitness and weight loss, and dietary control of chronic diseases.

[0003] Currently, the detection and display of food nutrient content mainly relies on digital nutrition scales, such as manually input digital nutrition scales for calculating food nutrient content. These scales have physical buttons and a list of food codes on the device. Users must first consult a food code manual and then manually input the corresponding food code using the buttons. The device then matches and displays the nutrient content based on a pre-set database. However, the cumbersome food code lookup process and complex manual input operation create a high barrier to entry for users, failing to meet the need for quick and convenient access to nutritional information.

[0004] Smart networked digital nutrition scales integrate wireless communication modules such as Bluetooth and Wi-Fi. By pairing with a mobile app, users select food items on the app, and the scale wirelessly transmits the weight data to the app. The app then calculates the nutritional components and sends the results back to the scale or the user's phone for display. Features include multi-ingredient accumulation and recipe saving. However, the addition of communication modules and control chips to the hardware of these scales increases costs. Furthermore, Bluetooth or Wi-Fi pairing complicates the user experience, limits compatibility and versatility, and makes them incompatible with existing household scales. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a food nutrition calculation method and system based on image recognition. It aims to achieve intelligent calculation of nutritional components through ordinary electronic scales and mobile terminals, reducing hardware modification costs and usage barriers, and improving user operation convenience and device compatibility.

[0006] This invention discloses a food nutrition calculation method based on image recognition, comprising: The target image is acquired using a mobile terminal, and the target image includes the display screen area and the weighing pan area of ​​the electronic scale. Image recognition is performed on the target image to extract food weight information and food type information; Based on the food weight information, the food type information, and a preset nutrition database, calculate the food nutritional composition information; The mobile terminal is used to display the food's weight information, food type information, and food nutritional information.

[0007] Preferably, acquiring the target image using a mobile terminal includes: The pre-configured shooting guide program in the mobile terminal is activated, and a shooting reference frame is overlaid on the viewfinder. In response to the user's shooting operation, a single shot is taken to capture the target image when both the display screen area and the weighing pan area of ​​the electronic scale are within the shooting reference frame.

[0008] Preferably, before performing image recognition on the target image, the method further includes: The target image is segmented, and based on edge detection and morphological operations, the display screen sub-image and the scale pan image are separated from the target image. The sub-image of the display screen and the image of the scale pan are preprocessed respectively. The preprocessing includes image enhancement, grayscale conversion and noise reduction.

[0009] Preferably, image recognition is performed on the target image to extract food weight information and food type information, including: Optical character recognition processing is performed on the sub-image of the display screen to extract the weight value and weight unit displayed in the display screen area, thereby obtaining the weight information of the food. The image of the weighing pan is input into a pre-trained food classification model, and the food type information corresponding to the image of the weighing pan is obtained based on image feature matching.

[0010] Preferably, after extracting the weight value and weight unit displayed in the display screen area, the method further includes: According to the preset unit conversion rules, the weight value is converted into a standard weight value in the standard weight unit; The standard weight value is verified. If the standard weight value is an abnormal value, a prompt message is displayed on the mobile terminal to prompt the user to re-acquire the target image.

[0011] Preferably, the image recognition-based food nutrition calculation method further includes: The food classification model is used to output the confidence score of the food category of the target food in the target image; If the confidence level of the food category is greater than or equal to a preset confidence threshold, the food type information is generated based on the food category. If the confidence level of the food category is less than a preset confidence threshold, the mobile terminal displays multiple candidate food categories to the user, and generates the food category information in response to the user selecting the target food category from the multiple candidate food categories.

[0012] Preferably, based on the food weight information, the food type information, and a preset nutrition database, the food nutritional information is calculated, including: Based on the food type information, the baseline data of the nutritional components of the target food per unit weight is queried from the preset nutrition database. Based on the food weight information and the nutritional component baseline data, the nutritional component information of the food is calculated, including total calories, protein content, fat content and carbohydrate content.

[0013] On the other hand, this invention discloses a food nutrition calculation system based on image recognition, used to execute a food nutrition calculation method based on image recognition. The food nutrition calculation system based on image recognition includes: The image acquisition module is configured to acquire a target image using a mobile terminal, the target image including the display screen area and the weighing pan area of ​​the electronic scale; The target recognition module is configured to: perform image recognition on the target image and extract food weight information and food type information; The nutrition calculation module is configured to calculate the nutritional composition information of the food based on the food weight information, the food type information, and a preset nutrition database. The result output module is configured to display the food weight information, food type information, and food nutritional information using the mobile terminal.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention eliminates the need for a dedicated smart nutrition scale. Users only need an existing electronic scale and a mobile device to quickly calculate the nutritional components of food. No hardware modifications to existing electronic scales are required, significantly reducing usage costs. It is also compatible with most common electronic scales, offering enhanced device compatibility. This invention extracts both food weight and type information simultaneously through image recognition. Information collection is completed with a single image capture, eliminating the need for manual input of food codes, step-by-step information entry, or Bluetooth / WiFi pairing. This simplifies the process, lowers the barrier to entry, and meets the need for quick and convenient access to food nutritional information in daily dietary management scenarios. By combining confidence level judgment with manual selection to identify food types, along with a weight anomaly verification mechanism, the accuracy of food nutrition calculation results is effectively improved, providing users with reliable nutritional data for reference. Attached Figure Description

[0015] Figure 1 A schematic flowchart of the food nutrition calculation method based on image recognition provided by the present invention; Figure 2 This is a schematic diagram of the structure of the food nutrition calculation system based on image recognition provided by the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] The present invention will now be described in further detail with reference to the accompanying drawings.

[0018] like Figure 1 As shown in the figure, this embodiment of the invention provides a food nutrition calculation method based on image recognition, including the following steps.

[0019] S1. Use a mobile terminal to collect target images.

[0020] In this embodiment of the invention, the electronic scale is a common household scale. It does not require an additional wireless communication module or intelligent control component; it only needs to be able to display the weight of food correctly to be compatible with this solution. When using the scale, the user places the food to be tested on the scale's pan. After the weight reading on the display stabilizes, the user activates the camera function of the corresponding application on the mobile terminal. The food to be tested can be grains, fruits and vegetables, meat, processed foods, etc. Only one type of food is on the scale pan at any given time. After enabling the pre-configured camera guidance program, a camera reference frame of the corresponding size is overlaid on the mobile terminal's viewfinder. The user simply adjusts the shooting angle and distance to ensure that both the display screen area and the scale pan area are completely within the camera reference frame, and presses the camera button to complete a single shot. This captures a target image that simultaneously includes both the display screen area and the scale pan area, eliminating the need for multiple separate shots to capture images of the two areas. Information collection is completed in one step, making the operation process much simpler.

[0021] Specifically, after the user opens the corresponding application on their mobile device and enters the food nutrition calculation function, the application automatically activates a pre-configured shooting guide program. A semi-transparent shooting reference frame is overlaid on the mobile device's camera viewfinder to prompt the user to adjust the shooting angle and position, ensuring the entire electronic scale is within the frame, and that both the scale's display area and the weighing pan containing the food to be tested are simultaneously and completely within the reference frame. Once the user confirms the framing is correct and triggers the shooting operation, the system, after detecting that the framing requirements are met, will directly execute a single shot, obtaining a target image that simultaneously includes both the electronic scale's display area and the weighing pan area, thus completing the target image acquisition.

[0022] S2. Perform image recognition on the target image to extract food weight information and food type information.

[0023] In this embodiment of the invention, after acquiring the target image, the target image is first processed by region segmentation, and based on edge detection and morphological operations, the display screen sub-image containing only the electronic scale reading and the scale plate image containing the target food are segmented and separated from the target image.

[0024] Specifically, in the process of region segmentation of the target image, the outer contour boundaries of the display screen and the weighing pan in the target image are first identified by the edge detection algorithm. Then, the noise and burrs of the contour edges are removed by combining morphological opening operations to obtain the accurate segmentation results of the two regions. Then, the display screen sub-image and the weighing pan image are cropped respectively.

[0025] Furthermore, preprocessing is performed on the segmented display screen sub-image and the scale pan image respectively. The preprocessing steps include contrast adjustment and enhancement processing, grayscale processing, and noise reduction processing to reduce the interference of image noise, uneven lighting, and other factors on the subsequent recognition results and improve the recognition accuracy.

[0026] Specifically, during image enhancement, a histogram equalization algorithm is used to adjust the image contrast, making the edges of the numbers on the display screen clearer and facilitating subsequent optical character recognition. Grayscale processing converts the three-channel color image into a single-channel grayscale image, reducing computation and improving recognition speed. Denoising processing uses Gaussian filtering to remove salt-and-pepper noise and Gaussian noise generated during image acquisition, further optimizing image quality.

[0027] In this embodiment of the invention, image recognition is performed on the target image to extract food weight information and food type information. Specifically, this includes: performing optical character recognition processing on the sub-image of the display screen to identify and extract the weight value and corresponding weight unit displayed in the display screen area, and combining them to obtain preliminary food weight information.

[0028] After obtaining the weight information, the identified weight value is converted into a standard weight value in the standard weight unit according to the preset unit conversion rules. Then, the standard weight value is verified. If the standard weight value is determined to be an abnormal value, such as zero or exceeding the reasonable weight range, a re-acquisition prompt message is immediately displayed on the mobile terminal, prompting the user to adjust the shooting position and re-acquire the target image to avoid abnormal data affecting the final calculation result.

[0029] Meanwhile, the image of the weighing pan is input into a pre-trained food classification model. The food classification model is a deep learning image classification model trained on a large number of publicly available food image datasets. It can extract the visual features of food based on the input weighing pan image, and output the recognition result and confidence level of the corresponding food category through feature matching, and finally obtain the corresponding food type information.

[0030] In the process of identifying food categories, this invention first determines whether the confidence level of the food category output by the food classification model meets the requirements: if the confidence level is greater than or equal to a preset confidence threshold, the identification result is directly used as the final food category information. If the confidence level is less than the preset confidence threshold, it indicates that the reliability of the model's identification result is insufficient. In this case, the mobile terminal will display multiple candidate food categories sorted by confidence level, allowing the user to manually select the matching target food category. Based on the user's selection, accurate food category information is then generated, ensuring both ease of operation and high identification accuracy.

[0031] S3. Calculate the nutritional information of the food based on the food weight information, the food type information, and a preset nutrition database.

[0032] In this embodiment of the invention, based on the food type information, the nutritional composition baseline data of the target food per unit weight is queried from the preset nutrition database. Based on the food weight information and the nutritional composition baseline data, the nutritional composition information of the food is calculated, including total calories, protein content, fat content, and carbohydrate content.

[0033] Specifically, the pre-stored nutrition database contains nutritional data per unit weight for various food categories, including grains, fruits and vegetables, meats, and processed foods. This includes the baseline content of core nutrients such as calories, protein, fat, and carbohydrates per 100 grams of edible portion. After obtaining the standard weight value and the specific food type, the system first retrieves the baseline nutritional data per unit weight of the edible portion of that food. Then, based on the standard weight value, it calculates the total calories, protein content, fat content, and carbohydrate content of the currently weighed food proportionally, thus completing the nutritional composition calculation.

[0034] S4. Display the food weight information, food type information, and food nutritional information using the mobile terminal.

[0035] In this embodiment of the invention, after calculating the food nutrient information, the mobile terminal application interface displays the collected food weight information, the identified food type information, and the finally calculated food nutrient information, making it convenient for users to quickly view the results and complete the food nutrient calculation process.

[0036] like Figure 2 As shown, this embodiment of the invention also provides a food nutrition calculation system based on image recognition, used to execute a food nutrition calculation method based on image recognition. The food nutrition calculation system based on image recognition includes an image acquisition module 201, a target recognition module 202, a nutrition calculation module 203, and a result output module 204.

[0037] The image acquisition module 201 is configured to acquire a target image using a mobile terminal. The target image includes the display screen area and the weighing pan area of ​​the electronic scale. The target recognition module 202 is configured to perform image recognition on the target image to extract food weight information and food type information. The nutrition calculation module 203 is configured to calculate food nutrient information based on the food weight information, the food type information, and a preset nutrition database. The result output module 204 is configured to display the food weight information, the food type information, and the food nutrient information using the mobile terminal.

[0038] As can be seen from the above technical solution, this application provides a food nutrition calculation method and system based on image recognition. The food nutrition calculation method includes: acquiring a target image using a mobile terminal, the target image including the display screen area and the weighing pan area of ​​an electronic scale; performing image recognition on the target image to extract food weight information and food type information; calculating food nutrient information based on the food weight information, food type information, and a preset nutrition database; and displaying the food weight information, food type information, and food nutrient information using the mobile terminal. This application achieves intelligent calculation of nutrient components through a common electronic scale and a mobile terminal, reducing hardware modification costs and the barrier to entry, and improving user convenience and device compatibility.

[0039] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A food nutrition calculation method based on image recognition, characterized in that, include: The target image is acquired using a mobile terminal, and the target image includes the display screen area and the weighing pan area of ​​the electronic scale. Image recognition is performed on the target image to extract food weight information and food type information; Based on the food weight information, the food type information, and a preset nutrition database, calculate the food nutritional composition information; The mobile terminal is used to display the food's weight information, food type information, and food nutritional information.

2. The food nutrition calculation method according to claim 1, characterized in that, The method of acquiring target images using a mobile terminal includes: The pre-configured shooting guide program in the mobile terminal is activated, and a shooting reference frame is overlaid on the viewfinder. In response to the user's shooting operation, a single shot is taken to capture the target image when both the display screen area and the weighing pan area of ​​the electronic scale are within the shooting reference frame.

3. The food nutrition calculation method according to claim 1, characterized in that, Before performing image recognition on the target image, the method further includes: The target image is segmented, and based on edge detection and morphological operations, the display screen sub-image and the scale pan image are separated from the target image. The sub-image of the display screen and the image of the scale pan are preprocessed respectively. The preprocessing includes image enhancement, grayscale conversion and noise reduction.

4. The food nutrition calculation method according to claim 3, characterized in that, The step of performing image recognition on the target image to extract food weight information and food type information includes: Optical character recognition processing is performed on the sub-image of the display screen to extract the weight value and weight unit displayed in the display screen area, thereby obtaining the weight information of the food. The image of the weighing pan is input into a pre-trained food classification model, and the food type information corresponding to the image of the weighing pan is obtained based on image feature matching.

5. The food nutrition calculation method according to claim 4, characterized in that, After extracting the weight value and weight unit displayed in the display area, the process also includes: According to the preset unit conversion rules, the weight value is converted into a standard weight value in the standard weight unit; The standard weight value is verified. If the standard weight value is an abnormal value, a prompt message is displayed on the mobile terminal to prompt the user to re-acquire the target image.

6. The food nutrition calculation method according to claim 4, characterized in that, Also includes: The food classification model is used to output the confidence score of the food category of the target food in the target image; If the confidence level of the food category is greater than or equal to a preset confidence threshold, the food type information is generated based on the food category. If the confidence level of the food category is less than a preset confidence threshold, the mobile terminal displays multiple candidate food categories to the user, and generates the food category information in response to the user selecting the target food category from the multiple candidate food categories.

7. The food nutrition calculation method according to claim 6, characterized in that, The step of calculating food nutritional information based on the food weight information, the food type information, and a preset nutrition database includes: Based on the food type information, the baseline data of the nutritional components of the target food per unit weight is queried from the preset nutrition database. Based on the food weight information and the nutritional component baseline data, the nutritional component information of the food is calculated, including total calories, protein content, fat content and carbohydrate content.

8. A food nutrition calculation system based on image recognition, used to execute the food nutrition calculation method based on image recognition as described in any one of claims 1-7, characterized in that, include: The image acquisition module is configured to acquire a target image using a mobile terminal, the target image including the display screen area and the weighing pan area of ​​the electronic scale; The target recognition module is configured to: perform image recognition on the target image and extract food weight information and food type information; The nutrition calculation module is configured to calculate the nutritional composition information of the food based on the food weight information, the food type information, and a preset nutrition database. The result output module is configured to display the food weight information, food type information, and food nutritional information using the mobile terminal.

Images