Food calorie calculation method and system based on image recognition

Abstract

The invention belongs to the technical field of image recognition, and discloses a food calorie calculation method and system based on image recognition, and the method comprises the steps: recognizing a food adhesion region, extracting the overall region feature of the food adhesion region, and deducing a food component combination contained in the adhesion region based on a mixed morphological feature library; and according to the deduced combination, a customized segmentation rule, an identification calibration parameter and a component calibration parameter are extracted, so that accurate segmentation, identification and component estimation of food components in the adhesion area are realized, the problems of morphological change and visual adhesion caused by food permeation or infiltration are effectively solved, and the accuracy of food calorie calculation is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and system for calculating food calories based on image recognition. Background Art

[0002] In the fields of modern health management and dietary tracking, the use of image processing technology to automatically identify food on a person's plate, estimate portion sizes, and subsequently calculate calorie intake has become a crucial technology. In particular, in centralized dining environments like cafeterias, deploying automated systems to capture images of food on a user's plate and calculate calories can provide users with convenient dietary information feedback. A typical system workflow involves: the user places the plate containing food in a designated image acquisition area; the system activates the image acquisition device to capture an image of the plate; the image is processed to identify the various food items on the plate and estimate their respective portion sizes; finally, a calorie database is consulted based on the food types and portion sizes to calculate the total calories.

[0003] However, in a real-world cafeteria setting, users often place different dishes within compartments or on a flat surface when placing food. Cafeterias offer a wide variety of dishes, including fluid or semi-fluid foods like soups, porridge, stews, and gravy dishes, as well as solid foods like rice, noodles, chunks of meat, and vegetables. Users may place fluid or semi-fluid foods adjacent to solid foods. Due to the physical properties of these foods, fluid or semi-fluid foods are prone to permeation or diffusion. For example, when brothy braised pork is placed adjacent to rice, the broth can seep into the rice, soaking the rice grains. This permeation or infiltration process alters the original shape, color, and texture of the solid food. Rice soaked in broth may become clumping, darkening in color, and blurring grain boundaries. This creates a visually continuous area between adjacent foods, with the boundaries no longer clearly defined in the image. This continuous area contains multiple food components, such as soaked rice and brothy braised pork. Due to permeation or infiltration, the food within the clumping area differs from its original, independent form. The original visual characteristics of food, such as texture, color, shape, etc., may be blurred or changed due to mutual penetration and mixing.

[0004] Existing image segmentation algorithms based on raw visual features of food (such as color, texture, edges, and shape) face significant challenges when processing such contiguous areas whose morphology has changed due to penetration or infiltration. They struggle to independently segment the individual food components within the area. The algorithm may treat the entire contiguous area as a single entity or produce a segmentation result that contains a mixture of multiple food components. For example, the system may be unable to distinguish between the areas of rice soaked in soup and the areas of braised pork. This inaccurate segmentation directly impacts the subsequent identification of the food components within the contiguous area. Even if rice and braised pork are identified, the infiltration and altered morphology of the rice may lead to misidentification of the food components or reduce recognition confidence. Furthermore, food portion estimation typically relies on the area of ​​the segmented area or volume estimation through morphological reconstruction. When food morphology changes due to infiltration, portion estimation methods based on area or morphology also face challenges. For example, the volume of rice soaked in soup may change, and its projected area on the image or the volume estimated by simple methods no longer accurately reflects its original portion size.

[0005] This inaccurate segmentation, identification, and portion estimation leads to deviations in the judgment of the types of food components and their respective portions in the adhesion area. Since food calorie calculation is based on the types of food and their corresponding portions, if the types and portions of the components in the adhesion area cannot be accurately assessed, the final calorie calculation result will be different from the actual calorie intake. This seriously affects the accuracy of the dietary management and calorie tracking data provided by the system, and limits its application value in the field of health management. Therefore, there is an urgent need for a technical solution that can effectively deal with the morphological changes and visual adhesion problems caused by food penetration or infiltration in the cafeteria plate environment, and realize accurate segmentation, identification and portion estimation of each food component in the adhesion area.

[0006] In view of the above problems, the existing technology is in urgent need of improvement. Summary of the Invention

[0007] The purpose of this application is to provide a food calorie calculation method and system based on image recognition, which can effectively deal with the problem of food adhesion and improve the accuracy of calorie calculation.

[0008] In a first aspect, the present application provides a food calorie calculation method based on image recognition, which is used to estimate the calories of food on a canteen plate. The method comprises the following steps:

[0009] A1. Obtain a food plate image, identify the food adhesion area, and extract regional features of the food adhesion area;

[0010] A2. Based on the regional features, feature matching is performed in a preset mixed morphological feature library to infer the food component combination contained in the food adhesion area;

[0011] A3. Based on the inferred food ingredient combination, the corresponding segmentation rules and identification calibration parameters are extracted from the preset segmentation rule library and the preset identification calibration parameter library;

[0012] A4. Segment the food adhesion region according to the extracted segmentation rules to separate the food component region inside the food adhesion region;

[0013] A5. Identify the food type of each food ingredient region and calibrate the food type identification result based on the extracted identification calibration parameters;

[0014] A6. Based on the calibrated food type recognition results, the corresponding component calibration parameters are extracted from the preset component calibration parameter library;

[0015] A7. Estimating the portion size of each food component region and calibrating the portion size estimation result according to the extracted portion calibration parameters;

[0016] A8. Calculate the total calories on the plate based on the calibrated food types and calibrated portion size estimates.

[0017] In a second aspect, the present application provides a food calorie calculation system based on image recognition, which is used to estimate the calories of food on a canteen plate. The system includes:

[0018] A feature acquisition module is used to acquire a food image of a plate, identify food adhesion areas therefrom, and extract regional features of the food adhesion areas;

[0019] a component inference module, configured to perform feature matching in a preset mixed morphology feature library based on the regional features, and infer the food component combination contained in the food adhesion region;

[0020] A first parameter acquisition module is used to extract corresponding segmentation rules and recognition calibration parameters from a preset segmentation rule library and a preset recognition calibration parameter library respectively according to the inferred food component combination;

[0021] a segmentation module, configured to segment the food adhesion region according to the extracted segmentation rules, and separate the food component region inside the food adhesion region;

[0022] an identification and calibration module, configured to identify the food type of each food component region and calibrate the food type identification result according to the extracted identification and calibration parameters;

[0023] A second parameter acquisition module is used to extract corresponding portion calibration parameters from a preset portion calibration parameter library according to the calibrated food type recognition result;

[0024] a portion calibration module, configured to estimate the portion size of each of the food component regions and calibrate the portion estimation results according to the extracted portion calibration parameters;

[0025] The calorie calculation module is used to calculate the total calories of the food on the plate based on the calibrated food types and the calibrated portion estimation results.

[0026] Beneficial effects: The present application provides a food calorie calculation method and system based on image recognition, which can effectively deal with the problem of food adhesion by identifying food adhesion areas, inferring ingredient combinations, and performing segmentation, identification and portion calibration, thereby having the advantage of improving the accuracy of calorie calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart of a method for calculating food calories based on image recognition provided in an embodiment of the present application.

[0028] Figure 2 A structural diagram of a food calorie calculation system based on image recognition provided in an embodiment of the present application.

[0029] Explanation of the reference numbers: 1. Feature acquisition module; 2. Component inference module; 3. First parameter acquisition module; 4. Segmentation module; 5. Identification and calibration module; 6. Second parameter acquisition module; 7. Quantity calibration module; 8. Calorie calculation module. DETAILED DESCRIPTION

[0030] The technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of this application.

[0031] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0032] refer to Figure 1 This application proposes a food calorie calculation method based on image recognition, which is used to estimate the calories of food on a canteen plate. The steps of this method include:

[0033] A1. Obtain a food plate image, identify the food adhesion area, and extract regional features of the food adhesion area;

[0034] A2. Based on the regional features, feature matching is performed in a preset mixed morphological feature library to infer the food component combination contained in the food adhesion area;

[0035] A3. Based on the inferred food ingredient combination, the corresponding segmentation rules and identification calibration parameters are extracted from the preset segmentation rule library and the preset identification calibration parameter library;

[0036] A4. Segment the food adhesion region according to the extracted segmentation rules to separate the food component region inside the food adhesion region;

[0037] A5. Identify the food type of each food ingredient region and calibrate the food type identification result based on the extracted identification calibration parameters;

[0038] A6. Based on the calibrated food type recognition results, the corresponding component calibration parameters are extracted from the preset component calibration parameter library;

[0039] A7. Estimating the portion size of each food component region and calibrating the portion size estimation result according to the extracted portion calibration parameters;

[0040] A8. Calculate the total calories on the plate based on the calibrated food types and calibrated portion size estimates.

[0041] Among them, the food adhesion area refers to the visually continuous and blurred boundary area in the food image of the plate due to the penetration or infiltration between foods, which contains multiple food components. It can be achieved by using technologies based on pixel visual continuity, color gradient analysis, texture change feature analysis and / or regional boundary fuzziness analysis. Its main purpose is to identify food areas that require special treatment.

[0042] Among them, regional features refer to data reflecting the overall visual attributes of the food adhesion area, such as color distribution, texture pattern and / or shape contour information, which can be implemented using technologies such as color histogram, local binary pattern (LBP) descriptor and / or shape moment, and are mainly used to quantitatively describe the visual performance of the food adhesion area.

[0043] Among them, the mixed morphological feature library refers to a data set that stores typical visual feature patterns of different food ingredient combinations in an adhesion state. It can be stored in the form of a database or a lookup table, where each entry contains a food ingredient combination and its corresponding regional feature pattern, which is mainly to provide a basis for inferring the internal components of the adhesion area.

[0044] Among them, feature matching refers to comparing the extracted regional features of the food adhesion area with the feature patterns in the mixed morphology feature library to find the most similar pattern. It can be achieved by using technologies such as Euclidean distance, cosine similarity or machine learning classifier. Its main purpose is to determine the similarity between the adhesion area and the known mixed pattern.

[0045] Among them, food ingredient combination inference refers to judging the types of food ingredients and their combinations that are most likely to be contained in the food adhesion area based on feature matching results. It can make decisions based on the pattern with the highest matching degree or in combination with other information (such as user historical preferences, other foods on the same plate). Its main purpose is to determine the internal composition of the adhesion area and provide guidance for subsequent processing.

[0046] Among them, the segmentation rule refers to an image segmentation algorithm or parameter set optimized for a specific food component combination, which may include a specific threshold, region growing criterion or pre-trained segmentation model parameters, which is mainly used to effectively separate the adhesion area into the individual food component areas within it.

[0047] Among them, the recognition calibration parameters refer to the rules or factors used to correct the food type recognition results, which may include recognition confidence adjustment rules or category mapping relationships, which are mainly used to improve the accuracy of food type recognition in the adhesion state.

[0048] Among them, the portion calibration parameter refers to the rule or factor used to correct the portion estimation result, which may include a volume expansion coefficient or a density correction factor, which is mainly used to improve the accuracy of food portion estimation in a sticky state.

[0049] The core innovation of this application lies in identifying food adhesion areas and extracting their overall regional features, inferring the combination of food components contained in the adhesion area based on a mixed morphological feature library, and extracting customized segmentation rules, identification calibration parameters and portion calibration parameters based on the inferred combination, thereby achieving accurate segmentation, identification and portion estimation of food components inside the adhesion area, effectively solving the problems of morphological changes and visual adhesion caused by food penetration or infiltration, and improving the accuracy of food calorie calculation.

[0050] Specifically, this method first captures an image of food on a plate and identifies areas of food adhesion caused by penetration or infiltration within the image. It also extracts the overall regional features of these areas. Based on the extracted regional features, the system performs feature matching within a pre-defined library of mixed morphological features to infer the most likely food component combination within the food adhesion area. Once the food component combination is determined, the system extracts corresponding segmentation rules and recognition calibration parameters from a pre-defined library of segmentation rules and recognition calibration parameters based on this combination. The system then uses the extracted segmentation rules to segment the food adhesion area, separating the individual food component regions within the adhesion area. The system then identifies the food type of each separated food component region and calibrates the identification results using the extracted recognition calibration parameters. Based on the calibrated food type identification results, the system extracts the corresponding portion calibration parameters from a pre-defined library of portion calibration parameters. Next, a preliminary portion size estimate is performed for each food component region, and the portion size estimates are calibrated using the extracted portion calibration parameters. Finally, the total calories of the food on the plate are calculated based on the calibrated food types and portion size estimates. The entire process overcomes the difficulty of directly processing complex adhesion areas by first inferring the whole area and then performing local customized processing, ensuring the accuracy of subsequent identification and estimation.

[0051] Through the above scheme, the present application can effectively identify and process food adhesion areas formed in cafeteria plates due to penetration or infiltration, overcome the problems of inaccurate segmentation, identification and portion estimation of traditional methods when processing such complex areas, and improve the assessment accuracy of the types and portions of various food components within the adhesion area, thereby significantly improving the overall accuracy of calorie calculation of food on the plate.

[0052] In some embodiments, step A1 comprises:

[0053] A101. Get the food image on the plate;

[0054] A102. Identify the plate area in the plate food image;

[0055] A103. Identify the food area within the plate area;

[0056] A104. Within the food region, identifying the food adhesion region based on visual continuity between pixels;

[0057] A105. Extract the region features reflecting the color, texture and / or shape attributes of the food adhesion region from the food adhesion region.

[0058] Among them, identifying the plate area refers to determining the image range where the plate is located in the acquired plate food image, which can be achieved by using template matching, edge detection, color segmentation or a trained target detection model.

[0059] Among them, identifying the food area refers to determining the image range containing food within the identified plate area, which can be achieved by using color threshold segmentation, texel analysis, clustering algorithm or semantic segmentation model.

[0060] Among them, based on the visual continuity between pixels, identifying food adhesion areas means that within the identified food area, by analyzing the similarity or gradient of visual attributes such as color, brightness, texture, etc. between adjacent pixels, those areas that are visually continuous and have blurred boundaries due to penetration or infiltration are determined. This can be achieved by using region growing algorithms, watershed algorithms, graph cut algorithms, or image segmentation models specially trained to identify infiltrated adhesion areas.

[0061] Specifically, the regional features may include the average color value, color histogram, local binary pattern (LBP) descriptor of the corresponding region, and shape descriptors such as the area, perimeter, and eccentricity of the region. These features can be extracted using existing technologies and are not limited here.

[0062] This solution details the implementation process for acquiring food plate images, identifying food adhesion areas, and extracting regional features from these areas. It aims to address the problem of accurately and effectively identifying and extracting features from food adhesion areas caused by penetration or infiltration in food plate images. By refining the image processing process step by step, from the overall to the local, the target area is gradually focused on and identified, and key features for subsequent analysis are extracted. First, a food plate image is acquired, which serves as the starting point for the entire processing process and provides the raw visual information to be analyzed. Next, the plate area is identified within the acquired food plate image. This step locates the plate within the complex background, thereby confining subsequent food identification and analysis to the plate interior, effectively eliminating background interference and improving processing efficiency and accuracy. Next, the food area is identified within the identified plate area. This step further narrows the processing scope by excluding non-food areas within the plate, allowing subsequent adhesion area identification to focus on the set of pixels containing the food. Finally, within the identified food area, the food adhesion area is identified based on visual continuity between pixels. This is the core technical approach for identifying adhesion areas in this solution. Unlike independent food chunks with clearly identified boundaries, adhesion areas formed by penetration or infiltration appear visually as pixels with high continuity and blurred boundaries. Through an analysis method specifically based on pixel visual continuity, these special, morphologically altered adhesion areas can be effectively distinguished from other food areas, laying the foundation for subsequent targeted processing. Finally, regional features reflecting the color, texture, and / or shape attributes of the identified food adhesion areas are extracted. These extracted features are visual representations of the adhesion areas and are used in subsequent steps to infer the combination of food components. Since the morphology of the adhesion area may be different from that of the original food, extracting these comprehensive regional features can more comprehensively reflect the visual characteristics of the adhesion area, providing key information for accurately inferring the combination of food components contained therein.

[0063] Through the synergistic effect of the above steps, this method can effectively locate and describe the food adhesion areas caused by infiltration from complex plate images, providing accurate input information for subsequent food composition inference, segmentation, identification and portion estimation, thereby improving the accuracy of the entire food calorie calculation method.

[0064] Preferably, step A104 may include:

[0065] identifying potential adhesion regions having visual continuity between pixels within the food region;

[0066] Analyzing pixel color gradients, pixel texture change characteristics, and / or region boundary fuzziness within the potential adhesion region;

[0067] Determining whether the potential adhesion area is an infiltration adhesion area based on the pixel color gradient, the pixel texture change characteristics and / or the region boundary fuzziness, and in combination with a preset infiltration adhesion feature pattern set;

[0068] The potential adhesion area judged as the infiltrated adhesion area is determined as the food adhesion area.

[0069] The pixel color gradient refers to the spatial rate of change of the pixel color value within a region, reflecting the smoothness or steepness of the color transition. Infiltration usually causes color diffusion, making the color gradient tend to be smooth.

[0070] Among them, the pixel texture change feature describes the regular or random changes in the grayscale or color distribution of pixels in the area. Infiltration may cause the original texture of the food to become blurred or mixed with other food textures.

[0071] Among them, the region boundary fuzziness measures the clarity of the color or grayscale transition of the region edge pixels. Infiltration will make the boundary unclear due to penetration.

[0072] The pre-set infiltration and adhesion feature pattern set is a knowledge base that stores the visual feature patterns of typical infiltration and adhesion areas. For example, it can be a set of feature threshold ranges, a set of rules, or a trained classification model. Based on the analyzed pixel color gradients, pixel texture variation characteristics, and / or region boundary fuzziness, combined with the pre-set infiltration and adhesion feature pattern set, potential adhesion areas can be judged to determine whether they meet the typical visual appearance of infiltration and adhesion.

[0073] Using the aforementioned technical means, this application first identifies potential adhesion regions within the food area, indicating visual continuity between pixels. These regions are candidate areas for possible adhesion. To distinguish true adhesion from visual continuity caused by other reasons, the visual features within these potential adhesion regions are analyzed in depth, extracting pixel color gradients, pixel texture variation characteristics, and region boundary blurriness. These features are typical manifestations of food adhesion in an image. Pixel color gradients reflect the degree of color diffusion and variation caused by liquid penetration, such as soup; pixel texture variation characteristics capture texture blurring or blending caused by penetration; and region boundary blurriness reflects the degree to which boundaries become unclear due to penetration. Subsequently, the potential adhesion regions are determined based on the analyzed pixel color gradients, pixel texture variation characteristics, and / or region boundary blurriness, combined with a pre-set set of adhesion and infiltration feature patterns. The pre-set adhesion and infiltration feature pattern set stores typical visual feature patterns of true adhesion and infiltration regions. By matching or comparing these patterns, it is possible to determine whether the potential region meets the characteristics of adhesion and infiltration. Ultimately, only potential adhesion areas that are judged to be infiltration adhesion areas are determined to be food adhesion areas. By introducing analysis and pattern judgment of infiltration-related visual features, this application can more accurately identify true food infiltration adhesion areas and avoid misidentifying visual continuity areas caused by non-infiltration as adhesion areas.

[0074] As a specific embodiment, within the food area, region growing, connected component analysis, or superpixel-based segmentation methods can be used to identify potential adhesion regions with visual continuity between pixels. For identified potential adhesion regions, the pixel color gradients within them can be calculated, for example, using the Sobel operator or Prewitt operator to calculate image gradients, and the gradient distribution characteristics within the region can be statistically analyzed. Pixel texture variation features can be extracted using local binary patterns (LBP) or Gabor filter banks to obtain a texture descriptor for the region. The fuzziness of region boundaries can be quantified by analyzing the gradient amplitude of pixels at the region boundary or the width of the edge response function. The preset set of infiltration and adhesion feature patterns can be a rule library that defines feature threshold ranges for infiltration and adhesion regions. For example, if the average color gradient within the region is below a certain threshold and the average gradient at the boundary is below a certain threshold, the region is judged to be infiltration and adhesion. Alternatively, the pattern set can be a trained machine learning classification model, such as a support vector machine (SVM) or a neural network, which uses the analyzed color gradient features, texture features, and boundary fuzziness features as input, and outputs a probability or category label for the region being an infiltration and adhesion region. According to the rule judgment result or the model output result, the potential adhesion area judged as the infiltration adhesion area is determined as the food adhesion area.

[0075] After identifying potential adhesion areas with visual continuity between pixels in the food area, further analyzing the pixel color gradients, pixel texture change characteristics and / or regional boundary fuzziness within these areas, and combining them with a preset set of infiltration adhesion feature patterns for judgment, the present application can more accurately distinguish between true food infiltration adhesion areas and visual continuity areas caused by non-infiltration. In this way, it is possible to avoid misjudging non-infiltration areas as adhesion areas, thereby improving the accuracy of food adhesion area identification. Accurate adhesion area identification provides a more reliable basis for subsequent food component inference, segmentation, identification and portion estimation, thereby improving the accuracy and reliability of the entire food calorie calculation method.

[0076] In some embodiments, step A2 comprises:

[0077] A201. Analyze the pixel color diffusion range, pixel texture blur and / or region boundary blur of the food adhesion region based on the region characteristics;

[0078] A202. Determine the level of wetness of the food adhesion area based on the pixel color diffusion range, the pixel texture blur and / or the region boundary blur;

[0079] A203. According to the regional characteristics and the infiltration level, feature matching is performed in a preset mixed morphological feature library; the mixed morphological feature library stores typical morphological feature patterns of different food component combinations at different infiltration levels;

[0080] A204. Based on the feature matching results, infer the food component combination contained in the food adhesion area.

[0081] The pixel color diffusion range refers to the degree of color penetration and mixing from the center to the edge or from one component to another in the food adhesion area. It can be quantified by analyzing color gradients, calculating color variance, or using color clustering.

[0082] The degree of pixel texture blur refers to the degree to which the texture details in the food adhesion area become unclear due to infiltration, which can be measured by calculating local contrast, analyzing texture energy, or using blur evaluation operators.

[0083] The fuzziness of regional boundaries refers to the clarity of the boundaries between the food adhesion area and other areas or different components within it. It can be evaluated by using edge detection algorithms (such as the Canny operator) and then analyzing the edge strength or using fuzzy edge detection technology.

[0084] Among them, the infiltration degree level refers to dividing the continuously changing food infiltration state into several discrete levels, such as light infiltration, moderate infiltration, heavy infiltration, etc., which can be determined based on the combined value of pixel color diffusion range, pixel texture blur degree and / or region boundary blur degree or through a classifier.

[0085] Among them, the mixed morphological feature library stores the typical visual feature patterns of various possible food ingredient combinations (such as rice + braised pork soup, noodles + meat sauce, etc.) at different levels of infiltration. It can be stored in the form of multidimensional feature vectors, template images or deep learning model parameters.

[0086] Feature matching is used to compare the regional features extracted from the current food adhesion area with the feature patterns stored in the mixed morphology feature library to identify the most similar patterns. This can be achieved using methods such as Euclidean distance, cosine similarity, support vector machines (SVM), or neural networks. A food ingredient combination refers to the specific composition of multiple food ingredients within the food adhesion area, such as a mixture of rice and braised pork broth or noodles and fried sauce.

[0087] This proposal addresses the problem that food adhesion regions, due to infiltration, exhibit diverse morphological changes, making it difficult to accurately infer the composition of food ingredients based solely on general regional features. An improved composition inference method is proposed. The key to this method is to assess and grade the degree of infiltration in the food adhesion region before feature matching. This method then combines regional features and infiltration levels during feature matching, utilizing a feature library containing typical morphological patterns at different infiltration levels to improve the accuracy of composition inference. Specifically, the method first analyzes the pixel color spread, pixel texture blur, and / or region boundary fuzziness of the food adhesion region based on regional features, extracting key visual cues reflecting the degree of infiltration from these regional features. The pixel color spread indicates areas where liquids such as soup have penetrated, while the pixel texture blur and region boundary fuzziness reflect changes in food morphology due to infiltration. These metrics are effective means of quantifying the infiltration state. Next, the infiltration level of the food adhesion region is determined based on the analyzed pixel color spread, pixel texture blur, and / or region boundary fuzziness. This step converts the continuously changing wetness state into discrete levels, such as light, moderate, and heavy, providing structured contextual information for subsequent feature matching. Different wetness levels correspond to different patterns of food morphology changes. Subsequently, feature matching is performed in a pre-set mixed morphology feature library based on the regional features and the determined wetness level. Unlike traditional single feature libraries, this mixed morphology feature library pre-stores typical morphological feature patterns of different food component combinations at different wetness levels. This means that for the same food component combination (such as rice and soup), the feature library will store its respective typical visual feature patterns in different wetness states, such as light, moderate, and heavy. By simultaneously inputting regional features and wetness levels for matching, the system can more accurately find the feature pattern that best matches the actual morphology of the current food adhesion area, thereby overcoming the recognition difficulties caused by morphological diversity caused by wetness. Finally, based on the feature matching results, the food component combination contained in the food adhesion area is inferred.

[0088] Because the matching process fully considers the impact of wetness on food morphology and utilizes a more refined feature library, the inferred food ingredient combinations are more accurate and reliable, laying a more solid foundation for subsequent segmentation, recognition, and portion estimation. This approach, based on identifying food adhesion regions and extracting their regional features, further refines the ingredient inference process. By introducing wetness assessment and grading, and utilizing a hybrid morphological feature library constructed for different wetness levels, the judgment of food ingredient combinations within adhesion regions is more precise, effectively addressing the morphological changes caused by food wetness and improving the robustness and accuracy of the overall method.

[0089] In a specific embodiment, the above steps can be implemented in the following manner. When analyzing the pixel color diffusion range, pixel texture blur, and / or region boundary blur of the food adhesion area, the image of the food adhesion area can be first converted to a grayscale image, and the variance of the image's Laplacian operator response can be calculated as a measure of texture blur. Simultaneously, the difference between the pixel color within the area and the average color of its local neighborhood can be calculated to reflect the color diffusion range. For region boundary blur, the Canny edge detector can be used to detect edges, and the gradient intensity distribution of edge pixels can be statistically analyzed to assess the blurriness. When determining the wetness level of the food adhesion area, the calculated pixel color diffusion range, pixel texture blur, and region boundary blur can be used as input features to train a classification model (e.g., a support vector machine or neural network). The model can map these features to a preset wetness level (e.g., Level 1: mild, Level 2: moderate, and Level 3: severe). When performing feature matching, the hybrid morphological feature library can be constructed as a multi-layer lookup table or a deep learning model. For example, the first-level index of the lookup table is the food ingredient combination type, and the second-level index is the wetness level. Each leaf node stores the typical feature vector of the corresponding combination at the corresponding level. During matching, the corresponding feature sub-library is first selected based on the inferred wetness level. Then, similarity is calculated between the regional features extracted from the current food adhesion area and the feature vectors in the sub-library. The food ingredient combinations corresponding to the feature vectors with the highest similarity and their matching degrees (i.e., similarity) are selected as the matching results. Alternatively, a multi-input deep learning model can be trained that accepts both regional features and wetness level as input and directly outputs a probability distribution of food ingredient combinations. The food ingredient combinations with the highest probability distribution and their matching degrees (i.e., probability distribution) are selected as the matching results. When inferring food ingredient combinations based on feature matching results, the food ingredient combination with the highest matching degree can be directly selected as the final inference result. Alternatively, if the matching result contains multiple candidate combinations and their matching degrees, further judgment can be made based on other information (e.g., information about other identified foods on the plate).

[0090] Preferably, after step A1 and before step A2, the method further comprises the following steps:

[0091] A9. Identify food type information in other food areas of the plate except the food adhesion area;

[0092] The feature matching result includes at least one candidate food component combination and a corresponding matching degree;

[0093] Step A204 includes:

[0094] B1. Obtain historical dining data with the user;

[0095] B2. Based on the food type information of other food areas identified in the plate and the historical dining data, obtain the confidence level that the food adhesion area contains each of the candidate food ingredient combinations;

[0096] B3. Infer the food component combination contained in the food adhesion area based on the feature matching result and the confidence level.

[0097] This solution aims to improve the accuracy of inferring the composition of food components within food adhesion areas, overcoming the potential shortcomings of relying solely on image features within the food adhesion area for matching and inference. Its core concept is to incorporate contextual information from other identified foods on the plate and the user's historical dining history as auxiliary criteria for judgment.

[0098] Specifically, after acquiring the image features of the food adhesion area and performing preliminary feature matching to obtain candidate food ingredient combinations and matching scores, but before finally inferring the food ingredient combinations, the algorithm also identifies food types in other food areas on the plate, excluding the food adhesion area. This information provides contextual information about the known food ingredients on the plate, which is valuable for inferring the possible food ingredients within the adhesion area. Based on this, this solution modifies the method of directly inferring food ingredient combinations based on feature matching results, refining it by obtaining the user's historical dining data. Based on the food type information of other identified food areas on the plate and the acquired historical dining data, the algorithm comprehensively calculates the likelihood that the food adhesion area contains each candidate food ingredient combination and obtains the corresponding confidence score. This confidence score incorporates the plausibility of the food combination on the plate and the user's historical dietary habits, effectively supplementing the results based solely on image feature matching. Finally, a comprehensive judgment is made based on the image feature matching results (including candidate combinations and matching scores) and the confidence score calculated based on the context and historical data to infer the final food ingredient combination contained in the food adhesion area. By combining the image feature matching results with the confidence calculated based on other information, this scheme can more accurately judge food ingredient combinations that may be similar in image features but have different actual probabilities.

[0099] In some possible implementations, the historical dining data includes food ingredient combination information and timestamps corresponding to the user's historical dining records;

[0100] Step B2 includes:

[0101] B201. Based on the food type information of other food areas identified in the plate, query the preset food pairing association rule library to obtain the food type information of other food areas and the matching correlation between each food ingredient in the candidate food ingredient combination;

[0102] B202. Based on the historical dining data, the frequency of occurrence of each candidate food ingredient combination in the historical dining records is counted, and the user's preference for each candidate food ingredient combination is calculated based on the frequency;

[0103] B203. Calculate the timeliness weight of each of the historical dining records according to the timestamp of each of the historical dining records in the historical dining data;

[0104] B204. Calculate the confidence level that the food adhesion region contains each candidate food component combination based on the collocation association, the preference level, and the timeliness weight.

[0105] The historical dining data refers to a data set that records the user's past dining habits, including the combination of food ingredients consumed by the user each time they ate and the time information of the meal.

[0106] Among them, the preset food pairing association rule library refers to a pre-established knowledge base that stores the association rules between different food types in pairing, such as which food combinations often appear together, or which food combinations are complementary in taste or nutrition; for canteens, their menus are highly fixed, so the pairing methods between different food types are relatively few and fixed, so the preset food pairing association rule library is more reliable.

[0107] Among them, the combination association degree refers to a quantitative indicator that measures the possibility or rationality of the co-occurrence of other known food types in the plate and the ingredients in the candidate food ingredient combination based on the food combination association rule library.

[0108] Among them, preference refers to the quantitative expression of the user's preference or habitual choice for a specific food ingredient combination obtained based on the user's historical dining data.

[0109] Among them, the timeliness weight refers to a weight factor calculated based on the timestamp of historical dining records, which is used to reflect the impact of the distance of the record from the current time on the user's preference. Generally, the more recent the record, the higher the weight.

[0110] Among them, confidence refers to a comprehensive evaluation value given on the possibility that the food adhesion area contains a specific candidate food ingredient combination after comprehensively considering multiple factors such as matching association, user preferences and timeliness of historical records.

[0111] This solution refines the process of determining the confidence level that a food adhesion region contains each candidate food ingredient combination, leveraging both the information from other identified food regions on the plate and the user's personalized historical dining data. Specifically, by querying a pre-defined food pairing association rule library, the known food types in the non-adhesive region of the current plate are used as contextual information to evaluate their compatibility with each ingredient in the candidate combinations in the adhesion region, thereby obtaining a combination relevance. This leverages universal principles of food pairing and provides context-based clues for determining the content of the adhesion region. Secondly, the user's preference for each candidate food ingredient combination is calculated by analyzing historical dining records. This reflects the user's personalized dietary habits; combinations that the user frequently consumes are more likely to appear on the current plate. Furthermore, considering that a user's dietary habits may change over time, a timeliness weight is calculated for historical records, giving a greater influence to recent dining records on preference levels. This improves the accuracy and dynamism of preference calculation based on historical data. Finally, the calculated combination relevance and timeliness-based preference are combined to calculate the confidence level that the food adhesion region contains each candidate food ingredient combination. This approach, which comprehensively considers the current meal context, the user's long-term habits, and recent preferences, enables a more comprehensive and accurate assessment of the likelihood of each candidate combination, providing a more reliable basis for subsequently inferring the most likely food ingredient combination. By integrating this multi-dimensional information, our solution effectively addresses the problem of fully utilizing this information to accurately calculate confidence levels, thereby improving the accuracy of inferred food ingredient combinations.

[0112] In one embodiment, historical dining data can be stored in a database, with each record containing the meal time, all food types identified on the plate, and their combinations. A pre-set food pairing association rule library can be a table or graph structure, storing association scores for food pairs or food combinations. These scores can be obtained through expert experience or mining large amounts of dining data. When calculating the association score, the rule library can be searched for rules related to each ingredient in the candidate food ingredient combination based on the other food types identified on the plate, and a comprehensive association score can be calculated based on the rules. For example, if a plate contains rice and Kung Pao Chicken, and the candidate combination is rice and braised pork broth, the system can query the association between rice, Kung Pao Chicken, and braised pork, as well as the association between Kung Pao Chicken and braised pork, to comprehensively determine the likelihood of rice and braised pork broth appearing on the plate. When calculating the preference score, the total number of occurrences of each candidate combination in the historical data can be counted and divided by the total number of meals to obtain the frequency as the preference score, or a more complex statistical model can be used. When calculating the timeliness weight, an exponential decay function can be used. For example, the weight is proportional to e minus λ multiplied by the time difference to the power of the time difference, where e is the base of the natural logarithm and λ is the decay coefficient. When calculating confidence, the collocation association and weighted preference (preference multiplied by the timeliness weight) can be weighted summed or input into a classifier (such as a support vector machine or neural network) for calculation. The classifier is trained to learn how to integrate these features to predict the confidence of the combination. The resulting confidence is a value between 0 and 1, with higher values ​​indicating a greater likelihood of the candidate combination.

[0113] In step B3, a weighted summation method may be used to perform weighted calculation on the matching degree and the confidence level to obtain a score value, and the candidate food component combination with the highest score value is determined as the final inference result.

[0114] In some embodiments, the recognition calibration parameters include recognition confidence calibration rules for inferred food component combinations;

[0115] Step A5 includes:

[0116] A501. For each of the food component regions, extracting sub-region features reflecting the color, texture and / or shape attributes of the food component region;

[0117] A502. Based on the extracted sub-region features, feature matching is performed in a preset food type feature library to obtain a preliminary food type recognition result and the corresponding recognition confidence;

[0118] A503. Calibrate the preliminary food type recognition result and the recognition confidence according to the recognition calibration parameters to obtain a calibrated food type recognition result.

[0119] The sub-region feature refers to a numerical value or vector that describes the visual characteristics of a specific region in an image, and can be represented by a feature of the same type as the region feature.

[0120] Among them, the food type feature library refers to a database that stores the correspondence between known food types and their typical visual features, which can store the standard feature patterns of each food in different forms.

[0121] Among them, the feature matching in step A502 refers to comparing the features extracted from the area to be identified with the known features in the feature library to determine which known category the area to be identified most likely belongs to. It can be achieved by distance-based matching, classifiers (such as support vector machines SVM, neural networks) or deep learning models. The preliminary food type recognition results and the corresponding recognition confidence refer to the food category predictions and their associated confidence scores directly output by the feature matching process before applying the calibration rules. Calibration refers to the process of correcting or adjusting the preliminary recognition results or confidence levels according to preset rules or models.

[0122] This solution aims to address the recognition accuracy issues caused by changes in food morphology within food adhesion regions by introducing rules for calibrating the recognition confidence of inferred food component combinations and defining the specific process for food type recognition and calibration. When performing food type recognition on each segmented food component region, step A501 first extracts subregion features reflecting the color, texture, and / or shape attributes of each segmented food component region. These features are a quantitative description of the region's visual appearance. Next, step A502 performs feature matching based on the extracted subregion features against a pre-set food type feature library, comparing the features of the current region with those of known food types in the library. This results in a preliminary food type recognition result and a recognition confidence level indicating the reliability of the result. However, since food morphology within the adhesion region may change due to infiltration, for example, rice may darken in color or blur in texture after being soaked in soup, this can lead to biased or low confidence in the preliminary recognition based on these local features. To correct for this bias, step A503 is a critical calibration step. This step uses the recognition calibration parameters (i.e., recognition confidence calibration rules) extracted from the food component combination inferred from the entire food adhesion region in the previous step to calibrate the preliminary recognition results and recognition confidences obtained in step A502. For example, if the adhesion region is inferred to contain rice and braised pork, and the preliminary recognition result identifies the soaked rice as some other food or gives a low confidence rating for the rice, the calibration rules can correct the preliminary result based on the pre-established knowledge that the visual characteristics of rice change when it adheres to braised pork, resulting in a low recognition confidence. For example, this can increase the confidence rating for the rice or correct the misidentified result to rice. This calibration, based on the inferred information about the overall composition of the adhesion region, can effectively correct for recognition bias caused by changes in local features, improving the accuracy and reliability of the final recognition results. By combining the inference of the overall composition of the adhesion region (completed in the previous step) with the recognition calibration of the segmented local regions, this solution can more accurately determine the true identity of each food component within the adhesion region, overcoming the limitations of relying solely on local features for recognition.

[0123] In one embodiment, step A501 can utilize a convolutional neural network (CNN) model to extract subregion features. This model, trained on a large number of food images, is capable of extracting discriminative deep features. Step A502 can utilize the classification layer of the CNN model as a feature matcher, inputting the extracted features into the classification layer and outputting a probability distribution for each food type. The food type with the highest probability is used as the preliminary recognition result, and the corresponding probability value is used as the recognition confidence. In step A503, the recognition calibration parameters can be stored as a lookup table or a small calibration network. The lookup table can provide calibrated recognition results and confidence levels based on the inferred food ingredient combination (e.g., rice + braised pork), the preliminary recognition result, and the confidence range. For example, the table can define: when the inferred combination is [rice, braised pork], the preliminary recognition is rice, and the confidence level is between 0.5 and 0.7, the post-calibration confidence level is adjusted to 0.85; when the preliminary recognition is potato, and the confidence level is less than 0.6, the post-calibration result is corrected to rice, and the confidence level is adjusted to 0.7. The calibration network can be a simple fully connected network, whose input includes the one-hot encoding of the preliminary recognition result, the preliminary confidence and the encoding of the inferred food component combination, and the output is the calibrated recognition result (such as through the softmax layer) and the calibrated confidence.

[0124] In some embodiments, the component calibration parameters include volume expansion coefficients and / or density correction factors for different levels of wetting;

[0125] Step A7 includes:

[0126] A701. For each of the food component regions, a preliminary portion estimation is performed based on the pixel area of ​​the food component region and the preset food thickness information to obtain a preliminary portion estimation result;

[0127] A702. Get the level of wetness of the food adhesion area;

[0128] A703. Select a corresponding volume expansion coefficient and / or density correction factor from the component calibration parameters according to the level of wetness of the food adhesion area;

[0129] A704. Calibrate the preliminary portion estimation result based on the selected volume expansion coefficient and / or density correction factor to obtain a calibrated portion estimation result.

[0130] The volume expansion coefficient refers to the proportional factor for the increase in volume of food due to absorption of liquid, which can be determined through experimental measurements or empirical data. The density correction factor refers to the correction ratio for the change in density of food due to absorption of liquid, which can be determined through experimental measurements or empirical data.

[0131] This solution addresses the problem of inaccurate portion estimation due to food infiltration in food adhesion areas. By introducing portion calibration parameters related to the degree of infiltration and refining the portion estimation and calibration steps, the accuracy of portion estimation is improved. Specifically, the portion calibration parameters are defined to include volume expansion coefficients and / or density correction factors at different levels of infiltration. This recognizes that the degree of food infiltration is different, and the degree of volume expansion or density change is also different. Therefore, different calibration parameters need to be provided according to the degree of infiltration to make the calibration more targeted. The volume expansion coefficient and density correction factor are effective means to quantify the effect of infiltration on food portion size, and can more accurately reflect the actual portion size of food after infiltration. In the specific portion estimation and calibration process,

[0132] First, a preliminary portion estimation is performed for each food component region based on the pixel area of ​​the food component region and preset food thickness information, resulting in a preliminary portion estimation result (e.g., directly multiplying the pixel area by the preset food thickness information to obtain a volume estimate). This is a basic method for portion estimation using image information and provides an initial value for subsequent calibration. Next, the wetness level of the food adhesion region is obtained. This wetness level reflects the degree to which the food is affected by wetness and is a key basis for accurate calibration. Then, based on the wetness level of the food adhesion region, a corresponding volume expansion coefficient and / or density correction factor is selected from the portion calibration parameters. This step links the wetness level with the calibration parameters, ensuring that the selected calibration parameters match the actual wetness of the current food adhesion region. Finally, the preliminary portion estimation result is calibrated based on the selected volume expansion coefficient and / or density correction factor to obtain a calibrated portion estimation result (e.g., multiplying the preliminary volume estimate by the volume expansion coefficient and / or density correction factor to obtain a calibrated volume estimate). By correcting the initial estimate using a volume expansion coefficient and / or density correction factor corresponding to the degree of sopping, the system more accurately reflects the actual portion size of the food, overcoming the shortcomings of traditional methods when dealing with sopping foods. By applying this more accurate portion estimation method to food components within the contiguous area of ​​the food, the accuracy of portion estimation for the entire plate is improved, laying the foundation for more accurate calorie calculations.

[0133] In one embodiment, portion estimation and calibration can be performed for the rice component region within the identified food adhesion region in the following manner. First, in step A701, the pixel area of ​​the rice component region on the image is calculated and multiplied by a preset average rice thickness (e.g., stored in a food thickness information library) to obtain a preliminary rice volume estimate. Then, in step A702, the wetness level of the food adhesion region is obtained. For example, by analyzing the region's color, texture, and other characteristics, the wetness level is determined to be "moderately wet," and its level is determined to be level 2. Next, in step A703, based on the obtained level 2 wetness level, the volume expansion coefficient (e.g., 1.15) and density correction factor (e.g., 1.08) for rice at level 2 wetness are retrieved from a preset portion calibration parameter library. Finally, in step A704, the preliminary rice volume estimate is multiplied by the retrieved volume expansion coefficient and density correction factor to obtain a calibrated rice volume estimate. This calibrated volume value more accurately reflects the actual portion size of the wet rice.

[0134] In some embodiments, step A8 includes:

[0135] A801. Obtain food type information and corresponding portion estimation results for other food areas in the plate except the food adhesion area;

[0136] A802. According to the calibrated food types and the food type information of the other food areas, query the preset food calorie database to obtain the calorie value of each food component per unit portion on the plate;

[0137] A803. Calculate the calories of each food component based on the caloric value per serving of each food component on the plate and the corresponding serving size estimate or calibrated serving size estimate for each food component;

[0138] A804. Sum up the calories of each food component in the plate to obtain the total calories of the food on the plate.

[0139] Among them, the food type information of other food areas in the plate except the food adhesion area adopts the recognition result of step A9, and the portion estimation results of these foods can be calculated using the pixel area of ​​the corresponding food area and the preset food thickness information. The specific process is similar to the estimation method of step A701.

[0140] Among them, the preset food calorie database refers to a data set that stores nutritional data of various foods, especially the calorie value contained in unit servings (for example, per 100 grams or per 100 milliliters). It can be stored in a local computing device or accessed through a network to a database on a remote server.

[0141] The solution of this application achieves a comprehensive consideration of the calories of all food components on the plate by refining the steps for calculating the total calories of food on the plate. First, by obtaining food type information and corresponding portion size estimates for food areas other than the food adhesion area on the plate, all identifiable food areas on the plate are included in the subsequent calorie calculation. This step supplements the processing of non-adhesive areas and overcomes the limitations of relying solely on the processing results of the adhesion area. Next, based on the calibrated food types obtained from the adhesion area and the food type information obtained from other non-adhesive food areas, a preset food calorie database is queried to obtain the unit serving calorie values ​​corresponding to all identified food components on the plate. This query process unifies food type information from different sources, providing the necessary basic data for subsequent calculations. The obtained unit serving calorie values ​​are then combined with the corresponding portion size information of each food component to calculate its calories. Here, specially calibrated portion size estimates are used for food components in the adhesion area, while direct portion size estimates are used for foods in other non-adhesive areas. This differentiated processing method, on the one hand, utilizes the more accurate calibration results obtained by complex processing of the adhesion area (including previous steps such as segmentation, identification calibration, and portion calibration). On the other hand, it also effectively utilizes the relatively direct identification and estimation results of the non-adhesion area, ensuring that the calories of all food components on the plate are calculated, and using calibrated data for key adhesion areas to improve the accuracy of the calorie calculation in this part. Finally, the calories calculated for all food components on the plate are summarized to obtain the total calories of the food on the entire plate. By combining the results of the detailed processing of the adhesion area with the conventional processing results of the non-adhesion area, this solution achieves a comprehensive calculation of the calories of all food on the plate and improves the accuracy of the total calorie estimate.

[0142] refer to Figure 2 The present application provides a food calorie calculation system based on image recognition, which is used to estimate the calories of food on a canteen plate. The system includes:

[0143] Feature acquisition module 1, used to acquire a food image of a plate, identify the food adhesion area, and extract regional features of the food adhesion area (refer to step A1 above for the specific process);

[0144] Composition inference module 2 is used to perform feature matching in a preset mixed morphology feature library based on the regional features to infer the food component combination contained in the food adhesion area (for the specific process, refer to step A2 above);

[0145] A first parameter acquisition module 3 is used to extract corresponding segmentation rules and recognition calibration parameters from a preset segmentation rule library and a preset recognition calibration parameter library according to the inferred food component combination (for the specific process, refer to step A3 above);

[0146] Segmentation module 4, for segmenting the food adhesion area according to the extracted segmentation rules, and separating the food component area inside the food adhesion area (for the specific process, refer to step A4 above);

[0147] Identification and calibration module 5, for identifying the food type of each food component region and calibrating the food type identification result according to the extracted identification calibration parameters (for the specific process, refer to step A5 above);

[0148] A second parameter acquisition module 6 is configured to extract corresponding portion calibration parameters from a preset portion calibration parameter library based on the calibrated food type recognition result (for the specific process, refer to step A6 above);

[0149] a portion calibration module 7 for estimating the portion size of each food component region and calibrating the portion size estimation result according to the extracted portion calibration parameters (for the specific process, refer to step A7 above);

[0150] The calorie calculation module 8 is used to calculate the total calories of the food on the plate based on the calibrated food types and the calibrated portion estimation results (for the specific process, please refer to step A8 above).

[0151] In some embodiments, the system further comprises:

[0152] The identification module is used to identify the food type information of other food areas in the plate except the food adhesion area (for the specific process, please refer to step A9 above).

[0153] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A food calorie calculation method based on image recognition, used to estimate the calorie of food on a canteen plate, characterized in that: The steps of the method include: A1. Obtain a food plate image, identify the food adhesion area, and extract regional features of the food adhesion area; A2. Based on the regional features, feature matching is performed in a preset mixed morphological feature library to infer the food component combination contained in the food adhesion area; A3. Based on the inferred food ingredient combination, the corresponding segmentation rules and identification calibration parameters are extracted from the preset segmentation rule library and the preset identification calibration parameter library; A4. Segment the food adhesion region according to the extracted segmentation rules to separate the food component region inside the food adhesion region; A5. Identify the food type of each food ingredient region and calibrate the food type identification result based on the extracted identification calibration parameters; A6. Based on the calibrated food type recognition results, the corresponding component calibration parameters are extracted from the preset component calibration parameter library; A7. Estimating the portion size of each food component region and calibrating the portion size estimation result according to the extracted portion calibration parameters; A8. Calculate the total calories on the plate based on the calibrated food types and calibrated portion size estimates.

2. The method for calculating food calories based on image recognition according to claim 1, characterized in that: Step A1 includes: A101. Get the food image on the plate; A102. Identify the plate area in the plate food image; A103. Identify the food area within the plate area; A104. Within the food region, identifying the food adhesion region based on visual continuity between pixels; A105. Extract the region features reflecting the color, texture and / or shape attributes of the food adhesion region from the food adhesion region.

3. The method for calculating food calories based on image recognition according to claim 2, characterized in that: Step A104 includes: identifying potential adhesion regions having visual continuity between pixels within the food region; Analyzing pixel color gradients, pixel texture change characteristics, and / or region boundary fuzziness within the potential adhesion region; Determining whether the potential adhesion area is an infiltration adhesion area based on the pixel color gradient, the pixel texture change characteristics and / or the region boundary fuzziness, and in combination with a preset infiltration adhesion feature pattern set; The potential adhesion area judged as the infiltrated adhesion area is determined as the food adhesion area.

4. The method for calculating food calories based on image recognition according to claim 1, characterized in that: Step A2 includes: A201. Analyze the pixel color diffusion range, pixel texture blur and / or region boundary blur of the food adhesion region based on the region characteristics; A202. Determine the level of wetness of the food adhesion area based on the pixel color diffusion range, the pixel texture blur and / or the region boundary blur; A203. According to the regional characteristics and the infiltration level, feature matching is performed in a preset mixed morphological feature library; the mixed morphological feature library stores typical morphological feature patterns of different food component combinations at different infiltration levels; A204. Based on the feature matching results, infer the food component combination contained in the food adhesion area.

5. The method for calculating food calories based on image recognition according to claim 4, characterized in that: After step A1 and before step A2, the following steps are further included: A9. Identify food type information in other food areas of the plate except the food adhesion area; The feature matching result includes at least one candidate food component combination and a corresponding matching degree; Step A204 includes: B1. Obtain historical dining data with the user; B2. Based on the food type information of other food areas identified in the plate and the historical dining data, obtain the confidence level that the food adhesion area contains each of the candidate food ingredient combinations; B3. Infer the food component combination contained in the food adhesion area based on the feature matching result and the confidence level.

6. The method for calculating food calories based on image recognition according to claim 5, characterized in that: The historical dining data includes food ingredient combination information and timestamps corresponding to the user's historical dining records; Step B2 includes: B201. Based on the food type information of other food areas identified in the plate, query the preset food pairing association rule library to obtain the food type information of other food areas and the matching correlation between each food ingredient in the candidate food ingredient combination; B202. Based on the historical dining data, the frequency of occurrence of each candidate food ingredient combination in the historical dining records is counted, and the user's preference for each candidate food ingredient combination is calculated based on the frequency; B203. Calculate the timeliness weight of each of the historical dining records according to the timestamp of each of the historical dining records in the historical dining data; B204. Calculate the confidence level that the food adhesion region contains each candidate food component combination based on the collocation association, the preference level, and the timeliness weight.

7. The method for calculating food calories based on image recognition according to claim 1, characterized in that: The identification calibration parameters include identification confidence calibration rules for the inferred food component combinations; Step A5 includes: A501. For each of the food component regions, extracting sub-region features reflecting the color, texture and / or shape attributes of the food component region; A502. Based on the extracted sub-region features, feature matching is performed in a preset food type feature library to obtain a preliminary food type recognition result and the corresponding recognition confidence; A503. Calibrate the preliminary food type recognition result and the recognition confidence according to the recognition calibration parameters to obtain a calibrated food type recognition result.

8. The method for calculating food calories based on image recognition according to claim 1, characterized in that: The component calibration parameters include volume expansion coefficients and / or density correction factors for different levels of wetting; Step A7 includes: A701. For each of the food component regions, a preliminary portion estimation is performed based on the pixel area of ​​the food component region and the preset food thickness information to obtain a preliminary portion estimation result; A702. Get the level of wetness of the food adhesion area; A703. Select a corresponding volume expansion coefficient and / or density correction factor from the component calibration parameters according to the level of wetness of the food adhesion area; A704. Calibrate the preliminary portion estimation result based on the selected volume expansion coefficient and / or density correction factor to obtain a calibrated portion estimation result.

9. The method for calculating food calories based on image recognition according to claim 1, characterized in that: Step A8 includes: A801. Obtain food type information and corresponding portion estimation results for other food areas in the plate except the food adhesion area; A802. According to the calibrated food types and the food type information of the other food areas, query the preset food calorie database to obtain the calorie value of each food component per unit portion on the plate; A803. Calculate the calories of each food component based on the caloric value per serving of each food component on the plate and the corresponding serving size estimate or calibrated serving size estimate for each food component; A804. Sum up the calories of each food component in the plate to obtain the total calories of the food on the plate.

10. A food calorie calculation system based on image recognition, used to estimate the calories of food on a canteen plate, characterized in that: The system includes: A feature acquisition module is used to acquire a food image of a plate, identify food adhesion areas therefrom, and extract regional features of the food adhesion areas; a component inference module, configured to perform feature matching in a preset mixed morphology feature library based on the regional features, and infer the food component combination contained in the food adhesion region; A first parameter acquisition module is used to extract corresponding segmentation rules and recognition calibration parameters from a preset segmentation rule library and a preset recognition calibration parameter library respectively according to the inferred food component combination; a segmentation module, configured to segment the food adhesion region according to the extracted segmentation rules, and separate the food component region inside the food adhesion region; an identification and calibration module, configured to identify the food type of each food component region and calibrate the food type identification result according to the extracted identification and calibration parameters; A second parameter acquisition module is used to extract corresponding portion calibration parameters from a preset portion calibration parameter library according to the calibrated food type recognition result; a portion calibration module, configured to estimate the portion size of each of the food component regions and calibrate the portion estimation results according to the extracted portion calibration parameters; The calorie calculation module is used to calculate the total calories of the food on the plate based on the calibrated food types and the calibrated portion estimation results.

Images