Intelligent school canteen nutrition meal serving system

Abstract

This invention discloses an intelligent school canteen nutrition meal planning system, belonging to the field of smart catering technology. It includes a food data acquisition module, a spectral nutrition mapping modeling module, a nutrition assessment module, and a nutrition meal planning module. This invention employs a food spectral nutrition modeling method based on continuous recipe sampling and probability distribution mapping to achieve comprehensive modeling of multiple possible food preparation states and establish a distributed correlation between spectral information and nutritional components. It uses a nutrition assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints to obtain stable and realistic nutrition assessment results in complex meal-taking scenarios. By collecting multi-dimensional information during the food-taking process and establishing nutritional correlations based on possible changes during food preparation, it achieves a stable estimate of actual nutrient intake, thereby realizing a shift from "rough meal planning" to "precise nutrition management" and improving the nutritional service level of school canteens.

Description

Technical Field

[0001] This invention belongs to the field of smart catering technology, specifically referring to an intelligent school canteen nutrition meal preparation system. Background Technology

[0002] An intelligent school canteen nutrition meal planning system collaboratively acquires images, spectral information, and weight data of dishes during students' meal collection process. By combining this data with the characteristics of changes in dish recipes to establish a nutritional mapping relationship, the system dynamically analyzes and comprehensively evaluates actual nutrient intake. The aim is to achieve accurate perception and quantitative analysis of students' actual nutritional intake, and based on this, provide individualized nutritional meal planning suggestions. This will improve the scientific and refined level of nutrition management in school canteens, and promote the optimization of students' dietary structure and healthy development.

[0003] However, in actual canteen operations, there are several technical challenges: the food preparation process is heavily influenced by manual operation, and the input of raw materials fluctuates randomly, resulting in the same dish exhibiting different nutritional compositions at different times. Traditional methods of estimation based on fixed nutrition tables or small samples are insufficient to reflect the actual food supply situation and cannot describe the range and uncertainty of nutrient variations, thus affecting the scientific nature and refined management of nutritional meal planning. Furthermore, traditional nutritional assessment methods rely heavily on a single source of information, making it difficult to simultaneously consider the appearance and structure of dishes, differences in internal components, and actual food consumption. The lack of effective coordination mechanisms between different information sources also leads to significant deviations and instability in assessment results. Finally, there are issues with accurately quantifying the nutritional content of dishes, inconsistent results from different estimation methods, and a lack of dietary guidance tailored to individual student differences, resulting in nutritional meal planning relying primarily on experience-based judgment and hindering refined management. Summary of the Invention

[0004] To address the above issues and overcome the shortcomings of existing technologies, this invention provides an intelligent school canteen nutrition meal planning system. It creatively employs a food spectral nutrition modeling method based on continuous recipe sampling and probability distribution mapping to achieve comprehensive modeling of multiple possible preparation states of dishes. By establishing a distributed correlation between spectral information and nutritional components, the nutritional assessment results can reflect the actual fluctuation range and trends, thereby improving the reliability and stability of nutritional estimation, enhancing the system's adaptability to complex canteen environments, and providing more scientific decision support for students' balanced nutrition. Furthermore, it creatively employs a nutritional assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints, collaboratively utilizing multiple types of information in the same assessment process. Furthermore, by employing a consistency constraint mechanism to dynamically adjust the differences between various information sources, the system enables mutual correction among these sources. This allows for stable and realistic nutritional assessment results even in complex meal-taking scenarios, effectively improving the scientific rigor and precision of school canteen meal planning. By collecting multi-dimensional information on the food-taking process and establishing nutritional correlations based on potential changes during food preparation, the system comprehensively utilizes information from different sources to achieve stable estimations of actual nutrient intake. Simultaneously, it analyzes and provides feedback on intake results based on individual student circumstances, thereby shifting from "rough meal planning" to "precise nutrition management," enhancing the nutritional service level of school canteens and improving students' health protection capabilities.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides an intelligent school canteen nutrition meal planning system, including a dish data acquisition module, a spectral nutrition mapping modeling module, a nutrition assessment module and a nutrition meal planning module;

[0006] The food data acquisition module is used to acquire food data, obtain single meal picking event data and plate-level multimodal data samples, and send the single meal picking event data to the nutrition assessment module and the plate-level multimodal data samples to the nutrition meal planning module.

[0007] The spectral nutrition mapping modeling module is used for spectral nutrition mapping modeling of dishes. It adopts a spectral nutrition modeling method based on continuous recipe sampling and probability distribution mapping to obtain structured spectral nutrition modeling results, and sends the structured spectral nutrition modeling results to the nutrition assessment module.

[0008] The nutrition assessment module is used for multimodal nutrition assessment. Based on single meal-taking event data and structured spectral nutrition modeling results, it adopts a nutrition assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints to obtain the nutritional estimation results of the dishes, and sends the nutritional estimation results of the dishes to the nutrition meal matching module.

[0009] The nutrition meal planning module is used for individualized nutrition meal planning to obtain individualized meal planning recommendations.

[0010] Furthermore, the food data collection is used to acquire multimodal synchronous data during the student's meal pick-up process and construct structured multimodal data samples. Specifically, by detecting the meal pick-up behavior, a meal pick-up transaction identifier is generated when the user starts picking up the meal. Image data, spectral data, and weight data of the corresponding dishes are collected, and associated and time-series aligned to construct single meal pick-up event data. Multiple meal pick-up events are then aggregated to form a plate-level multimodal data sample.

[0011] Furthermore, the spectral nutrition modeling of dishes is used to establish the mapping relationship between the spectral features of dishes and their nutritional components. Specifically, it adopts a spectral nutrition modeling method based on continuous recipe sampling and probability distribution mapping to obtain structured spectral nutrition modeling results.

[0012] The structured spectral nutrition modeling results include a set of recipe samples, corresponding spectral nutrition pairing data, as well as a spectral feature probability distribution model, a nutritional condition distribution model, and a spectral nutrition mapping relationship model.

[0013] The method for modeling the spectral nutrition of dishes based on continuous recipe sampling and probability distribution mapping includes the following steps: parameterized modeling of dish composition, continuous sampling of multiple recipes, joint acquisition of spectral nutrition, enhancement of spectral features, modeling of spectral distribution, and modeling of spectral nutrition mapping relationship.

[0014] The parameterized modeling of the dish composition is used to construct the structural constraint space of the dish composition. Specifically, it involves parameterizing each ingredient in the dish, defining each ingredient and its corresponding proportion range, modeling the dish composition, obtaining a set of dish parameters, and thus forming the constraint space of the dish composition.

[0015] The multi-recipe continuous sampling is used to generate dish samples under different ratio conditions. Specifically, for each dish, continuous random sampling is performed within the corresponding ratio range of each ingredient to generate each ingredient ratio parameter. Multiple sets of ingredient ratio combinations are constructed to obtain a set of dish recipe samples.

[0016] The aforementioned spectral nutrition joint acquisition is used to establish the correspondence between spectral data and nutritional data. Specifically, it involves acquiring spectral reflectance data for each dish recipe sample under different wavelength conditions and simultaneously obtaining the corresponding nutritional component data. The spectral reflectance data and the corresponding nutritional component data are then paired and associated to construct spectral nutrition paired data.

[0017] The spectral feature enhancement is used to improve the ability of spectral features to express nutritional changes. Specifically, it involves calculating the derivative of the spectral reflectance data and standardizing it to obtain standardized spectral feature data. Then, it extracts the main feature components from the standardized spectral feature data through a dimensionality reduction method and introduces nonlinear kernel mapping for enhancement to obtain a spectral enhanced feature vector.

[0018] The spectral distribution modeling is specifically based on spectral nutrient pairing data. By performing probability density modeling on the spectral enhancement feature vector, multiple spectral distribution clusters are constructed to obtain a spectral feature probability distribution model. The mean, covariance, and weight of each spectral distribution cluster are estimated. Then, combined with the nutrient component data corresponding to each spectral distribution cluster, conditional probability modeling is performed to obtain a nutrient conditional distribution model. The correlation between spectral distribution and nutrient distribution is established, realizing the transformation from single-point mapping to distribution modeling.

[0019] The spectral nutrient mapping model is specifically based on the spectral feature probability distribution model. It constructs a spectral nutrient mapping mechanism consisting of attribution probability calculation and probability weighting fusion to realize the conditional expectation mapping from spectral feature vectors to nutrient components, thereby obtaining the spectral nutrient mapping model.

[0020] Furthermore, the multimodal nutritional assessment is used to integrate visual, spectral, and weight information to achieve adaptive nutritional component assessment with uncertainty constraints. Specifically, based on single meal-taking event data and structured spectral nutritional modeling results, a nutritional assessment method based on visual-spectral collaborative modeling and cross-modal consistency constraints is adopted to obtain the nutritional estimation results of the dishes.

[0021] The nutritional assessment method based on visual-spectral collaborative modeling and cross-modal consistency constraints includes the following steps: visual branch modeling, spectral branch modeling, baseline nutritional estimation branch modeling, unified confidence modeling, dynamic gating fusion weight calculation, adaptive adjustment of fusion weights, and multimodal fusion estimation.

[0022] The visual branch modeling is used to perform structured visual analysis and nutritional estimation of food images. Specifically, it involves inputting food images from a single food collection event into a convolutional neural network for feature extraction to obtain a visual feature vector. Based on the visual feature vector, it outputs the food category probability, ingredient ratio vector, volume estimate, and visual uncertainty parameter. The visual nutritional estimation result is obtained by combining the food weight value, ingredient ratio vector, and nutritional vector per unit mass.

[0023] The spectral branch modeling is used to perform probabilistic modeling and uncertainty characterization of spectral components in spectral data. Specifically, it extracts spectral feature vectors from single meal-taking event data, inputs the spectral feature vectors into the spectral nutrition mapping relationship model, obtains the conditional expectation of nutrient components as the spectral nutrition estimation result, and performs weighted variance calculation by combining the nutrient distribution information corresponding to each distribution cluster to obtain the corresponding conditional variance as the spectral uncertainty parameter.

[0024] The benchmark nutrient estimation branch modeling is used to estimate the benchmark nutrient based on statistical nutrient density, specifically by calculating the benchmark nutrient estimation result based on the average nutrient density of the dish and the weight value of the dish.

[0025] The unified confidence modeling specifically involves calculating the visual weight estimate based on the volume estimate and the food volume density, obtaining the weight consistency error by calculating the difference between the visual weight estimate and the food weight, and then performing exponential mapping processing on the visual uncertainty parameter, spectral uncertainty parameter, and weight consistency error to obtain visual confidence, spectral confidence, and weight confidence, and concatenating the three types of confidence to form a unified confidence vector.

[0026] Wherein, the visual credibility is used to characterize the reliability of the food structure and nutritional estimation results obtained based on image analysis; the spectral credibility is used to characterize the stability and distribution uncertainty of the nutritional estimation results obtained based on spectral feature inference; and the weight credibility is used to characterize the degree of physical consistency between the visual estimation results and the actual weighing results.

[0027] The dynamic gating fusion weight calculation is used to adaptively determine the fusion weight of each modality based on the unified confidence vector. Specifically, it involves inputting the unified confidence vector into the learnable gating model and normalizing it using the Softmax function to obtain the visual nutrient fusion weight, spectral nutrient fusion weight, and baseline nutrient fusion weight.

[0028] The adaptive adjustment of the fusion weights is used to detect the consistency of multimodal estimation and dynamically adjust the fusion weights. Specifically, it calculates the difference between the visual nutrient estimation result and the spectral nutrient estimation result to obtain the cross-modal consistency error. When the consistency error exceeds a preset threshold, the weights are corrected; otherwise, the fusion weights remain unchanged.

[0029] The weight correction specifically involves exponentially decaying the visual nutrient fusion weights and spectral nutrient fusion weights, and then normalizing and redistributing all fusion weights by combining them with the baseline nutrient fusion weights that have not undergone exponential decay.

[0030] The multimodal fusion estimation is used to generate the final nutritional component estimation result. Specifically, based on the fusion weights that have been adaptively adjusted by the fusion weights, the visual nutritional estimation result, the spectral nutritional estimation result, and the baseline nutritional estimation result are weighted and summed to obtain the nutritional estimation result of the dish.

[0031] Furthermore, the personalized nutritional meal planning specifically involves summarizing the nutritional estimation results of dishes corresponding to each meal-taking event in the plate-level multimodal data sample based on the nutritional estimation results of the dishes obtained from the multimodal nutritional assessment, obtaining the total nutritional intake of a single meal, statistically analyzing the nutritional intake of the user across multiple meals, calculating the difference between the user's current nutritional intake and the target nutritional requirements, generating a target nutritional compensation vector, and based on the current canteen food nutritional data, selecting dish combinations that meet the target nutritional compensation vector from the candidate dish set to generate a personalized meal planning recommendation scheme.

[0032] The beneficial effects achieved by the present invention using the above solution are as follows:

[0033] (1) In view of the fact that the food preparation process is greatly affected by manual operation and the raw material input has random fluctuations, resulting in the same dish exhibiting different nutritional compositions at different times, and that the traditional estimation method based on fixed nutrition tables or a small number of samples is difficult to reflect the actual food supply situation and cannot describe the range and uncertainty of nutrient composition changes, thus affecting the scientific nature and refined management level of nutritional meal planning, this solution creatively adopts a food spectral nutrition modeling method based on continuous formula sampling and probability distribution mapping to achieve coverage modeling of multiple possible food preparation states. By establishing a distributed correlation between spectral information and nutrient composition, the nutritional assessment results can reflect the actual fluctuation range and trend of change, thereby improving the reliability and stability of nutritional estimation, enhancing the system's adaptability to complex canteen environments, and providing more scientific decision support for students' balanced dietary nutrition.

[0034] (2) In view of the technical problems that traditional nutritional assessment methods mainly rely on a single source of information, making it difficult to simultaneously take into account the appearance and structure of dishes, differences in internal components and actual meal quantity, and lack an effective coordination mechanism between different information sources, which easily leads to large deviations and insufficient stability of assessment results, this solution creatively adopts a nutritional assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints. In the same assessment process, multiple information sources are used collaboratively, and the differences between different information sources are dynamically adjusted through the consistency constraint mechanism, so that each information source can correct each other. Thus, stable and realistic nutritional assessment results can still be obtained in complex meal-taking scenarios, effectively improving the scientific and refined level of school canteen nutritional meal planning.

[0035] (3) In response to the problems of difficulty in accurately quantifying the nutrition of dishes, inconsistent results of different estimation methods, and lack of dietary guidance methods for individual student differences, which lead to the technical problem that nutritional meal planning mainly relies on experience judgment and is difficult to achieve refined management, this solution collects multi-dimensional information on the process of taking dishes and establishes nutritional correlations in combination with possible changes in the preparation of dishes. On this basis, it makes comprehensive use of information from different sources to achieve a stable estimate of actual nutritional intake. At the same time, it analyzes and provides feedback on the intake results based on individual student conditions, thereby realizing the transformation from "rough meal planning" to "precise nutrition management" and improving the nutritional service level of school canteens and the health protection capabilities of students. Attached Figure Description

[0036] Figure 1 A schematic diagram of a module for an intelligent school canteen nutrition meal preparation system provided by the present invention;

[0037] Figure 2 A flowchart illustrating the spectral nutrient mapping modeling module;

[0038] Figure 3 This is a flowchart of the nutrition assessment module.

[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0040] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0042] Example 1, see Figure 1 The present invention provides an intelligent school canteen nutrition meal planning system, including a dish data acquisition module, a spectral nutrition mapping modeling module, a nutrition assessment module and a nutrition meal planning module;

[0043] The food data acquisition module is used to acquire food data, obtain single meal picking event data and plate-level multimodal data samples, and send the single meal picking event data to the nutrition assessment module and the plate-level multimodal data samples to the nutrition meal planning module.

[0044] The spectral nutrition mapping modeling module is used for spectral nutrition mapping modeling of dishes. It adopts a spectral nutrition modeling method based on continuous recipe sampling and probability distribution mapping to obtain structured spectral nutrition modeling results, and sends the structured spectral nutrition modeling results to the nutrition assessment module.

[0045] The nutrition assessment module is used for multimodal nutrition assessment. Based on single meal-taking event data and structured spectral nutrition modeling results, it adopts a nutrition assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints to obtain the nutritional estimation results of the dishes, and sends the nutritional estimation results of the dishes to the nutrition meal matching module.

[0046] The nutrition meal planning module is used for individualized nutrition meal planning to obtain individualized meal planning recommendations.

[0047] Example 2, see Figure 1 This embodiment is based on the above embodiment. The food data collection is used to obtain multimodal synchronous data during the student's meal picking process and construct structured multimodal data samples. Specifically, by detecting the meal picking behavior, a meal picking transaction identifier is generated when the user starts picking up the meal. Image data, spectral data and weight data of the corresponding dishes are collected, and they are associated, bound and time-series aligned to construct single meal picking event data. Multiple meal picking events are aggregated to form a plate-level multimodal data sample.

[0048] The detection of food picking behavior can be achieved through one or more of the following methods: identity recognition device, weight change detection, or visual motion recognition.

[0049] The association binding and time sequence alignment are achieved through a timestamp synchronization mechanism or a unified data cache queue;

[0050] The aggregation of multiple food pickup events specifically involves aggregating multiple food pickup events corresponding to the same food pickup transaction identifier within a preset time window, preferably 3 to 7 seconds.

[0051] Example 3, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. The food spectral nutrition modeling is used to establish the mapping relationship between food spectral features and nutritional components. Specifically, it adopts a food spectral nutrition modeling method based on continuous formula sampling and probability distribution mapping to obtain structured spectral nutrition modeling results.

[0052] The structured spectral nutrition modeling results include a set of recipe samples, corresponding spectral nutrition pairing data, as well as a spectral feature probability distribution model, a nutritional condition distribution model, and a spectral nutrition mapping relationship model.

[0053] The method for modeling the spectral nutrition of dishes based on continuous recipe sampling and probability distribution mapping includes the following steps: parameterized modeling of dish composition, continuous sampling of multiple recipes, joint acquisition of spectral nutrition, enhancement of spectral features, modeling of spectral distribution, and modeling of spectral nutrition mapping relationship.

[0054] The parameterized modeling of the dish composition is used to construct the structural constraint space of the dish composition. Specifically, it involves parameterizing each ingredient in the dish, defining each ingredient and its corresponding proportion range, modeling the dish composition, obtaining a set of dish parameters, and thus forming the constraint space of the dish composition.

[0055] The calculation formula for the set of dish parameters is:

[0056] ;

[0057] In the formula, D j It is the set of parameters for the j-th dish, where j is the dish index, i is the ingredient index, and x is the ingredient index. i It is the identifier for the i-th raw material. It is the lower limit of the proportion of the i-th raw material. is the upper limit of the proportion of the i-th ingredient, and M is the quantity of the ingredients corresponding to the dish;

[0058] The multi-recipe continuous sampling is used to generate dish samples under different ratio conditions. Specifically, for each dish, continuous random sampling is performed within the corresponding ratio range of each ingredient to generate each ingredient ratio parameter. Multiple sets of ingredient ratio combinations are constructed to obtain a set of dish recipe samples.

[0059] The calculation formula for the recipe sample is as follows:

[0060] ;

[0061] In the formula, X k This is the k-th recipe sample, where k is the sample index, r1 is the ratio parameter of the first ingredient, r2 is the ratio parameter of the second ingredient, and r... M This is the proportioning parameter for the Mth raw material;

[0062] The aforementioned spectral nutrition joint acquisition is used to establish the correspondence between spectral data and nutritional data. Specifically, it involves acquiring spectral reflectance data for each dish recipe sample under different wavelength conditions and simultaneously obtaining the corresponding nutritional component data. The spectral reflectance data and the corresponding nutritional component data are then paired and associated to construct spectral nutrition paired data.

[0063] The nutritional data includes, but is not limited to, the content of fat, protein, carbohydrates, and sodium.

[0064] The spectral feature enhancement is used to improve the ability of spectral features to express nutritional changes. Specifically, it involves calculating the derivative of the spectral reflectance data and standardizing it to obtain standardized spectral feature data. Then, it extracts the main feature components from the standardized spectral feature data through a dimensionality reduction method and introduces nonlinear kernel mapping for enhancement to obtain a spectral enhanced feature vector.

[0065] The dimensionality reduction method may specifically employ principal component analysis, linear discriminant analysis, or dimensionality reduction based on eigenvalue decomposition.

[0066] The nonlinear kernel mapping can specifically employ radial basis function kernel mapping, Gaussian kernel mapping, or polynomial kernel mapping.

[0067] The spectral distribution modeling is specifically based on spectral nutrient pairing data. By performing probability density modeling on the spectral enhancement feature vector, multiple spectral distribution clusters are constructed to obtain a spectral feature probability distribution model. The mean, covariance, and weight of each spectral distribution cluster are estimated. Then, combined with the nutrient component data corresponding to each spectral distribution cluster, conditional probability modeling is performed to obtain a nutrient conditional distribution model. The correlation between spectral distribution and nutrient distribution is established, realizing the transformation from single-point mapping to distribution modeling.

[0068] The spectral nutrient mapping relationship modeling is specifically based on the spectral feature probability distribution model, constructing a spectral nutrient mapping mechanism composed of attribution probability calculation and probability weighting fusion, so as to realize the conditional expectation mapping from spectral feature vectors to nutrient components, and obtain the spectral nutrient mapping relationship model.

[0069] The attribution probability, used to characterize the probability that a spectral feature vector belongs to each spectral distribution cluster, is calculated using the following formula:

[0070] ;

[0071] In the formula, F represents the probability of the spectral feature vector belonging to the 'a'-th spectral distribution cluster, where 'a' is the first index of the spectral distribution cluster, and F is the spectral feature vector, specifically the spectral feature representation after enhancement. It is the weighting coefficient of the a-th spectral distribution cluster. C is the mean of the a-th spectral distribution cluster. a Let be the covariance matrix of the a-th spectral distribution cluster, and b be the second index of the spectral distribution cluster. These are the weighting coefficients of the b-th spectral distribution cluster. C is the mean of the b-th spectral distribution cluster.b It is the covariance matrix of the b-th spectral distribution cluster;

[0072] The calculation formula for the probability-weighted fusion is as follows:

[0073] ;

[0074] In the formula, It is the expected output of nutrient components corresponding to the spectral feature vector. It is the nutrient mean vector corresponding to the a-th spectral distribution cluster.

[0075] By performing the above operations, this solution addresses the technical problem that the food preparation process is greatly affected by manual operation and the raw material input has random fluctuations, resulting in the same dish exhibiting different nutritional compositions at different times. Traditional estimation methods based on fixed nutrition tables or small samples are insufficient to reflect the actual meal supply situation and cannot describe the range and uncertainty of nutrient composition changes, thus affecting the scientific nature and refined management level of nutritional meal planning. This solution creatively adopts a food spectral nutrition modeling method based on continuous formula sampling and probability distribution mapping to achieve comprehensive modeling of multiple possible food preparation states. By establishing a distributed correlation between spectral information and nutrient composition, the nutritional assessment results can reflect the actual fluctuation range and trend of changes, thereby improving the reliability and stability of nutritional estimation, enhancing the system's adaptability to complex canteen environments, and providing more scientific decision support for students' balanced dietary nutrition.

[0076] Example 4, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. The multimodal nutrition assessment is used to integrate visual, spectral and weight information to achieve adaptive assessment of nutritional components with uncertainty constraints. Specifically, based on single meal-taking event data and structured spectral nutrition modeling results, a nutrition assessment method based on visual-spectral collaborative modeling and cross-modal consistency constraints is adopted to obtain the nutritional estimation results of the dishes.

[0077] The nutritional assessment method based on visual-spectral collaborative modeling and cross-modal consistency constraints includes the following steps: visual branch modeling, spectral branch modeling, baseline nutritional estimation branch modeling, unified confidence modeling, dynamic gating fusion weight calculation, adaptive adjustment of fusion weights, and multimodal fusion estimation.

[0078] The visual branch modeling is used to perform structured visual analysis and nutritional estimation of food images. Specifically, it involves inputting food images from a single food collection event into a convolutional neural network for feature extraction to obtain a visual feature vector. Based on the visual feature vector, it outputs the food category probability, ingredient ratio vector, volume estimate, and visual uncertainty parameter. The visual nutritional estimation result is obtained by combining the food weight value, ingredient ratio vector, and nutritional vector per unit mass.

[0079] The raw material ratio vector is specifically obtained by performing multi-class regression prediction on the visual feature vector and then normalizing it.

[0080] The visual uncertainty parameter is specifically obtained by introducing an uncertainty estimation branch at the output of the convolutional neural network to predict the variance of the visual feature vector.

[0081] The nutrient vector per unit mass is derived from nutrient composition data and is used to characterize the nutrient content corresponding to each unit mass of raw materials.

[0082] The formula for calculating the visual nutrition estimation result is as follows:

[0083] ;

[0084] In the formula, N v This is the result of visual nutritional estimation, where W is the weight of the dish and r is the ingredient proportion vector. It is a nutrient vector per unit mass;

[0085] The spectral branch modeling is used to perform probabilistic modeling and uncertainty characterization of spectral components in spectral data. Specifically, it extracts spectral feature vectors from single meal-taking event data, inputs the spectral feature vectors into the spectral nutrition mapping relationship model, obtains the conditional expectation of nutrient components as the spectral nutrition estimation result, and performs weighted variance calculation by combining the nutrient distribution information corresponding to each distribution cluster to obtain the corresponding conditional variance as the spectral uncertainty parameter.

[0086] The formula for calculating the spectral uncertainty parameter is as follows:

[0087] ;

[0088] In the formula, It is the spectral uncertainty parameter, specifically the conditional variance, N. s This is the result of spectral nutrient estimation;

[0089] The baseline nutrient estimation branch modeling is used to estimate baseline nutrients based on statistical nutrient density. Specifically, it calculates the baseline nutrient estimation result based on the average nutrient density of the dish and its weight. The calculation formula is as follows:

[0090] ;

[0091] In the formula, N rule This is the baseline nutrient estimate result. It is the average nutritional density of the dish;

[0092] The average nutritional density of the dishes is obtained from a preset dish attribute database based on the dish category, or by statistical analysis based on historical dish samples; the preset dish attribute database is used to store statistical information on nutritional components and physical property parameters corresponding to different dish categories;

[0093] The unified confidence modeling specifically involves calculating the visual weight estimate based on the volume estimate and the food volume density, obtaining the weight consistency error by calculating the difference between the visual weight estimate and the food weight, and then performing exponential mapping processing on the visual uncertainty parameter, spectral uncertainty parameter, and weight consistency error to obtain visual confidence, spectral confidence, and weight confidence, and concatenating the three types of confidence to form a unified confidence vector.

[0094] The formula for calculating the visual weight estimate is as follows:

[0095] ;

[0096] In the formula, It is a visual weight estimate. V is the bulk density of the food, and V is the estimated volume.

[0097] The specific volume density of the dish is obtained from a preset dish attribute database according to the dish category, or through statistical analysis based on historical dish samples;

[0098] The formula for calculating the weight consistency error is as follows:

[0099] ;

[0100] In the formula, e w It is a weight consistency error;

[0101] The calculation formula for the exponential mapping process is as follows:

[0102] ;

[0103] In the formula, c v It represents visual credibility, and exp(·) is the natural exponential function. It is the visual attenuation coefficient. It is the visual uncertainty parameter, c s It is the reliability of the spectrum. It is the spectral attenuation coefficient, c w It's about the reliability of the weight. It is the weight attenuation coefficient;

[0104] Wherein, the visual credibility is used to characterize the reliability of the food structure and nutritional estimation results obtained based on image analysis; the spectral credibility is used to characterize the stability and distribution uncertainty of the nutritional estimation results obtained based on spectral feature inference; and the weight credibility is used to characterize the degree of physical consistency between the visual estimation results and the actual weighing results.

[0105] The dynamic gating fusion weight calculation is used to adaptively determine the fusion weight of each modality based on the unified confidence vector. Specifically, it involves inputting the unified confidence vector into the learnable gating model and normalizing it using the Softmax function to obtain the visual nutrient fusion weight, spectral nutrient fusion weight, and baseline nutrient fusion weight.

[0106] The adaptive adjustment of the fusion weights is used to detect the consistency of multimodal estimation and dynamically adjust the fusion weights. Specifically, it calculates the difference between the visual nutrient estimation result and the spectral nutrient estimation result to obtain the cross-modal consistency error. When the consistency error exceeds a preset threshold, the weights are corrected; otherwise, the fusion weights remain unchanged.

[0107] The weight correction specifically involves exponentially decaying the visual nutrient fusion weights and spectral nutrient fusion weights, and then normalizing and redistributing all fusion weights by combining them with the baseline nutrient fusion weights that have not undergone exponential decay.

[0108] The calculation formula for the exponential decay process is as follows:

[0109] ;

[0110] In the formula, It is the visual nutrient fusion weight after exponential decay. It is the visual nutrition fusion weight. It is the attenuation coefficient, e vs It is cross-modal consistency error. It is the exponentially decaying spectral nutrient fusion weight. It is the spectral nutrition fusion weight;

[0111] The multimodal fusion estimation is used to generate the final nutritional component estimation result. Specifically, based on the fusion weights that have been adaptively adjusted by the fusion weights, the visual nutritional estimation result, the spectral nutritional estimation result, and the baseline nutritional estimation result are weighted and summed to obtain the nutritional estimation result of the dish.

[0112] By performing the above operations, this solution addresses the technical problems of traditional nutritional assessment methods, which mainly rely on a single source of information, making it difficult to simultaneously consider the appearance and structure of dishes, differences in internal components, and actual meal quantities. Furthermore, the lack of an effective coordination mechanism between different information sources easily leads to significant deviations and insufficient stability in assessment results. This solution creatively adopts a nutritional assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints. In the same assessment process, multiple information sources are used collaboratively, and the differences between different information sources are dynamically adjusted through a consistency constraint mechanism. This allows each information source to correct each other, thereby obtaining stable and realistic nutritional assessment results even in complex meal-taking scenarios. This effectively improves the scientific and refined level of nutritional meal planning in school canteens.

[0113] Example 5, see Figure 1 This embodiment is based on the above embodiment. The individualized nutritional meal planning specifically involves summarizing the nutritional estimation results of dishes corresponding to each meal-taking event in the multimodal nutritional assessment based on the nutritional estimation results of dishes obtained from the multimodal nutritional assessment, obtaining the total nutritional intake of a single meal, and statistically analyzing the nutritional intake of users across multiple meals to calculate the difference between the user's current nutritional intake and the target nutritional requirements, generating a target nutritional compensation vector, and selecting dish combinations that meet the target nutritional compensation vector from the candidate dish set based on the current canteen dish nutritional data to generate an individualized meal planning recommendation scheme.

[0114] In one implementation, the personalized meal recommendation scheme is output through a terminal device. Based on the user's current nutritional intake, it recommends corresponding dish combinations and meal suggestions. When it detects that the intake of a specific nutrient is insufficient or excessive, it prioritizes recommending dishes containing the corresponding nutrient for supplementation or substitution.

[0115] By implementing the above operations, this solution addresses the technical challenges of accurately quantifying the nutritional content of food, inconsistent results from different estimation methods, and a lack of dietary guidance tailored to individual student differences. These challenges stem from the reliance on experience-based judgment in nutritional meal planning, hindering refined management. This solution involves collecting multi-dimensional information during the food preparation process and establishing nutritional correlations based on potential changes during food preparation. By comprehensively utilizing information from different sources, it achieves a stable estimate of actual nutrient intake. Furthermore, it analyzes and provides feedback on intake results based on individual student circumstances, thereby shifting from "rough meal planning" to "precise nutritional management," improving the nutritional service level of school canteens and enhancing students' health protection capabilities.

[0116] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0117] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

[0118] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. An intelligent school canteen nutrition meal preparation system, characterized in that: It includes a dish data acquisition module, a spectral nutrition mapping modeling module, a nutrition assessment module, and a nutrition meal planning module; The food data acquisition module is used to acquire food data, obtain single meal picking event data and plate-level multimodal data samples, and send the single meal picking event data to the nutrition assessment module and the plate-level multimodal data samples to the nutrition meal planning module. The spectral nutrition mapping modeling module is used for spectral nutrition mapping modeling of dishes. It adopts a spectral nutrition modeling method based on continuous recipe sampling and probability distribution mapping to obtain structured spectral nutrition modeling results, and sends the structured spectral nutrition modeling results to the nutrition assessment module. The method for modeling the spectral nutrition of dishes based on continuous recipe sampling and probability distribution mapping includes the following steps: parameterized modeling of dish composition, continuous sampling of multiple recipes, joint acquisition of spectral nutrition, enhancement of spectral features, modeling of spectral distribution, and modeling of spectral nutrition mapping relationship. The nutrition assessment module is used for multimodal nutrition assessment. Based on single meal-taking event data and structured spectral nutrition modeling results, it adopts a nutrition assessment method based on visual spectral collaborative modeling and cross-modal consistency constraints to obtain the nutritional estimation results of the dishes, and sends the nutritional estimation results of the dishes to the nutrition meal matching module. The nutritional assessment method based on visual-spectral collaborative modeling and cross-modal consistency constraints includes the following steps: visual branch modeling, spectral branch modeling, baseline nutritional estimation branch modeling, unified confidence modeling, dynamic gating fusion weight calculation, adaptive adjustment of fusion weights, and multimodal fusion estimation. The nutrition meal planning module is used for individualized nutrition meal planning to obtain individualized meal planning recommendations.

2. The intelligent school canteen nutrition meal preparation system according to claim 1, characterized in that: The parameterized modeling of the dish composition is used to construct the structural constraint space of the dish composition. Specifically, it involves parameterizing each ingredient in the dish, defining each ingredient and its corresponding proportion range, modeling the dish composition, obtaining a set of dish parameters, and thus forming the constraint space of the dish composition. The multi-recipe continuous sampling is used to generate dish samples under different ratio conditions. Specifically, for each dish, continuous random sampling is performed within the corresponding ratio range of each ingredient to generate the ratio parameters of each ingredient. Multiple sets of ingredient ratio combinations are constructed to obtain a set of dish recipe samples.

3. The intelligent school canteen nutrition meal preparation system according to claim 2, characterized in that: The aforementioned spectral nutrition joint acquisition is used to establish the correspondence between spectral data and nutritional data. Specifically, it involves acquiring spectral reflectance data for each dish recipe sample under different wavelength conditions and simultaneously obtaining the corresponding nutritional component data. The spectral reflectance data and the corresponding nutritional component data are then paired and associated to construct spectral nutrition paired data. The spectral feature enhancement is used to improve the ability of spectral features to express nutritional changes. Specifically, it involves calculating the derivative of the spectral reflectance data and standardizing it to obtain standardized spectral feature data. Then, it extracts the main feature components from the standardized spectral feature data through a dimensionality reduction method and introduces nonlinear kernel mapping for enhancement to obtain a spectral enhanced feature vector.

4. The intelligent school canteen nutrition meal preparation system according to claim 3, characterized in that: The spectral distribution modeling is specifically based on spectral nutrient pairing data. By performing probability density modeling on the spectral enhancement feature vector, multiple spectral distribution clusters are constructed to obtain a spectral feature probability distribution model. The mean, covariance, and weight of each spectral distribution cluster are estimated. Then, combined with the nutrient component data corresponding to each spectral distribution cluster, conditional probability modeling is performed to obtain a nutrient conditional distribution model. The correlation between spectral distribution and nutrient distribution is established, realizing the transformation from single-point mapping to distribution modeling. The spectral nutrient mapping model is specifically based on the spectral feature probability distribution model. It constructs a spectral nutrient mapping mechanism consisting of attribution probability calculation and probability weighting fusion to realize the conditional expectation mapping from spectral feature vectors to nutrient components, thereby obtaining the spectral nutrient mapping model.

5. The intelligent school canteen nutrition meal preparation system according to claim 4, characterized in that: The visual branch modeling is used to perform structured visual analysis and nutritional estimation of food images. Specifically, it involves inputting food images from a single food collection event into a convolutional neural network for feature extraction to obtain a visual feature vector. Based on the visual feature vector, it outputs the food category probability, ingredient ratio vector, volume estimate, and visual uncertainty parameter. The visual nutritional estimation result is obtained by combining the food weight value, ingredient ratio vector, and nutritional vector per unit mass. The spectral branch modeling is used to perform probabilistic modeling and uncertainty characterization of spectral components in spectral data. Specifically, it extracts spectral feature vectors from single meal-taking event data, inputs the spectral feature vectors into the spectral nutrition mapping relationship model, obtains the conditional expectation of nutrient components as the spectral nutrition estimation result, and performs weighted variance calculation by combining the nutrient distribution information corresponding to each distribution cluster to obtain the corresponding conditional variance as the spectral uncertainty parameter. The benchmark nutrient estimation branch modeling is used to estimate the benchmark nutrient based on statistical nutrient density, specifically by calculating the benchmark nutrient estimation result based on the average nutrient density of the dish and the weight value of the dish.

6. The intelligent school canteen nutrition meal preparation system according to claim 5, characterized in that: The unified confidence modeling specifically involves calculating the visual weight estimate based on the volume estimate and the food volume density, obtaining the weight consistency error by calculating the difference between the visual weight estimate and the food weight, and then performing exponential mapping processing on the visual uncertainty parameter, spectral uncertainty parameter, and weight consistency error to obtain visual confidence, spectral confidence, and weight confidence, and concatenating the three types of confidence to form a unified confidence vector. Wherein, the visual credibility is used to characterize the reliability of the food structure and nutritional estimation results obtained based on image analysis; the spectral credibility is used to characterize the stability and distribution uncertainty of the nutritional estimation results obtained based on spectral feature inference; and the weight credibility is used to characterize the degree of physical consistency between the visual estimation results and the actual weighing results. The dynamic gating fusion weight calculation is used to adaptively determine the fusion weight of each modality based on the unified confidence vector. Specifically, it involves inputting the unified confidence vector into the learnable gating model and normalizing it using the Softmax function to obtain the visual nutrient fusion weight, spectral nutrient fusion weight, and baseline nutrient fusion weight.

7. The intelligent school canteen nutrition meal preparation system according to claim 6, characterized in that: The adaptive adjustment of the fusion weights is used to detect the consistency of multimodal estimation and dynamically adjust the fusion weights. Specifically, it calculates the difference between the visual nutrient estimation result and the spectral nutrient estimation result to obtain the cross-modal consistency error. When the consistency error exceeds a preset threshold, the weights are corrected; otherwise, the fusion weights remain unchanged. The weight correction specifically involves exponentially decaying the visual nutrient fusion weights and spectral nutrient fusion weights, and then normalizing and redistributing all fusion weights by combining them with the baseline nutrient fusion weights that have not undergone exponential decay. The multimodal fusion estimation is used to generate the final nutritional component estimation result. Specifically, based on the fusion weights that have been adaptively adjusted by the fusion weights, the visual nutritional estimation result, the spectral nutritional estimation result, and the baseline nutritional estimation result are weighted and summed to obtain the nutritional estimation result of the dish.

8. The intelligent school canteen nutrition meal preparation system according to claim 7, characterized in that: The personalized nutritional meal planning specifically involves summarizing the nutritional estimation results of dishes corresponding to each meal-taking event in the multimodal nutritional assessment based on the nutritional estimation results of the dishes obtained from the plate-level multimodal data sample, obtaining the total nutritional intake of a single meal, and statistically analyzing the nutritional intake of the user across multiple meals to calculate the difference between the user's current nutritional intake and the target nutritional requirements, generating a target nutritional compensation vector, and selecting dish combinations that meet the target nutritional compensation vector from the candidate dish set based on the current canteen dish nutritional data to generate a personalized meal planning recommendation scheme.

9. The intelligent school canteen nutrition meal preparation system according to claim 8, characterized in that: The food data collection is used to acquire multimodal synchronous data during the student's meal pick-up process and construct structured multimodal data samples. Specifically, it involves detecting the meal pick-up behavior, generating a meal pick-up transaction identifier when a user starts picking up food, collecting the image data, spectral data, and weight data of the corresponding dishes, associating and binding them with time sequence alignment, constructing single meal pick-up event data, and aggregating multiple meal pick-up events to form a plate-level multimodal data sample.

Images