Diet data analysis method, system and device based on intelligent equipment

Abstract

The invention provides a diet data analysis method, system and device based on intelligent equipment, and aims to solve the problems that traditional diet analysis depends on manpower, recognition is single, dynamic states in meals are ignored, and food materials and cooking processes cannot be reversely deduced. Comprising the steps of pre-meal static multi-mode scanning, physical decoupling analysis, in-meal dynamic interaction analysis, cross-spatio-temporal data fusion and probability traceability. The system comprises the intelligent tableware integrated with multiple modules / sensors, a data processing unit for executing an algorithm and a result display unit for displaying a report. According to the method, the component identification precision and the nutrition estimation accuracy are improved, the whole-process accurate analysis of the diet data from before meal to during meal is realized, and reliable data support is provided for diet health management.

Description

Technical Field

[0001] This invention belongs to the field of dietary health and relates to a method, system, and device for dietary data analysis based on intelligent devices. Background Technology

[0002] To ensure daily nutritional needs and dietary health, recipes are an important basis for residents' balanced diets. It is necessary to identify dishes to obtain their components. This is especially true in the field of Chinese cuisine, where the same dishes are more difficult to identify than ordinary objects due to differences in cooking methods, heat control, and the high degree of similarity in the appearance of some different dishes.

[0003] Traditional dietary analysis often relies on manual weighing, subjective recording, or single image recognition, only obtaining basic information such as the weight and general category of dishes. It cannot accurately capture the deep physical properties and microscopic components of dishes, focusing only on the state of the dishes before the meal and completely ignoring the dynamic interaction process during the user's meal, such as the changes in texture when the ingredients are picked up, the separation of components, and the actual intake. This kind of analysis, which only looks at the starting point and ignores the process, cannot reflect the user's true eating behavior and nutritional intake.

[0004] Traditional food analysis cannot deduce the original ingredient list and cooking process from the finished dish; it can only rely on user memory or recipes for inference, which is highly subjective and prone to error. Summary of the Invention

[0005] To address the problems existing in the background technology, this application proposes a method, system, and device for analyzing dietary data based on smart devices.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows: On the one hand, the present invention provides a method for analyzing dietary data based on smart devices, including the following steps: Pre-meal static multimodal scanning involves acquiring multimodal data of the dishes using hyperspectral imaging, 3D structured light scanning, and thermal imaging. Physical decoupling analysis decomposes the collected multimodal data into physical property primitives, which include texture, geometric shape, spectral characteristics and temperature; Dynamic interactive analysis during meals: Smart tableware monitors mechanical changes, visual trajectory changes, and weight changes during the user's eating process. Cross-temporal and spatial data fusion aligns and merges static scan data with dynamic interactive data on a timeline; Probabilistic tracing involves reverse matching of physical property primitives using a cooking physics simulator to deduce the original ingredient list and cooking process.

[0007] Furthermore, the hyperspectral imaging scan identifies the distribution and content of oil, sugar, starch, protein, and water in the dish by capturing hundreds of narrow-band image data.

[0008] Furthermore, the 3D structured light scanning acquires a millimeter-precision 3D model of the dish, which is used to accurately measure the volume of the ingredients and analyze the surface texture.

[0009] Furthermore, the thermal imaging temperature field analysis uses a thermistor array or thermal imager to plot the temperature distribution of the dish, which is used to infer the oil temperature and the state of the ingredients.

[0010] Furthermore, the physical decoupling analysis step uses a geometric constraint optimization algorithm to recombine the physical property primitives of the multimodal data decomposition in three-dimensional space.

[0011] Furthermore, the dynamic interactive analysis during the meal includes measuring the hardness, elasticity, adhesion, and crispness of the ingredients using miniature piezoresistive and torque sensors built into the smart chopsticks.

[0012] Furthermore, the dynamic interactive analysis also includes continuously filming the chopsticks picking up food using a camera inside the bowl to analyze the component separation process and the internal structure exposure process.

[0013] Furthermore, the dynamic interactive analysis also includes using a high-precision pressure sensor inside the smart bowl to record weight changes in real time and accurately calculate the mass of each component.

[0014] Furthermore, the cooking physics simulator has a built-in physical model of the cooking process, which derives the original ingredients and cooking process by reverse matching the physical attribute primitives with the initial ingredient state.

[0015] Furthermore, the method ultimately generates a component traceability report that includes main ingredients, sauce components, seasonings, nutritional estimates, and confidence analyses.

[0016] On the other hand, the present invention also provides a diet data analysis system based on a smart device for implementing the above method, comprising: The smart tableware integrates a hyperspectral imaging module, a 3D structured light scanning module, a thermal imaging module, and a high-precision pressure sensor; it also integrates a miniature piezoresistive sensor, a torque sensor, and a communication module. The data processing unit is configured to perform physical decoupling analysis, dynamic interaction analysis, and probabilistic tracing algorithms. The results display unit is used to display the ingredient traceability report.

[0017] On the other hand, the present invention also provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the program to implement the above-described method.

[0018] Compared with the prior art, this application has the following beneficial effects: This invention utilizes hyperspectral imaging to accurately identify the spatial distribution of components such as oils, sugars, and proteins through hundreds of narrow-band images; 3D structured light scanning acquires millimeter-precision 3D models to accurately calculate food volume; and thermal imaging creates temperature distribution maps to infer oil temperature and food doneness. Through a geometric constraint optimization algorithm, multimodal data is decomposed into physical property primitives such as texture, geometric shape, spectral characteristics, and temperature, and then reconstructed in three-dimensional space to fully restore the physical and component characteristics of the dish. Compared to traditional single-dimensional analysis, this method improves component identification accuracy by over 80%, reduces nutrient estimation error from ±30% to ±5%, and can accurately distinguish component differences in different regions within a dish.

[0019] The miniature piezoresistive / torque sensor of the smart tableware of this invention measures the hardness, elasticity, and adhesion of food in real time, reflecting the eating experience and actual edibility of the dishes. The high-precision pressure sensor of the smart tableware records weight changes in real time. This dynamic process monitoring upgrades the nutritional intake analysis from estimating the total amount before meals to accurately calculating the actual intake, improving the matching degree with the real situation.

[0020] This invention aligns pre-meal static multimodal data with in-meal dynamic data along a timeline. For example, if a user picks up a high-oil area on the surface of a dish, the system can combine hyperspectral oil distribution data to accurately calculate the oil intake of that bite. If the user only eats the staple food and leaves the vegetables, the system can also correct the nutritional calculation results by matching the weight changes with the 3D model.

[0021] This invention's cooking physics simulator incorporates a physical model of the cooking process. By reverse matching physical property primitives and combining the temperature distribution of thermal imaging with the characteristics of oil in hyperspectral imaging, it reverse-engineers the preparation of dishes. It also matches the original ingredients using 3D geometric shape and texture data. Detailed Implementation

[0022] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] Example: On one hand, this invention provides a dietary data analysis method based on smart devices. This embodiment uses the dietary management of Mr. Zhang (30 years old, a fitness enthusiast who needs to strictly control his daily protein intake to ≥120g and sodium intake to ≤2000mg) as a scenario, and the smart tableware used is bowls and chopsticks. Mr. Zhang eats a takeout "Kung Pao Chicken" for dinner. The system needs to accurately analyze its ingredients (chicken, peanuts, cucumber, chili, scallions, sauce, etc.), cooking method, and nutritional content to provide data support for the day's dietary record. The method includes the following steps: Pre-meal static multimodal scanning involves acquiring multimodal data of the dishes using hyperspectral imaging, 3D structured light scanning, and thermal imaging. Physical decoupling analysis decomposes the collected multimodal data into physical property primitives, which include texture, geometric shape, spectral characteristics and temperature; Dynamic interactive analysis during meals: Smart tableware monitors mechanical changes, visual trajectory changes, and weight changes during the user's eating process. Cross-temporal and spatial data fusion aligns and merges static scan data with dynamic interactive data on a timeline; Probabilistic tracing involves reverse matching of physical property primitives using a cooking physics simulator to deduce the original ingredient list and cooking process.

[0024] Kung Pao Chicken has a complex composition: it contains fried peanuts (high in fat), stir-fried chicken (high in protein), various vegetables (fiber), and mixed sauces (high in sugar and sodium). The ingredients are of different sizes and stick together (such as the chicken coated with sauce and peanuts mixed with scallions). Traditional image recognition is prone to misjudging the proportion of ingredients, especially in distinguishing between "fried peanuts" and "cashews" and "the sugar and salt content in the sauce". It is necessary to rely on the "dynamic multimodal deconstruction-source analysis" framework to achieve accurate deconstruction.

[0025] The bowl's rim integrates a miniature hyperspectral camera (5nm resolution) in the 400-1700nm band, acquiring 3 frames per second, covering the visible to near-infrared spectrum for chemical component identification. A ToF (Time-of-Flight) sensor with 0.1mm accuracy is embedded in the center of the bowl's bottom to construct a 3D model of the dish. An 8×8 pixel thermistor array is built into the bowl's wall, measuring temperatures from -20 to 150℃ with ±0.5℃ accuracy, recording temperature distribution. High-precision pressure sensors with a range of 0-500g and 0.1g accuracy are deployed at the four corners of the bowl's bottom to monitor weight changes in real time.

[0026] The chopsticks feature a built-in piezoresistive force sensor (range 0-5N, accuracy 0.01N) and torque sensor (range 0-0.5N·m) at the tip to measure gripping force and deformation. A 2-megapixel macro lens with a focal length of 5-10mm is integrated in the middle section of the chopsticks to capture detailed images of food being gripped.

[0027] When the Kung Pao Chicken was placed in the smart bowl, the system automatically triggered a 30-second static scan. It identified a strong signal in the blocky region (chicken meat) at 760nm (protein characteristic peak); a strong signal in the granular region (peanuts) at 1200nm (oil characteristic peak); a medium signal in the strip-shaped region (cucumber and scallion segments) at 900nm (moisture characteristic peak); and a strong signal in the liquid region (sauce) covering the surface of the chicken at 1500nm (sugar characteristic peak). The preliminary composition analysis indicated that the chicken (high protein, medium moisture), peanuts (high fat), cucumber / scallion segments (high moisture, high fiber), and sauce (high sugar, high sodium).

[0028] 3D models were generated: the average volume of the largest chunk (chicken) was approximately 1.2 cm³ / piece (18 pieces in total), the volume of the granular objects (peanuts) was approximately 0.3 cm³ / piece (12 pieces in total), the volume of the strip-shaped objects (cucumber / scallion segments) was approximately 0.8 cm³ / segment (15 segments in total), and the average thickness of the sauce coating was 0.2 cm. Surface texture analysis included smooth peanut surfaces with minor charring marks on the edges (demonstrated deep-frying characteristics) and rough chicken surfaces (demonstrated stir-frying characteristics).

[0029] Temperature distribution: Chicken center temperature 78℃, peanut temperature 82℃ (higher than chicken, due to longer residual heat retention after frying), sauce temperature 65℃, cucumber slice temperature 55℃ (due to high moisture content and rapid heat dissipation). The high-temperature characteristics of the peanuts match the frying process, ruling out the possibility of "raw peanuts".

[0030] The above data was decomposed into 5 categories of physical property units: Unit 1: {Texture: tender and springy, Geometry: irregular blocky, Spectrum: high protein (strong at 760nm), Temperature: 78℃} → Preliminary identification: chicken; Unit 2: {Texture: firm, Geometry: near-spherical particles, Spectrum: high fat (strong at 1200nm), Temperature: 82℃} → Preliminary identification: fried peanuts; Unit 3: {Texture: crisp and tender, Geometry: strip-shaped, Spectrum: high moisture (strong at 900nm), Temperature: 55℃} → Preliminary identification: cucumber / scallion segments; Unit 4: {Texture: viscous, Geometry: thin film covering, Spectrum: high sugar (strong at 1500nm), Temperature: 65℃} → Preliminary identification: sauce; Unit 5: {Texture: relatively firm, Geometry: thin strip-shaped, Spectrum: medium fiber, Temperature: 60℃} → Preliminary identification: chili segments.

[0031] When using smart chopsticks for dining, the system initiates real-time dynamic monitoring for approximately 15 minutes. When picking up chicken, a force of 1.2N was applied, resulting in a 0.3mm deformation. This elasticity coefficient matches the characteristics of well-cooked, stir-fried chicken, ruling out the softness of stewed meat or the hardness of raw meat. When picking up peanuts, the force-displacement curve showed a steep peak at the moment of crushing; after a 0.8N force, the peanuts deformed 0.1mm and then suddenly broke, with a vibration frequency of 200Hz, consistent with the crispness of fried peanuts, ruling out the softness of boiled peanuts. When picking up chicken covered in sauce, the adhesion force upon removing the chopsticks was 0.5N, higher than the 0.2N for pure chicken, suggesting a sauce viscosity of approximately 500cP, corresponding to a medium consistency and thickened with starch.

[0032] When chopsticks picked up a piece of chicken, three peanuts and a piece of scallion were lifted and detached. The system, through trajectory comparison, determined that the peanuts, scallion, and chicken were physically mixed rather than stuck together, correcting the error of overlapping volumes observed in static scanning. Opening a larger piece of chicken, a macro lens captured the internal fibrous structure. Combined with hyperspectral scanning, this confirmed uniform protein distribution, eliminating any undercooked areas. The system also recorded the weight change of each piece picked up in real time.

[0033] Align the pre-meal static data with the in-meal dynamic data on the timeline, using the clamping action as the time anchor point, and reverse-engineer the data using a cooking physics simulator.

[0034] The peanuts were deduced as follows: combining "high temperature of 82℃ + brittleness vibration frequency of 200Hz + oil spectrum", the simulator matched the process characteristics of "frying at 180℃ for 30 seconds" rather than roasting, because roasting temperature is usually ≤150℃, and the brittleness is lower.

[0035] The chicken is designed based on the process of "78℃ temperature + 1.2N clamping force deformation + fiber structure" and "stir-frying at 120℃ oil for 2 minutes" (70% cooked, meeting the standard of "tender but not raw" for Kung Pao Chicken).

[0036] The sauce is estimated to contain "500 cP viscosity + high sugar spectrum + 0.5 N adhesion" based on the simulator. The formula is "20% soy sauce, 15% sugar, 5% starch, and 60% water" (the starch thickening process increases the viscosity).

[0037] The system outputs the final analysis results, which include the confidence level.

[0038] It also includes using a high-precision pressure sensor inside the smart bowl to record weight changes in real time and accurately calculate the mass of each component.

[0039] On the other hand, the present invention also provides a diet data analysis system based on a smart device for implementing the above method, comprising: The smart tableware integrates a hyperspectral imaging module, a 3D structured light scanning module, a thermal imaging module, and a high-precision pressure sensor; it also integrates a miniature piezoresistive sensor, a torque sensor, and a communication module. The data processing unit is configured to perform physical decoupling analysis, dynamic interaction analysis, and probabilistic tracing algorithms. The results display unit is used to display the ingredient traceability report.

[0040] On the other hand, the present invention also provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the program to implement the above-described method.

[0041] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for analyzing dietary data based on smart devices, characterized in that, Includes the following steps: Pre-meal static multimodal scanning involves acquiring multimodal data of the dishes using hyperspectral imaging, 3D structured light scanning, and thermal imaging. Physical decoupling analysis decomposes the collected multimodal data into physical property primitives, which include texture, geometric shape, spectral characteristics and temperature; Dynamic interactive analysis during meals: Smart tableware monitors mechanical changes, visual trajectory changes, and weight changes during the user's eating process. Cross-temporal and spatial data fusion aligns and merges static scan data with dynamic interactive data on a timeline; Probabilistic tracing involves reverse matching of physical property primitives using a cooking physics simulator to deduce the original ingredient list and cooking process.

2. The method according to claim 1, characterized in that, The hyperspectral imaging scan identifies the distribution and content of oil, sugar, starch, protein, and water in dishes by capturing hundreds of narrow-band image data.

3. The method according to claim 1, characterized in that, The 3D structured light scanning acquires a millimeter-precision 3D model of the dish, which is used to accurately measure the volume of the ingredients and analyze the surface texture.

4. The method according to claim 1, characterized in that, The thermal imaging temperature field analysis uses a thermistor array or thermal imager to draw a temperature distribution map of the dish, which is used to infer the oil temperature and the state of the ingredients.

5. The method according to claim 1, characterized in that, The physical decoupling analysis step uses a geometric constraint optimization algorithm to recombine the physical property primitives of multimodal data decomposition in three-dimensional space.

6. The method according to claim 1, characterized in that, The dynamic interactive analysis during the meal includes measuring the hardness, elasticity, adhesion, and crispness of the ingredients using miniature piezoresistive and torque sensors built into the smart tableware. The smart tableware uses a high-precision pressure sensor to record weight changes inside the bowl in real time and accurately calculate the mass of each component.

7. The method according to claim 1, characterized in that, The dynamic interactive analysis also includes continuously filming the chopsticks picking up food using a camera inside the bowl to analyze the component separation process and the internal structure exposure process.

8. The method according to claim 1, characterized in that, The cooking physics simulator has a built-in physical model of the cooking process. It derives the original ingredients and cooking process by reverse matching the physical property primitives with the initial ingredient state. The method ultimately generates a component traceability report that includes main ingredients, sauce components, seasonings, nutritional estimates, and confidence analyses.

9. A diet data analysis system based on a smart device that implements the method of any one of claims 1-8, characterized in that, include: The smart tableware integrates a hyperspectral imaging module, a 3D structured light scanning module, a thermal imaging module, and a high-precision pressure sensor; it also integrates a miniature piezoresistive sensor, a torque sensor, and a communication module. The data processing unit is configured to perform physical decoupling analysis, dynamic interaction analysis, and probabilistic tracing algorithms. The results display unit is used to display the ingredient traceability report.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the processor executes the program, it implements the method as described in any one of claims 1-8.