Eating pattern analysis system, method, and program
The eating pattern analysis system quantitatively measures and analyzes eating patterns using camera images to categorize and extract characteristic eating patterns, enhancing dietary guidance and nutritional epidemiology research.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- KDDI CORP
- Filing Date
- 2023-09-28
- Publication Date
- 2026-07-22
AI Technical Summary
Existing methods fail to objectively quantify and analyze eating patterns of a large number of diners, making it difficult to provide individualized dietary guidance and contribute to nutritional epidemiology research.
An eating pattern analysis system that photographs food consumption, estimates the remaining rate of each food item, categorizes time series data into representative patterns, and extracts characteristic eating patterns using statistical methods like latent class mixture models and principal component analysis.
Enables the categorization of eating habits, broadening the scope of dietary guidance and contributing to nutritional epidemiology research by revealing healthy eating patterns in real-life scenarios.
Smart Images
Figure 0007893793000001 
Figure 0007893793000002 
Figure 0007893793000003
Abstract
Description
Technical Field
[0001] The present invention relates to a eating pattern analysis system, method, and program, and particularly to a system, method, and program for analyzing eating patterns based on a camera image of food taken by a diner.
Background Art
[0002] It is said that eating vegetables first is a healthy way of eating, but this has been demonstrated by intervention studies with instructions on eating methods. Also, the subjects are often diabetic patients who are receiving clinical guidance.
[0003] However, in real life, diners consume various foods in different amounts and in their own ways of eating. In such a dining scene of general diners, it is not clear what eating patterns exist and how they affect disease risks and health.
[0004] If the pattern of a healthy way of eating in real life can be clarified, it has great social significance such as greatly expanding the scope of diet guidance. Specifically, if the eating behavior of a diner can be evaluated by combining not only the content of the meal but also the pattern of eating, the speed of eating, the number of chewing times, etc., individualized detailed diet guidance will be possible.
[0005] In addition, nutritional epidemiology research studies the relationship between food / nutrient intake and disease risks, but there is no study that objectively measures eating methods and the speed of eating in observational studies and evaluates their relationship with disease risks. Therefore, if the pattern of eating methods can be classified, it will also greatly contribute to the development of nutritional epidemiology research.
[0006] Prior art for analyzing eating habits includes Patent Document 1 and Non-Patent Document 1. Patent Document 1 discloses a meal system and analysis system that allows diners to experience the enjoyment of eating by stimulating their vision and other senses. Non-Patent Document 2 discloses a technology that uses a camera to film the progress of food and drink consumption during a meal and analyzes eating habits through video analysis. [Prior art documents] [Patent Documents]
[0007] [Patent Document 1] Japanese Patent Publication No. 2021-049337 [Non-patent literature]
[0008] [Non-Patent Document 1] Real-time meal management system (Kyocera Corporation) (https: / / www.kyocera.co.jp / ceatec / #anc11: Confirmed September 17, 2023) [Overview of the project] [Problems that the invention aims to solve]
[0009] To clarify individual eating habits, it is necessary to observe and measure actual mealtimes. Similarly, to clarify eating patterns present in the general population, with the Japanese population being the largest sample size, it is necessary to observe and measure the actual eating habits of a large number of diners.
[0010] However, there is no established method for observing and measuring the eating habits of a group of diners. Because the time it takes for each diner to eat and the content of their meals vary, it is difficult to quantitatively measure and statistically analyze the actual eating habits of a large number of diners under the same conditions.
[0011] Furthermore, there was no clear definition of "eating patterns" that could be useful for dietary guidance in the general population, nor was there an established method for analyzing them.
[0012] The objective of the present invention is to solve the above technical problems and to provide a system, method, and program that can quantitatively measure and analyze eating patterns useful for dietary guidance based on camera images of food consumed by a person. [Means for solving the problem]
[0013] To achieve the above objective, the present invention is characterized by having the following configuration in an eating pattern analysis system for analyzing the eating patterns of a group of diners.
[0014] (1) The system comprises means for photographing the food being consumed by each eater in a group of eaters, means for estimating the remaining rate of each food item for each eater in a time series based on the photographed images, means for categorizing the time series of each eater's remaining rate into several representative eating patterns, and means for extracting characteristic eating patterns from the categorized representative eating patterns.
[0015] (2) The categorization means comprises means for normalizing the time series scale of the survival rate for each eater by mealtime, and means for extracting a number of representative eating patterns for each food based on the normalized time series of the survival rate for the eater group, so that the time series of the survival rate for each eater is represented by one of the representative eating patterns for each food.
[0016] Furthermore, the present invention can be realized not only as an eating pattern analysis system equipped with such characteristic means, but also as an eating pattern analysis method in which such characteristic means are used as steps, or as an eating pattern analysis program that causes a computer to execute each step. [Effects of the Invention]
[0017] According to the present invention, since patterns of eating habits can be categorized, the scope of dietary guidance can be greatly broadened by clarifying patterns of healthy eating habits in real life, and as a result, it will be possible to make a significant contribution to the development of nutritional epidemiology research.
Brief Description of the Drawings
[0018] [Figure 1] It is a functional block diagram showing the configuration of the main part of a eating pattern analysis system according to an embodiment of the present invention. [Figure 2] It is a diagram schematically showing an example of a method for estimating a residual rate time series for each food item by a residual rate estimation unit. [Figure 3] [[ID=I3]]It is a diagram schematically showing an example in which a typification unit extracts a representative pattern of residual rate time series data. [Figure 4] It is a diagram schematically showing an example of typifying residual rate time series data. [Figure 5] It is a diagram schematically showing an example of a method (Part 1) for extracting a characteristic eating pattern from the typified eating patterns. [Figure 6] It is a diagram schematically showing an example of a method (Part 2) for extracting a characteristic eating pattern from the typified eating patterns.
Embodiments for Carrying Out the Invention
[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of the main part of an eating pattern analysis system 1 according to an embodiment of the present invention, and includes a photographing unit 10, a classification unit 20, a residual rate estimation unit 30, a typification unit 40, and an eating pattern extraction unit 50 as main components.
[0020] )]] Such an eating pattern analysis system 1 can be configured by implementing an application (program) that realizes each function described in detail below on a general-purpose computer or server equipped with a CPU, ROM, RAM, bus, interface, etc. Alternatively, it can also be configured as a dedicated machine or single-function machine in which part of the application is hardwareized or softwareized.
[0021] The camera unit 10 is equipped with a 3D camera and an RGB-D camera capable of acquiring depth information, and it photographs each food item consumed by the diner during the mealtime, from the start to the end of the meal.
[0022] The classification unit 20 divides each food item into a small number of meal items based on the video footage of the food. In this embodiment, each food item is classified into one of four meal items: "staple food," "main dish," "side dish," and "other," based on its attributes such as nutrients and taste. Note that the classification of each food item may be performed manually in advance.
[0023] In this embodiment, foods relatively rich in carbohydrates are classified as "staple foods," foods relatively rich in protein are classified as "main dishes," and foods relatively rich in dietary fiber and vitamins are classified as "side dishes." Foods that cannot be classified into any of these categories are classified as "other."
[0024] The survival rate estimation unit 30 estimates the survival rate of each meal item for each diner in a time series, based on the video footage of the food and the volume changes measured for each meal item based on the classification results.
[0025] In this embodiment, as shown in Figure 2, the survival rate of each meal item, categorized as "staple food," "main dish," "side dish," and "other," is estimated at predetermined intervals during the meal period from the start to the end of eating. This survival rate estimation is performed for each eater (ID1, ID2, etc.) targeting a large number of eaters (eating group), and the estimated results of the survival rate time series are stored in a database (not shown).
[0026] The categorization unit 40 comprises a normalization unit 401 and a representative pattern extraction unit 402, and categorizes the numerous time-series data on the remaining rates of "staple food," "main dish," "side dish," and "other" obtained for each diner (ID1, ID2, etc.) into multiple representative eating patterns.
[0027] In this embodiment, the normalization unit 401 first normalizes the time scale of the survival rate time series data for each diner to the meal duration from the start time to the end time of eating. Therefore, for the same diner, the time scales of the survival rate time series data for "staple food," "main dish," "side dish," and "other" will be the same, but they will differ between diners with long meal times and those with short meal times.
[0028] The start time of a meal can be detected as the time when the remaining percentage of any meal item begins to decrease. The end time of a meal can be detected as the time when the remaining percentage of all meal items or the main meal item becomes zero, or as the time when the last change in the remaining percentage is detected after a period of no changes in the remaining percentage have been detected. The duration of the meal can be detected as the time from the start time to the end time.
[0029] The representative pattern extraction unit 402 applies a latent class mixture model (LCMM) or a similar statistical analysis method for classifying time series data to the survival rate time series data of all eaters for each meal item, and represents the survival rate time series data of all eaters with one of a few eating patterns.
[0030] In this embodiment, as shown in Figure 3 as an example, the time-series data of the remaining rate for each meal item, categorized as "staple food," "main dish," "side dish," and "other," is represented by one of several representative patterns (in this embodiment, three patterns: pattern 1, pattern 2, and pattern 3).
[0031] As described above, when the time-series data of each diner's survival rate for each meal item is represented by one of four representative patterns, the categorization unit 40 generates eating pattern data that centrally manages which representative pattern represents the time-series data of the survival rates of each diner (ID) for "staple food," "main dish," "side dish," and "other."
[0032] In the example in Figure 4, the time-series data on the survival rate of staple foods for meal recipient ID 1 is represented by Staple Food Pattern 2 (Staple Food 2), the time-series data on the survival rate of main dishes is represented by Main Dish Pattern 2 (Main Dish 2), the time-series data on the survival rate of side dishes is represented by Side Dish Pattern 1 (Side Dish 1), and the time-series data on the survival rate of other foods is represented by Other Pattern 1 (Other 1).
[0033] Similarly, the time-series data on the survival rate of staple foods for meal recipient ID 2 is represented by Staple Food Pattern 2, the time-series data on the survival rate of main dishes is represented by Main Dish Pattern 2, the time-series data on the survival rate of side dishes is represented by Side Dish Pattern 3, and the time-series data on the survival rate of other foods is represented by Other Pattern 1 (Other 1).
[0034] The eating pattern extraction unit 50 extracts eating patterns characteristic of the group of eaters based on the categorized eating patterns. Here, two feature extraction methods will be explained as examples.
[0035] Figure 5 shows an example of feature extraction based on principal component analysis and factor loadings. First, principal component analysis is applied to the data of the categorized eating patterns, and the dimensionality is reduced to a number of potential composite variables (in this embodiment, the first to fourth principal components) in order of least information loss.
[0036] Next, the influence of each principal component is quantified by calculating the factor loadings using principal component analysis. Specifically, the influence of each composite variable is quantified based on whether the factor loading is close to "1", "-1" (both strongly correlated with the principal component), or "0" (weakly correlated with the principal component).
[0037] Figure 6 shows an example of creating combinations by arbitrarily selecting characteristic patterns from the representative patterns of the categorized eating patterns described above. By comprehensively combining all the selected characteristic patterns, a large number of "eating patterns" can be created.
[0038] In this embodiment, we focus on "staple food," "main dish," and "side dish" from the four meal items, excluding "other." From the three representative patterns extracted for each of these three meal items, we select two characteristic patterns for each. By comprehensively combining these selected representative patterns, characteristic eating patterns (1) to (8) are extracted. Alternatively, we may focus on all four meal items and create patterns.
[0039] In the illustrated example, staple food patterns 1 and 2 are selected for the staple food, main dish patterns 2 and 3 are selected for the main dish, and patterns 1 and 2 are selected for the side dish. Therefore, the combination "staple food pattern 1 - main dish pattern 2 - side dish pattern 1" is extracted as "eating method (1)". Similarly, the combination "staple food pattern 1 - main dish pattern 2 - side dish pattern 2" is extracted as "eating method (2)".
[0040] In this way, by arbitrarily selecting characteristic patterns from representative patterns of categorized eating patterns and comprehensively creating combinations thereof, it becomes easier to search for typical eating patterns corresponding to the eating patterns of the person being supported.
[0041] In the above embodiment, the food consumed by the diner was described as being classified into "staple foods" rich in carbohydrates, "main dishes" rich in protein, "side dishes" rich in dietary fiber and vitamins, and "others" based on their nutrients. However, the present invention is not limited to this, and classifications may also be made based on other major nutrients such as lipids and sugars.
[0042] Furthermore, instead of focusing on nutrients, foods can be classified based on taste (sweet, bitter, umami, salty, sour), on usage (rice, side dishes, soups, desserts, etc.), on calories per unit weight, on the classification of six basic food groups, on the classification of ten food groups, on the classification of three color food groups, or in combination of these.
[0043] According to this embodiment, it is possible to categorize the eating patterns of a group of people and extract characteristic eating patterns, thereby greatly expanding the scope of dietary guidance by revealing healthy eating patterns in real life.
[0044] Furthermore, according to this embodiment, since the eating patterns of eaters are pre-converted into data where the time series scale of the survival rate is normalized by mealtime, it becomes possible to handle and analyze data from diverse eater groups with varying amounts and rates of food intake in a similar manner.
[0045] Furthermore, according to the above embodiment, eating patterns can be categorized, thereby revealing healthy eating patterns in real life and greatly expanding the scope of dietary guidance. As a result, it becomes possible to contribute to Goal 3 of the United Nations-led Sustainable Development Goals (SDGs), "Ensure healthy lives and promote well-being for all at all ages." [Explanation of symbols]
[0046] 10...Photography unit, 20...Classification unit, 30...Survival rate estimation unit, 40...Typology unit, 401...Regulation unit, 402...Representative pattern extraction unit
Claims
1. In a system for analyzing the eating patterns of a group of diners, A means of photographing the food being consumed by each person in a group of diners, A means for estimating the retention rate of each food item for each diner over time, based on the aforementioned captured video footage, A method for classifying the time series of survival rates for each diner using multiple representative eating patterns, The system comprises means for extracting characteristic eating patterns from the aforementioned categorized representative eating patterns, The aforementioned means of categorization is, A method for normalizing the time scale of the survival rate time series for each diner by mealtime, The system comprises means for extracting multiple representative eating patterns for each food item based on the time series of the normalized survival rate of the group of eaters, and for representing the time series of the survival rate of each eater for each food item with one of the representative eating patterns. The means for extracting the characteristic eating patterns is a eating pattern analysis system characterized by extracting eating patterns characteristic of the group of eaters from the factor loadings of the principal component analysis of the typified representative eating patterns.
2. In a eating pattern analysis system for analyzing the eating patterns of a group of diners, A means of photographing the food being consumed by each person in a group of diners, A means for estimating the retention rate of each food item for each diner over time, based on the aforementioned captured video footage, A method for classifying the time series of survival rates for each diner using multiple representative eating patterns, The system comprises means for extracting characteristic eating patterns from the aforementioned categorized representative eating patterns, The aforementioned means of categorization is, A method for normalizing the time scale of the survival rate time series for each diner by mealtime, The system comprises means for extracting multiple representative eating patterns for each food item based on the time series of the normalized survival rate of the group of eaters, and for representing the time series of the survival rate of each eater for each food item with one of the representative eating patterns. The means for extracting the characteristic eating patterns is an eating pattern analysis system characterized by selecting some representative eating patterns for each food item from the categorized representative eating patterns, and comprehensively combining the selected representative eating patterns to extract characteristic eating patterns.
3. The system includes means for classifying each food item shown in the captured video into multiple types of food items based on the attributes of the food item, The estimation means estimates the time series of the remaining rate for each food item, The eating pattern analysis system according to claim 1 or 2, characterized in that the categorization means represents the time series of the survival rate of each eater with a typical eating pattern for each meal item.
4. The eating pattern analysis system according to claim 3, characterized in that the means for classification classifies each food item as having one or a combination of its nutrients, taste, use, calories per unit weight, and classification of food group as attributes.
5. The eating pattern analysis system according to claim 4, characterized in that the means for classification classifies foods that are relatively rich in carbohydrates into one meal item, foods that are relatively rich in protein into another meal item, and foods that are relatively rich in dietary fiber and vitamins into yet another meal item.
6. In a method of analyzing eating patterns in which a computer analyzes the eating patterns of a group of diners, The steps include taking a photograph of the food being consumed by each person in the group of diners, The steps include: estimating the retention rate of each food item for each person eating, based on the aforementioned captured video footage, The first step is to categorize the time series of survival rates for each diner using several representative eating patterns, This includes the step of extracting characteristic eating patterns from the aforementioned categorized representative eating patterns. The aforementioned step of categorization is, The steps include: normalizing the time scale of the survival rate time series for each meal participant by meal time, The process includes the step of extracting multiple representative eating patterns for each food item based on the time series of the normalized survival rates of the group of eaters, and then representing the time series of the survival rates of each eater for each food item with one of the representative eating patterns. The method for analyzing eating patterns is characterized in that, in the step of extracting the characteristic eating patterns, the eating patterns characteristic of the group of eaters are extracted from the factor loadings of the principal component analysis of the typified representative eating patterns.
7. In a method of analyzing eating patterns in which a computer analyzes the eating patterns of a group of diners, The steps include taking a photograph of the food being consumed by each person in the group of diners, The steps include: estimating the retention rate of each food item for each person eating, based on the aforementioned captured video footage, The first step is to categorize the time series of survival rates for each diner using several representative eating patterns, This includes the step of extracting characteristic eating patterns from the aforementioned categorized representative eating patterns. The aforementioned step of categorization is, The steps include: normalizing the time scale of the survival rate time series for each meal participant by meal time, The process includes the step of extracting multiple representative eating patterns for each food item based on the time series of the normalized survival rates of the group of eaters, and then representing the time series of the survival rates of each eater for each food item with one of the representative eating patterns. The step of extracting the characteristic eating patterns is characterized by selecting some representative eating patterns for each food item from the categorized representative eating patterns, and then comprehensively combining the selected representative eating patterns to extract characteristic eating patterns.
8. In a program that analyzes the eating patterns of a group of diners, The procedure for photographing the food being consumed by each person in a group of diners, A procedure for estimating the retention rate of each food item for each person eating, based on the aforementioned captured video footage, A procedure for categorizing the time series of survival rates for each diner using multiple representative eating patterns, The procedure for extracting characteristic eating patterns from the aforementioned categorized representative eating patterns is then performed on a computer. The aforementioned categorization procedure is The procedure for normalizing the time scale of the survival rate time series for each meal participant by meal time, The procedure includes extracting multiple representative eating patterns for each food item based on the time series of the normalized survival rates of the group of eaters, and then representing the time series of the survival rates of each eater with one of the representative eating patterns for each food item. The procedure for extracting the characteristic eating patterns is a program for analyzing eating patterns, characterized by extracting eating patterns characteristic of the group of eaters from the factor loadings of the principal component analysis of the typified representative eating patterns.
9. In a program that analyzes the eating patterns of a group of diners, The procedure for photographing the food being consumed by each person in a group of diners, A procedure for estimating the retention rate of each food item for each person eating, based on the aforementioned captured video footage, A procedure for categorizing the time series of survival rates for each diner using multiple representative eating patterns, The procedure for extracting characteristic eating patterns from the aforementioned categorized representative eating patterns is then performed on a computer. The aforementioned categorization procedure is The procedure for normalizing the time scale of the survival rate time series for each meal participant by meal time, The procedure includes extracting multiple representative eating patterns for each food item based on the time series of the normalized survival rates of the group of eaters, and then representing the time series of the survival rates of each eater with one of the representative eating patterns for each food item. The procedure for extracting the characteristic eating patterns is characterized by selecting some representative eating patterns for each food item from the categorized representative eating patterns, and then comprehensively combining these selected representative eating patterns to extract characteristic eating patterns.