Eating manner pattern analysis system, method and program

Abstract

To provide a system, a method, and a program capable of quantitatively measuring and analyzing an eating manner pattern helpful for dietary guidance on the basis of a camera image photographing food an eater eats and drinks.SOLUTION: An eating manner pattern analysis system 1 for analyzing an eating manner pattern of a group of eaters includes: a photography part 10 for photographing food being taken in every eater of a group of eaters; a classification part 20 for sorting respective foods projected in the photographed video into many kinds of dietary items on the basis of attributes of the foods; a residual rate estimation part 30 for estimating a residual rate of each food item time serially every eater on the basis of the photographed video; a categorization part 40 for categorizing the time series of the residual rate of each eater in a plurality of representative eating manner patterns; and an eating manner pattern extraction part 50 for extracting a characteristic eating manner pattern from the categorized eating manner patterns.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system, method and program for analyzing eating patterns, and more particularly to a system, method and program for analyzing eating patterns based on camera images of food ingested by a diner. [Background technology]

[0002] It is said that eating vegetables first is a healthy diet, but this has only been demonstrated in intervention studies, where the subjects were often diabetic patients receiving clinical guidance.

[0003] However, in real life, diners consume a variety of foods in different amounts and in different eating patterns. It is unclear what eating patterns exist in typical eating situations and how these patterns affect disease risk and health.

[0004] If healthy eating patterns in real life could be clarified, it would have great social significance, as it would greatly expand the scope of dietary advice. Specifically, if a person's eating behavior could be evaluated by combining not only the content of the meal but also the eating pattern, eating speed, number of chews, etc., it would be possible to provide detailed, individualized dietary advice.

[0005] In addition, while nutritional epidemiology studies have investigated the relationship between food and nutrient intake and disease risk, there have been no observational studies that have objectively measured eating patterns and eating speed to evaluate their relationship with disease risk. Therefore, if eating patterns can be categorized, it will greatly contribute to the development of nutritional epidemiology research.

[0006] Prior art for analyzing eating patterns includes Patent Document 1 and Non-Patent Document 1. Patent Document 1 discloses a meal system and an analysis system that stimulates the diner's vision and allows the diner to experience the pleasure of eating. Non-Patent Document 2 discloses a technology that uses a camera to capture the process of food and drink being consumed during a meal and analyzes eating patterns through video analysis. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2021-049337 [Non-patent literature]

[0008] [Non-Patent Document 1] Real-time Meal Management System (Kyocera Corporation) (https: / / www.kyocera.co.jp / ceatec / #anc11: confirmed on September 17, 2023) Summary of the Invention [Problem to be solved by the invention]

[0009] In order to clarify the eating habits of individuals, it is necessary to observe and measure actual mealtimes, and in order to clarify the eating patterns that exist in the general population, with the Japanese people being the largest population, it is necessary to observe and measure the actual eating habits of a large number of diners.

[0010] However, there are no established methods for observing and measuring the eating habits of a group of diners, and because the time it takes to eat and the contents of a meal vary from person to person, it is difficult to quantitatively measure the actual eating habits of a large number of diners under the same conditions and perform statistical processing.

[0011] In addition, the "eating patterns" that may exist in the general population and are useful for dietary advice have not been clearly defined, and no established method has been established for analyzing them.

[0012] An object of the present invention is to solve the above-mentioned technical problems and provide a system, method, and program that can quantitatively measure and analyze eating patterns that are useful for dietary guidance based on camera images of the food a diner is eating. [Means for solving the problem]

[0013] In order to achieve the above object, the present invention is characterized in that an eating pattern analysis system for analyzing the eating patterns of a group of diners comprises the following components.

[0014] (1) The system comprises a means for photographing food being ingested by each diner in a group of diners, a means for estimating the remaining rate of each food over time for each diner based on the photographed images, a means for categorizing the time series of the remaining rate for each diner into a plurality of representative eating patterns, and a means for extracting characteristic eating patterns from the categorized representative eating patterns.

[0015] (2) The categorization means comprises a means for normalizing the scale of the time series of the survival rate for each diner by meal time, and a means for extracting a plurality of representative eating patterns for each food based on the time series of the normalized survival rate for the group of diners, so that the time series of the survival rate for each diner is represented by one of the representative eating patterns for each food.

[0016] The present invention can be realized not only as an eating pattern analysis system equipped with such characteristic means, but also as an eating pattern analysis method that includes such characteristic means as steps, or as an eating pattern analysis program for causing a computer to execute each step. Effect of the Invention

[0017] According to the present invention, food eating patterns can be categorized, which can greatly broaden the scope of dietary advice by clarifying healthy eating patterns in real life, thereby making a significant contribution to the development of nutritional epidemiology research. [Brief description of the drawings]

[0018] [Figure 1] 1 is a functional block diagram showing a configuration of a main part of an eating pattern analysis system according to an embodiment of the present invention. [Diagram 2] 13 is a diagram illustrating an example of a method for estimating a time series of a remaining rate for each meal item by a remaining rate estimating unit. FIG. [Diagram 3] 11 is a diagram illustrating an example in which a categorizing unit extracts a representative pattern of survival rate time series data. FIG. [Figure 4] FIG. 13 is a diagram illustrating an example of categorizing survival rate time series data. [Diagram 5] FIG. 1 is a diagram showing a schematic example of a method (part 1) for extracting characteristic eating patterns from typified eating patterns. [Figure 6] FIG. 13 is a diagram showing a schematic example of a method (part 2) for extracting characteristic eating patterns from typified eating patterns. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Fig. 1 is a functional block diagram showing the configuration of the main parts of an eating pattern analysis system 1 according to an embodiment of the present invention, and the main components are a photographing unit 10, a classification unit 20, a residual rate estimation unit 30, a categorization unit 40, and an eating pattern extraction unit 50.

[0020] Such an eating pattern analysis system 1 can be configured by implementing applications (programs) that realize the functions detailed below on a general-purpose computer or server equipped with a CPU, ROM, RAM, bus, interface, etc. Alternatively, it can be configured as a dedicated machine or a single-function machine in which part of the application is implemented as hardware or software.

[0021] The photographing unit 10 includes a 3D camera or an RGB-D camera capable of acquiring depth information, and photographs each food ingested by the diner during the mealtime from when the diner starts to ingest the food to when the diner finishes ingesting the food.

[0022] The classification unit 20 classifies each food into a small number of meal items based on the captured image of the food. In this embodiment, each food is classified into one of four meal items, "staple food," "main dish," "side dish," and "other," based on attributes such as nutrients and taste. Note that the classification of each food may be performed manually in advance.

[0023] In this embodiment, foods relatively rich in carbohydrates are classified as a meal item "staple food," foods relatively rich in protein are classified as a meal item "main dish," and foods relatively rich in dietary fiber and vitamins are classified as a meal item "side dish." Foods that cannot be classified as any of the meal items are classified as "other."

[0024] The remaining rate estimation unit 30 estimates the remaining rate of each meal item in time series for each diner, based on the image of the food and the volume transition measured for each meal item based on the results of the classification.

[0025] In this embodiment, the remaining rate of each meal item, "staple food," "main dish," "side dish," and "other," is estimated at a predetermined cycle during the meal period from the start to the end of eating, as shown in Figure 2. This remaining rate estimation is performed for each diner (ID1, ID2, ...) of a large number of diners (a group of diners), and the estimated results of the remaining rate time series are stored in a database (not shown).

[0026] The categorization unit 40 includes a normalization unit 401 and a representative pattern extraction unit 402, and categorizes a large number of time-series data on the remaining rates of "staple food," "main dish," "side dish," and "other," obtained for each diner (ID1, ID2, ...), into a plurality of representative eating patterns.

[0027] In this embodiment, the normalization unit 401 first normalizes the time scale of the time series data of the remaining rate for each diner by the meal time from the start time to the end time of eating. Therefore, for the same diner, the time scale of the time series data of the remaining rate of "staple food," "main dish," "side dish," and "other" will be the same, but will be different between a diner who takes a long meal and a diner who takes a short meal.

[0028] The start time of the meal can be detected as the time when the remaining rate of any meal item starts to decrease. The end time of the meal can be detected as the time when the remaining rates of all meal items or major meal items become zero, or the time when the last change in the remaining rate is detected after no change has been detected for a certain period of time. The meal duration can be detected as the time from the start time to the end time of the meal.

[0029] The representative pattern extraction unit 402 applies a latent class mixture model (LCMM) or a similar statistical analysis method for categorizing time-series data to the time-series data of the remaining rate of all diners for each meal item, and represents the time-series data of the remaining rate of all diners with one of a small number of eating patterns.

[0030] In this embodiment, as shown in an example in FIG. 3, the remaining rate time series data for each meal item, "staple food," "main dish," "side dish," and "other," is represented by one of several representative patterns (in this embodiment, three patterns: pattern 1, pattern 2, and pattern 3).

[0031] In this manner, once the remaining rate time series data for each diner for each meal item is represented by one of the four representative patterns, the categorization unit 40 generates eating pattern data that centrally manages which representative pattern represents each remaining rate time series data for each diner (ID)'s "staple food," "main dish," "side dish," and "other," as an example shown in Figure 4.

[0032] In the example of Figure 4, the time series data of the staple food remaining rate for diner ID 1 is represented by staple food pattern 2 (staple food 2), the time series data of the main dish remaining rate is represented by main dish pattern 2 (main dish 2), the time series data of the side dish remaining rate is represented by side dish pattern 1 (side dish 1), and the other remaining rate time series data is represented by other pattern 1 (other 1).

[0033] Similarly, the time series data of the staple food remaining rate for diner ID 2 is represented by staple food pattern 2, the time series data of the main dish remaining rate is represented by main dish pattern 2, the time series data of the side dish remaining rate is represented by side dish pattern 3, and the other remaining rate time series data is represented by other pattern 1 (other 1).

[0034] The eating pattern extraction unit 50 extracts eating patterns characteristic of a diner group based on the categorized eating patterns. Here, two characteristic extraction methods will be described as examples.

[0035] FIG. 5 is a diagram showing an example of feature extraction based on principal component analysis and factor loadings. First, principal component analysis is applied to the data of the categorized eating patterns, and the dimensions are reduced to multiple latent synthetic variables (in this embodiment, the first to fourth principal components) in order of least information loss.

[0036] Next, the influence of each principal component is quantified by calculating the factor loading through principal component analysis. In other words, the influence of each composite variable is quantified based on whether the factor loading is closer to "1," "-1" (both of which are strongly correlated with the principal component), or "0" (weakly correlated with the principal component).

[0037] FIG. 6 shows an example of randomly selecting characteristic patterns from the representative patterns of the categorized eating patterns and creating combinations of these. All selected characteristic patterns are comprehensively combined to create a large number of "eating patterns."

[0038] In this embodiment, the focus is on the four meal items, excluding "other," namely "staple food," "main dish," and "side dish," and two characteristic patterns are selected from each of the three representative patterns extracted for these three meal items. These selected representative patterns are then comprehensively combined to extract characteristic eating patterns (1) to (8). It is also possible to create patterns by focusing on all four meal items.

[0039] In the illustrated example, staple food patterns 1 and 2 are selected for staple foods, main dish patterns 2 and 3 are selected for main dishes, and patterns 1 and 2 are selected for side dishes, so the combination of "staple food pattern 1-main dish pattern 2-side dish pattern 1" is extracted as "way of eating (1)". Similarly, the combination of "staple food pattern 1-main dish pattern 2-side dish pattern 2" is extracted as "way of eating (2)".

[0040] In this way, by randomly selecting characteristic patterns from the representative patterns of categorized eating patterns and creating a comprehensive list of combinations, it becomes easy to search for a typical eating pattern that corresponds to the eating pattern of the person being supported.

[0041] In the above embodiment, the food consumed by a diner is classified based on its nutrients into "staple foods" rich in carbohydrates, "main dishes" rich in protein, "side dishes" rich in dietary fiber and vitamins, and "others." However, the present invention is not limited to this, and classification may also be based on the other five major nutrients, such as lipids and carbohydrates.

[0042] Furthermore, rather than focusing on nutrients, food may be classified based on taste (sweet, bitter, umami, salty, sour), or based on usage (rice, side dish, soup, dessert, etc.), or based on calories per unit weight, or based on the classification of six basic food groups, or based on the classification of ten food groups, or based on the classification of three color food groups, or a combination of these.

[0043] According to this embodiment, the food eating patterns of a group of diners can be categorized and characteristic eating patterns can be extracted, thereby greatly broadening the scope of dietary guidance by clarifying healthy eating patterns in real life.

[0044] Furthermore, according to this embodiment, the eating patterns of diners are converted in advance into data in which the time series scale of survival rates is normalized by meal time, making it possible to similarly handle and analyze data from a group of diners with diverse food intake amounts and intake speeds.

[0045] According to the above embodiment, food eating patterns can be categorized, which can clarify healthy eating patterns in real life and greatly expand the scope of dietary advice. As a result, it is possible to contribute to Goal 3 of the United Nations Sustainable Development Goals (SDGs) "Ensure healthy lives and promote well-being for all at all ages." [Explanation of symbols]

[0046] 10: imaging unit, 20: classification unit, 30: survival rate estimation unit, 40: categorization unit, 401: normalization unit, 402: representative pattern extraction unit

Claims

1. In an eating pattern analysis system for analyzing eating patterns of a group of diners, a means for photographing food being consumed by each diner in a group of diners; A means for estimating the remaining rate of each food for each diner in time series based on the captured image; A means for categorizing the time series of survival rates of each diner into a number of representative eating patterns; and means for extracting a characteristic eating pattern from the categorized representative eating patterns.

2. The means for categorizing includes: A means for normalizing the time scale of the time series of survival rates for each diner by meal time; and a means for extracting a plurality of representative eating patterns for each food item based on the time series of the normalized survival rate of the eating group; 2. The eating pattern analysis system according to claim 1, wherein the time series of the survival rate of each diner is represented for each food by any one of the representative eating patterns.

3. The eating pattern analysis system according to claim 2, characterized in that the means for extracting the characteristic eating pattern extracts the eating pattern characteristic of the diner group from the factor loading of a principal component analysis of the categorized representative eating patterns.

4. The eating pattern analysis system according to claim 2, characterized in that the means for extracting the characteristic eating pattern selects a portion of representative eating patterns for each food from the categorized representative eating patterns, and extracts a characteristic eating pattern by comprehensively combining the selected representative eating patterns.

5. A means for classifying each food captured in the captured image into a plurality of types of meal items based on the attributes of the food, The estimating means estimates a time series of the remaining rate for each of the meal items, 5. The eating pattern analysis system according to claim 1, wherein the categorizing means represents the time series of the survival rate of each diner by a representative eating pattern for each meal item.

6. 6. The eating pattern analysis system according to claim 5, wherein the classification means classifies each food into attributes based on any one or a combination of nutrients, taste, use, calorie content per unit weight, and food group classification.

7. 7. The eating pattern analysis system of claim 6, wherein the classification means classifies foods relatively rich in carbohydrates into one meal item, foods relatively rich in protein into another meal item, and foods relatively rich in dietary fiber and vitamins into yet another meal item.

8. A method for analyzing eating patterns in which a computer analyzes eating patterns of a group of diners, comprising: Photographing the food being consumed by each diner in the diner group; A remaining rate of each food item is estimated in time series for each diner based on the captured image; The time series of survival rates for each diner was categorized into several representative eating patterns, An eating pattern analysis method, comprising: extracting characteristic eating patterns from the categorized representative eating patterns.

9. The eating pattern analysis method according to claim 8, characterized in that, in the categorization, a time scale of the time series of the remaining rate for each diner is normalized by meal time, a plurality of representative eating patterns are extracted for each food based on the time series of the normalized remaining rate for a group of diners, and the time series of the remaining rate for each diner is represented by one of the representative eating patterns for each food.

10. In an eating pattern analysis program that analyzes the eating patterns of a group of diners, taking a photograph of the food being consumed by each diner in the diner group; a step of estimating a remaining rate of each food for each diner in time series based on the captured image; A procedure for categorizing the time series of each diner's survival rate into several representative eating patterns; A step of extracting a characteristic eating pattern from the categorized representative eating patterns; An eating pattern analysis program characterized by causing a computer to execute the above.

11. The eating pattern analysis program according to claim 10, characterized in that the categorizing step includes a step of normalizing the time scale of the time series of the remaining rate for each diner by meal time, and a step of extracting a plurality of representative eating patterns for each food based on the time series of the normalized remaining rate for a group of diners, and the time series of the remaining rate for each diner is represented by one of the representative eating patterns for each food.

Images