Meal management device, method, and program

Abstract

To provide a meal management device with which it is possible to conduct objective meal guidance by a simple structure.SOLUTION: The present invention executes: a first process (12) for detecting the type and amount of food at each time of day from an image in which at least the food a user is eating is captured (11); a second process (14) for estimating the eating speed of the user at each time of day from history (13) of the detected type and amount; and a third process (17) for outputting advice information related to whether the eating speed is appropriate, on the basis of the result of having compared (15) the eating speed estimated at each time of day with the appropriate eating speed (16) of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to a meal management device, method, and program. [Background technology]

[0002] Existing methods related to generating advice information about diet include, for example, the technologies described in Patent Documents 1 to 3. Patent Document 1 relates to a lifestyle monitoring device that can generate and provide more comprehensive information about lifestyle, such as diet, sleep, and exercise, from electromyography sensors and wristbands attached to the user's head. Patent Document 2 relates to a system that provides advice about diet from a camera that films the user eating and a device that measures biological data. In particular, it prioritizes providing advice on items that greatly contribute to changes in weight and blood glucose levels. Patent Document 3 relates to a system that generates chewing information from a device attached to the eater's head, determines whether the person is eating too quickly or not chewing enough based on the number of chews, and notifies the user using a wirelessly connected transmission medium. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-189513 [Patent Document 2] Japanese Patent Publication No. 2017-54163 [Patent Document 3] Japanese Patent Publication No. 2018-033568 [Non-patent literature]

[0004] [Non-Patent Document 1] Ministry of Agriculture, Forestry and Fisheries website Home > Organization & Policy > Consumer & Safety > Promotion of Food Education > Food Education for Everyone > Topics by Generation & Lifestyle > Middle-aged and Elderly Men > Eating Slowly [Searched February 16, 2023], Internet<URL:https: / / www.maff.go.jp / j / syokuiku / minna_navi / topics / topics4_02.html> [Overview of the project] [Problems that the invention aims to solve]

[0005] However, conventional technology has not been able to provide a pace-setting (meal speed adjustment) function as objective dietary guidance tailored to each user in a simple configuration that does not require additional specialized equipment such as biosensors beyond general-purpose devices such as mobile terminals, thus reducing the cost burden.

[0006] For example, as disclosed in Non-Patent Document 1, the physical and health benefits of eating slowly are generally known. However, there are no clear methods or techniques to address this issue other than relying on personal motivation, such as the individual consciously trying to eat slowly or a dietary counselor providing educational leaflets to the individual to help them understand the necessity of eating slowly. Furthermore, even if the individual consciously tries to eat slowly, objectively speaking, they may still be eating too quickly.

[0007] Although the methods described in Patent Documents 1 to 3 are technologies for generating dietary advice, none of them relate to objective meal pacing functions. Furthermore, all of them require additional specialized equipment such as biosensors, making the overall equipment costly and potentially prohibitively expensive for everyday use by the average user. Additionally, the need to attach sensors and other equipment could make them difficult to use easily.

[0008] In particular, Patent Document 2 determines the optimal eating speed and order that have the best effect on biological data (weight, blood glucose level), and if it determines that the eating speed is having a negative impact, it provides advice during the meal such as "Eat slowly." However, this relates to the speed of the entire meal and does not enable real-time adjustments tailored to the content of the meal at that moment. Furthermore, it was not possible to obtain objective information on how "slow" the eating should be.

[0009] Furthermore, Patent Document 3 describes a method for determining whether a user is eating too quickly or not chewing properly by attaching a device to the user's head while they are eating, based on the number of chews. However, since the number of chews and the amount of food consumed are not linked, and the determination is not made according to the content of the meal, there is room for further improvement in order to achieve objective pacing. In addition, it also necessitates that the user attach a device to their head.

[0010] To achieve appropriate pacing based on the content of meals, the following points need to be considered, but conventional technologies have not taken these into account.

[0011] In the National Health and Nutrition Survey, obesity is defined as body mass index (BMI (weight [kg] / height [m])). 2Studies have shown that men with a body mass index (BMI) of 25 or higher tend to eat faster than non-obese men. Improving eating speed and making chewing a habit can help suppress appetite by making it easier for the brain to signal fullness with smaller portions of food. While a mealtime of around 20 minutes is considered desirable, it actually varies depending on the content of the meal. In other words, it is important to eat an appropriate amount over 20 minutes, and it is not recommended to eat all meals in 20 minutes regardless of the amount or content of the food. For example, eating 1000kcal over 20 minutes puts less strain on the digestive system than eating 500kcal over 20 minutes, even though the total time is the same. The appropriate mealtime depends on the content of the meal, but simply telling fast eaters to "eat slowly" makes it difficult for them to grasp and maintain an appropriate pace.

[0012] In view of the problems of the prior art described above, the present invention aims to provide a dietary management device, method, and program that enable objective dietary guidance with a simple configuration. [Means for solving the problem]

[0013] To achieve the above objective, the present invention is a meal management device characterized by performing: a first process of detecting the type and amount of food at each time from at least images of the food being eaten by the user; a second process of estimating the user's eating speed at each time from the history of the detected type and amount; and a third process of outputting advice information regarding whether the eating speed is appropriate based on the result of comparing the estimated eating speed at each time with the user's appropriate eating speed. The present invention is also characterized by a method and program corresponding to the device. [Effects of the Invention]

[0014] According to the present invention, by analyzing an image that can be obtained by a camera that is easy to use with a general terminal without requiring a special device such as a biosensor, the eating speed of a user is estimated, and advice information is output based on comparison with an appropriate eating speed at each time. Therefore, objective eating guidance can be provided with a simple configuration.

Brief Description of Drawings

[0015] [Figure 1] It is a functional block diagram of a meal management device according to an embodiment. [Figure 2] It is a schematic diagram showing a state of shooting by a meal management device. [Figure 3] It is a diagram showing an example of a video that is the object to be shot and analyzed by a meal management device as an example for explanation. [Figure 4] It is a flowchart of the operation of a meal management device according to an embodiment. [Figure 5] It is a diagram showing an example of a meal speed management table constructed for a target user. [Figure 6] It is a flowchart of the processing of a detection unit according to an embodiment. [Figure 7] It is a flowchart of the processing of a speed estimation unit according to an embodiment. [Figure 8] It is a diagram showing a schematic example of visual advice information displayed on a screen. [Figure 9] It is a diagram showing an example of visually outputting information on the ease of eating for each type of food as advice information. [[ID=D34]] [Figure 10] It is a diagram showing the hardware configuration of a general computer.

Modes for Carrying Out the Invention

[0016] Figure 1 is a functional block diagram of a meal management device 10 according to one embodiment, which comprises a shooting unit 11, a detection unit 12, a history management unit 13, a speed estimation unit 14, a determination unit 15, a meal speed management table construction unit 16, and an advice output unit 17. The meal management device 10 can be implemented as a general-purpose information terminal device such as a smartphone or tablet. As schematically shown in Figure 2, the shooting function of the camera, which is the hardware constituting the shooting unit 11, captures images of the user while they are eating, and by analyzing the frame images P(t) of the video at each time t (t=1,2,...), it is possible to provide the user with real-time advice information regarding their eating speed in visual or auditory format as needed at each time t.

[0017] Figure 2 is a schematic diagram showing an example of how the meal management device 10 captures images. The meal management device 10, which is a general-purpose information terminal such as a tablet device, is placed on the table TB where user U is eating, and the shooting unit 11, which is configured as the front camera, captures images of user U while they are eating. In this embodiment, the shooting of the meal management device 10 only needs to capture the food F that user U is eating (the food F is within the field of view R of the camera of the shooting unit 11), and it is not necessary for user U themselves to be captured while they are eating.

[0018] Figure 3 shows an example of video footage captured and analyzed by the meal management device 10. It shows frame images P(k) within the video footage captured at each time point k=1, 2, ..., T (where k=1 is the start time of the meal and k=T is the end time of the meal) for the cases k=1, t-1, t, T (where t is any time during the meal). In the example in Figure 3, the food (dish) consists of four items: white rice in a bowl, miso soup, fish, and a small bowl of salad. The video shows how the food decreases as the meal progresses.

[0019] Note that the speech bubbles attached to each frame image P(k) in Figure 3 do not represent the content of the image, but rather schematically represent the results of analyzing the image in the meal management device 10 using the method described later. Figure 3 will be referred to again as an example for explanation in the following explanation.

[0020] Figure 4 is a flowchart of the operation of a meal management device 10 according to one embodiment. In step S1, the device accepts user information input, and the meal speed management table construction unit 16 pre-constructs a database necessary for managing the user's eating speed, before proceeding to step S2. In step S2, the shooting unit 11, which is configured as a camera, starts shooting the user's eating habits, and then proceeds to step S3. In step S3, the meal management device 10 analyzes the frame image P(t) at the current time t of the video being captured in real time by the shooting unit 11 at each time t to estimate the user's eating speed. If it is determined that the eating speed is too fast, the device provides the user with advice information as needed, and then proceeds to step S4. Step S4 is a step that manages the real-time processing timing of the meal management device 10, and updates the current time t to the next time t+1 before returning to step S3. In this way, the processing in step S3 is repeatedly executed at each real-time time t=1,2,3..., making it possible to provide the user with advice information in real time as needed.

[0021] The following describes in detail each functional part of the meal management device 10 shown in Figure 1, which specifically implements each step in Figure 4 described above.

[0022] <Step S1...Meal speed management table construction section 16> Figure 5 shows an example of a meal speed management table TB constructed for a target user by the meal speed management table construction unit 16. As shown in the figure, the meal speed management table construction unit 16 can construct a meal speed management table TB (hereinafter abbreviated as "table TB") for a user by receiving input from the user for each question item in any format using existing methods, such as selecting from a menu or directly entering text.

[0023] In the table TB shown in Figure 5, each item in item group M1 is personal information related to the user and their meals, and its setting value can be obtained by accepting input from the user. On the other hand, as indicated in the figure as "calculated value," each item in item group M2 can be automatically calculated by the meal speed management table construction unit 16 by referring to the setting value entered in item group M1. The meal speed management table construction unit 16 saves the setting value entered in item group M1 and the calculated setting value in item group M2 as table TB. (Note that Figure 5 is a schematic example of table TB, and specific values ​​in the setting value column are omitted except for some.)

[0024] Specifically, as shown in item group M1, the system accepts initial input of the user's height, age, gender, and physical activity level (for example, three levels: "Level 1: Low, Level 2: Normal, Level 3: High"), and based on this input, the setting values ​​for each item in item group M2 are calculated as follows.

[0025] ● The "standard weight W" is calculated from height as follows: W[kg]=(Height[m]) 2 ×22 [kg / m 2 ]

[0026] This calculation is based on general knowledge, such as that disclosed in Non-Patent Document 2 below. [Non-Patent Literature 2] "What is the daily calorie requirement for adult men? Secrets to staying healthy are also explained." [Retrieved February 16, 2023], Internet<URL:https: / / medipalette.lotte.co.jp / diet / 1952>

[0027] ● The "daily required calorie count Z" is calculated by multiplying the standard weight W by the estimated required calorie count per kg of body weight, which is based on age, gender, and physical activity level, as follows: Z[kcal] = W[kg] × Required calories per 1kg [kcal / kg]

[0028] Here, the "required calories per kilogram" (CAL) can be calculated as a value CAL = CAL(x1, x2, x3) corresponding to the three input variables: age x1, gender x2, and physical activity level x3, using a predetermined formula or function based on general knowledge, such as disclosed in Non-Patent Document 3 below. [Non-Patent Literature 3] "Ministry of Health, Labour and Welfare, 'Dietary Reference Intakes for Japanese (2020 Edition)'" [Retrieved February 16, 2023], Internet <URL:https: / / www.mhlw.go.jp / content / 10904750 / 000586556.pdf?_ga=2.68837526.666361972.1652509222-1074234362.1652509222>

[0029] ● Determine the values ​​obtained by distributing the above "daily required calorie count Z" to breakfast, lunch, and dinner. Using the entered calorie ratios for breakfast, lunch, and dinner (r1:r2:r3), calculate the values ​​A, B, and C by distributing Z according to these ratios, as follows. Note that these "required calories" refer to "appropriate portion sizes." The calories needed for breakfast A = Z * r1 / (r1 + r2 + r3) The calories needed for lunch are B = Z * r² / (r¹ + r² + r³) The calories needed for dinner are C = Z * r3 / (r1 + r2 + r3)

[0030] Regarding the calorie ratio r1:r2:r3 for breakfast, lunch, and dinner, the user may choose to input their individual settings in item group M1, or they may choose to use a standard default value, such as "r1:r2:r3=3:3:4", without accepting such input. (Alternatively, if the user indicates during input for item group M1 that they do not wish to set individual calorie ratios, this default value may be adopted.) This default value is based on general knowledge, such as that disclosed in Non-Patent Document 4 below. [Non-Patent Literature 4] "Chrononutrition Reveals the Ideal Ratio of Three Meals: 3-3-4" [Retrieved February 16, 2023], Internet<URL:https: / / style.nikkei.com / article / DGXKZO16220180R10C17A5NZBP01 / ?page=2>

[0031] Furthermore, to accommodate users' eating habits, the system may accept settings for two or four meals a day, rather than just three meals a day (breakfast, lunch, and dinner). The system may also accept user-defined settings for calorie ratios, and the total daily calorie requirement Z can be distributed to each meal in the same way as with three meals. For example, individual settings can be customized to accommodate preferences such as always eating a larger breakfast, snacking, or working night shifts.

[0032] ● Based on the required calories A, B, and C mentioned above, and assuming a predetermined understanding that a meal should ideally take about 20 minutes, the appropriate meal times for breakfast, lunch, and dinner are set to, for example, 15 minutes, 15 minutes, and 20 minutes respectively. The appropriate eating speeds a, b, and c [kcal / min], defined for the calories contained in the eaten meal, are calculated as follows. (Note that this calculation can be performed similarly for settings of 2 or 4 meals, not just 3, using the appropriate meal times for each given meal.) The appropriate eating speed for breakfast is a = A [kcal] / 15 [min]. The appropriate eating speed for lunch is b = B [kcal] / 15 [min]. The appropriate eating speed for dinner is c = C [kcal] / 20 [min].

[0033] ● The above speeds a, b, and c are appropriate eating speeds based on calories, but as shown in item group M3, the appropriate eating speeds α, β, and γ [cm] are based on the volume consumed. 3 You may record [ / min] in table TB. The optimal breakfast eating speed α = VA [cm] 3 ] / 15[min] The appropriate eating speed for lunch is β = VB [cm] 3 ] / 15[min] The appropriate eating speed for dinner is γ = VC [cm] 3 ] / 20[min]

[0034] Regarding the volumes VA, VB, and VC for breakfast, lunch, and dinner, as will be explained later, they can be obtained by analyzing the images taken at the time the meal footage was actually acquired in step S2. (Therefore, unlike item groups M1 and M2, the setting values ​​for item group M3 cannot be calculated in step S1 as pre-processing before video acquisition, but since this information can be recorded in table TB, it is explained here.)

[0035] <Step S2...Photography Department 11> The shooting unit 11 can be implemented as a camera with its hardware configuration, and acquires video by shooting so that at least the food F of user U eating is captured, as explained in the schematic example in Figure 2, and outputs images F(t) at each time t=1, 2, 3, ... in real time to the detection unit 12. The detection unit 12 and beyond estimate the eating speed and provide advice information as needed, but for this processing, it is sufficient to detect the decrease in the food F being photographed, as shown in the schematic example in Figure 3, by comparing each frame image F(t) with the frame image F(t-1) at the immediately preceding time t-1.

[0036] Therefore, even if the frame rate of the camera constituting the shooting unit 11 is, for example, 30fps (frames per second) or 60fps, the real-time intervals between each time t=1,2,3,... of each frame image F(t) output to the detection unit 12 and beyond as the target of analysis can be set in advance to intervals such as every 30 seconds or every minute, so that it can be recognized between images F(t) and F(t-1) at adjacent time intervals that the food F has decreased at least to some extent as a result of being eaten by the user U. (That is, the processing timing of each time t in step S4 above can be set in advance so that adjacent times (each time t and the next time t+1) have an interval of 30 seconds or so in real time.) The shooting unit 11 may shoot video and output each frame image F(t) at 30-second or 1-minute intervals to the detection unit 12, or it may shoot still images at 30-second or 1-minute intervals and output these as each frame image F(t) to the detection unit 12.

[0037] <Step S3...Detection unit 12, history management unit 13, speed estimation unit 14, determination unit 15, advice output unit 17> The following is an overview of the processing performed by each functional unit in step S3.

[0038] The detection unit 12 applies object recognition to the frame image F(t) to detect the type of food K(t) and its quantity V(t) from the frame image F(t), and outputs it to the history management unit 13. The history management unit 13 stores the type and quantity output from the detection unit 12 at each time t, and manages the meal history up to the current time t as Hist(t)={(K(k),V(k))|k=1,2,3,…,t}, and provides this meal history Hist(t) for reference to the next speed estimation unit 14.

[0039] The rate estimation unit 14 refers to the meal history Hist(t) and estimates the eating rate v(t) at the current time t as a rate based on at least the volume of the type and amount (volume) of food eaten between the previous time t-1 and the current time t, using the food state remaining at the current time t (K(k), V(k)) and the food state remaining at the previous time t-1 (K(t-1), V(t-1)). This rate is then output to the determination unit 15.

[0040] The determination unit 15 compares the current eating speed v(t) with the appropriate eating speed, which is the user information recorded in table TB of the eating speed management table construction unit 16 that has been constructed and stored for the user who is eating. By doing so, it obtains a determination result regarding whether the eating speed v(t) is too fast or not, and outputs it to the advice output unit 17. The advice output unit 17 presents the user with advice information corresponding to the determination result at the current time t in any manner, such as visually and / or audibly, thereby realizing the provision of real-time advice to the user during the meal.

[0041] The details of each functional unit that performs the processing in step S3 will be further explained below.

[0042] <Detection unit 12> Figure 6 is a flowchart of the processing of the detection unit 12 according to one embodiment. In step S11, a frame image F(t) at the current time t is acquired as input from the video of the imaging unit 11. In step S12, object recognition processing is applied to the frame image F(t) to identify food items in the image F(t) and obtain information on the identified type K(t) of each food item. In step S13, for each of the identified types K(t), volume recognition processing is applied to each food item region in the frame image F(t) to calculate the volume V(t) of that food item. In step S14, the type K(t) and volume V(t) obtained by applying the recognition processing to the image F(t) are output to the history management unit 13 for storage, and the flow in Figure 6 is terminated.

[0043] For object recognition and volume recognition in the detection unit 12, any existing method such as a deep learning network may be used. For example, a large number of food images, along with their correct labels for food type, may be prepared in advance for training. A deep learning network can then be trained using an existing method such as backpropagation to obtain the parameters for the food type recognition network, and the food type K(t) can be recognized using the network with these parameters. Similarly, correct labels for the volume of these training food images may also be prepared, and the parameters for the food volume recognition network can be similarly obtained. The food volume V(t) can then be recognized using the network with these parameters. Alternatively, instead of applying the "food type recognition network" to image P(t) in step S12 and then applying the "volume recognition network" to each food region within the resulting image P(t) in step S13, a "food type and volume recognition network" that can recognize these simultaneously may be pre-trained and used.

[0044] Furthermore, object recognition and volume recognition may be implemented using methods other than deep learning networks, such as similar image search. That is, multiple food region images, each composed of different types and volumes, may be prepared as reference image data to serve as the correct answer. Image regions that are judged to be similar to the reference food region images may be detected within image P(t), and the type and volume pre-associated with the similarly judged reference food region image may be used as the recognition result. Any existing method can be used for similar image search, such as extracting various features from an image and determining images with similar features in the feature space as similar images.

[0045] The example in Figure 3 above also shows the food type K(k) and volume V(k) detected by the detection unit 12 for the image P(k) at each time k=1, t-1, t, and T. Specifically, there are four types of food K(k) at each time k, and they are detected as K(k)={white rice Ka, miso soup Kb, fish Kc, salad Kd} (where Ka, Kb, Kc, and Kd are indices for the respective food types). At the same time, four volume values ​​are detected for each of these four types of food as V(k)={Va(k), Vb(k), Vc(k), Vd(k)}.

[0046] <Regarding the setting of item group M3 by the detection unit 12> Furthermore, in order to obtain the setting value of item group M3 in table TB of Figure 4 mentioned above, the detection unit 12 should perform the following additional processing. That is, from the detection result of image P(1) at the start of the meal at time t=1, if the meal is breakfast, the total product VA should be calculated as follows and recorded in table TB. VA = Va(1) + Vb(1) + Vc(1) + Vd(1)

[0047] Here, the user may manually specify whether the current meal is breakfast, lunch, or dinner, or it may be automatically determined by the clock function.

[0048] <History Management Department 13> The history management unit 13 stores the food type and volume, which are the detection results for each time obtained by the detection unit 12 as described above, as a meal history Hist(t)={(K(k),V(k))|k=1,2,3,…,t} up to the current time t, making it available for reference by the speed estimation unit 14.

[0049] <Speed ​​estimation unit 14> FIG. 7 is a flowchart of the processing of the speed estimation unit 14 according to an embodiment. In step S21, information on the meal history Hist(t)={(K(k),V(k))|k = 1,2,3,…,t} up to the current time t is acquired from the history management unit 13. In step S22, information (K(t),V(t)) on the food state remaining at the current time t is acquired from within the meal history Hist(t). In step S23, information (K(t - t1),V(t - t1)) on the food state at a time t1 before the current time t is acquired from within the meal history Hist(t). Regarding this fixed time t1, any time such as t1 = 1,2,… etc. may be set in advance. Hereinafter, for the sake of explanation, assuming that t1 = 1 is set, information (K(t - 1),V(t - 1)) on the food state at the time t - 1 immediately before the current time t is acquired.

[0050] In step S24, from the information obtained in steps S22 and S23 above, the volume of each type of food that has decreased due to the user eating over a fixed time up to the current time t is calculated. In step S25, the sum for each food type of the decreased volume is calculated as the eating speed v(t). In the example of FIG. 3, the eating speed v(t) can be calculated as follows in equation (1) as the sum of the decreased volumes for each food type from the previous time t - 1 to the current time t. (Normally, as each food decreases along with the progress of the meal, 0≦Va(t)≦Va(t - 1) etc. holds, so by calculating with -v(t)≦0, a positive value speed v(t) in the decreasing direction is calculated such that v(t)≧0. The same applies to the speed v(t) by other than equation (1).) -v(t)={Va(t)-Va(t - 1)}+{Vb(t)-Vb(t - 1)}+{Vc(t)-Vc(t - 1)}+{Vd(t)-Vd(t - 1)}…(1)

[0051] Alternatively, the eating speed v(t) may be calculated as the calorie speed of the eaten portion that can be calculated from the decreased volume as follows in equation (2). Here, Ca, Cb, Cc, Cd [kcal / cm 3 are the calories per unit volume of white rice, miso soup, fish, and salad [kcal / cm 3The value can be stored in a database and referenced to calculate the calorie rate. -v(t)=Ca*{Va(t)-Va(t-1)}+Cb*{Vb(t)-Vb(t-1)} +Cc*{Vc(t)-Vc(t-1)}+Cd*{Vd(t)-Vd(t-1)} …(2)

[0052] Alternatively, the eating speed may be calculated by multiplying it by a weight w that takes into account the ease of eating each type of food, using the following equation (1-w) as a variation of equation (1), or the following equation (2-w) as a variation of equation (2). -v(t)=wa*{Va(t)-Va(t-1)}+wb*{Vb(t)-Vb(t-1)} +wc*{Vc(t)-Vc(t-1)}+wd*{Vd(t)-Vd(t-1)} …(1-w) -v(t)=wa*Ca*{Va(t)-Va(t-1)}+wb*Cb*{Vb(t)-Vb(t-1)} +wc*Cc*{Vc(t)-Vc(t-1)}+wd*Cc*{Vd(t)-Vd(t-1)} …(2-w)

[0053] Here, wa, wb, wc, and wd are predetermined weighting coefficients corresponding to the ease of eating white rice, miso soup, fish, and salad, respectively. These values ​​are stored in a database, and by referencing them, volumetric velocity and calorie velocity that take ease of eating into account can be calculated.

[0054] The weight coefficients can be set to predetermined values ​​for each type of food based on the following considerations. For example, in the case of bread, considering that it is natural to eat it faster than usual due to its low density, a weight of a value less than 1, such as 0.5, should be set. Conversely, in the case of hard foods (such as rice crackers) or foods that are difficult to chew (such as offal), a weight greater than 1 should be set, as it is natural to eat them more slowly than usual.

[0055] Note that the above equations (1), etc., use the example of four types of food, "white rice, miso soup, fish, and salad," as shown in Figure 3. Similarly, for any number of types of meals other than these four examples of "white rice, miso soup, fish, and salad," the eating rate v(t) can be calculated using the same calculation as equations (1), etc., based at least on the sum of the volume reductions.

[0056] <Determination unit 15 and advice output unit 17> The determination unit 15 compares the eating speed v(t) obtained by the speed estimation unit 14 with a threshold TH obtained by referring to table TB in the eating speed management table construction unit 16. If "v(t)>TH" is true and the eating speed is inappropriately high, the determination unit 15 outputs a determination result to that effect to the advice output unit 17, and the advice output unit 17 outputs information to the user indicating that the eating speed is too fast and inappropriate as advice information for the current time t.

[0057] For the threshold TH, you should obtain and use a value from table TB that corresponds to whether the meal in question is breakfast, lunch, or dinner, and the definition of the eating rate v(t) (whether it is a volumetric velocity such as equation (1) or a calorie velocity such as equation (2)). For example, if the meal in question is breakfast and the eating rate v(t) is defined as a volumetric velocity, you can refer to the value α from item group M3 and use TH=α. Alternatively, if the meal in question is breakfast and the eating rate v(t) is defined as a calorie velocity, you can refer to the value A from item group M2 and use TH=A.

[0058] Incidentally, the processing of using weights according to the aforementioned equations (1-w) and (2-w) can also be interpreted as relaxing or making stricter the determination process of the determination unit 15 when not using weights. That is, when food is hard and it tends to be more difficult to eat, multiplying by a weight of 0 < w < 1 corrects in the direction of reducing the eating speed. This is equivalent to not using weights for the eating speed v(t) and increasing the determination threshold to TH / w. Therefore, it corresponds to allowing fluctuations in the threshold determination in a form that allows the eating speed to be relatively fast for foods that are difficult to eat. Similarly, conversely, when food is soft and it tends to be easier to eat, multiplying by a weight of w > 1 corrects in the direction of increasing the eating speed. This is equivalent to not using weights for the eating speed v(t) and correcting the determination threshold to be smaller as TH / w. Therefore, it corresponds to allowing fluctuations in the threshold determination in a form that encourages the eating speed to be relatively slow.

[0059] Also, the determination in the determination unit 15 and the output of advice information in the advice output unit 17 corresponding thereto may perform more detailed determination according to the comparison of the eating speed v(t) and the threshold TH, and output corresponding advice information. As specific examples, the following various modes are possible.

[0060] The eating speed v(t) estimated from the amount of decrease in the amount of food eaten may be displayed on the display screen to show how much faster / slower it is compared to the appropriate eating speed (corresponding threshold TH) (for example, red means too fast, green means appropriate, blue means too slow, etc.). When exceeding the appropriate eating speed, advice to eat slowly and a pacing sound may be output from a voice device or an imaging device.

[0061] In situations where the eater is in an environment where they cannot make noise, or where sound pacing is ineffective, a metronome can be displayed on the screen, outputting an animation that shows the tempo of the eating speed with a ticking sound. A schematic example of this is shown in Figure 8 as Example EX2. Here, the degree of deviation from the appropriate eating speed is indicated by an arrow on a bar. If the eating speed is too fast, the metronome activates, a pacing sound is emitted, and the metronome needle moves to set the eating speed tempo. Advice such as "Your pace is too fast! Please eat slowly." is also displayed.

[0062] When the eating speed v(t) drops to the appropriate eating speed (threshold TH), the system outputs advice to the eater to maintain that speed. If the error is around ±5-10% of the ideal time, the system will say, "Your eating time was excellent, neither too fast nor too slow. Keep it up." If the error is around ±10-20%, the system will say, "That was a good pace. Keep it up." If the error is around ±20-40%, the system will say, "Almost there. Try to be a little more conscious of your pace." If the error is more than ±40%, the system will say, "Are you chewing well? Paying attention to your eating pace will bring you closer to better health." The system displays advice based on how close the eating speed v(t) is to the threshold TH.

[0063] Similarly, as an additional embodiment, the advice output unit 17 may refer to the amount corresponding to the meal from the item group M2 of table TB mentioned above, and then display advice such as "The meal volume is perfect" if it is within ±5-10% of the ideal amount, "The meal volume is just right" if the error is within ±10-20%, "The meal volume is just right" if the error is within ±20-40%, "The meal volume is a little small. Let's take in nutrients according to your appropriate amount" if the error is within ±40% or more, and "Let's reconsider the meal volume" at least at the start of the meal. Example EX1 in Figure 8 is an example of displaying on the screen whether the amount of food is appropriate at the start of the meal.

[0064] The ideal amount can be determined by using the required calories A, B, and C for each meal, which are calculated by allocating the required calories Z calculated in Table TB. On the other hand, the actual calorie amount of a meal to be compared with this ideal amount can be determined by using the following total calorie amount, which can be referenced when calculating equation (2) for the image P(1) taken at the start of meal photography t=1. Ca*Va(1)+Cb*Vb(1)+Cc*Vc(1)+Cd*Vd(1)

[0065] Furthermore, if there is a significant difference between the appropriate calorie intake for each meal and the calorie value calculated from the actual meal image (image P(1) at the start of the meal at t=1), the system may refer to the results of determining the type of food from image P(t) and output advice regarding quantity, such as "Let's reduce the amount of pork cutlet by two pieces" or "Let's add one more side dish."

[0066] As described above, according to the embodiments of the present invention, compared to conventional approaches, the user only needs to prepare an existing device they own, such as a tablet or smartphone, that can record video, be placed on the dining table, and run the application according to the embodiments of the present invention. This allows for real-time advice on eating speed at low cost, without the hassle or burden of attaching equipment. Furthermore, it enables users to objectively and subjectively learn what constitutes a "slow" eating speed and to develop the habit of "eating an appropriate amount slowly, tailored to their individual needs." This can lead to improvements in eating speed and quantity, and is expected to have a positive effect on health management, such as preventing obesity, by maintaining healthy digestion in the gastrointestinal tract.

[0067] The following sections will explain various supplementary examples, variations, and additional examples.

[0068] (1) The meal management device 10 of this embodiment makes it possible to encourage the user to eat at an appropriate pace, thereby contributing to United Nations Sustainable Development Goal (SDG) 3, "Ensure healthy lives and promote well-being for all at all ages."

[0069] (2) Depending on the weight w of equation (1-w) or equation (2-w), the region of the food of that type recognized on image P(t) may be color-coded or displayed, and this information may be output from the advice output unit 17 as supplementary advice information indicating ease of eating. Figure 9 shows an example of such an advice display indicating ease of eating for image P(t) in Figure 3.

[0070] (3) As an additional embodiment of the advice output unit 17, the estimated eating speed v(t) may be displayed in a graph format, allowing the user to conduct an overall review after the meal. For example, if a dinner took the prescribed 20 minutes, the user can see that they were quick at certain times (specifically when eating XX), or a little slow at other times (when eating something they dislike), and can consider this when adjusting their eating speed for future meals. In this case, information about the state of the remaining food (K(t), V(t)) may be linked to the eating speed v(t) graph.

[0071] (4) Hardware configuration Figure 10 shows an example of a hardware configuration in a typical computer. The meal management device 10 can be implemented as one or more computer devices 70 having such a configuration. When implementing the meal management device 10 with two or more computer devices 70, information necessary for processing may be sent and received via a network. The computer device 70 includes a CPU (Central Processing Unit) 71 that executes predetermined instructions, a GPU (Graphics Processing Unit) 72 as a dedicated processor that executes some or all of the execution instructions of the CPU 71 on behalf of or in cooperation with the CPU 71, RAM 73 as main memory that provides a work area to the CPU 71 (and GPU 72), ROM 74 as auxiliary storage, a communication interface 75, a display 76, an input interface 77 that accepts user input via a mouse, keyboard, touch panel, etc., a camera 81, a speaker 82, a light 83, a vibration element 84, and a bus BS for sending and receiving data between them.

[0072] Each functional unit of the meal management device 10 can be implemented by a CPU 71 and / or GPU 72 that read and execute a predetermined program corresponding to the function of each unit from ROM 74. Both the CPU 71 and GPU 72 are types of arithmetic units (processors). When display-related processing is performed, the display 76 also operates in conjunction, and when communication-related processing related to data transmission and reception is performed, the communication interface 75 also operates in conjunction.

[0073] When providing advice visually in the advice output unit 17, this can be done by using the display on the display 76 or the illumination of the light 83. When providing advice audibly, this can be done by using the speaker 82. When providing alert output due to vibration, this can be done by using a vibration element 84, which consists of a vibrator or the like. [Explanation of Symbols]

[0074] 10...Meal management device, 11...Photography unit, 12...Detection unit, 13...History management unit, 14...Speed ​​estimation unit, 15...Determination unit, 16...Meal speed management table construction unit, 17...Advice output unit

Claims

1. A first process that detects the type and quantity of food at each time point from images taken of the food the user is eating, A second process is performed to estimate the user's eating speed at each time point based on the history of the detected type and quantity. A third process is performed which, based on the results of comparing the estimated eating speed at each time point with the appropriate eating speed for the user, outputs advice information regarding whether the eating speed is appropriate. In the second process, the eating speed is estimated by multiplying the detected quantity by a weight that is predetermined for each of the detected types, with the weight being smaller for types of food that are easier to eat. A meal management device characterized in that, in the third process, the estimated eating speed is corrected to the lower side when the food is of a type that is easier to eat.

2. The meal management device according to claim 1, further characterized in that it performs a fourth process of assigning weight information to each type of food detected in the image and displaying it to the user.

3. The meal management device according to claim 1, characterized in that the second process estimates the sum of the decreases in the amount of each type detected from the previous time to the current time as the volume velocity of the meal, and sets the estimated volume velocity as the estimated meal velocity.

4. The meal management device according to claim 1, characterized in that the second process estimates the meal rate as the calorie rate of the meal by referring to the calorie amount per volume of each type from the sum of the decreases in the amount of each type detected from the previous time to the current time.

5. The meal management device according to claim 1, characterized in that, in the third process, if the previously estimated meal rate exceeds the appropriate meal rate, it outputs advice information indicating that the meal rate is too fast and inappropriate.

6. A first process that detects the type and quantity of food at each time point from images taken of the food the user is eating, A second process is performed to estimate the user's eating speed at each time point based on the history of the detected type and quantity. A meal management method characterized in that a computer performs a third process which outputs advice information regarding whether the meal speed is appropriate based on the result of comparing the estimated meal speed with the appropriate meal speed for the user at each time point, In the second process, the eating speed is estimated by multiplying the detected quantity by a weight that is predetermined for each of the detected types, with the weight being smaller for types of food that are easier to eat. A method for managing meals, characterized in that, in the third process, the estimated eating rate is corrected to a lower value when the food is of a type that is easier to eat.

7. A program characterized by causing a computer to function as a meal management device according to any one of claims 1 to 5.

Images