Measurement device, terminal, meal measurement system, measurement method, and program

Abstract

To improve the accuracy of meal measurement more than before.SOLUTION: The measurement apparatus includes an in-meal information acquisition unit configured to acquire a meal action image that is an image of a meal action of each measurement subject included in the time-series captured images, an amount estimation unit configured to output an eating and drinking amount that is an amount of food and drink eaten and drunk for each meal action of the measurement subject using the meal action image, and a meal information generation unit configured to generate meal information based on the eating and drinking amount for each meal action output by the amount estimation unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosed technology relates to dietary measurement technology. [Background technology]

[0002] Dietary measurement technology is a technology for measuring the diet of a subject. Patent Document 1 discloses a method for calculating the rate at which a meal is completed. Specifically, Patent Document 1 states that one indicator of "participants' reactions as a result of the situation at the venue and the organizer's actions to liven things up" at a remote drinking party is "the number of times each participant takes chopsticks to each food or drink in the food and drink set is counted from each participant's image, and based on the pre-set number of chopstick movements per food or drink, it is determined how much of the food and drink set the participant has eaten, and the proportion (percentage (%)) of the entire food and drink set that has been eaten is quantified as the completion rate." "Note that the number of chopstick movements per food or drink is based on the assumption that everyone is eating the same meal, but the number of mouthfuls it takes to finish one meal is determined in advance by linking with the database of the provider that lists the food or drink in question." (See paragraph

[0084] of Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-184145 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the number of mouthfuls a person can finish eating food or drink in may vary depending on the person being measured, and therefore, the technology described in Patent Document 1 has a problem in that measurement results, such as the rate at which food is eaten completely, tend to be inaccurate and the measurement results are low.

[0005] The present disclosure is intended to solve the above-mentioned problems, and aims to improve the accuracy of measurement results of meal measurements compared to conventional methods. [Means for solving the problem]

[0006] The measuring device disclosed herein includes a meal information acquisition unit that acquires eating behavior images, which are images of the eating behavior of each measured subject included in a time series of captured images; a quantity estimation unit that uses the eating behavior images to output the amount of food and drink eaten, which is the amount of food and drink eaten for each eating behavior of the measured subject; and a meal information generation unit that generates meal information based on the amount of food and drink eaten for each eating behavior output by the quantity estimation unit. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to achieve an effect of improving the accuracy of the measurement results of meal measurement compared to the prior art. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a measurement device according to a first embodiment of the present disclosure. [Figure 2] FIG. 2 is a flowchart illustrating an example of processing performed by the measurement device according to the first embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram showing a configuration example of a diet measurement system 1 (1A) including a measurement device 10 (10A) according to the first embodiment of the present disclosure. [Figure 4] FIG. 4 is a flowchart illustrating an example of processing performed by the mealtime information acquisition unit according to the first embodiment of the present disclosure. [Figure 5] FIG. 5 is a flowchart illustrating an example of processing by the eating behavior detection unit according to the first embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart illustrating an example of processing by the quantity estimating unit according to the first embodiment of the present disclosure. [Figure 7] FIG. 7 is a diagram showing a configuration example of a dietary measurement system 1 (1B) including a measurement device 10 (10B) according to the second embodiment of the present disclosure. [Figure 8] FIG. 8 is a flowchart showing an example of processing by the measurement device 10 (10B) according to the second embodiment of the present disclosure. [Figure 9] FIG. 9 is a flowchart showing an example of processing by the registration information acquisition unit 500 (500B) according to the second embodiment of the present disclosure. [Figure 10] FIG. 10 is a flowchart showing an example of the processing of the mealtime information acquisition section 100 (100B) according to the second embodiment of the present disclosure. [Figure 11] FIG. 11 is a flowchart showing an example of processing by the quantity estimator 200 (200B) according to the second embodiment of the present disclosure. [Figure 12] FIG. 12 is a flowchart showing an example of processing by the diet information generating unit 300 (300B) according to the second embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram showing a configuration example of a diet measurement system 1 (1C) including a measurement device 10 (10C) according to the third embodiment of the present disclosure. [Figure 14] FIG. 14 is a flowchart showing an example of processing by the quantity estimator 200 (200C) according to the third embodiment of the present disclosure. [Figure 15] FIG. 15 is a flowchart showing an example (first example) of the process of the diet information generating unit 300 (300C) according to the third embodiment of the present disclosure. [Figure 16] FIG. 16 is a diagram showing a first example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. [Figure 17] FIG. 17 is a diagram showing a second example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. [Figure 18] FIG. 18 is a flowchart showing an example (second example) of the process of the diet information generating unit 300 (300C) according to the third embodiment of the present disclosure. [Figure 19] FIG. 19 is a diagram showing a third example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. [Figure 20]FIG. 20 is a diagram showing a fourth example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. [Figure 21] FIG. 21 is a diagram showing a configuration example of a diet measurement system 1 (1D) including a measurement device 10 (10D) according to the fourth embodiment of the present disclosure. [Figure 22] FIG. 22 is a flowchart showing an example of the processing of the mealtime information acquisition section 100 (100D) according to the fourth embodiment of the present disclosure. [Figure 23] FIG. 23 is a flowchart showing an example of processing by the quantity estimating unit 200 (200D) according to the fourth embodiment of the present disclosure. [Figure 24] FIG. 24 is a diagram showing a configuration example of a diet measurement system 1 (1E) including a measurement device 10 (10E) according to the fifth embodiment of the present disclosure. [Figure 25] FIG. 25 is a flowchart showing an example of processing by the quantity estimator 200 (200E) according to the fifth embodiment of the present disclosure. [Figure 26] FIG. 26 is a diagram showing a configuration example of a diet measurement system 1 (1F) including a measuring device 10 (10F) according to the sixth embodiment of the present disclosure. [Figure 27] FIG. 27 is a flowchart showing an example of processing by the imaging command unit 700 (700F) according to the sixth embodiment of the present disclosure. [Figure 28] FIG. 28 is a diagram showing a configuration example of a dietary measurement system 1 (1G) including a measuring device 10 (10G) according to the seventh embodiment of the present disclosure. [Figure 29] FIG. 29 is a flowchart showing an example of processing by the illumination command unit 800 (800G) according to the seventh embodiment of the present disclosure. [Figure 30] FIG. 30 is a diagram showing a configuration example of a diet measurement system 1 (1H) including a measurement device 10 (10H) according to the eighth embodiment of the present disclosure. [Figure 31] FIG. 31 is a flowchart showing an example of processing by the measuring device 10 (10H) according to the eighth embodiment of the present disclosure. [Figure 32]FIG. 32 is a diagram illustrating a configuration example of the state estimation unit 340 (340-1) according to the eighth embodiment of the present disclosure. [Figure 33] FIG. 33 is a flowchart illustrating an example of the state estimation unit 340 (340-1) according to the eighth embodiment of the present disclosure. [Figure 34] FIG. 34 is a diagram illustrating a configuration example of the state estimation unit 340 (340-2) according to the eighth embodiment of the present disclosure. [Figure 35] FIG. 35 is a flowchart illustrating an example of the state estimation unit 340 (340-2) according to the eighth embodiment of the present disclosure. [Figure 36] FIG. 36A is a diagram illustrating a configuration example of a subject state detection unit 140 (140-3) according to the eighth embodiment of the present disclosure, and FIG. 36B is a diagram illustrating a configuration example of a state estimation unit 340 (340-3). [Figure 37] FIG. 37 is a flowchart illustrating an example of the state estimation unit 340 (340-3) according to the eighth embodiment of the present disclosure. [Figure 38] FIG. 38A is a diagram illustrating a configuration example of a subject state detection unit 140 (140-4) according to the eighth embodiment of the present disclosure, and FIG. 38B is a diagram illustrating a configuration example of a state estimation unit 340 (340-4). [Figure 39] FIG. 39 is a flowchart illustrating an example of the state estimation unit 340 (340-4) according to the eighth embodiment of the present disclosure. [Figure 40] FIG. 40A is a diagram illustrating a configuration example of a subject state detection unit 140 (140-5) according to the eighth embodiment of the present disclosure, and FIG. 40B is a diagram illustrating a configuration example of a state estimation unit 340 (340-5). [Figure 41] FIG. 41 is a flowchart illustrating an example of the state estimation unit 340 (340-5) according to the eighth embodiment of the present disclosure. [Figure 42] FIG. 42 is a diagram illustrating a configuration example of the state estimation unit 340 (340-6) according to the eighth embodiment of the present disclosure. [Figure 43] FIG. 43 is a flowchart illustrating an example of the state estimation unit 340 (340-6) according to the eighth embodiment of the present disclosure. [Figure 44]FIG. 44 is a diagram illustrating a configuration example of the state estimation unit 340 (340-7) according to the eighth embodiment of the present disclosure. [Figure 45] FIG. 45 is a flowchart illustrating an example of the state estimation unit 340 (340-7) according to the eighth embodiment of the present disclosure. [Figure 46] FIG. 46 is a diagram illustrating a configuration example of the state estimation unit 340 (340-8) according to the eighth embodiment of the present disclosure. [Figure 47] FIG. 47 is a flowchart illustrating an example of the state estimation unit 340 (340-8) according to the eighth embodiment of the present disclosure. [Figure 48] FIG. 48 is a diagram illustrating a configuration example of the state estimation unit 340 (340-9) according to the eighth embodiment of the present disclosure. [Figure 49] FIG. 49 is a flowchart illustrating an example of the state estimation unit 340 (340-9) according to the eighth embodiment of the present disclosure. [Figure 50] FIG. 50 is a diagram illustrating a configuration example of the state estimation unit 340 (340-10) according to the eighth embodiment of the present disclosure. [Figure 51] FIG. 51 is a flowchart illustrating an example of the state estimation unit 340 (340-10) according to the eighth embodiment of the present disclosure. [Figure 52] FIG. 52 is a diagram showing a configuration example of a diet measurement system 1 (1L) including a measuring device 10 (10L) according to the ninth embodiment of the present disclosure. [Figure 53] FIG. 53 is a diagram illustrating a configuration example of a meal support unit 360 (360L) according to the ninth embodiment of the present disclosure. [Figure 54] FIG. 54 is a flowchart showing an example of processing in the meal information generating unit 300 (300L) according to Embodiment 9 of the present disclosure. [Figure 55] FIG. 55 is a flowchart showing an example of processing by the meal support unit 360 (360L) in the meal information generation unit 300 (300L) according to Embodiment 9 of the present disclosure. [Figure 56] FIG. 56 is a diagram showing a configuration example of a meal measurement system 1 (1M) including a measuring device 10 (10M) according to the tenth embodiment of the present disclosure. [Figure 57] FIG. 57 is a diagram illustrating a configuration example of a status notification unit 370 according to the tenth embodiment of the present disclosure. [Figure 58] FIG. 58 is a flowchart showing an example of processing in the diet information generating unit 300 (300M) according to the tenth embodiment of the present disclosure. [Figure 59] FIG. 59 is a flowchart showing an example of processing by the status reporting unit 370 according to the tenth embodiment of the present disclosure. [Figure 60] FIG. 60 is a diagram illustrating a first example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. [Figure 61] FIG. 61 is a diagram illustrating a second example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1 In the first embodiment, a configuration example having a basic configuration for controlling meal measurement will be described.

[0011] An example of the configuration of a measurement device according to the first embodiment of the present disclosure will be described. FIG. 1 is a diagram illustrating an example of the configuration of a measurement device according to a first embodiment of the present disclosure. The measurement device 10 performs dietary measurements on the diet of a subject. The measuring device 10 shown in FIG. 1 includes a meal information acquisition unit 100, a quantity estimation unit 200, and a meal information generation unit 300.

[0012] The mealtime information acquisition section 100 acquires information relating to the eating behavior of the user as a subject to be measured. Specifically, the mealtime information acquisition section 100 acquires eating behavior images, which are images of eating behaviors of each measurement subject included in the time-series captured images. The subject to be measured is a user who is included in the captured image and is the subject of dietary measurement, which is measurement related to meals. Eating movements are movements related to eating by the subject, such as hand and arm movements to scoop up food or drink from dishes on a table (dining table) using a tool and bring it to the mouth, as well as chewing and swallowing after bringing the food or drink to the mouth. The eating action image is, for example, an image relating to eating actions detected using images captured by an external device, or an image relating to eating actions detected internally by the mealtime information acquisition section 100 using images captured.

[0013] The quantity estimation unit 200 outputs the quantity of food and drink consumed for each eating action by the user as the measurement subject. The quantity estimation section 200 outputs the amount of food and drink eaten by the subject for each eating action using the eating action image. The quantity estimation unit 200 uses the eating action image to estimate the amount of food and drink brought to the subject's mouth based on the image of food and drink included in the eating action image, and outputs the estimated amount of food and drink as the amount of food and drink eaten. For example, the eating action image includes a state in which the subject scoops food and drink from a bowl with a spoon and brings it to his / her mouth. The amount of food and drink can be estimated from the food and drink in this eating action image.

[0014] The meal information generating section 300 generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimating section. The meal information generation unit 300 generates meal information based on any one or a combination of the amount of food and drink eaten for each eating action, the amount of food and drink accumulated in chronological order, the amount of food eaten for each meal, or the amount of food eaten accumulated in chronological order. The dietary information may be the above-mentioned information itself, or may be the result of analyzing the above-mentioned information. The analysis result may be information about diet based on the above-mentioned information, such as trends in dietary amount, dietary content, and dietary habits.

[0015] In addition to the above components, the measuring device 10 also includes a control unit (not shown), a storage unit (not shown), and a communication unit (not shown). A control unit (not shown) controls the entire measuring device 10 and each of its components. The control unit (not shown) starts up the measuring device 10 in accordance with, for example, an external command. The control unit (not shown) also controls the state of the measuring device 10 (operating state = start-up, shutdown, sleep, etc.). A storage unit (not shown) stores each piece of data used in the measurement device 10. The storage unit (not shown) stores, for example, the output (output data) from each component in the measurement device 10, and outputs data requested by each component to the component that made the request. A communication unit (not shown) communicates with an external device, for example, between the measurement device 10 and a peripheral device (for example, a server when the measurement device 10 is a terminal device). For example, if the measurement device 10 and the server are not connected by wire, the communication unit (not shown) has a function of communicating between the measurement device 10 and the server. The control unit (not shown), the storage unit (not shown), and the communication unit (not shown) are the same in the embodiments described below.

[0016] A processing example of the measurement device according to the first embodiment of the present disclosure will be described. FIG. 2 is a flowchart showing an example of processing performed by the measurement device 10 according to the first embodiment of the present disclosure. The process shown in FIG. 2 is a measurement method performed by the measurement device 10. For example, the measuring device 10 shown in FIG. 1 starts the process shown in FIG. 2 (“Start”) when a measurement start condition is met, such as when the power of the measuring device is turned from OFF to ON, or when the meal measurement execution button on the measuring device is pressed, making it possible for the measuring device 10 to measure meals.

[0017] The measuring device 10 then executes a mealtime information acquisition process (step ST100). In the mealtime information acquisition process, the mealtime information acquisition unit 100 of the measuring device 10 acquires mealtime behavior images, which are images of the mealtime behavior of each measurement subject included in the time-series captured images.

[0018] The measuring device 10 then executes a quantity estimation process (step ST200). In the quantity estimation process, the quantity estimation unit 200 of the measuring device 10 outputs the amount of food and drink consumed by the subject for each eating action using the eating action image.

[0019] The measuring device 10 then executes a meal information generation process (step ST300). In the meal information generation process, the meal information generation unit 300 of the measuring device 10 generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit 200.

[0020] The measuring device 10 then executes a meal information output process (step ST400). In the meal information output process, the meal information generation unit 300 of the measuring device 10 outputs the generated meal information to an output destination such as a storage unit (not shown), a meal information output unit (described later), a record database (described later), or a display device (not shown).

[0021] The measuring device 10 then executes an end determination process (step ST400 "End?"). In the termination determination process, a control unit (not shown) of the measuring device 10 determines whether to terminate the processing of the measuring device 10. The control unit (not shown) determines whether to terminate the processing of the measuring device 10 in accordance with, for example, an external termination command or an execution program. When the control unit (not shown) determines not to end the processing of the measuring device 10 (step ST400 "NO"), the process proceeds to step ST100, and the process is repeated from step ST100. When the control unit (not shown) determines that the processing of the measuring device 10 is to be ended (step ST400 "YES"), the measuring device 10 ends the processing ("end").

[0022] Next, a configuration example of a system including the measurement device according to the first embodiment will be described. FIG. 3 is a diagram showing a configuration example of a diet measurement system 1 (1A) including a measurement device 10 (10A) according to the first embodiment of the present disclosure. The dietary measurement system 1 (1A) is a system that measures the diet of a subject. The diet measurement system 1A includes a measurement device 10 (10A) and an information source device 20 (20A).

[0023] The information source device 20A acquires information used for processing by the measurement device 10A and outputs it to the measurement device 10A. In the description, the information source device 20A is separate from the measurement device 10A, but it may also be configured as an integrated device, for example, included in the measurement device 10A. The information source device 20A shown in FIG. The photographing device 21 photographs the place where the meal is to be eaten and outputs the photographed image. The photographing device 21 is, for example, a camera placed on a table where meals are eaten. In this case, the photographing device 21 may be configured so that the height of the photographing position or the photographing direction can be changed. Alternatively, the image capturing device 21 may be, for example, a camera installed on the ceiling above a table (dining table). In this case, the image capturing device 21 may be configured to be able to change the image capturing direction. The image capturing device 21 may be capable of outputting images other than those captured using visible light, as long as it can output captured images that allow the measurement device 10A to function.

[0024] The measuring device 10A performs dietary measurement of the subject's diet using captured images. The measuring device 10A shown in FIG. 3 includes a meal information acquisition unit 100 (100A), a quantity estimation unit 200 (200A), a meal information generation unit 300 (300A), and a meal information output unit 400 (400A).

[0025] The mealtime information acquisition section 100A acquires mealtime information, which is information relating to the eating behavior of the user as a subject to be measured. The mealtime information acquiring section 100A acquires, as mealtime information, mealtime behavior images, which are images of the mealtime behavior of each measurement subject included in the time-series captured images. The mealtime information acquisition section 100A shown in FIG. The eating action detection section 110 of the mealtime information acquisition section 100A receives photographed images in chronological order, and detects eating action images using the received photographed images in chronological order. The eating information acquisition section 100A acquires eating action images detected by the eating action detection section 110.

[0026] The quantity estimation unit 200A outputs the quantity of food and drink consumed for each eating action by the user as the measurement subject. The amount estimation section 200A shown in FIG.

[0027] The intake amount estimation unit 210 of the amount estimation unit 200A estimates the intake amount, which is the amount of food and drink eaten for each eating action of the subject, using the eating action image. The intake amount estimation unit 210 uses the eating action image to perform image analysis based on the images of food and drink contained in the eating action image, estimates the amount of food and drink brought to the mouth of the subject as the analysis result, and outputs the estimated intake amount of food and drink. Alternatively, the intake amount estimation unit 210 may be configured to perform image analysis processing using AI (Artificial Intelligence) technology. In this case, the intake amount estimation unit 210 performs AI image analysis using an image of an eating action as input, and outputs the amount of food and drink consumed.

[0028] Like the meal information generating section 300 already described, the meal information generating section 300A generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimating section. Specifically, the meal information generation unit 300A generates meal information based on any one or a combination of the amount of food and drink eaten for each eating action, the amount of food and drink accumulated over time, the amount of food eaten for each meal, or the amount of food eaten accumulated over time. The dietary information may be the above-mentioned information itself, or may be the result of analyzing the above-mentioned information. The analysis result may be information about diet based on the above-mentioned information, such as trends in dietary amount, dietary content, and dietary habits.

[0029] The meal information output unit 400A outputs the meal information generated by the meal information generation unit 300A. The meal information output unit 400A outputs the meal information to an output destination such as a storage unit (not shown), a record database (described below), or an external processing device (not shown). Alternatively, the meal information output section 400A outputs the meal information to an output destination such as a display device included in the measurement device.

[0030] Here, the meal measurement system of the present disclosure can be configured to include a terminal device and a server. In the meal measurement system, the terminal device and the server may be configured to cooperate to realize the measurement device. For example, the terminal device is configured to include a meal information acquisition unit 100A, a quantity estimation unit 200A, and a meal information output unit 400A, and the server is configured to include a meal information generation unit 300A. Specifically, for example, the quantity estimation unit 200A of the terminal device uses eating behavior images to output to the server the amount of food and drink consumed, which is the amount of food and drink consumed for each eating behavior of the subject. The server's meal information generation unit 300A accepts the amount of food and drink consumed for each eating behavior output by the quantity estimation unit 200A, and outputs meal information based on the accepted amount of food and drink to the terminal device. The terminal device's meal information output unit 400A accepts the meal information output by the server's meal information generation unit 300A, and outputs the accepted meal information to a display device or the like. By configuring it in this manner, the processing load on the terminal device can be reduced, and it is possible for the meal measurement service provider using the server to perform analysis using information from many users, thereby improving the accuracy and quality of the meal information provided using the analysis results. The dietary measurement system can also be configured to include a terminal device and a server, which also applies to the embodiments described below.

[0031] Next, a processing example in the measurement device in the diet measurement system according to the first embodiment of the present disclosure will be described. FIG. 4 is a flowchart showing an example of processing by the measurement device 10A according to the first embodiment of the present disclosure. The process shown in FIG. 4 is a control method performed by the measuring device 10A. For example, the measuring device 10A shown in FIG. 3 starts the process shown in FIG. 4 when a measurement start condition is met, such as when the power of the measuring device 10A is turned from OFF to ON, or when the meal measurement execution button on the measuring device 10A is pressed, making it possible for the measuring device 10A to perform meal measurement ("Start").

[0032] The measuring device 10A then executes a mealtime information acquisition process (step ST1100). In the mealtime information acquisition process, the mealtime information acquisition unit 100A of the measuring device 10A acquires mealtime behavior images, which are images of the mealtime behavior of each measurement subject included in the time-series captured images.

[0033] Here, a detailed example of the processing of the mealtime information acquisition section 100A will be described. FIG. 5 is a flowchart showing an example of processing by the mealtime information acquisition unit 100A according to the first embodiment of the present disclosure. Upon receiving the captured image, mealtime information acquisition section 100A starts processing (“START”).

[0034] The mealtime information acquisition unit 100A executes an eating action detection process (step ST1110). In the eating action detection process, the eating action detection unit 110 of the mealtime information acquisition unit 100A receives captured images in chronological order from the imaging device 21 of the information source device 20A. The eating action detection unit 110 detects eating action images using the received chronologically-series captured images. The eating action detection unit 110 outputs eating action images, which are chronological images related to the detected eating actions, and then ends the process ("end").

[0035] Returning to the explanation of Figure 4. The mealtime information acquisition section 100A outputs the detected eating action image to the amount estimation section 200A.

[0036] The measuring device 10A then executes an amount estimation process (step ST1200). In the amount estimation process, the amount estimation unit 200A of the measuring device 10A estimates the amount of food and drink eaten by the subject for each eating action using the eating action image.

[0037] Here, a detailed example of the processing of the quantity estimator 200A will be described. FIG. 6 is a flowchart showing an example of processing by the quantity estimator 200A according to the first embodiment of the present disclosure. Once the quantity estimation unit 200A starts processing ("START"), it then executes the intake amount estimation process (step ST1120). In the intake amount estimation process, the intake amount estimation unit 210 of the quantity estimation unit 200A receives the eating action image output by the eating information acquisition unit 100A. The intake amount estimation unit 210 uses the eating action image to perform image analysis based on the images of food and drink contained in the eating action image. The intake amount estimation unit 210 estimates the type and amount of food and drink brought to the mouth of the measurement subject as the analysis results, and outputs the estimated type and amount of food and drink. Alternatively, the intake amount estimation unit 210 may be configured to perform image analysis processing using AI (Artificial Intelligence) technology. In this case, the intake amount estimation unit 210 estimates the intake amount for each type of food and drink by performing AI image analysis using eating action images as input. The intake amount estimation unit 210 outputs the intake amount for each type of food and drink as the estimation result. The amount estimation unit 200A ends the process after outputting the amount of food and drink ("end"). In addition, when the meal measurement system 1A has the above-mentioned configuration including a terminal device and a server, the quantity estimation unit 200A of the terminal device outputs the amount of food and drink consumed, which is the amount of food and drink consumed for each meal action of the subject, to the server (the meal information generation unit 300A of the server) using the meal action image.

[0038] Returning to the explanation of Figure 4. The measuring device 10A then executes a meal information generation process (step ST1300). In the meal information generation process, the meal information generation unit 300A of the measuring device 10A generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit. Here, when the meal measurement system 1A has the above-mentioned configuration including a terminal device and a server, the meal information generation unit 300A of the server accepts the amount of food and drink consumed for each eating action output by the amount estimation unit 200A, and outputs meal information based on the accepted amount of food and drink consumed to the terminal device.

[0039] The measuring device 10A then executes a meal information output process (step ST1400). In the meal information output process, the meal information output unit 400A of the measuring device 10A outputs the meal information generated by the meal information generation unit 300A. The meal information output unit 400A outputs the meal information to an output destination, such as a storage unit (not shown), a record database (described below), an external processing device (not shown), or a display device included in the measuring device. Here, when the meal measurement system 1A has the above-mentioned configuration including a terminal device and a server, the meal information output unit 400A of the terminal device receives the meal information output by the meal information generation unit 300A of the server, and outputs the received meal information to a display device or the like.

[0040] The measuring apparatus 10A then executes an end determination process (step ST1500 "End?"). In the termination determination process, a control unit (not shown) of the measuring device 10A determines whether to terminate the processing of the measuring device 10A. The control unit (not shown) determines whether to terminate the processing of the measuring device 10A in accordance with, for example, an external termination command or an execution program. If the control unit (not shown) determines not to end the processing of the measuring device 10A ("NO" in step ST1500), the process proceeds to step ST1100, and the processing is repeated from step ST1100. When the control unit (not shown) determines that the processing of the measuring device 10A is to be ended (step ST1500 "YES"), the measuring device 10A ends the processing ("end").

[0041] This embodiment shows a configuration including the following. a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink eaten by the subject for each eating action using the eating action image; a meal information generation unit that generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit; A measuring device comprising: As a result, the present disclosure has the effect of providing a measurement device that enables the accuracy of meal measurement to be improved compared to conventional devices.

[0042] This embodiment shows a configuration including the following. a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink consumed by the subject for each eating action using the eating action image to a server outside the device; a meal information output unit that receives from the server meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit and outputs the meal information; A terminal device comprising: As a result, the present disclosure has the effect of providing a terminal device that enables the accuracy of meal measurement to be improved compared to conventional devices.

[0043] This embodiment shows a configuration including the following. A meal measurement system including a terminal device and a server, The terminal device a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs to the server an amount of food and drink consumed by the subject for each eating action using the eating action image; The server a meal information generation unit that receives the amount of food and drink consumed for each eating behavior output by the amount estimation unit, and outputs meal information based on the received amount of food and drink consumed to the terminal device. The terminal device A meal information output unit is provided which receives the meal information output by the server and outputs the received meal information. A meal measurement system characterized by: As a result, the present disclosure has the effect of providing a meal measurement system that enables the accuracy of meal measurement to be improved compared to conventional systems.

[0044] This embodiment shows a configuration including the following. A measurement method using a measurement device, a mealtime information acquisition step in which a mealtime information acquisition unit of the measurement device acquires mealtime behavior images, which are images of mealtime behaviors of each measurement subject included in time-series captured images; an amount estimation step in which a quantity estimation unit of the measuring device uses the eating behavior image to output an eating amount, which is the amount of food and drink eaten by the measurement subject for each eating behavior; a meal information generating step in which a meal information generating unit of the measuring device generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimating unit; A measuring method comprising: As a result, the present disclosure has the effect of providing a measurement method that enables the accuracy of meal measurement to be improved compared to conventional methods.

[0045] This embodiment shows a configuration including the following. A measurement method using a terminal device, a mealtime information acquisition step in which a mealtime information acquisition unit of the terminal device acquires mealtime behavior images, which are images of mealtime behaviors of each measurement subject included in time-series captured images; an amount estimation step in which the amount estimation unit of the terminal device outputs an amount of food and drink eaten by the subject for each eating action using the eating action image to a server outside the device; a meal information output step in which the meal information output unit of the terminal device receives from the server meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit and outputs the meal information; A measuring method comprising: As a result, the present disclosure has the effect of providing a measurement method that enables the accuracy of meal measurement to be improved compared to conventional methods.

[0046] This embodiment shows a configuration including the following. Computer, a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink eaten by the subject for each eating action using the eating action image; a meal information generation unit that generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit; A measuring device comprising: A program characterized by operating as As a result, the present disclosure has the effect of providing a program that causes a computer to operate as a measurement device that enables the accuracy of meal measurement to be improved compared to conventional methods.

[0047] This embodiment shows a configuration including the following. Computer, a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink consumed by the subject for each eating action using the eating action image to a server outside the device; a meal information output unit that receives from the server meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit and outputs the meal information; A terminal device comprising: A program characterized by operating as As a result, the present disclosure has the effect of providing a program that causes a computer to operate as a terminal device that enables the accuracy of meal measurement to be improved compared to conventional methods.

[0048] This embodiment further includes the following configuration. The mealtime information acquisition unit Accepting photographed images in time series, and detecting and acquiring the eating action image using the accepted photographed images in time series. Characterized by

[0049] The measuring device according to claim 1. As a result, the present disclosure further has the effect of providing a control device that enables the accuracy of meal measurement to be improved compared to conventional methods, for example, by using images captured by an existing camera. Furthermore, the present disclosure achieves the same effect as the above by applying the above configuration to a system including a measurement device, the above terminal device in the system, the above measurement method, or the above program.

[0050] Embodiment 2 In the above-described first embodiment, a configuration example including a form in which the area of ​​a person to be measured is detected using a captured image has been described. In the second embodiment, a configuration example that enables further improvement in the accuracy of detecting the area of ​​the measurement subject will be described. In the second embodiment, among the components of the second embodiment, the components and their functions that are similar to the components of the first embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0051] Next, a configuration example of a measurement device according to a second embodiment of the present disclosure will be described. FIG. 7 is a diagram showing a configuration example of a dietary measurement system 1 (1B) including a measurement device 10 (10B) according to the second embodiment of the present disclosure. The diet measurement system 1 (1B) shown in FIG. 7 includes a measurement device 10 (10B) and an information source device 20 (20B).

[0052] The information source device 20B is configured in the same manner as the information source device 20 already described, and the information source device 20B shown in FIG.

[0053] The measuring device 10B performs dietary measurement of the subject's diet using the captured images, or performs dietary measurement of the subject's diet using the captured images and registered information. The measuring device 10B shown in Figure 7 is configured to include a meal information acquisition unit 100 (100B), a quantity estimation unit 200 (200B), a meal information generation unit 300 (300B), a meal information output unit 400 (400B), a registration information acquisition unit 500 (500B), and a record database unit 600 (600B).

[0054] The registration information acquisition unit 500B acquires information used auxiliary for meal measurement by the measurement device 10B from pre-registered registration information. The registration information acquiring unit 500B acquires, for example, registered measurement subject information including a photographing area for each measurement subject stored in advance. The registered measurement subject information includes, for example, information for identifying the measurement subject, the time when the measurement subject sits down, the position (area) where the measurement subject sits down, and a facial image of the measurement subject. Moreover, the registration information acquisition unit 500B is configured to receive and store the registration information. The registration information acquisition unit 500B shown in FIG. 7 includes a registration acceptance unit 510 and a registration information storage unit 520. The registration reception unit 510 receives registration information. When the registration reception unit 510 receives a registration request from an operator, it displays an image (GUI) for input and receives the registration information based on the operation by the operator on the image. With this configuration, it is possible to easily change the seat, for example, when a seat needs to be changed due to a visitor or the like. The registration receiving unit 510 receives, for example, registered measurement subject information including an imaging area for each measurement subject. The registered measurement subject information may also include, for example, a face image for each measurement subject. The registration receiving unit 510 also receives, for example, registered food and drink type information indicating the type of food and drink. The registered food and drink type information is information based on a menu for each type of food and drink in a meal, for example. The registration information storage unit 520 stores the registration information accepted by the registration acceptance unit 510. The registration information storage unit 520 stores, for example, registered measurement object information, registered food and drink type information, or registered measurement object information and registered food and drink type information.

[0055] The record database unit 600B records the amount of food and drink consumed and dietary information for each subject to be measured. The record database unit 600B shown in Fig. 7 is configured to include an intake amount record database 610 and a meal record database 620. The intake amount record database 610 of the record database unit 600B chronologically accumulates the intake amount for each type of food and drink output for each eating behavior by the amount estimation unit 200B. Furthermore, the meal record database 620 of the record database unit 600B records the meal information output by the meal information generation unit 300B for each meal. Furthermore, the record database unit 600B may be further configured to record information acquired by the meal information acquisition unit 100B.

[0056] The mealtime information acquiring section 100B acquires eating behavior images, which are images of eating behaviors of each measurement subject included in the time-series captured images. The eating information acquiring section 100B shown in FIG. 7 includes an eating motion detecting section 110 and a measurement subject information acquiring section 120. The eating motion detecting section 110 detects an eating state of a person. When multiple subjects are included in the time-series captured images, the measurement subject information acquiring section 120 acquires measurement subject information relating to the region of each of the multiple subjects. Furthermore, when the registered measurement subject information relating to the measurement subject is registered as registration information, the measurement subject information acquiring unit 120 acquires the measurement subject information including the registered measurement subject information using the registered measurement subject information acquired by the registration information acquiring unit 500B. Specifically, for example, the measurement subject information acquiring unit 120 performs recognition by comparing the captured image with the image of the measurement subject included in the registered measurement subject information. The eating motion detection section 110 of the mealtime information acquisition section 100B receives the captured images in chronological order and detects eating motion images using the received chronologically captured images. The eating motion detection section 110 further detects eating motion images using the registered measurement subject information acquired by the registration information acquisition section 500B.

[0057] The quantity estimation unit 200B outputs the quantity of food and drink consumed for each eating behavior by the user as the measurement subject. The quantity estimation unit 200B further outputs the amount of food and drink consumed for each type of food and drink. The amount estimation unit 200B shown in FIG. 7 includes an eating amount estimation unit 210 and a food and drink type estimation unit 220.

[0058] The food type estimation unit 220 of the quantity estimation unit 200B estimates the type of food, which is the type of each food, using the eating action image. The food type estimation unit 220 uses the eating action image to perform image analysis based on the image of the food included in the eating action image, estimates the food type as the analysis result, and outputs the estimated food type. Alternatively, the food and drink type estimation unit 220 of the quantity estimation unit 200B may be configured to perform image analysis processing using AI (Artificial Intelligence) technology. In this case, the eating amount estimation unit 210 performs AI image analysis using an eating action image as input, and outputs the type of food and drink. Furthermore, the food and drink type estimation unit 220 of the quantity estimation unit 200B further outputs the type of food and drink using the registered information. The food and drink type estimation unit 220 outputs the type of food and drink using the registered food and drink type information, which is the registered information acquired by the registered information acquisition unit 500B, and the eating action image. This allows the food and drink type estimation unit 220 to improve the accuracy of the food and drink type estimation result.

[0059] The intake amount estimation section 210 of the quantity estimation section 200B estimates the intake amount, which is the amount of food and drink eaten for each eating action of the subject, using the food type and eating action image estimated by the food and drink type estimation section 220. This allows the quantity estimation section 200B to estimate the intake amount for each type of food and drink for each eating action.

[0060] The meal information generating unit 300B generates meal information based on the amount of food and drink consumed for each eating action and for each type of food and drink output by the amount estimating unit 200B. Specifically, the meal information generating unit 300B generates meal information based on any one or a combination of the amount of food and drink consumed for each eating action, the amount of food and drink accumulated in chronological order, the amount of food consumed for each meal, and the amount of food and drink accumulated in chronological order. The meal information generating section 300B calculates the total amount of food and drink consumed for each meal based on the amount of food and drink accumulated in chronological order, and generates meal information indicating the amount of food consumed using the total amount of food and drink consumed. The diet information generating section 300B shown in FIG. The total amount of food and drink calculation section 310 calculates the amount of food and drink eaten for each meal using the amounts of food and drink accumulated in the food and drink amount recording database 610 of the recording database section 600.

[0061] The meal information output section 400B outputs the meal information generated by the meal information generation section 300B, similar to the meal information output section 400A already described.

[0062] Next, a processing example of the measurement device according to the second embodiment of the present disclosure will be described. FIG. 8 is a flowchart showing an example of processing by the measurement device 10 (10B) according to the second embodiment of the present disclosure. The process shown in FIG. 8 is a control method performed by the measuring device 10B. The measuring device 10B shown in FIG. 7 starts the process shown in FIG. 8 when a measurement start condition is met, such as when the power of the measuring device 10B is turned from OFF to ON, or when the meal measurement execution button on the measuring device 10B is pressed, making it possible for the measuring device 10B to perform meal measurement. ("Start")

[0063] The measurement device 10B then executes a registration information acquisition process (step ST2100). In the registration information acquisition process, if registration information is registered, the registration information acquisition unit 500B of the measurement device 10B acquires the registration information and outputs it to the meal information acquisition unit 100B. In the description here, the registration information is registered measurement subject information including the position or area of ​​each measurement subject, or registered food and drink type information.

[0064] Here, an example of the registration process in the registration information acquisition unit 500 (500B) will be described. FIG. 9 is a flowchart showing an example of processing by the registration information acquisition unit 500 (500B) according to the second embodiment of the present disclosure. When a registration is requested by a user who is to be registered, the registration information acquisition unit 500B starts the process shown in FIG. 9 (“Start”). Next, registration information acquisition unit 500B executes a registration acceptance process (step ST2710). In the registration acceptance process, upon accepting a registration request from an operator, registration acceptance unit 510 of registration information acquisition unit 500B displays an image (GUI) for input and accepts registration information based on an operation by the operator on the image. Next, registration information acquiring section 500B executes a registration information storing process (step ST2720). In the registration information storing process, registration information storing section 520 of registration information acquiring section 500B stores the registration information accepted by registration accepting section 510. After storing the registration information, registration information acquisition unit 500B ends the process and waits until the next command is received ("end").

[0065] Returning to the explanation of Figure 8. The measuring device 10B executes a mealtime information acquisition process (step ST2200). In the mealtime information acquisition process, the mealtime information acquisition unit 100B of the measuring device 10B acquires mealtime behavior images, which are images of the mealtime behavior of each measurement subject included in the time-series captured images.

[0066] Here, a detailed example of the processing of the mealtime information acquisition section 100B will be described. FIG. 10 is a flowchart showing an example of the processing of the mealtime information acquisition section 100 (100B) according to the second embodiment of the present disclosure. Upon receiving the captured image, mealtime information acquisition section 100B starts processing (“START”). The mealtime information acquiring section 100B executes a measurement subject information acquiring process (step ST2110). In the measurement subject information acquiring process, when multiple measurement subjects are included in the time-series captured images, the measurement subject information acquiring section 120 of the mealtime information acquiring section 100B acquires measurement subject information relating to the regions of each of the multiple measurement subjects. In addition, when registered measurement subject information relating to the measurement subject is registered as registered information, the measurement subject information acquisition unit 120 acquires measurement subject information including the registered measurement subject information using the registered measurement subject information acquired by the registered information acquisition unit 500B. The measurable object information acquiring section 120 outputs the acquired measurable object information to the eating motion detecting section 110. The mealtime information acquisition unit 100B executes eating action detection processing (step ST2120). In the eating action detection processing, the eating action detection unit 110 of the mealtime information acquisition unit 100B receives time-series captured images output by the imaging device 21 of the information source device 20B, and also receives registered measurable object information output by the registration information acquisition unit 500B. The eating action detection unit 110 detects eating action images using the time-series captured images and the registered measurable object information. For example, when food or drink on a table (dining table) is brought to the mouth in the time-series captured images, the eating action detection unit 110 regards the captured images during this period as eating action images, acquires eating action information including the eating action images, and outputs it as mealtime information. After outputting the eating action image as the eating information, eating information acquisition section 100B ends the process ("end").

[0067] Returning to the explanation of Figure 8. The measuring device 10B executes the amount estimation process (step ST2300). In the amount estimation process, the amount estimation unit 200B of the measuring device 10B outputs the amount of food or drink consumed for each eating action by the user as the person to be measured. The quantity estimation unit 200B further outputs the amount of food and drink consumed for each type of food and drink.

[0068] Here, a detailed example of the processing of the quantity estimator 200B will be described. FIG. 11 is a flowchart showing an example of processing by the quantity estimator 200 (200B) according to the second embodiment of the present disclosure. The amount estimation unit 200B starts processing ("start"), for example, when it acquires an eating action image output by the eating information acquisition unit 100B.

[0069] The quantity estimation unit 200B executes a food type estimation process (step ST2210). In the food type estimation process, the food type estimation unit 220 of the quantity estimation unit 200B estimates the type of food, which is the type of each food, using the eating action image. Specifically, the food type estimation unit 220 acquires the eating action image output by the eating information acquisition unit 100B. The food type estimation unit 220 estimates the type of each food based on the image of the food brought to the mouth by the subject using the eating action image, and outputs food type information indicating the food type.

[0070] The quantity estimation unit 200B executes an intake amount estimation process (step ST2220). In the intake amount estimation process, the intake amount estimation unit 210 of the quantity estimation unit 200B estimates the intake amount, which is the amount of food and drink eaten for each eating action of the measurement subject, using the food type information and eating action image output by the food type estimation unit 220. Specifically, the intake amount estimation unit 210 acquires food and drink type information output by the food and drink type estimation unit 220, and acquires eating action images output by the eating information acquisition unit 100B. Using the food and drink type information and the eating action images, the intake amount estimation unit 210 estimates the amount of food and drink consumed for each eating action of the subject based on images of food and drink brought to the subject's mouth by using the food and drink type information and the eating action images, and outputs intake amount information including the amount of food and drink consumed. The intake amount estimation unit 210 outputs intake amount information such as the time of the eating action, the type of food (for example, the name of the dish "nikujaga"), and the amount of food and drink consumed, such as "30g."

[0071] The amount estimation unit 200B combines the food type information and the intake amount information, outputs the intake amount information for each food type to the diet information generation unit 300B, and then ends the process ("end").

[0072] Returning to the explanation of Figure 8. The measuring device 10B executes a meal information generation process (step ST2400). In the meal information generation process, the meal information generation unit 300B of the measuring device 10B generates meal information based on the amount of food consumed for each eating behavior and for each type of food output by the amount estimation unit 200B. The meal information generation unit 300B refers to the meal amount recording database 610 of the recording database unit 600B, and calculates the total amount of food and drink consumed, which is the total amount of food and drink consumed for each type of food and drink, using the amount of food and drink consumed during the meal time from the start time of the meal to the end time of the meal.

[0073] Here, a detailed example of the processing of the meal information generating unit 300B will be described. FIG. 12 is a flowchart showing an example of processing by the diet information generating unit 300 (300B) according to the second embodiment of the present disclosure. The meal information generating unit 300B starts processing when it acquires the food and drink amount information output by the amount estimating unit 200B ("START"). The meal information generation unit 300B executes a process to calculate the total amount of food and drink (step ST2310). In this process, the total amount of food and drink calculation unit 310 of the meal information generation unit 300B refers to the food and drink amount recording database 610 of the recording database unit 600B, and obtains the amount of food and drink consumed for each eating behavior during the meal from the meal start time to the meal end time. The total amount of food and drink calculation unit 310 adds up the obtained amounts of food and drink to calculate the total amount of food and drink consumed. The total amount of food and drink calculation unit 310 outputs the calculated total amount of food and drink consumed as meal information for the meal. After outputting the meal information to the meal information output unit 400B, the meal information generation unit 300B ends the process and waits until the next process ("end").

[0074] Returning to the explanation of Figure 8. The measuring device 10B executes a meal information output process (step ST2500). In the meal information output process, the meal information output unit 400B of the measuring device 10B acquires the meal information output by the meal information generation unit 300B, and outputs the total amount of food and drink indicated in the meal information.

[0075] The measuring equipment 10B then executes an end determination process (step ST2600 "end?"). In the termination determination process, a control unit (not shown) of the measuring device 10B determines whether to terminate the processing of the measuring device 10. The control unit (not shown) determines whether to terminate the processing of the measuring device 10B in accordance with, for example, an external termination command or an execution program. When the control unit (not shown) determines not to end the processing of the measuring device 10B (2600 "NO"), the process proceeds to step ST2100, and the processing is repeated from step ST2100. When the control unit (not shown) determines that the processing of the measuring device 10B is to be ended (2600 "YES"), the measuring device 10B ends the processing ("End").

[0076] As described in this embodiment, by registering information that is known in advance, such as the person to be measured and the seat in which the person to be measured sits, and recognizing the person to be measured using the registered information, accuracy can be improved compared to when the registered information is not used. In this case, for example, if the captured image covers a table and its surroundings, and it is known in advance that A and B usually sit across from each other and that only A eats breakfast, it is possible to register the following: "Time: 7:00, Position: Left, Subject: A," "Time: 12:00, Position: Left, Subject: A," "Time: 12:00, Position: Right, Subject: B," "Time: 19:00, Position: Left, Subject: A," "Time: 19:00, Position: Right, Subject: B." Furthermore, by accumulating the amount of food and drink consumed for each meal action over the duration of the meal, the total amount of food and drink consumed for each meal can be recorded as the amount of food eaten, thereby providing highly accurate amounts of food eaten.

[0077] This embodiment further includes the following configuration. a registration information acquisition unit that acquires registered measurement subject information including a photographing area for each measurement subject that has been stored in advance; Furthermore, The mealtime information acquisition unit and detecting the eating action image using the registered measurement subject information acquired by the registration information acquisition unit. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that can improve the accuracy of detecting the subject and their eating behavior by, for example, pre-storing the sitting position (area) of each subject within the shooting range, thereby making it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0078] This embodiment further includes the following configuration. The meal information output by the meal information generation unit includes: The amount of food and drink consumed for each meal is accumulated in chronological order. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that makes it possible to improve the accuracy of meal measurement, such as the total amount of food and drink consumed at each meal, compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0079] This embodiment further includes the following configuration. The quantity estimator Furthermore, the amount of food and drink consumed is output for each type of food and drink. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that makes it possible to obtain the amount of food and drink consumed for each type even when there are multiple types of food and drink, for example, when different dishes are served in each dish. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0080] This embodiment further includes the following configuration. The meal information output by the meal information generation unit includes: The amount of food and drink consumed for each eating behavior is accumulated in chronological order, and the total amount of food and drink consumed for each meal and for each type of food and drink is included. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that makes it possible to improve the accuracy of meal measurement, such as the total amount of food and drink consumed at each meal, compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0081] Embodiment 3 In the above-described embodiment, the amount of food and drink consumed is estimated based on images of food and drink related to the eating behavior of the subject. In the third embodiment, a configuration example will be described in which the accuracy of meal measurement is improved by estimating the amount of food and drink eaten using the amount of food and drink remaining. In the third embodiment, among the components of the third embodiment, those components and their functions that are similar to those of the components of the first or second embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0082] Next, a configuration example of a measurement device according to a third embodiment of the present disclosure will be described. FIG. 13 is a diagram showing a configuration example of a diet measurement system 1 (1C) including a measurement device 10 (10C) according to the third embodiment of the present disclosure. The diet measurement system 1 (1C) shown in FIG. 13 includes a measurement device 10 (10C) and an information source device 20 (20C).

[0083] The information source device 20C is configured similarly to the information source device 20 already described, and the information source device 20C shown in FIG.

[0084] The measurement device 10C performs dietary measurements related to the diet of a subject. The measuring device 10C shown in Figure 13 is configured to include a meal information acquisition unit 100 (100C), a quantity estimation unit 200 (200C), a meal information generation unit 300 (300C), a meal information output unit 400 (400C), a registration information acquisition unit 500 (500C), a record database unit 600 (600C), and a food and drink type database 630.

[0085] The registration information acquisition unit 500C acquires information used auxiliary for meal measurement by the measurement device 10C from pre-registered registration information. The registration information acquisition unit 500C acquires a pre-stored registered initial amount, which is the amount of each food and drink before starting a meal. The registration information acquisition unit 500C is configured to receive and store the registration information. The registration information acquisition unit 500C shown in FIG. 13 includes a registration acceptance unit 510 and a registration information storage unit 520. The registration reception unit 510 receives registration information. When the registration reception unit 510 receives a registration request from an operator, it displays an image (GUI) for input and receives the registration information based on an operation by the operator on the image. The registration receiving unit 510 of the registration information acquiring unit 500C receives, as the registration information, for example, an initial registration amount, which is the amount of each food or drink before starting a meal. The registration information storage unit 520 stores the registration information accepted by the registration accepting unit 510. The registration information storage unit 520 of the registration information acquisition unit 500C stores, for example, the registration initial amount, which is the amount of each food or drink before starting a meal. The initial registered amount is the amount of food and drink before the meal begins that is registered based on the menu, for example, when the amount of each dish is predetermined according to the menu.

[0086] The record database unit 600C records the amount of food and drink consumed and dietary information for each subject to be measured. The record database section 600C shown in FIG. 13 is configured to include an eating and drinking amount record database 610 and a meal record database 620. The food intake record database 610 is similar to the food intake record database 610 already described. The meal record database 620 is similar to the meal record database 620 already described.

[0087] The food and drink type database 630 is a database used to generate meal information. The food and drink type database 630 stores information such as identification information, food and drink names, ingredients, nutrients, and calorie content in association with each other. The identification information is information for identifying each combination of data, and is information for making it possible to identify each type of food and drink. The food and drink name is the name of each type of food and drink, for example, the name of a dish. Food ingredients are ingredients contained in food and drink, and indicate, for example, ingredients and their amounts for each dish. Food and drink nutrients are nutrients contained in food and drink, and indicate the nutrients per unit amount contained in each dish, for example. The calorific value of food and drink is the calorific value of the food and drink, and indicates, for example, the calorific value per unit amount. By using such information, more detailed meal information can be generated for each meal.

[0088] The mealtime information acquisition section 100C acquires eating behavior images, which are images of eating behaviors of each measurement subject included in the time-series captured images. The eating information acquiring section 100C shown in FIG. 13 includes an eating motion detecting section 110 and a measurement subject information acquiring section 120. The eating motion detecting section 110 detects an eating state of a person. The eating motion detection section 110 has the same configuration as the eating motion detection section 110 already described. The measurable object information acquiring section 120 has the same configuration as the measurable object information acquiring section 120 already described.

[0089] The quantity estimation unit 200C outputs the quantity of food and drink consumed for each eating behavior by the user as the measurement subject. The quantity estimation unit 200C uses the captured image to obtain the amount of remaining food and drink, which is the amount of uneaten food and drink, and further uses the uneaten amount to estimate the amount of food and drink eaten. Alternatively, the quantity estimation unit 200C uses the uneaten amount to estimate the eating rate, and then uses the eating rate to estimate the amount of food and drink eaten. The quantity estimation unit 200C further estimates the amount of food and drink consumed using the initial registered amount acquired by the registration information acquisition unit 500C. Alternatively, the quantity estimation unit 200C estimates the rate of food and drink consumed using the initial registered amount and the amount of food and drink not consumed, and estimates the amount of food and drink consumed using the rate of food and drink consumed. The quantity estimation section 200C shown in FIG. 13 includes an ingested amount estimation section 210, a food and drink type estimation section 220, an unattached amount acquisition section 230, and a difference amount calculation section 240. The food and drink type estimation unit 220 is configured in the same manner as the food and drink type estimation unit 220 already described. The uneaten amount acquisition section 230 uses the photographed images in time series to acquire the uneaten amount, which is the amount of food and drink, for each remaining food and drink in time series. Each time the difference amount calculation unit 240 acquires the amount of uneaten food, it calculates the difference amount between the amount of uneaten food acquired in the current eating action and the amount of uneaten food acquired in the previous eating action. The intake amount estimation unit 210 further estimates the intake amount using the difference amount output by the difference amount calculation unit 240. This allows the intake amount estimation unit 210 to improve the accuracy of estimating the intake amount not only based on the eating action image but also by using the reduction in the amount of uneaten food and drink for each eating action. Alternatively, this allows the intake amount estimation unit 210 to improve the accuracy of estimating the intake amount not only based on the eating action image but also by using the eating rate, which is the reduction rate of uneaten food and drink for each eating action. Furthermore, the intake amount estimation unit 210 estimates the intake amount using the initial registered amount acquired by the registration information acquisition unit 500C. This allows the intake amount estimation unit 210 to more accurately estimate the amount of uneaten food before the meal starts, thereby improving the accuracy of estimating the intake amount.

[0090] The meal information generating section 300C is configured to be able to generate meal information that is different from the meal information generated by the meal information generating section 300 already described. The meal information output by meal information generating section 300C includes a completion rate for each meal and each food and drink, which indicates the ratio of the total amount of food and drink obtained by accumulating the amount of food and drink eaten for each eating behavior to the amount of each food and drink before the meal started. The meal information output by the meal information generating unit 300C includes a completion rate for each meal and each food and drink, which indicates the ratio of the total meal amount obtained by accumulating the amount of food and drink eaten for each eating behavior to the initial registered amount. The meal information generating section 300C shown in FIG. 13 includes a total amount of food and drink calculating section 310, a complete eating rate calculating section 320, and a processed information calculating section 330.

[0091] The total amount of food and drink calculation section 310 calculates the total amount of food and drink in the same manner as the total amount of food and drink calculation section 310 already described.

[0092] The eating completion rate calculation unit 320 calculates the rate at which food is completed per meal. The eating completion rate calculation unit 320 refers to the recording database unit 600B and calculates the rate at which food is completed per meal using the amount of food and drink not eaten before the meal started and the meal amount, which is the total amount of food and drink eaten during the meal. The eating completion rate indicates the ratio of the total amount of food and drink eaten, which is the accumulated amount of food and drink eaten for each eating action, to the amount of food and drink not eaten before the meal started. The amount of food and drink not consumed before the meal begins may be the amount of food and drink not consumed obtained from the captured image before the meal begins by the uneaten amount acquisition unit 230, or it may be the initial registered amount obtained by the registration information acquisition unit 500C, or it may be a value calculated using both. The meal amount, which is the total amount of food and drink consumed during a mealtime, is the total amount of food and drink calculated by total food and drink amount calculation section 310. The meal information generating section 300C generates meal information including the completion rate calculated by the completion rate calculating section 320.

[0093] The processed information calculation unit 330 refers to the recording database unit 600B and calculates the processed information based on any one or a combination of the amount of food and drink eaten for each eating action, the amount of food and drink accumulated over time, the amount of food eaten for each meal, or the amount of food eaten accumulated over time. Examples of the processing information include the amount of food eaten or the rate at which food is eaten completely for each ingredient per meal, or the amount of food eaten or the rate at which food is eaten completely for each nutrient per meal. The meal information generation unit 300C generates meal information including the processed information calculated by the processed information calculation unit 330.

[0094] The meal information output section 400C outputs the meal information generated by the meal information generation section 300B, similar to the meal information output section 400A already described.

[0095] A processing example of the control device according to the third embodiment of the present disclosure will be described. FIG. 14 is a flowchart showing an example of processing by the quantity estimator 200 (200C) according to the third embodiment of the present disclosure. When the quantity estimation unit 200C starts the process ("START"), it then executes the food type estimation process (step ST3210). In the food type estimation process, the food type estimation unit 220 of the quantity estimation unit 200C estimates the food type, which is the type of each food, using the eating action image, in the same way as the food type estimation process already described.

[0096] The quantity estimation unit 200C then executes an uneaten amount acquisition process (step ST3220). In the uneaten amount acquisition process, the uneaten amount acquisition unit 230 of the quantity estimation unit 200C uses the time-series captured images to acquire the uneaten amount, which is the amount of food and drink, for each remaining food and drink in time series.

[0097] The quantity estimation unit 200C then executes a difference amount calculation process (step ST3230). In the difference amount calculation process, the difference amount calculation unit 240 of the quantity estimation unit 200C calculates the difference amount between the uneaten amount acquired in the current eating action and the uneaten amount acquired in the previous eating action each time an uneaten amount is acquired.

[0098] The quantity estimation unit 200C executes an intake amount estimation process (step ST3240). In the intake amount estimation process, the intake amount estimation unit 210 of the quantity estimation unit 200C estimates the intake amount, which is the amount of food and drink eaten by the measurement subject for each eating action, using the food and drink type information and the eating action image. The intake amount estimation unit 210 further estimates the intake amount by correcting the difference amount output by the difference amount calculation unit 240. Furthermore, the intake amount estimation unit 210 can estimate the intake amount using the registered initial amount. The intake amount estimation unit 210 can set an initial value for the amount of unavoidable intake using the registered initial amount, and can set a more accurate initial value.

[0099] The quantity estimation unit 200C then executes an end determination process (step ST3250 "End?"). In the termination determination process, the quantity estimation unit 200C determines whether to terminate the processing of the quantity estimation unit 200C. For example, it determines whether to terminate the processing of the quantity estimation unit 200C in accordance with an external termination command or an execution program. When the quantity estimation unit 200C determines not to end the process ("NO" in step ST3250), the process proceeds to step ST3210, and the process is repeated from step ST3210. When the control unit (not shown) determines that the processing of the amount estimation unit 200C should be ended ("YES" in step ST3250), the amount estimation unit 200C ends the processing ("end").

[0100] Next, a detailed first example of the processing of the diet information generating unit according to the third embodiment of the present disclosure will be described. FIG. 15 is a flowchart showing an example (first example) of the process of the diet information generating unit 300 (300C) according to the third embodiment of the present disclosure. The meal information generation unit 300C starts processing ("start"), for example, when a setting is made to generate meal information, or when a request is made to generate meal information.

[0101] Next, meal information generation unit 300C executes a process to calculate the rate of completion of meal (step ST3320). In the process, meal completion rate calculation unit 320 of meal information generation unit 300C calculates the rate of completion of meal per meal using the amount of food and drink not eaten before the start of the meal and the meal amount, which is the total amount of food and drink eaten during the meal time. Specifically, the eating completion rate calculation section 320 first refers to the eating and drinking amount recording database 610 in the recording database section 600B, and adds up the eating and drinking amounts accumulated in chronological order to calculate the eating amount, which is the total amount of eating and drinking during the mealtime. The completion rate calculation unit 320 then obtains the amount of food and drink that has not been eaten before the meal started. For example, the completion rate calculation unit 320 obtains the amount of food and drink that has not been eaten, estimated from the captured image before the meal started by the uneaten amount acquisition unit 230. Alternatively, the completion rate calculation unit 320 obtains, for example, the registered initial amount acquired by the registration information acquisition unit 500C as the amount of food and drink that has not been eaten before the meal started. Alternatively, the completion rate calculation unit 320 may use both the amount of food and drink that has not been eaten estimated by the uneaten amount acquisition unit 230 and the registered initial amount acquired by the registration information acquisition unit 500C for calculation. Next, the completion rate calculation section 320 calculates the completion rate per meal using the amount of food and drink not eaten before the meal started and the meal amount, which is the total amount of food and drink eaten during the meal time.

[0102] The meal information generation unit 300C outputs the meal information including the completion rate per meal calculated by the completion rate calculation unit 320 to the meal information output unit 400C, and then ends the process ("end"). The meal information generation unit 300C may output the completion rate output by the above-described processing as meal information as is, or may further generate and output meal information such as that shown in FIG. 16 or FIG. 17.

[0103] FIG. 16 is a diagram showing a first example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. 16 is composed of a date 3110, a time period 3120, and a completion rate 3130. When the meal information 3100 is stored in the recording database unit 600C, the date 3110, the time period 3120, and the completion rate 3130 are recorded in association with each other.

[0104] FIG. 17 is a diagram showing a second example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. 17 is composed of a date 3210, a time period 3220, and a completion rate for each dish (food type) 3230. The meal information 3200 is stored in the recording database unit 600C, and is recorded in association with the date 3210, the time period 3220, and the completion rate 3230.

[0105] Next, a second detailed example of the processing by the diet information generating unit according to the third embodiment of the present disclosure will be described. FIG. 18 is a flowchart showing an example (second example) of the process of the diet information generating unit 300 (300C) according to the third embodiment of the present disclosure. The meal information generation unit 300C starts processing ("start"), for example, when a setting is made to generate meal information, or when a request is made to generate meal information.

[0106] Next, the meal information generation unit 300C executes a processed information calculation process (step ST3330). In the processed information calculation process, the processed information calculation unit 330 of the meal information generation unit 300C refers to the record database unit 600B and calculates processed information based on any one or a combination of the amount of food and drink eaten for each eating behavior, the amount of food and drink accumulated in chronological order, the amount of food eaten which is the total amount of food and drink eaten for each meal, and the amount of food eaten which is accumulated in chronological order. Specifically, for example, the processed information calculation unit 330 first refers to the food and drink amount recording database 610 in the recording database unit 600B and obtains the total amount of food and drink consumed for each meal. The processed information calculation unit 330 then refers to the food and drink type database 630 and calculates the ingredients for each food and drink and the amount of each ingredient. The processed information calculation unit 330 then calculates the amount of food consumed for each ingredient. Specifically, for example, the processed information calculation unit 330 first refers to the food and drink amount record database 610 in the record database unit 600B and obtains the total amount of food and drink consumed for each meal. The processed information calculation unit 330 then refers to the food and drink type database 630 and calculates the nutrients for each food and drink and the amount of each nutrient. The processed information calculation unit 330 then calculates the amount of food consumed for each nutrient. The meal information generation unit 300C generates meal information including the processed information calculated by the processed information calculation unit 330.

[0107] The meal information generation unit 300C generates meal information including the processing information calculated by the processing information calculation unit 330, outputs the generated meal information to the meal information output unit 400C, and then ends the process ("end"). The meal information generation unit 300C may output the processed information output by the above-described processing as meal information as is, or may further generate and output meal information such as that shown in FIG. 19 or FIG. 20.

[0108] FIG. 19 is a diagram showing a third example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. 19 is composed of a date 3310, a time period 3320, and an amount of food consumed by each ingredient (food type) 3330. When the meal information 3300 is stored in the recording database unit 600C, the date 3310, the time period 3320, and the amount of food consumed by each ingredient (food type) 3330 are recorded in association with each other.

[0109] FIG. 20 is a diagram showing a fourth example of meal information generated by the meal information generating unit 300 (300C) according to the third embodiment of the present disclosure. 20 is composed of a date 3410, a time period 3420, and an amount of food and drink consumed for each nutrient 3430. The diet information 3400 is stored in the recording database unit 600C, whereby the date 3410, the time period 3420, and the amount of food and drink consumed for each nutrient 3430 are recorded in association with each other.

[0110] This embodiment further includes the following configuration. The quantity estimator Using the captured image, an uneaten amount, which is the amount of food and drink remaining, is obtained, and the uneaten amount is further used to estimate the amount of food and drink. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that can estimate the amount of food and drink consumed taking into account the amount of food and drink remaining, thereby enabling the accuracy of meal measurement to be improved compared to conventional methods. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0111] This embodiment further includes the following configuration. The amount estimation unit estimates an eating rate using the uneaten amount, and estimates the amount of eating using the eating rate. A measuring device characterized by: Accordingly, the present disclosure further provides: The effect is that it is possible to provide a measuring device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0112] This embodiment further includes the following configuration. The meal information output by the meal information generation unit includes: a complete eating rate for each meal and each food and drink, which indicates a ratio of the total amount of food and drink eaten by each eating action to the amount of food and drink eaten before the start of the meal; A measuring device characterized by: Accordingly, the present disclosure further provides: The effect is that it is possible to provide a measuring device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0113] This embodiment further includes the following configuration. a registration information acquisition unit that acquires a registered initial amount, which is the amount of each food or drink stored before starting a meal; Furthermore, The quantity estimator Furthermore, the intake amount is estimated using the registered initial amount acquired by the registration information acquisition unit. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that can improve the accuracy of meal measurement compared to conventional methods, for example, by registering meal menus and amounts that have been decided in advance. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0114] This embodiment further includes the following configuration. The meal information output by the meal information generation unit includes: a complete eating rate for each meal and each food and drink indicating a ratio of the total amount of food and drink accumulated for each eating action to the registered initial amount; A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measurement device that enables the accuracy of meal measurement to be improved compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0115] Embodiment 4 In the third embodiment, the amount of uneaten food and drink for each type of food and drink is acquired and used for meal measurement. In this case, the accuracy of meal measurement may vary depending on whether the food and drink is served in a personal dish or a shared dish. In the fourth embodiment, a configuration example that can further take eating utensils into consideration will be described. In the fourth embodiment, among the components of the fourth embodiment, the components and functions similar to those of the components of the first, second, or third embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0116] A configuration example of a measurement device according to a fourth embodiment of the present disclosure will be described. FIG. 21 is a diagram showing a configuration example of a diet measurement system 1 (1D) including a measurement device 10 (10D) according to the fourth embodiment of the present disclosure. The diet measurement system 1 (1D) shown in FIG. 21 includes a measurement device 10 (10D) and an information source device 20 (20D).

[0117] The information source device 20D is configured similarly to the information source device 20 already described, and the information source device 20D shown in FIG.

[0118] The measurement device 10D performs dietary measurements related to the diet of a subject. The measuring device 10D includes a meal information acquisition unit 100 (100D), a quantity estimation unit 200 (200D), a meal information generation unit 300 (300D), a meal information output unit 400 (400D), a registration information acquisition unit 500 (500D), and a record database unit 600 (600D).

[0119] The registration information acquisition unit 500D acquires information used as an auxiliary for meal measurement by the measuring device 10D from pre-registered registration information. The registration information acquisition unit 500D further acquires pre-stored registered eating utensil information including the positions and types of food and drink containers. The registered tableware information may further include pre-stored types of tableware that will be used for eating. Tableware is cutlery, such as spoons, chopsticks, and forks.

[0120] The registration information acquisition unit 500C is configured to receive and store the registration information. The registration information acquisition unit 500D shown in FIG. 21 includes a registration acceptance unit 510 and a registration information storage unit 520.

[0121] The registration reception unit 510 receives registration information. When the registration reception unit 510 receives a registration request from an operator, it displays an image (GUI) for input and receives the registration information based on an operation by the operator on the image. The registration accepting unit 510 of the registration information acquiring unit 500C accepts, as registration information, registration eating utensil information indicating, for example, dishes or utensils (cutlery) that will be used for eating, or dishes and utensils (cutlery).

[0122] The registration information storage unit 520 stores the registration information accepted by the registration accepting unit 510. The registration information storage unit 520 of the registration information acquisition unit 500C stores registered eating utensil information indicating, for example, dishes or utensils (cutlery), or dishes and utensils (cutlery), that will be used for eating. The registered tableware information is information that can be registered when, for example, the type and area (position) of the tableware or utensils (cutlery) used for eating, or the type and area (position) of the tableware and utensils (cutlery), are predetermined, and indicates the type and area (position) of the tableware or utensils (cutlery). The type is, for example, personal tableware, shared tableware, spoon, chopsticks, fork, etc., and the area (position) is the area (position) in the shooting area.

[0123] The record database unit 600D records the amount of food and drink consumed and dietary information for each subject to be measured. The record database section 600D shown in FIG. 21 is configured to include an eating and drinking amount record database 610 and a meal record database 620. The food intake record database 610 is similar to the food intake record database 610 already described. The meal record database 620 is similar to the meal record database 620 already described.

[0124] The eating information acquiring section 100D acquires eating behavior images, which are images of eating behaviors of each measurement subject included in the time-series captured images. Furthermore, the mealtime information acquiring section 100D further acquires eating utensil information including the type of tableware, which is the type of tableware for each use of the food and drink contained in the photographed image. Furthermore, the mealtime information acquisition unit 100C further acquires eating utensil information indicating the dishes or utensils, or the dishes and utensils, used for the meal. The eating information acquiring section 100D shown in FIG. 21 includes an eating motion detecting section 110, a measurement target information acquiring section 120, and an eating utensil information acquiring section .

[0125] The eating motion detection section 110 has the same configuration as the eating motion detection section 110 already described. The measurable object information acquiring section 120 has the same configuration as the measurable object information acquiring section 120 already described. The eating utensil information acquisition unit 130 acquires eating utensil information using the photographed image or eating action image, or registration information (registered eating utensil information).

[0126] The quantity estimation unit 200D outputs the quantity of food and drink eaten for each eating action by the user as the measurement subject. Furthermore, the amount estimation unit 200D further calculates and acquires the amount of food not yet eaten using the type of tableware indicated in the eating utensil information. The quantity estimation section 200D shown in FIG. 21 includes an ingested amount estimation section 210, a food and drink type estimation section 220, an unattached amount acquisition section 230, and a difference amount calculation section 240.

[0127] The food and drink type estimation unit 220 is configured in the same manner as the food and drink type estimation unit 220 already described.

[0128] The uneaten amount acquisition unit 230 uses a time series of captured images to chronologically acquire the uneaten amount, which is the amount of food and drink for each remaining food and drink. Furthermore, the uneaten amount acquisition unit 230 further acquires the uneaten amount for each eating action according to the type of dish used, using the eating utensil information acquired by the eating utensil information acquisition unit 130. When the type of dish used in the eating utensil information indicates a shared dish used by multiple subjects, the uneaten amount acquisition unit 230 calculates the uneaten amount for each eating action of each subject using the shared dish. This allows the uneaten amount acquisition unit 230 to calculate the uneaten amount by taking into account the amounts eaten by multiple subjects, for example, when a shared dish (shared dish) is used by multiple subjects, thereby improving the accuracy of the difference amount calculated by the difference amount calculation unit 240.

[0129] Each time the difference amount calculation unit 240 acquires the amount of uneaten food, it calculates the difference amount between the amount of uneaten food acquired in the current eating action and the amount of uneaten food acquired in the previous eating action. The intake amount estimation section 210 is configured in the same manner as the intake amount estimation section 210 of the amount estimation section 200C already described.

[0130] The meal information generation unit 300D may be configured in the same manner as any one of the meal information generation units 300 already described, and detailed description thereof will be omitted here.

[0131] The meal information output section 400D may be configured in the same manner as any one of the meal information output sections 400 already described, and detailed description thereof will be omitted here.

[0132] Next, a processing example of the mealtime information acquisition unit according to the fourth embodiment of the present disclosure will be described. FIG. 22 is a flowchart showing an example of the processing of the mealtime information acquisition section 100 (100D) according to the fourth embodiment of the present disclosure. When the mealtime information acquiring section 100D starts the process (“START”), it then executes a measurable subject information acquiring process (step ST4110). In the measurable subject information acquiring process, when multiple measurable subjects are included in the time-series captured images, the measurable subject information acquiring section 120 of the mealtime information acquiring section 100D acquires measurable subject information relating to the regions of each of the multiple measurable subjects. In addition, when registered measured subject information relating to the measured subject is registered as registered information, the measured subject information acquisition unit 120 acquires measured subject information including the registered measured subject information using the registered measured subject information acquired by the registered information acquisition unit 500D. The measurable object information acquiring section 120 outputs the acquired measurable object information to the eating motion detecting section 110. The mealtime information acquisition unit 100D executes eating action detection processing (step ST4120). In the eating action detection processing, the eating action detection unit 110 of the mealtime information acquisition unit 100D receives time-series captured images output by the imaging device 21 of the information source device 20D, and also receives registered measurable object information output by the registration information acquisition unit 500B. The eating action detection unit 110 detects eating action images using the time-series captured images and the registered measurable object information. For example, when food or drink on a table (dining table) is brought to the mouth in the time-series captured images, the eating action detection unit 110 regards the captured images during this period as eating action images, acquires eating action information including the eating action images, and outputs it as mealtime information. The eating information acquisition unit 100D executes eating utensil information acquisition processing (step ST4130). In the eating utensil information acquisition processing, the eating utensil information acquisition unit 130 of the eating information acquisition unit 100D acquires eating utensil information including utensil types, which are types of food utensils for each purpose included in the captured image. The eating utensil information acquisition unit 130 acquires eating utensil information using the captured image, eating action image, or registered information (registered eating utensil information). After outputting the eating action image and eating utensil information as eating information, eating information acquisition section 100D ends the process ("end").

[0133] Next, a processing example of the mealtime information acquisition unit according to the fourth embodiment of the present disclosure will be described. FIG. 23 is a flowchart showing an example of processing by the quantity estimating unit 200 (200D) according to the fourth embodiment of the present disclosure. When the quantity estimation unit 200D starts the process ("START"), it then executes a food type estimation process (step ST4210). In the food type estimation process, the food type estimation unit 220 of the quantity estimation unit 200D estimates the type of food, which is the type of each food, using the eating action image, in the same way as the food type estimation process already described. The quantity estimation unit 200D then executes an uneaten amount acquisition process (step ST4220). In the uneaten amount acquisition process, the uneaten amount acquisition unit 230 of the quantity estimation unit 200D uses the time-series captured images to acquire the uneaten amount, which is the amount of food and drink, for each remaining food and drink in time series. The quantity estimation unit 200D then executes a difference amount calculation process (step ST4230). In the difference amount calculation process, the difference amount calculation unit 240 of the quantity estimation unit 200D calculates the difference amount between the uneaten amount acquired in the current eating action and the uneaten amount acquired in the previous eating action each time an uneaten amount is acquired. The quantity estimation unit 200D then executes an intake amount estimation process (step ST4240). In the intake amount estimation process, the intake amount estimation unit 210 of the quantity estimation unit 200D estimates the intake amount, which is the amount of food and drink eaten by the measurement subject for each eating action, using the food and drink type information and the eating action image. The intake amount estimation unit 210 further estimates the intake amount by correcting the difference amount output by the difference amount calculation unit 240. Furthermore, the intake amount estimation unit 210 can estimate the amount of intake and drink using the eating and drink utensil information. The intake and drink amount estimation unit 210 calculates the difference amount according to the type of dish using the eating and drink utensil information, taking into account the amounts of intake and drink by multiple subjects when the type of dish is a shared dish, for example. This is because, when one subject is eating food or drink from a shared dish, an error may occur in the difference amount if another subject eats or drinks from the shared dish, so the error can be reduced by subtracting the intake and drink amounts of the other subjects from the difference amount. The quantity estimation unit 200D then executes an end determination process (step ST4250 "End?"). In the termination determination process, the quantity estimation unit 200D determines whether to terminate the processing of the quantity estimation unit 200D, for example, in accordance with an external termination command or an execution program. When the quantity estimation unit 200D determines not to end the process ("NO" in step ST4250), the process proceeds to step ST3210, and the process is repeated from step ST3210. When the control unit (not shown) determines that the processing of the amount estimation unit 200D is to be ended (step ST4250 "YES"), the amount estimation unit 200D ends the processing ("end").

[0134] According to this embodiment, the accuracy of calculation and estimation of the total amount eaten can be improved. The measuring device of this embodiment can identify the subject and determine the type of food eaten and the amount of food eaten each time (amount eaten). By analyzing the captured images, it is possible to determine what was picked up with the tips of chopsticks, spoon, or fork, how much was scooped, and how much was eaten, based on the volume, color, texture, and shape, and to determine the type of food and the amount eaten each time. It is also possible to configure the device to obtain the color, texture, shape, and aroma of food and drink, and have the AI ​​determine a menu that is closest to that information.

[0135] The measuring device of this embodiment can determine the difference between images taken before and after eating from the dish, and can assist in estimating how much food has been eaten (volume, weight), thereby improving the accuracy of the amount eaten. The measuring device of this embodiment may be configured to extract the amount or percentage of food completely eaten by comparing images of the food taken before and after eating.

[0136] The measuring device of this embodiment can take into consideration cases where people eat from shared dishes when identifying a subject and detecting the amount of food eaten. The measuring device of this embodiment is configured to register and determine information that enables subject selection in advance, so that when shared dishes such as platter dishes or hotpot dishes are eaten, it is possible to determine who ate, when, and how much from the dish. In other words, the measuring device of this embodiment is configured to record when and how much food and drink each subject put into their mouth for both individually served food and drink and food and drink served in a shared dish, and to estimate and calculate the total amount ultimately eaten.

[0137] This embodiment further includes the following configuration. The mealtime information acquisition unit Furthermore, tableware information including the type of tableware, which is the type of tableware for each purpose of the food and drink contained in the photographed image, is acquired, The quantity estimator Furthermore, the amount of food not yet eaten is calculated and acquired using the type of tableware indicated in the tableware information. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a control device that can improve the accuracy of the amount of unavoidable food intake depending on the type of dish, such as a large plate, a small plate, an individual plate, or a shared plate, thereby enabling the accuracy of meal measurement to be improved compared to conventional methods. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0138] This embodiment further includes the following configuration. The system further includes a registration information acquisition unit that acquires registered tableware information including the positions of food and drink containers and the types of the containers stored in advance, The quantity estimator Furthermore, the amount of food not yet eaten is calculated and acquired using the registered eating utensil information acquired by the registration information acquisition unit. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that can improve the accuracy of recognizing food and drink containers and enable the accuracy of food measurement to be improved compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0139] This embodiment further includes the following configuration. When the type of utensil included in the eating utensil information indicates a shared utensil that is a utensil shared by multiple measurement subjects, The quantity estimator The amount of food and drink not consumed is calculated and acquired for each eating and drinking behavior of each of the measurement subjects using the shared appliance, and the amount of food and drink consumed by the measurement subjects who have performed the eating and drinking behavior is estimated. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a measuring device that can measure food intake by taking into account the amount of food and drink consumed by each subject using a shared device, thereby enabling the accuracy of food measurement to be improved compared to conventional methods. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0140] Embodiment 5. In the above-described embodiment, if the subject spits out food or drink that he or she has brought to his or her mouth, the accuracy of meal measurement may be reduced. In the fifth embodiment, a configuration example in which the amount of food or drink returned is taken into consideration will be described. In the fifth embodiment, among the components of the fifth embodiment, those components and their functions that are similar to those of the components of the first, second, third, or fourth embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0141] A configuration example of a control device and a diet measurement system including the control device according to a fifth embodiment of the present disclosure will be described. FIG. 24 is a diagram showing a configuration example of a diet measurement system 1 (1E) including a measurement device 10 (10E) according to the fifth embodiment of the present disclosure. The diet measurement system 1 (1E) shown in FIG. 24 includes a measurement device 10 (10E) and an information source device 20 (20E).

[0142] The information source device 20E is configured similarly to the information source device 20 already described, and the information source device 20E shown in FIG.

[0143] The measurement device 10E measures the amount of food eaten by the subject, and estimates the amount of food and drink eaten, taking into account the amount of food and drink regurgitated by the subject when the subject regurgitates food and drink that has been brought to the mouth. The measuring device 10E shown in Figure 21 is configured to include a meal information acquisition unit 100 (100E), a quantity estimation unit 200 (200E), a meal information generation unit 300 (300E), a meal information output unit 400 (400E), a registration information acquisition unit 500 (500E), and a record database unit 600 (600E). The measuring apparatus 10E shown in FIG. 21 differs from the measuring apparatus 10D already described, particularly in the configuration of the quantity estimating section 200E.

[0144] The quantity estimation unit 200E further estimates the amount of food and drink regurgitated by the subject using the time-series captured images, and calculates and outputs the amount of food and drink regurgitated using the estimated amount regurgitated. The amount estimating section 200E shown in FIG. 21 is configured to further include a return amount estimating section 250 compared to the amount estimating section 200E already described. The return amount estimation unit 250 estimates the amount of food or drink returned by the subject using the time-series captured images. The return amount estimation unit 250 analyzes the time-series captured images to detect the subject's return of food or drink, and analyzes the images of the return of food or drink to estimate the amount of food or drink returned.

[0145] A processing example of the quantity estimator according to the fifth embodiment of the present disclosure will be described. FIG. 25 is a flowchart showing an example of processing by the quantity estimator 200 (200E) according to the fifth embodiment of the present disclosure. When the quantity estimation unit 200E starts the process ("START"), it then executes the food type estimation process (step ST5210). In the food type estimation process, the food type estimation unit 220 of the quantity estimation unit 200E estimates the food type, which is the type of each food, using the eating action image, in the same way as the food type estimation process already described.

[0146] The quantity estimation unit 200E then executes an uneaten amount acquisition process (step ST5220). In the uneaten amount acquisition process, the uneaten amount acquisition unit 230 of the quantity estimation unit 200E uses the time-series captured images to acquire the uneaten amount, which is the amount of food and drink, for each remaining food and drink in time series.

[0147] The quantity estimation unit 200E then executes a difference amount calculation process (step ST5230). In the difference amount calculation process, the difference amount calculation unit 240 of the quantity estimation unit 200E calculates the difference amount between the uneaten amount acquired in the current eating action and the uneaten amount acquired in the previous eating action each time an uneaten amount is acquired.

[0148] The quantity estimation unit 200E then executes a returning action determination process (step ST5240 "Returning action?"). In the returning action determination process, the returning amount estimation unit 250 of the quantity estimation unit 200E analyzes the time-series captured images to determine whether the subject has performed a returning action of food or drink. When the return amount estimation unit 250 of the amount estimation unit 200E determines that the operation is not a return operation ("NO" in step ST5240), the amount estimation unit 200E proceeds to the process of step ST5260.

[0149] When the return amount estimation unit 250 of the amount estimation unit 200E determines that the operation is a return operation (step ST5240 “YES”), The amount estimation unit 200E then executes a return amount estimation process (step ST5250). In the return amount estimation process, the return amount estimation unit 250 of the amount estimation unit 200E analyzes the image of the return action extracted from the captured image and estimates the return amount of food or drink.

[0150] The quantity estimation unit 200E then executes an intake amount estimation process (step ST5260). In the intake amount estimation process, the intake amount estimation unit 210 of the quantity estimation unit 200E performs the intake amount estimation process of the intake amount estimation unit 210 already explained, and further uses the returned amount estimated by the returned amount estimation unit 250 to output an intake amount by subtracting the returned amount, for example. The quantity estimation unit 200E then executes an end determination process (step ST5270 "end?"). In the termination determination process, the quantity estimation unit 200E determines whether to terminate the processing of the quantity estimation unit 200E, for example, in accordance with an external termination command or an execution program. When the quantity estimation unit 200E determines not to end the process ("NO" in step ST5270), the process proceeds to step ST5210, and the process is repeated from step ST5210. When the control unit (not shown) determines that the processing of the amount estimation unit 200E should be ended ("YES" in step ST5270), the amount estimation unit 200E ends the processing ("end").

[0151] According to the configuration of this embodiment, even if the subject spits out the food after putting it in their mouth, the amount of regurgitation can be estimated and subtracted. Also, the amount of regurgitation can be estimated for each type of food or drink.

[0152] This embodiment further includes the following configuration. The quantity estimator Furthermore, the amount of food and drink regurgitated by the subject is estimated using the time-series captured images, and the estimated amount of food and drink regurgitated is used to calculate and output the amount of food and drink. A measuring device characterized by: As a result, the present disclosure has the effect of providing a measuring device that can improve the accuracy of meal measurement compared to conventional methods by taking into account the amount of food or drink that the subject spits out after bringing it to their mouth.

[0153] Embodiment 6 In the above-described embodiment, if the captured image does not sufficiently include the subject or food, the accuracy of the meal measurement may be poor. In the sixth embodiment, a configuration example in which the photographing position can be controlled based on the acquired image will be described. In the sixth embodiment, among the components of the sixth embodiment, those components and their functions that are similar to those of the components of the first, second, third, fourth, or fifth embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0154] A configuration example of a control device and a diet measurement system including the control device according to a sixth embodiment of the present disclosure will be described. FIG. 26 is a diagram showing a configuration example of a diet measurement system 1 (1F) including a measuring device 10 (10F) according to the sixth embodiment of the present disclosure. The diet measurement system 1 (1F) shown in FIG. 26 includes a measurement device 10 (10F) and an information source device 20 (20F).

[0155] 26 is configured to include an image capturing device 21. The information source device 20F is provided with an image capturing position control device 22 that can change the image capturing position of the image capturing device 21. The information source device 20F also is provided with an image capturing angle control device 23 that can change the image capturing direction of the image capturing device 21.

[0156] The measuring device 10F performs dietary measurements of the diet of a subject. The measuring device 10F differs from the measuring device 10 already described in particular in that it has a configuration that can control the imaging position, imaging direction, or both of the imaging device 21. The measuring device 10F shown in Figure 26 is configured to include a meal information acquisition unit 100 (100F), a quantity estimation unit 200 (200F), a meal information generation unit 300 (300F), a meal information output unit 400 (400F), a registration information acquisition unit 500 (500F), a recording database unit 600 (600F), and an imaging command unit 700 (700F). The measuring device 10F shown in FIG. 26 differs from the measuring device 10 already described in particular in that it includes an imaging command unit 700F.

[0157] The imaging command unit 700F controls at least one of the imaging position and imaging direction of the imaging device 21. The photographing command unit 700F outputs a control signal that commands the photographing position, photographing direction, or photographing position and photographing direction of the photographing device that captured the photographed image including the eating action image, based on the photographed image or eating action image. The imaging command unit 700F shown in FIG. 26 includes an imaging state determination unit 710, an imaging control amount calculation unit 720, and an imaging control command output unit 730.

[0158] The imaging state determination unit 710 of the imaging command unit 700F determines the imaging state based on the captured image or the eating behavior image. The imaging state determination unit 710 determines whether the imaging state is such that the eating behavior of the subject can be determined. Specifically, for example, the positions of the body parts necessary for determining the eating behavior of the subject, such as the hands, arms, and face, of the subject are determined. Also, for example, the positions of food, drink, or bowls are determined.

[0159] The imaging control amount calculation unit 720 of the imaging command unit 700F calculates the control amount of the imaging position, imaging direction, or both, so as to achieve an imaging state in which the eating behavior of the person being measured can be determined, using the determination result by the imaging state determination unit 710. Specifically, the control amount of the imaging position, imaging direction, or both, which can capture the positions of parts necessary for estimating the eating behavior of the person being measured, such as the hands, arms, and face of the person being measured, is calculated. Also, the control amount of the imaging position, imaging direction, or both, which can capture the positions of food, drink, or containers, for example, is calculated.

[0160] The shooting control command output unit 730 of the shooting command unit 700F outputs a control signal to instruct the shooting position control device 22 or the shooting angle control device 23 using the control amount of the shooting position, the shooting direction, or both calculated by the shooting control amount calculation unit 720.

[0161] A processing example of a meal measurement system including a control device according to a fifth embodiment of the present disclosure will be described. FIG. 27 is a flowchart showing an example of processing by the imaging command unit 700 (700F) according to the sixth embodiment of the present disclosure. The imaging command unit 700F starts the process shown in FIG. 27 when a measurement start condition is met, such as when the power of the measuring device 10F is turned from OFF to ON, when the execute button for meal measurement on the measuring device 10F is pressed, or when an imaging adjustment command is received, and the measuring device 10F is ready to perform meal measurement. ("Start")

[0162] The imaging command unit 700F then executes imaging state determination processing (step ST6610). In the imaging state determination processing, the imaging state determination unit 710 of the imaging command unit 700F determines the imaging state based on the eating behavior image acquired by the mealtime information acquisition unit 100F. The imaging state determination unit 710 determines whether the imaging state is such that the eating behavior of the subject can be determined. Specifically, it determines the positions of parts necessary for determining the eating behavior of the subject, such as the hands, arms, and face of the subject. The imaging state determination unit 710 outputs the determination result to the imaging control amount calculation unit 720.

[0163] The imaging command unit 700F then executes imaging mechanism control amount calculation processing (step ST6620). In the imaging mechanism control amount calculation processing, upon receiving the determination result from the imaging state determination unit 710, the imaging control amount calculation unit 720 of the imaging command unit 700F calculates, using the determination result from the imaging state determination unit 710, control amounts for the imaging position, the imaging direction, or both, so as to achieve an imaging state in which the eating behavior of the subject can be determined. Specifically, for example, the control amounts for the imaging position, the imaging direction, or both, so as to capture the positions of parts necessary for determining the eating behavior of the subject, such as the hands, arms, and face of the subject, can be calculated. The imaging control amount calculation unit 720 outputs the control amounts to the imaging control command output unit 730.

[0164] The imaging command unit 700F then executes imaging mechanism control command output processing (step ST6630). In the imaging mechanism control command output processing, upon receiving the control amount output by the imaging control amount calculation unit 720, the imaging control command output unit 730 of the imaging command unit 700F outputs a control signal to instruct at least one of the imaging position control device 22 or the imaging angle control device 23 using the control amount. The imaging position control device 22 or the imaging angle control device 23 changes the imaging position or the imaging angle of the imaging device 21 according to a command from the imaging command unit 700F.

[0165] Next, the imaging command unit 700F executes an end determination process (step ST6650 "end?"). In the termination determination process, the imaging command unit 700F determines whether to terminate the processing of the imaging command unit 700F. The imaging command unit 700F determines whether to terminate the processing of the imaging command unit 700F in accordance with, for example, an external termination command or an execution program. When it is determined that the processing of the imaging command unit 700F is not to be ended ("NO" in step ST6650), the processing proceeds to step ST6610, and the processing is repeated from step ST6610. If the imaging command unit 700F determines to end the process ("YES" in step ST6650), the imaging command unit 700F ends the process ("end").

[0166] This embodiment has the following functions. -Variable camera angles and positions -Camera function that can be separated from the main unit (separated from tablet: model image, ceiling installation, etc.) - Multiple camera setup eliminates blind spots For example, depending on the number of subjects, the position of the camera installed in the tablet can be automatically slid up and down, and the angle and direction can be automatically changed so that the camera position does not block the capture of subjects or objects. Note that when there are a large number of people, etc., and many blind spots occur, the camera (photography device) can be separated from the main body of the tablet etc. (measurement device, terminal device) and placed separately on the ceiling, or multiple cameras can be installed. According to the measuring device of this embodiment, the accuracy of the acquired video information (photographed image) can be improved.

[0167] This embodiment further includes the following configuration. an imaging control unit that outputs, based on the eating action image, a control signal for instructing the imaging position, imaging direction, or imaging position and imaging direction of an imaging device that captured the captured image including the eating action image; A measuring device comprising: As a result, the present disclosure further has the effect of providing a measuring device that makes it possible to include in the captured image the subject being measured or food and drink, which is necessary for estimating the amount of food eaten, thereby enabling the accuracy of dietary measurement to be improved compared to conventional methods. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0168] Embodiment 7 In the above-described embodiment, for example, if the area in the captured image is not bright enough, the accuracy of the meal measurement may not be sufficient. In the seventh embodiment, a configuration example will be described in which the accuracy of food measurement is improved by controlling the illumination based on the acquired image. In the seventh embodiment, among the components of the seventh embodiment, those components and their functions that are similar to those of the components of the first, second, third, fourth, fifth, or sixth embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0169] A configuration example of a control device and a diet measurement system including the control device according to a seventh embodiment of the present disclosure will be described. FIG. 28 is a diagram showing a configuration example of a dietary measurement system 1 (1G) including a measuring device 10 (10G) according to the seventh embodiment of the present disclosure. The diet measurement system 1 (1G) shown in FIG. 28 includes a measurement device 10 (10G), an information source device 20 (20G), and an illumination device 30.

[0170] The information source device 20G is configured similarly to the information source device 20 already described, and the information source device 20G shown in FIG.

[0171] The lighting device 30 illuminates a part of or the entire area of ​​the area photographed by the photographing device 21. The lighting device 30 is controlled by receiving a command from the measuring device 10G.

[0172] The measurement device 10G performs dietary measurements of the diet of a subject. The measurement device 10G differs from the measurement device 10 already described in particular in that it has a function of controlling the lighting device 30. The measuring device 10G shown in Figure 28 is configured to include a meal information acquisition unit 100 (100G), a quantity estimation unit 200 (200G), a meal information generation unit 300 (300G), a meal information output unit 400 (400G), a registration information acquisition unit 500 (500G), a record database unit 600 (600G), and a lighting command unit 800 (800G). A measurement apparatus 10G shown in FIG. 28 differs from the measurement apparatus 10 already described in particular in that it includes an illumination command unit 800 (800G).

[0173] The lighting command unit 800G controls the lighting device 30. Based on the photographed image or the eating action image, the lighting command unit 800G controls the lighting device 30 to illuminate the photographed area where the photographed image including the eating action image was photographed. The lighting command unit 800G controls the lighting device 30 so that the brightness of the photographed image or the eating action image becomes brightness suitable for estimating the amount of eating and drinking. The illumination command section 800G shown in FIG. 28 includes an illumination state determination section 810, an illumination control amount calculation section 820, and an illumination control command output section 830. The illumination state determination unit 810 determines the illumination state based on the captured image or the eating action image. Specifically, for example, the illumination state determination unit 810 determines the illuminance. Based on the lighting state determined by the lighting state determination unit 810, the lighting control amount calculation unit 820 calculates a lighting control amount such that the brightness of the captured image or eating action image becomes suitable for estimating the amount of food and drink consumed. The lighting control command output unit 830 uses the lighting control amount calculated by the lighting control amount calculation unit 820 to output a control signal to instruct the lighting device 30 .

[0174] A processing example of a meal measurement system including a control device according to a seventh embodiment of the present disclosure will be described. FIG. 29 is a flowchart showing an example of processing by the illumination command unit 800 (800G) according to the seventh embodiment of the present disclosure. When lighting command unit 800G starts processing (“START”), it then executes lighting state determination processing (step ST7710). In the processing, lighting state determination unit 810 of lighting command unit 800G determines the lighting state based on the captured image or the eating action image. Specifically, for example, lighting state determination unit 810 determines the illuminance. Illumination state determination section 810 determines whether the illuminance is equal to or greater than the illuminance threshold value using the illuminance, and if the illuminance is not equal to or greater than the illuminance threshold value, proceeds to the process of step ST7720.

[0175] Illumination command unit 800G then executes a lighting control amount calculation process (step ST7720). In the lighting control amount calculation process, lighting control amount calculation unit 820 of lighting command unit 800G calculates a lighting control amount based on the lighting state determined by lighting state determination unit 810, such that the brightness of the captured image or eating action image becomes brightness suitable for estimating the amount of food and drink eaten.

[0176] Next, lighting command unit 800G executes a lighting control command output process (step ST7730). In the lighting control command output process, lighting control command output unit 830 of lighting command unit 800G outputs a control signal to instruct lighting device 30 using the lighting control amount calculated by lighting control amount calculation unit 820.

[0177] Illumination command unit 800G then executes an end determination process (step ST7740 "end?"). In the termination determination process, the lighting command unit 800G determines whether to terminate the processing of the lighting command unit 800G. The lighting command unit 800G determines whether to terminate the processing of the lighting command unit 800G, for example, in accordance with an external termination command or an execution program. Alternatively, the lighting command unit 800G determines, for example, whether the illuminance is equal to or greater than an illuminance threshold, and determines to terminate the processing if the illuminance is equal to or greater than the illuminance threshold, and determines not to terminate the processing if the illuminance is not equal to or greater than the illuminance threshold. When it is determined that the processing of the illumination command unit 800G is not to be ended ("NO" in step ST7740), the processing proceeds to step ST7710, and the processing is repeated from step ST7710. When the lighting command unit 800G determines to end the process ("YES" in step ST7740), the lighting command unit 800G ends the process ("end").

[0178] In this embodiment, a configuration example has been shown in which the accuracy of food measurement is improved by stabilizing the shooting environment. By automatically recognizing and correcting the ambient lighting temperature setting, a consistent lighting environment can be created, improving measurement accuracy. Even in an environment with variable lighting, it is possible to consistently detect not only food but also facial expressions, complexion, etc. Note that the ceiling-mounted camera in the above-described embodiment may be configured to have LED lighting attached so that brightness can be adjusted from the ceiling.

[0179] This embodiment further includes the following configuration. an illumination control unit that controls, based on the eating action image, an illumination device that illuminates a photographing area in which a photographed image including the eating action image is photographed; A measuring device comprising: As a result, the present disclosure further has the effect of providing a measuring device that can improve the accuracy of food measurement compared to conventional devices by controlling the brightness of the captured image. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0180] Embodiment 8 In the above-described embodiment, the dietary information is output based on the amount of food and drink consumed. In the eighth embodiment, a configuration example in which the state of a subject to be measured is estimated based on the amount of food and drink consumed or dietary information will be further described. In the eighth embodiment, among the components of the eighth embodiment, those components and their functions that are similar to those of the components of the first, second, third, fourth, fifth, sixth, or seventh embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0181] A configuration example of a control device and a diet measurement system including the control device according to an eighth embodiment of the present disclosure will be described. FIG. 30 is a diagram showing a configuration example of a diet measurement system 1 (1H) including a measurement device 10 (10H) according to the eighth embodiment of the present disclosure. The diet measurement system 1 (1H) shown in FIG. 30 includes a measurement device 10 (10H) and an information source device 20 (20H).

[0182] 30 includes a biosensor 24 in addition to the image capturing device 21. The biosensor 24 is a sensor that acquires biometric information of the subject.

[0183] The measuring device 10H performs dietary measurements of the subject's diet. The measuring device 10H differs from the measuring device 10 already described in that it has a function of estimating the subject's condition using dietary information such as the amount of food eaten and the amount of drink consumed. The measuring device 10H shown in Figure 30 is configured to include a meal information acquisition unit 100 (100H), a quantity estimation unit 200 (200H), a meal information generation unit 300 (300H), a meal information output unit 400 (400H), a registration information acquisition unit 500 (500H), and a record database unit 600 (600H). The measuring device 10H shown in Figure 30 differs from the measuring device 10 already described in particular in that the meal information generation unit 300H is equipped with a state estimation unit 340 and the meal information acquisition unit 100H is equipped with a subject state detection unit 140.

[0184] The diet information generating unit 300H includes a state estimating unit 340. The condition estimation unit 340 estimates the condition of the measurement subject regarding his / her physical condition. The condition estimation section 340 uses the accumulated amount of food and drink and the dietary information to output condition information indicating the state of the measurement subject's physical condition. The meal information generating unit 300H outputs the state information output by the state estimating unit 340 as the meal information. The condition estimation unit 340 may be further configured to estimate the condition of the physical condition of the measurement subject using registered information pre-registered in the registered information acquisition unit 500H. In this case, the registered information may be, for example, information on food preferences, information on chronic illnesses, etc.

[0185] The subject state detection unit 140 detects the state of the subject to be measured using the captured image or the eating action image. The subject state detection unit 140 analyzes the captured image or the eating action image and estimates and detects the state of the subject to be measured contained in the captured image or the eating action image. Specifically, for example, the subject state detection unit 140 detects the state of the subject to be measured, such as chewing, swallowing, or choking. Furthermore, for example, the subject state detection unit 140 may be configured to detect the state of the subject to be measured, such as facial expression.

[0186] A processing example of a meal measurement system including a control device according to an eighth embodiment of the present disclosure will be described. FIG. 31 is a flowchart showing an example of processing by the measuring device 10 (10H) according to the eighth embodiment of the present disclosure. The measuring device 10H starts processing when it is set in advance to execute state estimation or when it receives a request to execute state estimation (“Start”).

[0187] The measuring device 10H then executes a state estimation process (step ST8801). In the state estimation process, the state estimation unit 340 in the diet information generation unit 300H of the measuring device 10H estimates the state of the subject's physical condition. The state estimation unit 340 refers to the record database unit 600H and uses the accumulated intake and drinking amounts and diet information to output state information indicating the state of the subject's physical condition.

[0188] The measuring device 10H then executes a meal information output process (step ST8802). In the process, the meal information generating unit 300H of the measuring device 10H outputs the state information output by the state estimating unit 340 as meal information.

[0189] After outputting the state information as meal information, the measuring device 10H ends the process and goes into standby mode ("end").

[0190] Next, a specific example of the state information will be described. [First specific example] The speed at which the person is eating is detected by the interval and time it takes for chopsticks to reach the mouth, and appetite and hunger levels are detected. FIG. 32 is a diagram illustrating a configuration example of the state estimation unit 340 (340-1) according to the eighth embodiment of the present disclosure. The state estimation unit 340 (340-1) calculates an eating rate, which is the rate at which the subject eats and drinks, and estimates the state of the subject based on the eating rate. The state estimation unit 340 (340-1) includes an eating speed calculation unit 341. The eating speed calculation unit 341 calculates the eating speed by detecting the interval or time it takes to bring food to the mouth based on the time-series captured images and eating behavior images. The state estimation unit 340 (340-1) estimates a state, such as appetite or hunger, based on the eating speed. Next, an example of processing by the state estimation unit 340 (340-1) will be described. FIG. 33 is a flowchart illustrating an example of the state estimation unit 340 (340-1) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-1) starts processing (“START”), it executes an eating speed calculation process (step ST8821). In the eating speed calculation process, the eating speed calculation unit 341 of the state estimation unit 340 (340-1) detects the interval or time it takes to bring food to the mouth based on the time-series captured images and eating behavior images, and calculates the eating speed. The state estimation unit 340 (340-1) then executes a first state estimation (appetite state, hunger state) process (step ST8822). In the first state estimation process, the state estimation unit 340 (340-1) acquires the eating speed calculated by the eating speed calculation unit 341. Next, the state estimation unit 340 (340-1) references the recording database unit 600 (600L) using the identification information of the subject to acquire dietary information related to the subject. Next, the state estimation unit 340 (340-1) estimates a state, such as appetite or hunger, using the eating speed and dietary information. After outputting the state information, the state estimation unit 340 (340-1) ends the process ("end"). As a result, the state estimation unit 340 (340-1) estimates that, for example, when the eating speed is faster than normal, the person has an appetite or is hungry.

[0191] [Second specific example] FIG. 34 is a diagram illustrating a configuration example of the state estimation unit 340 (340-2) according to the eighth embodiment of the present disclosure. The state estimation unit 340 (340-2) estimates the state of the subject, such as mood or likes and dislikes, based on the eating speed and facial expression of the subject. The state estimation unit 340 (340-2) includes an eating speed calculation unit 341 and a facial expression estimation unit 342. The eating speed calculation unit 341 calculates the eating speed in the same manner as the eating speed calculation unit 341 described above. The facial expression estimation unit 342 estimates the facial expression of the subject based on the time-series captured images and eating behavior images. Next, an example of processing by the state estimation unit 340 (340-2) will be described. FIG. 35 is a flowchart illustrating an example of the state estimation unit 340 (340-2) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-2) starts the process ("START"), it then executes an eating speed calculation process (step ST8821). In the eating speed calculation process, the eating speed calculation unit 341 of the state estimation unit 340 (340-2) The state estimation unit 340 (340-2) then executes a facial expression estimation process (step ST8822). In the process, the facial expression estimation unit 342 of the state estimation unit 340 (340-2) calculates the eating speed in the same manner as the eating speed calculation unit 341 described above. The state estimation unit 340 (340-2) then executes a second state estimation (likes and dislikes, mood) process (step ST8823). In the second state estimation process, the facial expression estimation unit 342 of the state estimation unit 340 (340-2) estimates the facial expression of the measurement subject based on the time-series captured images and eating action images. The state estimation unit 340 (340-2) estimates the state of the subject, such as mood or likes and dislikes, based on the eating speed and facial expression of the subject using the eating speed calculated by the eating speed calculation unit 341, the facial expression estimated by the facial expression estimation unit 342, and the meal information. The state estimation unit 340 (340-2) estimates the likes and dislikes and mood of the meal by linking the eating speed of each menu item with the video of the subject's facial expression. For example, if the eating speed is faster than usual and the facial expression is smiling while eating, it is estimated that the subject is eating a favorite food or drink. Also, for example, if the eating speed is slower than usual and the facial expression is grim while eating, it is estimated that the subject is eating a disliked food or drink or is in a bad mood. The state estimation unit 340 (340-2) outputs state information indicating the estimation result and then ends the process ("end").

[0192] [Third specific example] FIG. 36A is a diagram illustrating a configuration example of a subject state detection unit 140 (140-3) according to the eighth embodiment of the present disclosure, and FIG. 36B is a diagram illustrating a configuration example of a state estimation unit 340 (340-3). The subject state detection unit 140 (140-3) detects the mastication of the subject to be measured. The subject state detection unit 140-1 is configured to include a chewing detection unit 141. The mastication detection unit 141 performs image analysis using the eating action image to detect the mastication of the measurement subject. The mastication detection unit 141 detects, for example, the number of mastications. The state estimation unit 340 (340-3) estimates the mastication state of the measurement subject. The state estimation unit 340 (340-3) estimates whether the subject is able to masticate and swallow. The state estimating unit 340 (340-3) includes a chewing tendency estimating unit 343. The mastication tendency estimation unit 343 estimates the mastication tendency of the subject over time. The state estimation unit 340 (340-3) estimates the state of mastication based on the mastication tendency estimated by the mastication tendency estimation unit 343. The state estimation unit 340 (340-3) outputs state information indicating the estimation result. Next, an example of processing by the state estimation unit 340 (340-3) will be described. FIG. 37 is a flowchart illustrating an example of the state estimation unit 340 (340-3) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-3) starts the process (“START”), it then executes a mastication information acquisition process (step ST8831). In the mastication information acquisition process, the mastication detection unit 141 of the state estimation unit 340 (340-3) performs image analysis using the eating action image to detect the mastication of the subject. The state estimation unit 340 (340-3) then executes a mastication degree estimation process (step ST8832). In the process, the mastication tendency estimation unit 343 of the state estimation unit 340 (340-3) estimates the mastication degree using the detection result by the mastication detection unit 141. The state estimation unit 340 (340-3) then executes a "third state estimation (estimating the mastication state)" process (step ST8833). In the third state estimation process, the state estimation unit 340 (340-3) estimates the time-series trend of the mastication degree, outputs state information indicating the estimation result, and then ends the process ("end").

[0193] [Fourth specific example] FIG. 38A is a diagram illustrating a configuration example of a subject state detection unit 140 (140-4) according to the eighth embodiment of the present disclosure, and FIG. 38B is a diagram illustrating a configuration example of a state estimation unit 340 (340-4). The subject state detection unit 140 (140-4) detects swallowing by the subject to be measured. The subject state detection unit 140 (140-4) includes a swallowing detection unit 142. The swallowing detection unit 142 acquires the number of swallows for each eating action or each meal using time-series captured images or eating action images. The state estimation unit 340 (340-4) includes a swallowing tendency estimation unit 344. The swallowing tendency estimation unit 344 performs image analysis using the eating behavior image to detect swallowing by the measurement subject. The mastication detection unit 141 detects, for example, the number of swallowings. Next, an example of processing by the state estimation unit 340 (340-4) will be described. FIG. 39 is a flowchart illustrating an example of the state estimation unit 340 (340-4) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-4) starts processing (“START”), it then executes swallowing information acquisition processing (step ST8841). In the swallowing information acquisition processing, the swallowing tendency estimation unit 344 of the state estimation unit 340 (340-4) acquires the number of swallows for each eating action or each meal in time series detected by the subject state detection unit 140 (140-4). The state estimation unit 340 (340-4) then executes a swallowing degree estimation process (step ST8842). In the swallowing degree estimation process, the swallowing tendency estimation unit 344 of the state estimation unit 340 (340-4) estimates the swallowing degree using the time-series number of swallowings. The state estimation unit 340 (340-4) then executes a "fourth state estimation (estimating likelihood of aspiration state occurring)" process (step ST8843). In the fourth state estimation process, the state estimation unit 340 (340-4) estimates likelihood of aspiration state occurring based on the tendency of the degree of swallowing estimated by the swallowing tendency estimation unit 344. The state estimation unit 340 (340-4) outputs state information indicating the estimation result and then ends the process ("end").

[0194] [Fifth Specific Example] FIG. 40A is a diagram illustrating a configuration example of a subject state detection unit 140 (140-5) according to the eighth embodiment of the present disclosure, and FIG. 40B is a diagram illustrating a configuration example of a state estimation unit 340 (340-5). The subject state detection unit 140 (140-5) includes a choking detection unit 143. The choking detection unit 144 performs image analysis using the eating behavior image to detect choking by the measurement subject. The choking detection unit 144 detects the number of choking incidents for each meal, for example. The state estimation unit 340 (340-5) estimates the likelihood of an aspiration state occurring based on choking. The state estimation unit 340 (340-5) includes a choking tendency estimation unit 345. The choking tendency estimation unit 345 estimates the chronological tendency of the measurement subject to choking based on the number of times the subject choked for each meal detected by the choking detection unit 143. The state estimation unit 340 (340-5) estimates the likelihood of an aspiration state occurring based on the choking tendency estimated by the choking tendency estimation unit 345. Next, an example of processing by the state estimation unit 340 (340-5) will be described. FIG. 41 is a flowchart illustrating an example of the state estimation unit 340 (340-5) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-5) starts the process (“START”), it then executes a choking information acquisition process (step ST8851). In the choking information acquisition process, the state estimation unit 340 (340-5) acquires the number of choking episodes for each meal detected by the choking detection unit 143. The state estimation unit 340 (340-5) then executes a choking degree estimation process (step ST8852). In the process, the choking tendency estimation unit 345 of the state estimation unit 340 (340-5) estimates the chronological tendency of the measurement subject to choking based on the number of times the subject choked for each meal. The state estimation unit 340 (340-5) then executes the "fifth state estimation (estimating likelihood of aspiration state)" process (step ST8853). In the process, the state estimation unit 340 (340-5) estimates likelihood of aspiration state based on the choking tendency estimated by the choking tendency estimation unit 345. The state estimation unit 340 (340-5) outputs an estimation result indicating likelihood of aspiration state. The state estimation unit 340 (340-5) outputs state information indicating the estimation result and then ends the process ("end"). The condition estimation unit 340 (340-5) outputs condition information for preventing aspiration based on the number of times the patient chokes.

[0195] [Sixth Specific Example] FIG. 42 is a diagram illustrating a configuration example of the state estimation unit 340 (340-6) according to the eighth embodiment of the present disclosure. The condition estimation unit 340 (340-6) estimates the physical condition of the subject. The state estimation unit 340 (340-6) includes a first tendency estimation unit 346. The first tendency estimation section 346 estimates the tendency of the physical condition of each subject based on the time-series dietary information. The condition estimation unit 340 (340-6) estimates the physical condition of the subject based on the tendency of the motor function. Next, an example of processing by the state estimation unit 340 (340-6) will be described. FIG. 43 is a flowchart illustrating an example of the state estimation unit 340 (340-6) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-6) starts the process (“START”), it then executes a record data acquisition process (step ST8861). In the record data acquisition process, the state estimation unit 340 (340-6) refers to the record database unit 600 (600H) using the identification information of the subject to acquire diet information related to the subject. The state estimation unit 340 (340-6) then executes a first trend estimation process (step ST8862). In the first trend estimation process, the first trend estimation unit 346 of the state estimation unit 340 (340-6) analyzes the diet information of the subject to be measured and estimates a trend related to the subject's physical condition. The state estimation unit 340 (340-6) then executes a sixth state estimation process (step ST8863). In the sixth state estimation process, the state estimation unit 340 (340-6) estimates a change in the physical condition of the measurement subject based on the tendency of the physical condition of the measurement subject. The state estimation unit 340 (340-6) outputs state information indicating the estimation result and then ends the process ("end").

[0196] [Seventh Specific Example] FIG. 44 is a diagram illustrating a configuration example of the state estimation unit 340 (340-7) according to the eighth embodiment of the present disclosure. The condition estimation unit 340 (340-7) estimates the condition of the subject's motor function. The state estimation unit 340 (340-7) includes a second tendency estimation unit 347. The second tendency estimation section 347 estimates a tendency regarding the motor function of the subject based on the time-series diet information of the subject. Next, an example of processing by the state estimation unit 340 (340-7) will be described. FIG. 45 is a flowchart illustrating an example of the state estimation unit 340 (340-7) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-7) starts the process (“START”), it then executes a record data acquisition process (step ST8871). In the record data acquisition process, the state estimation unit 340 (340-7) refers to the record database unit 600 (600H) using the identification information of the subject to acquire time-series diet information of the subject. The state estimation unit 340 (340-7) then executes a "second trend estimation (determine a trend related to motor function)" process (step ST8872). In the second trend estimation process, the second trend estimation unit 347 of the state estimation unit 340 (340-7) estimates a trend of motor function, such as the motor function of the fingertips, using time-series dietary information related to the subject. ↓ The state estimation unit 340 (340-7) then executes a "seventh state estimation (estimating the state of motor function)" process (step ST8873). In the seventh state estimation process, the state estimation unit 340 (340-7) estimates the state of motor function based on the tendency of motor function estimated by the second tendency estimation unit 347. The state estimation unit 340 (340-7) outputs state information indicating the estimation result and then ends the process ("end").

[0197] [Eighth Specific Example] FIG. 46 is a diagram illustrating a configuration example of the state estimation unit 340 (340-8) according to the eighth embodiment of the present disclosure. The state estimation unit 340 (340-8) estimates the cognitive state. The state estimation unit 340 (340-8) includes a third tendency estimation unit 348. The third tendency estimation unit 348 estimates the cognitive tendency based on the dietary information of the measurement subject. Next, an example of processing by the state estimation unit 340 (340-8) will be described. FIG. 47 is a flowchart illustrating an example of the state estimation unit 340 (340-8) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-8) starts the process (“START”), it then executes a record data acquisition process (step ST8881). In the record data acquisition process, the state estimation unit 340 (340-8) refers to the record database unit 600 (600L) using the identification information of the subject to acquire diet information related to the subject. The state estimation unit 340 (340-8) then executes a "third tendency estimation (determining a tendency related to cognition)" process (step ST8882). In the third tendency estimation process, the third tendency estimation unit 348 of the state estimation unit 340 (340-8) estimates a tendency related to cognition based on the dietary information of the subject. The third tendency estimation unit 348 estimates, for example, tendencies such as likes and dislikes of food and drink, and tendencies in the amount of food eaten for each type of food and drink. The reason for estimating such tendencies is that, as an early symptom of dementia, for example, a tendency to dislike vegetables and stop eating them becomes more pronounced. The state estimation unit 340 (340-8) then executes an "eighth state estimation (estimating a cognitive state)" process (step ST8883). In the eighth state estimation process, the state estimation unit 340 (340-8) estimates a cognitive state based on the tendency related to cognition estimated by the third tendency estimation unit 348. The state estimation unit 340 (340-8) outputs state information indicating the estimation result and then ends the process ("end").

[0198] [Ninth Specific Example] FIG. 48 is a diagram illustrating a configuration example of the state estimation unit 340 (340-9) according to the eighth embodiment of the present disclosure. The state estimation unit 340 (340-9) estimates the aspiration state from the tendency of the type of food or drink. The state estimation unit 340 (340-9) includes a fourth tendency estimation unit 349. The fourth tendency estimation unit 349 Next, an example of processing by the state estimation unit 340 (340-9) will be described. FIG. 49 is a flowchart illustrating an example of the state estimation unit 340 (340-9) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-9) starts the process (“START”), it then executes a recorded data acquisition process (step ST8891). In the recorded data acquisition process, the fourth tendency estimation unit 349 of the state estimation unit 340 (340-9) refers to the record database unit 600 (600H) using the identification information of the subject to acquire time-series dietary information of the subject. The state estimation unit 340 (340-9) then executes a "fourth trend estimation (estimating a trend in food types)" process (step ST8892). In the fourth trend estimation process, the fourth trend estimation unit 349 of the state estimation unit 340 (340-9) estimates a trend in food types based on the time-series diet information. The state estimation unit 340 (340-9) then executes the "ninth state estimation (estimating likelihood of occurrence of aspiration state)" process (step ST8893). In the ninth state estimation process, the state estimation unit 340 (340-9) estimates likelihood of occurrence of aspiration state based on the tendency of the type of food and drink estimated by the fourth tendency estimation unit 349. After outputting the state information, the state estimation unit 340 (340-9) ends the process ("end").

[0199] [10th Specific Example] FIG. 50 is a diagram illustrating a configuration example of the state estimation unit 340 (340-10) according to the eighth embodiment of the present disclosure. When the subject has no appetite, the condition estimation unit 340 (340-10) acquires biological information, estimates the cause of the lack of appetite and the state of physical condition, and outputs the information. The state estimation unit 340 (340-10) is configured to include a biological information acquisition unit 350. The biological information acquisition unit 350 acquires biological information output by the biological sensor 24. The biological information acquisition unit 350 may be configured to be input by an operator of the measurement device 10M. The biological information includes, for example, body temperature, blood pressure, SP02, heart rate, etc. Next, an example of processing by the state estimation unit 340 (340-10) will be described. FIG. 51 is a flowchart illustrating an example of the state estimation unit 340 (340-10) according to the eighth embodiment of the present disclosure. When the state estimation unit 340 (340-10) starts processing ("START"), it then executes a recorded data acquisition process (step ST8901). In the recorded data acquisition process, the state estimation unit 340 (340-10) refers to the record database unit 600 (600L) using the identification information of the subject to be measured and acquires dietary information related to the subject. If the state estimation unit 340 (340-10) analyzes the dietary information and determines that the subject has no appetite, it proceeds to the process of step ST8902. The state estimation unit 340 (340-10) then executes a biological information acquisition process (step ST8902). In the biological information acquisition process, the biological information acquisition unit 350 of the state estimation unit 340 (340-10) acquires biological information of the measurement subject recorded in chronological order. The state estimation unit 340 (340-10) then executes a tenth state estimation process (to estimate the cause or physical condition) (step ST8903). In the tenth state estimation process, the state estimation unit 340 (340-10) estimates the cause or physical condition using the diet information and biological information of the measurement subject. The state estimation unit 340 (340-10) outputs the estimation result. The state estimation unit 340 (340-10) outputs state information indicating the estimation result and then ends the process ("end").

[0200] According to this embodiment, various conditions of the subject can be estimated from dietary information including the amount of food and drink consumed and the amount of food eaten by the subject. For example, by estimating the following conditions and providing them as dietary information, it becomes possible to provide support, guidance, advice, etc. regarding dietary habits. ·appetite Hunger Full stomach Mood (happy / unhappy) Likes and dislikes ·Chewing tendency Swallowing tendency Motor function of the arms and fingers ·Dementia determination Possibility of aspiration ·physical condition It becomes possible to estimate such a state and output it as state information. As a result, the appetite state of the person being measured can be automatically grasped, so for example, if the person being measured is elderly, it can prevent nutritional imbalance, nutrient deficiency, and over-nutrition, allowing them to maintain their health and prevent deterioration of physical condition, illness, fractures, etc. in advance.

[0201] This embodiment further includes the following configuration. a state estimation unit that uses the accumulated amount of food and drink and the dietary information to output state information indicating the state of the measurement subject's physical condition; Equipped with the meal information generation unit outputs the state information output by the state estimation unit as the meal information. A measuring device characterized by: As a result, the present disclosure further has the effect of providing a control device that makes it possible to improve the accuracy of dietary measurement compared to conventional methods and to estimate the condition of the person being measured. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0202] Embodiment 9 In the above-described embodiment, the form of outputting dietary information including the state of the subject estimated based on the amount of food and drink consumed by the subject has been described. In the ninth embodiment, a configuration example for outputting advice on the dietary habits of a subject will be further described. In the ninth embodiment, among the components of the ninth embodiment, those components and their functions that are similar to those of the components of the first, second, third, fourth, fifth, sixth, seventh, or eighth embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0203] A configuration example of a control device and a diet measurement system including the control device according to a ninth embodiment of the present disclosure will be described. FIG. 52 is a diagram showing a configuration example of a diet measurement system 1 (1L) including a measuring device 10 (10L) according to the ninth embodiment of the present disclosure.

[0204] The diet measurement system 1 (1L) shown in FIG. 52 includes a measurement device 10 (10L) and an information source device 20 (20L).

[0205] The information source device 20H shown in FIG. 52 is configured to include an image capturing device 21, similar to the information source device 20 already described.

[0206] The measuring device 10L measures the diet of the subject. The measuring device 10L differs from the measuring device 10 already described in that it has a function to provide dietary support to the subject using dietary information such as the amount of food eaten and the amount of food consumed. The measuring device 10L shown in Figure 52 is configured to include a meal information acquisition unit 100 (100L), a quantity estimation unit 200 (200L), a meal information generation unit 300 (300L), a meal information output unit 400 (400L), a registration information acquisition unit 500 (500L), and a record database unit 600 (600L). The measuring device 10L shown in FIG. 52 differs from the measuring device 10 already described in particular in that the diet information generating unit 300L includes a diet support unit 360.

[0207] The meal information generating unit 300L includes a meal support unit 360. The dietary support unit 360 uses the accumulated amount of food and drink consumed and dietary information to generate and output advice regarding the dietary habits of the subject. The meal information generation unit 300L outputs meal information including the advice output by the meal support unit 360.

[0208] FIG. 53 is a diagram illustrating a configuration example of a meal support unit 360 (360L) according to the ninth embodiment of the present disclosure. The dietary support unit 360 uses the accumulated amount of food and drink consumed and dietary information to generate and output advice regarding the dietary habits of the subject. The dietary support unit 360 includes an n-th dietary advice output unit (1≦n≦N, N≧2) 361-n.

[0209] The n-th dietary advice output unit 361-n (1≦n≦N, N≧2) uses the accumulated amount of food and drink and dietary information to generate advice regarding the dietary habits of the subject.

[0210] The diet information generation unit 300L outputs the diet information including the advice output by the diet support unit 360.

[0211] Here, a processing example of the measurement device according to the ninth embodiment of the present disclosure will be described. FIG. 54 is a flowchart showing an example of processing in the meal information generating unit 300 (300L) according to Embodiment 9 of the present disclosure. When the diet information generation unit 300L starts the process ("START"), the diet information generation unit 300L then executes dietary support information output processing (step ST9000). In the dietary support information output processing, the dietary support unit 360 of the diet information generation unit 300L generates advice regarding the dietary habits of the measurement subject using the accumulated amount of food and drink and the dietary information. Once the diet information generating unit 300L has output the diet information including the dietary support information, it ends the process ("end").

[0212] Here, a detailed example of the processing of the meal support unit 360 will be described. FIG. 55 is a flowchart showing an example of processing by the meal support unit 360 in the meal information generation unit 300 (300L) according to Embodiment 9 of the present disclosure. When the dietary support unit 360 starts the dietary support information output process (“START”), it then executes the measurement subject information acquisition process (step ST9100). In the measurement subject information acquisition process, the n-th dietary advice output unit 361-n (1≦n≦N, N≧2) of the dietary support unit 360 refers to the record database unit 600 (600L) using the identification information of the measurement subject, and acquires the measurement subject information of the measurement subject. The dietary support unit 360 then executes a recorded data search process (step ST9200). In the recorded data search process, the n-th dietary advice output unit 361-n (1≦n≦N, N≧2) of the dietary support unit 360 searches for and acquires dietary information related to the diet of the subject indicated in the measurement subject information. The dietary support unit 360 then executes a recorded data analysis process (step ST9300). In the recorded data analysis process, the n-th dietary advice output unit 361-n (1≦n≦N, N≧2) of the dietary support unit 360 analyzes the acquired dietary information and obtains an analysis result. In the analysis, the n-th dietary advice output unit 361-n (1≦n≦N, N≧2) can use the registered information acquired by the registered information acquisition unit 500M. The dietary support unit 360 then executes advice data acquisition processing (step ST9400). In the advice data acquisition processing, the n-th dietary advice output unit 361-n (1≦n≦N, N≧2) of the dietary support unit 360 acquires advice data based on the analysis results. When using the registration information acquired by the registration information acquisition unit 500M, the n-th dietary advice output unit 361-n (1≦n≦N, N≧2) can provide advice such as the following, for example. Advice on how to limit overeating to prevent obesity and chronic disease It not only alerts the user to eat less, but also suggests appropriate portions of foods that may have an adverse effect on the body when the subject starts eating, helping to prevent obesity and chronic diseases. If the subject overeats, it will suggest recommended dishes and portions so that the subject can adjust their intake at future meals. Additionally, if a person has been eating an unbalanced diet over a certain period of time, the system will display the body shape that would result if the person continued to eat the same foods. By suggesting other foods to balance the diet, limiting portions, or suggesting other meal recipes and restaurants, the system will encourage the person to adopt a nutritionally balanced diet and prevent obesity and illness. Advice on the order in which food and drink are eaten to prevent blood sugar rise, obesity, aspiration, and overeating By analyzing time-series photographs, accumulated time-series food and drink amounts, time-series meal amounts, or registered information, the system obtains information on what, when, how quickly, and how much was eaten, and generates situations and advice to prevent blood sugar rise, obesity, aspiration, and overeating.Advice may be a combination of, for example, the order in which food is eaten and nutritional improvements, the number of times chewed, posture, eating movements and speed (speed at which chopsticks are brought to the mouth, chewing speed, swallowing speed), time of day eaten (morning, noon, evening), etc.

[0213] The dietary support unit 360 then executes dietary support information generation processing (step ST9500). In the dietary support information generation processing, the dietary support unit 360 generates and outputs dietary support information including advice generated by the n-th dietary advice output unit 361-n (1≦n≦N, N≧2). The dietary support unit 360 also records the dietary support information including advice in the recording database unit 600 (600L).

[0214] The meal support unit 360 then executes an end determination process (step ST9600 "End?"). In the termination determination process, the meal support unit 360 determines whether to terminate the processing of the meal support unit 360. The meal support unit 360 determines whether to terminate the processing of the meal support unit 360, for example, in accordance with an external termination command or an execution program. If the meal support unit 360 determines not to end the processing of the meal support unit 360 (step ST9600 "NO"), the meal support unit 360 proceeds to the processing of step ST9100, and repeats the processing from step ST9100. If the meal support unit 360 determines that the processing of the meal support unit 360 is to be ended (step ST9600 "YES"), the meal support unit 360 ends the processing ("End").

[0215] This embodiment further includes the following configuration. a dietary support unit that generates and outputs advice regarding the dietary habits of the subject using the accumulated amount of food and drink and the dietary information; Equipped with the diet information generation unit outputs the diet information including the advice output by the diet support unit. A measuring device characterized by: As a result, the present disclosure further has the effect of making it possible to improve the accuracy of dietary measurement compared to conventional methods, and to provide a measuring device that can generate dietary information to support dietary habits based on the amount of food and drink eaten or the amount of food eaten with improved accuracy. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0216] Embodiment 10 In the above-described ninth embodiment, a form has been described in which advice on the dietary habits of a subject is generated from dietary information. In the tenth embodiment, a configuration example will be described in which a notification or emergency call is made when an abnormality occurs in a subject to be measured based on meal information. In the tenth embodiment, among the components of the tenth embodiment, those components and their functions that are similar to those of the components of the first, second, third, fourth, fifth, sixth, seventh, eighth, or ninth embodiments already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0217] A configuration example of a control device and a diet measurement system including the control device according to a tenth embodiment of the present disclosure will be described. FIG. 56 is a diagram showing a configuration example of a meal measurement system 1 (1M) including a measuring device 10 (10M) according to the tenth embodiment of the present disclosure.

[0218] The diet measurement system 1 (1M) shown in FIG. 56 includes a measurement device 10 (10L) and an information source device 20 (20M).

[0219] The information source device 20M shown in FIG. 56 is configured to include an image capturing device 21 and a biometric sensor 24, similar to the information source device 20H already described.

[0220] The measuring device 10M measures the diet of a subject. The measuring device 10M differs from the measuring device 10 already described in particular in the following respects: It differs in that it has the function of determining abnormalities in the subject using dietary information such as the amount of food eaten and the amount of drink eaten, and then leaving the determination result as is. The measuring device 10M shown in Figure 52 is configured to include a meal information acquisition unit 100 (100M), a quantity estimation unit 200 (200M), a meal information generation unit 300 (300M), a meal information output unit 400 (400M), a registration information acquisition unit 500 (500M), and a recording database unit 600 (600M). The measuring device 10M shown in FIG. 52 differs from the measuring device 10 already described in particular in that the diet information generating unit 300M includes a status notifying unit 370.

[0221] The meal information generating unit 300M includes a status notifying unit 370. The condition notification section 370 uses the accumulated amount of food and drink and the dietary information to estimate an abnormality in the subject, and notifies the result of the estimation to a pre-stored output destination.

[0222] FIG. 57 is a diagram illustrating a configuration example of a status notification unit 370 according to the tenth embodiment of the present disclosure. The status notification unit 370 The status notification unit 370 includes an analysis unit 371 , an abnormality information output unit 372 , and an emergency notification unit 373 . The analysis unit 371 performs analysis using the accumulated amount of food and drink and dietary information, and estimates abnormalities in the subject. When the analysis result of the analysis unit 371 indicates an abnormality, the abnormality information output unit 372 generates and outputs abnormality information. Furthermore, the abnormality information output unit 372 outputs the abnormality information to a previously stored output destination. If the abnormality information output by the abnormality information output unit 372 is of a high level of urgency or more, the emergency notification unit 373 notifies a pre-stored emergency notification destination.

[0223] Next, a processing example of the measurement device according to the tenth embodiment of the present disclosure will be described. FIG. 58 is a flowchart showing an example of processing in the diet information generating unit 300 (300M) according to the tenth embodiment of the present disclosure. When the meal information generating unit 300M starts the process (“START”), it then executes a state information output process (step ST10010). In the state information output process, when the state notification unit 370 of the meal information generating unit 300M receives the identification information of the subject, it references the recording database unit 600M and outputs the state information of the subject.

[0224] The meal information generation unit 300M then executes a notification determination process (step ST10020). In the notification determination process, the state notification unit 370 of the meal information generation unit 300M executes the notification determination process. After executing the notification determination process, the meal information generation unit 300M ends the process and goes into standby.

[0225] Here, a detailed example of the notification determination process will be described. FIG. 59 is a flowchart showing an example of processing by the status reporting unit 370 according to the tenth embodiment of the present disclosure. When the status reporting unit 370 in the diet information generating unit 300M starts the process (“START”), it then executes a measurement subject information acquisition process (step ST10110). In the measurement subject information acquisition process, the analysis unit 371 of the status reporting unit 370 acquires measurement subject information by referring to the record database unit 600M in response to a request from the measurement subject, for example.

[0226] The status reporting unit 370 then executes a recorded data search process (step ST10120). In the recorded data search process, the analysis unit 371 of the status reporting unit 370 uses the measurement subject information and further refers to the record database unit 600M to search for recorded data related to the measurement subject indicated in the measurement subject information.

[0227] The status reporting unit 370 then executes a recorded data analysis process (step ST10130). In the recorded data analysis process, the analysis unit 371 of the status reporting unit 370 analyzes the recorded data as the search result. At this time, the analysis unit 371 may perform the analysis using the registered information acquired by the registered information acquisition unit 500M. The analysis unit 371 outputs the physical condition of the measurement subject as the analysis result.

[0228] The status notification unit 370 then executes a status information generation process (step ST10140). In the status information generation process, the abnormality information output unit 372 of the status notification unit 370 generates status information indicating the physical condition of the measurement subject according to the analysis result by the analysis unit 371.

[0229] The status notification unit 370 then executes an abnormality determination process (step ST10150 "Abnormal?"). In the abnormality determination process, the abnormality information output unit 372 of the status notification unit 370 determines whether an abnormality exists based on the state indicated in the status information and a preset state. If the abnormality information output unit 372 determines that an abnormality exists (step ST10150 "YES"), it then executes abnormality notification processing (step ST10160). In the abnormality notification processing, the abnormality information output unit 372 outputs abnormality information indicating an abnormal state to a pre-stored output destination. The pre-stored output destination is, for example, the meal information output unit 400M or a terminal device for family use. If the anomaly information output unit 372 determines that there is no anomaly ("NO" in step ST10150), the process proceeds to step ST10190.

[0230] Status notification unit 370 determines whether an emergency call is necessary (step ST10170 "Emergency call necessary?"). In this process, the emergency notification unit 373 of the status notification unit 370 determines whether or not an emergency notification is necessary based on the status indicated in the abnormality information and a preset status. If the emergency call is not made ("NO" in step ST10170), the emergency call unit 373 proceeds to the process of step ST10190.

[0231] If an emergency call is required (step ST10170 "YES"), the status notification unit 370 executes emergency call processing (step ST10180). In the emergency call processing, the emergency call unit 373 of the status notification unit 370 makes a call to a pre-stored emergency call destination.

[0232] Next, the status notification unit 370 executes an end determination process (step ST10190 "end?"). In the termination determination process, the status notification unit 370 determines whether to terminate the processing of the status notification unit 370. The status notification unit 370 determines whether to terminate the processing of the status notification unit 370 in accordance with, for example, an external termination command or an execution program. When the status notification unit 370 determines not to end the processing of the status notification unit 370 ("NO" in step ST10190), the processing proceeds to step ST10100, and the processing is repeated from step ST10100. When the status notification unit 370 determines that the processing of the status notification unit 370 is to be ended ("YES" in step ST10190), the status notification unit 370 ends the processing ("End").

[0233] This embodiment is particularly suitable for emergency situations. For example, if the subject's eating state does not fall within a preset eating state for a set period of time, the subject is notified of this. For example, if the subject's diet does not meet a predetermined dietary standard for a set period of time, the system can notify family members and affiliated institutions. In addition, in the event of an emergency, emergency contact can be made. In addition, because it accurately measures dietary status every day, it can continuously notify the person being measured, such as the elderly, and their family members of their daily nutritional intake status and changes, encouraging them to be aware of improving their eating habits.

[0234] This embodiment further includes the following configuration. a status notification unit that estimates an abnormality in the subject using the accumulated amount of food and drink and the dietary information and notifies the result of the estimation to a pre-stored output destination; A measuring device comprising: As a result, the present disclosure further has the effect of providing a measuring device that can determine and notify abnormalities based on the amount of food eaten or the amount of food consumed with improved accuracy, thereby enabling the accuracy of food measurement to be improved compared to conventional methods. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a measurement device, the above measurement method, or the above program.

[0235] Here, a hardware configuration for realizing the functions of the present disclosure will be described. FIG. 60 is a diagram illustrating a first example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. FIG. 61 is a diagram illustrating a second example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. Measurement device 10 (10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, 10L, 10M) of the present disclosure, , are realized by hardware such as that shown in FIG. 60 or FIG. 61, respectively.

[0236] Measuring device 10 (10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, 10L, 10M), , each of which is configured by, for example, a processor 10001, a memory 10002, an input / output interface 10003, and a communication circuit 10004, as shown in FIG. The processor 10001 and the memory 10002 are, for example, installed in a computer. The memory 10002 stores the computer Meal information acquisition unit 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G, 100H, 100L, 100M), eating behavior detection unit 110, measurement subject information acquisition unit 120, eating utensil information acquisition unit 130, subject state detection unit 140 (140-1, 140-2, 140-3), chewing detection unit 141, swallowing detection unit 142, choking detection unit 143, amount estimation unit 200 (200A, 200B, 200C, 200D, 200E, 200F, 200G, 200H, 200L, 200M), eating and drinking amount estimation unit 210, food and drink type estimation unit 220, uneaten amount acquisition unit 230, difference amount calculation unit 240, returned amount estimation unit 250, meal information generation unit 300 (300A, 300B, 300C, 300D, 300E, 300F, 300G, 300H, 300L, 300M), total amount of food and drink calculation unit 310, complete eating rate calculation unit 320, processing information calculation unit 330, state estimation unit 340 (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10), eating speed calculation unit 341, A facial expression estimation unit 342, a chewing tendency estimation unit 343, a swallowing tendency estimation unit 344, a choking tendency estimation unit 345, a first tendency estimation unit 346, a second tendency estimation unit 347, a third tendency estimation unit 348, a fourth tendency estimation unit 349, a biological information acquisition unit 350, a dietary support unit 360 (360L), an n-th dietary advice output unit (1≦n≦N, N≧2) 361-n, a status notification unit 370, an analysis unit 371, an abnormality information output unit 372, an emergency notification unit 373, a dietary information output unit 400 (400A, 400B, 400C, 400D, 400E, 400F, 40 The memory stores a program for causing the memory to function as a control unit (not shown), a registration information acquisition unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), a registration information acquisition unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), a registration reception unit 510, a photography command unit 700 (700F), a photography state determination unit 710, a photography control amount calculation unit 720, a photography control command output unit 730, a lighting command unit 800 (800G), a lighting state determination unit 810, a lighting control amount calculation unit 820, a lighting control command output unit 830, and a control unit (not shown).The processor 10001 reads and executes the program stored in the memory 10002, thereby controlling the mealtime information acquisition unit 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G, 100H, 100L, 100M), the eating behavior detection unit 110, the measurement subject information acquisition unit 120, the eating utensil information acquisition unit 130, the subject state detection unit 140 (140-1, 140-2, 140-3), the mastication detection unit 141, the swallowing detection unit 142, the choking detection unit 143, the quantity estimation unit 200 (200A, 200B, 200C, 200D, 200E, 200F, 200G, 200H, 200L, 200M), the eating behavior detection unit 110, the measurement subject information acquisition unit 120, the eating utensil information acquisition unit 130, the subject state detection unit 140 (140-1, 140-2, 140-3), the chewing detection unit 141, the swallowing detection unit 142, the choking detection unit 143, the quantity estimation unit 200 (200A, 200B, 200C, 200M), the eating behavior detection unit 110, the measurement subject information acquisition unit 120, the eating utensil information acquisition unit 130, the subject state detection unit 140 (140-1, 140-2, 140-3), the chewing detection unit 141, the swallowing detection unit 142, the choking detection unit 143, the quantity estimation unit 200 (200A, 200B, 200 D, 200E, 200F, 200G, 200H, 200L, 200M), intake amount estimation unit 210, food and drink type estimation unit 220, uneaten amount acquisition unit 230, difference amount calculation unit 240, return amount estimation unit 250, meal information generation unit 300 (300A, 300B, 300C, 300D, 300E, 300F, 300G, 300H, 300L, 300M), total intake amount calculation unit 310, completion rate calculation unit 320, processing information calculation unit 330, state estimation unit 340 (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10), an eating speed calculation unit 341, a facial expression estimation unit 342, a chewing tendency estimation unit 343, a swallowing tendency estimation unit 344, a choking tendency estimation unit 345, a first tendency estimation unit 346, a second tendency estimation unit 347, a third tendency estimation unit 348, a fourth tendency estimation unit 349, a biological information acquisition unit 350, a dietary support unit 360 (360L), an n-th dietary advice output unit (1≦n≦N, N≧2) 361-n, a status notification unit 370, an analysis unit 371, an abnormality information output unit 372, an emergency notification unit 373, a dietary information output unit 400 (400A, 400B, 400C, 400D, 400E, 400F, 400G, 400H, 400L, 400M), registration information acquisition unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), registration acceptance unit 510, photography command unit 700 (700F), photography state determination unit 710, photography control amount calculation unit 720, photography control command output unit 730, lighting command unit 800 (800G), lighting state determination unit 810, lighting control amount calculation unit 820, lighting control command output unit 830, and the functions of a control unit not shown are realized. In addition, the memory 10002 or other memory not shown implements a registration information storage unit 520, a record database unit 600 (600A, 600B, 600C, 600D, 600E, 600F, 600G, 600H, 600L, 600M), a food and drink quantity record database 610, a meal record database 620, a food and drink type database 630, and a storage unit not shown. Furthermore, the communication circuit 10004 realizes a communication unit (not shown).

[0237] The processor 10001 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, or a digital signal processor (DSP). Memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory, or a magnetic disk such as a hard disk or flexible disk, or an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk. The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a state where they can transmit data to each other. The processor 10001, the memory 10002, and the communication circuit 10004 are also connected in a state where they can transmit data to other hardware via the input / output interface 10003.

[0238] Alternatively, in the measuring device 10 (10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, 10L, 10M), the eating information acquisition unit 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G, 100H, 100L, 100M), the eating behavior detection unit 110, the measurement subject information acquisition unit 120, the eating utensil information acquisition unit 130, the subject state detection unit 140 (140-1, 140-2, 140-3), the mastication detection unit 141, the swallowing detection unit 142, the choking detection unit 143, the quantity estimation unit 200 (200A, 200B, 200C, 200 D, 200E, 200F, 200G, 200H, 200L, 200M), intake amount estimation unit 210, food and drink type estimation unit 220, uneaten amount acquisition unit 230, difference amount calculation unit 240, return amount estimation unit 250, meal information generation unit 300 (300A, 300B, 300C, 300D, 300E, 300F, 300G, 300H, 300L, 300M), total intake amount calculation unit 310, completion rate calculation unit 320, processing information calculation unit 330, state estimation unit 340 (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10), an eating speed calculation unit 341, a facial expression estimation unit 342, a chewing tendency estimation unit 343, a swallowing tendency estimation unit 344, a choking tendency estimation unit 345, a first tendency estimation unit 346, a second tendency estimation unit 347, a third tendency estimation unit 348, a fourth tendency estimation unit 349, a biological information acquisition unit 350, a dietary support unit 360 (360L), an n-th dietary advice output unit (1≦n≦N, N≧2) 361-n, a status notification unit 370, an analysis unit 371, an abnormality information output unit 372, an emergency notification unit 373, a dietary information output unit 400 (400A, 400B, 400C, 400D, 400E, 400F, 400 The functions of the image capturing unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), the registration information acquisition unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), the registration reception unit 510, the photography command unit 700 (700F), the photography state determination unit 710, the photography control amount calculation unit 720, the photography control command output unit 730, the lighting command unit 800 (800G), the lighting state determination unit 810, the lighting control amount calculation unit 820, the lighting control command output unit 830, and the control unit not shown may be realized by a dedicated processing circuit 20001, as shown in FIG.

[0239] The processing circuit 20001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), an SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration). In addition, the memory 20002 or other memory not shown implements a registration information storage unit 520, a record database unit 600 (600A, 600B, 600C, 600D, 600E, 600F, 600G, 600H, 600L, 600M), a food and drink quantity record database 610, a meal record database 620, a food and drink type database 630, and a storage unit not shown. Memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory, or a magnetic disk such as a hard disk or flexible disk, or an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk. Furthermore, the communication circuit 20004 realizes a communication unit (not shown). The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a state where they can transmit data to each other. In addition, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a state where they can transmit data to each other and to other hardware via the input / output interface 20003. In addition, in the measuring device 10 (10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, 10L, 10M), a mealtime information acquisition unit 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G, 100H, 100L, 100M), a meal behavior detection unit 110, a measurement target information acquisition unit 120, a eating utensil information acquisition unit 130, a target person state detection unit 140 (140-1, 140-2, 140-3), a chewing detection unit 141, a swallowing detection unit 142, a choking detection unit 144, a quantity estimation unit 200 (200A, 200B, 200C, 200D, 200E, 200F, 200G, 200H, 200L, 200M), 00E, 200F, 200G, 200H, 200L, 200M), intake amount estimation unit 210, food and drink type estimation unit 220, uneaten amount acquisition unit 230, difference amount calculation unit 240, return amount estimation unit 250, meal information generation unit 300 (300A, 300B, 300C, 300D, 300E, 300F, 300G, 300H, 300L, 300M), total intake amount calculation unit 310, completion rate calculation unit 320, processing information calculation unit 330, state estimation unit 340 (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10), an eating speed calculation unit 341, a facial expression estimation unit 342, a chewing tendency estimation unit 343, a swallowing tendency estimation unit 344, a choking tendency estimation unit 345, a first tendency estimation unit 346, a second tendency estimation unit 347, a third tendency estimation unit 348, a fourth tendency estimation unit 349, a biological information acquisition unit 350, a dietary support unit 360 (360L), an n-th dietary advice output unit (1≦n≦N, N≧2) 361-n, a status notification unit 370, an analysis unit 371, an abnormality information output unit 372, an emergency notification unit 373, a dietary information output unit 400 (400A, 400B, 400C, 400D, 400E, 400F, 4 The functions of the image capturing unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), the registration information acquisition unit 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), the registration reception unit 510, the photography command unit 700 (700F), the photography state determination unit 710, the photography control amount calculation unit 720, the photography control command output unit 730, the lighting command unit 800 (800G), the lighting state determination unit 810, the lighting control amount calculation unit 820, the lighting control command output unit 830, and the control unit (not shown) may be realized by separate processing circuits, or may be realized together by a processing circuit.

[0240] Or, in the measuring device 10 (10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, 10L, 10M), a mealtime information acquisition unit 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G, 100H, 100L, 100M), a mealtime behavior detection unit 110, a measurement target information acquisition unit 120, a eating utensil information acquisition unit 130, a target person state detection unit 140 (140-1, 140-2, 140-3), a chewing detection unit 141, a swallowing detection unit 142, a choking detection unit 143, a quantity estimation unit 200 (200A, 200B, 200C, 200D, 200E, 200F , 200G, 200H, 200L, 200M), eaten amount estimation unit 210, food and drink type estimation unit 220, uneaten amount acquisition unit 230, difference amount calculation unit 240, returned amount estimation unit 250, meal information generation unit 300 (300A, 300B, 300C, 300D, 300E, 300F, 300G, 300H, 300L, 300M), total eaten amount calculation unit 310, complete eating rate calculation unit 320, processing information calculation unit 330, state estimation unit 340 (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10), eating speed calculation unit 341 , facial expression estimation unit 342, chewing tendency estimation unit 343, swallowing tendency estimation unit 344, choking tendency estimation unit 345, first tendency estimation unit 346, second tendency estimation unit 347, third tendency estimation unit 348, fourth tendency estimation unit 349, biological information acquisition unit 350, dietary support unit 360 (360L), nth dietary advice output unit (1≦n≦N, N≧2) 361-n, status notification unit 370, analysis unit 371, abnormality information output unit 372, emergency notification unit 373, dietary information output unit 400 (400A, 400B, 400C, 400D, 400E, 400F, 400G, 400H, 400L, 400M), registration information acquisition The functions of the units 500 (500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M), the registration reception unit 510, the photography command unit 700 (700F), the photography state determination unit 710, the photography control amount calculation unit 720, the photography control command output unit 730, the lighting command unit 800 (800G), the lighting state determination unit 810, the lighting control amount calculation unit 820, the lighting control command output unit 830, and a control unit (not shown) may be realized by the processor 10001 and the memory 10002, and the remaining functions may be realized by the processing circuit 20001.

[0241] It should be noted that, within the scope of this disclosure, the embodiments may be freely combined, any component of each embodiment may be modified, or any component of each embodiment may be omitted.

[0242] The present disclosure can improve the accuracy of meal measurement compared to conventional methods, and is therefore suitable for use in, for example, a measurement device that performs meal measurement and a system that includes a measurement device. [Explanation of symbols]

[0243] 1 (1A, 1B, 1C, 1D, 1E, 1F, 1G, 1H, 1L, 1M) Meal measurement system, 10 (10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, 10L, 10M) Measuring device, 20 (20A, 20B, 20C, 20D, 20E, 20F, 20G, 20H, 20L, 20M) Information source device, 21 Shooting device, 22 Shooting position control device, 23 Shooting angle control device, 24 Biometric sensor, 30 Lighting device, 100 (100A, 100B, 100C, 100D, 100E, 100F, 100G, 100H, 100L, 100M) Meal information acquisition unit, 110 Eating behavior detection unit, 120 Measurement object information acquisition unit, 130, eating utensil information acquisition unit, 140, subject state detection unit, 141, chewing detection unit, 142, swallowing detection unit, 143, choking detection unit, 200 (200A, 200B, 200C, 200D, 200E, 200F, 200G, 200H, 200L, 200M), amount estimation unit, 210, eaten amount estimation unit, 220, food type estimation unit, 230, unaware amount acquisition unit, 240, difference amount calculation unit, 250, regurgitated amount estimation unit, 300 (300A, 300B, 300C, 300D, 300E, 300F, 300G, 300H, 300L, 300M), meal information generation unit, 310, total eaten amount calculation unit, 320 Eating completion rate calculation unit, 330 processed information calculation unit, 340 (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10) state estimation unit, 341 eating speed calculation unit, 342 facial expression estimation unit, 343 chewing tendency estimation unit, 344 aspiration degree estimation unit, 345 choking tendency estimation unit, 346 first tendency estimation unit, 347 second tendency estimation unit, 348 third tendency estimation unit, 349 fourth tendency estimation unit, 350 (350L) biological information acquisition unit, 360 meal support unit, 361-n nth meal advice output unit (1≦n≦N, N≧2), 370 state notification unit, 371 analysis unit, 372 Abnormality information output unit, 373 Emergency notification unit, 400 (400A, 400B, 400C, 400D, 400E, 400F, 400G, 400H, 400L, 400M) Meal information output unit, 500 (500A, 500B, 500C, 500D, 500E, 500F, 500G, 500H, 500L, 500M) Registration information acquisition unit, 510 Registration acceptance unit, 520 Registration information storage unit, 600 (600A, 600B, 600C,600D, 600E, 600F, 600G, 600H, 600L, 600M) Recording database section, 610 Food and drink amount recording database, 620 Meal record database, 630 Food and drink type database, 700 (700F) Photography command section, 710 Photography state determination section, 720 Photography control amount calculation section, 730 Photography control command output section, 800 (800G) Lighting command section, 810 Lighting state determination section, 820 Lighting control amount calculation section, 830 Lighting control command output section, 3100, 3200, 3300, 3400 Meal information, 3110, 3210, 3310, 3410 Date, 3120, 3220, 3320, 3420 Time period, 3130 Eating completion rate, 3230 Completion rate for each dish (food type), 3330 intake amount for each ingredient (food type), 3430 intake amount for each nutrient, 10001 processor, 10002 memory, 10003 input / output interface, 10004 communication circuit, 20001 processing circuit, 20002 memory, 20003 input / output interface, 20004 communication circuit.,

Claims

1. a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink eaten by the subject for each eating action using the eating action image; a meal information generation unit that generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit; A measuring device comprising:

2. The mealtime information acquisition unit Accepting photographed images in time series, and detecting and acquiring the eating action image using the accepted photographed images in time series.

2. The measuring device according to claim 1.

3. a registration information acquisition unit that acquires registered measurement subject information including a photographing area for each measurement subject that has been stored in advance; Furthermore, The mealtime information acquisition unit and detecting the eating action image using the registered measurement subject information acquired by the registration information acquisition unit.

3. The measuring device according to claim 1 or 2.

4. The meal information output by the meal information generation unit includes: The amount of food and drink consumed for each meal is accumulated in chronological order.

3. The measuring device according to claim 1 or 2.

5. The quantity estimator, Furthermore, the amount of food and drink consumed is output for each type of food and drink.

3. The measuring device according to claim 1 or 2.

6. The meal information output by the meal information generation unit includes: The amount of food and drink consumed for each eating behavior is accumulated in chronological order for each meal, and the total amount of food and drink consumed for each type of food and drink is included.

6. The measuring device according to claim 5.

7. The quantity estimator, Using the captured image, an uneaten amount, which is the amount of food and drink remaining, is obtained, and the uneaten amount is further used to estimate the amount of food and drink.

3. The measuring device according to claim 1 or 2.

8. The amount estimation unit estimates an eating rate using the uneaten amount, and estimates the amount of eating using the eating rate.

8. The measuring device according to claim 7.

9. The meal information output by the meal information generation unit includes: a complete eating rate for each meal, which indicates a ratio of the total amount of food and drink accumulated for each eating action to the amount of food and drink before the start of the meal; 8. The measuring device according to claim 7.

10. a registration information acquisition unit that acquires a registered initial amount, which is the amount of each food or drink stored before starting a meal; Furthermore, The quantity estimator, Furthermore, the intake amount is estimated using the registered initial amount acquired by the registration information acquisition unit.

8. The measuring device according to claim 7.

11. The meal information output by the meal information generation unit includes: a complete eating rate for each meal and each food and drink indicating a ratio of the total amount of food and drink accumulated for each eating action to the registered initial amount; The measuring device according to claim 10 .

12. The mealtime information acquisition unit Furthermore, tableware information including the type of tableware, which is the type of tableware for each purpose of the food and drink contained in the photographed image, is acquired, The quantity estimator, Furthermore, the amount of food not yet eaten for each type of tableware is calculated and acquired using the type of tableware indicated in the tableware information.

8. The measuring device according to claim 7.

13. The system further includes a registration information acquisition unit that acquires registered tableware information including the positions of food and drink containers and the types of the containers stored in advance, The quantity estimator, Furthermore, the amount of food not yet eaten is calculated and acquired using the registered eating utensil information acquired by the registration information acquisition unit. The measuring device according to claim 12 .

14. When the type of utensil included in the eating utensil information indicates a shared utensil that is a utensil shared by multiple measurement subjects, The quantity estimator, The amount of food and drink not consumed is calculated and acquired for each eating and drinking behavior of each of the measurement subjects using the shared appliance, and the amount of food and drink consumed by the measurement subjects who have performed the eating and drinking behavior is estimated. The measuring device according to claim 12 .

15. The quantity estimator, Furthermore, the amount of food and drink regurgitated by the subject is estimated using the time-series captured images, and the estimated amount of food and drink regurgitated is used to calculate and output the amount of food and drink.

3. The measuring device according to claim 1 or 2.

16. an imaging control unit that outputs, based on the eating action image, a control signal for instructing the imaging position, the imaging direction, or the imaging position and the imaging direction of the imaging device that captured the photographed image including the eating action image; 3. The measuring device according to claim 1, further comprising:

17. an illumination control unit that controls, based on the eating action image, an illumination device that illuminates a photographing area in which a photographed image including the eating action image is photographed; 3. The measuring device according to claim 1, further comprising:

18. a state estimation unit that uses the accumulated amount of food and drink and the dietary information to output state information indicating the state of the measurement subject's physical condition; Equipped with the meal information generation unit outputs the state information output by the state estimation unit as the meal information.

3. The measuring device according to claim 1 or 2.

19. a dietary support unit that generates and outputs advice regarding the dietary habits of the subject using the accumulated amount of food and drink and the dietary information; Equipped with the diet information generation unit outputs the diet information including the advice output by the diet support unit.

3. The measuring device according to claim 1 or 2.

20. a status notification unit that estimates an abnormality in the subject using the accumulated amount of food and drink and the dietary information and notifies the result of the estimation to a pre-stored output destination; 3. The measuring device according to claim 1, further comprising:

21. a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink consumed by the subject for each eating action using the eating action image to a server outside the device; a meal information output unit that receives from the server meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit and outputs the meal information; A terminal device comprising:

22. A meal measurement system including a terminal device and a server, The terminal device a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs to the server an amount of food and drink consumed by the subject for each eating action using the eating action image; The server a meal information generation unit that receives the amount of food and drink consumed for each eating behavior output by the amount estimation unit, and outputs meal information based on the received amount of food and drink consumed to the terminal device. The terminal device A meal information output unit is provided which receives the meal information output by the server and outputs the received meal information. A meal measurement system characterized by:

23. A measurement method using a measurement device, a mealtime information acquisition step in which a mealtime information acquisition unit of the measurement device acquires mealtime behavior images, which are images of mealtime behaviors of each measurement subject included in time-series captured images; an amount estimation step in which a quantity estimation unit of the measuring device uses the eating behavior image to output an eating amount, which is the amount of food and drink eaten by the measurement subject for each eating behavior; a meal information generating step in which a meal information generating unit of the measuring device generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimating unit; A measuring method comprising:

24. A measurement method using a terminal device, a mealtime information acquisition step in which a mealtime information acquisition unit of the terminal device acquires mealtime behavior images, which are images of mealtime behaviors of each measurement subject included in time-series captured images; an amount estimation step in which the amount estimation unit of the terminal device outputs an amount of food and drink eaten by the subject for each eating action using the eating action image to a server outside the device; a meal information output step in which the meal information output unit of the terminal device receives from the server meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit and outputs the meal information; A measuring method comprising:

25. Computer, a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink eaten by the subject for each eating action using the eating action image; a meal information generation unit that generates meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit; A measuring device comprising: A program characterized by operating as

26. Computer, a mealtime information acquisition unit that acquires mealtime behavior images, which are images of mealtime behaviors of each subject included in time-series captured images; an amount estimation unit that outputs an amount of food and drink consumed by the subject for each eating action using the eating action image to a server outside the device; a meal information output unit that receives from the server meal information based on the amount of food and drink consumed for each eating behavior output by the amount estimation unit and outputs the meal information; A terminal device comprising: A program characterized by operating as

Images